Awarded DECI Projects – DECI 12

Find below the results of DECI-12 (Distributed European Computing Initiative) call.

Projects were from the following research areas:

Astro Sciences Projects

Project Title: Galactic Chemodynamics in the Era of Gaia

Project Leader: Brad Gibson, University of Central Lancashire, Preston, UK

Resource Awarded: 4 288 000 core hours on CSC – Sisu, CSC – Sisu_XC40 and EPCC – Archer


  • Dr. Christopher Few – University of Exeter, School of Physics, Exeter, UK
  • Dr. Daisuke Kawata – University College London, London, UK
  • Dr. Chiaki Kobayashi – University of Herdfortshire, Centre for Astrophysics Research, Hatfield, UK

We propose to run two families of simulations of Milky Way-type systems, as a pathway to the goal of more realistic Milky Way simulations; the codes employed are our particle-based GCD+ and our new chemistry-enhanced, grid-based code, RAMSES-CH. One family is based upon “zoom” style cosmological simulations, for which we will run 8 simulations with different merger histories and environments, and explore how these characteristics impact upon the formation and evolution of each galaxy’s sub-components, including their stellar and gaseous discs, bulge, and stellar and hot coronal halo. The other family is a controlled simulation of the Galactic disc; using high resolution simulations. We will study the formation processes of the detailed structures in the Milky Way-type galaxies such as spiral arms and their effects on gas and stellar dynamics and star formation. Our codes are unique in terms of their self-consistent treatment of chemical evolution in non-linear dynamical simulations, which allow them to predict the distribution of elements in stars and gas as a function of position, kinematics, and age, allowing their comparison directly with observations. The associated chemical information will allow us to self-consistently generate mock observational data, using our stellar population synthesis package. This will link directly to the anticipated returns from future missions and experiments, such as ESA’s Gaia and the Gaia-ESO Survey, and allow us to directly compare our models with the observed kinematical and chemical properties of stars in the Milky Way. The mock observational data and simulated outputs will be made publicly available.

Project Title: Galaxies with Light And Matter interaction

Project Leader: Dr. Karl Joakim Rosdahl, Leiden University, Leiden Observatory, Leiden, The Netherlands

Resource Awarded: 4 384 000 core hours on EPCC – Archer


  • M.Sc. Alexander Richings – Leiden University, Leiden Observatory, Leiden, The Netherlands
  • Prof. Joop Schaye – Leiden University, Leiden Observatory, Leiden, The Netherlands
  • Prof. Romain Teyssier – University of Zurich, Zürich, Switzerland

The highly complicated problem of the formation and evolution of galaxies requires the use of three dimensional hydrodynamical simulations. Simulation work has added tremendous insight into the interplay of mechanisms which regulate galaxies. Yet many problems remain unaccounted for, the most impending one being that simulated galaxies are too compact and too efficient at forming stars, compared to observations. The inclusion of radiation feedback, i.e. the interaction of stellar radiation and the gas that fills the galaxies and their environment, is thought an important piece of the puzzle in solving the problem and gaining further understanding. Yet, in part due to the complexity and computational cost involved, simulation treatments of radiation hydrodynamics (RHD), needed to model this interaction, have been largely out of reach, although recent simulation work has seen a rise in variously well motivated non-RHD approximations of radiation feedback.
The researchers have recently developed a fast and robust RHD version of the widely used hydrodynamical simulation code RAMSES. With the code, they will perform cosmological simulations of galaxy formation and evolution in order to study how stellar radiation impacts galaxies and their environment.
The main points to be studied with the simulations are i) the role of the radiation in regulating the condensation of gas and subsequent transformation into stars, ii) in generating outflows, also known as galactic winds, which are routinely observed around galaxies, iii) how radiation and outflows interact with accretion of gas into the galaxies, and iv) the impact of radiation on the observable signatures of galaxies and their environment. The output of this work has the potential to advance our understanding of the evolution of galaxies and interaction with their environment, and to assist in interpreting observations of the interstellar and circumgalactic medium, i.e. of galaxies and their environment.

Project Title: Our Neighbourhood in the Universe: From the First Stars to the Present Day

Project Leader: Dr. Ilian Iliev, University of Sussex, Physics and Astronomy, Sussex, UK

Resource Awarded: 8 073 800 core hours on CINECA – GALILEO, CINECA – PLX and EPCC – Archer

The process of reionization, driven by radiation from the first galaxies had profound effects on the formation of early cosmic structures and has left a lasting impression on subsequent galaxy and star formation. This process is not yet well understood, however, due to its complex nature and scarcity of observational data. Detailed numerical modelling is critical for understanding these complex interactions, as well as for guiding and interpreting the observations of this epoch. Due to its proximity to us, our local volume is by far the best-studied patch in the Universe, with a wealth of data available. The Epoch of Reionization (EoR) leaves imprints on the smallest galaxies which can only be observed today in the nearby universe. This proposal focuses on the feedback effects of the First Stars on the formation of early cosmic structures and the resulting observational signatures. The questions we will be looking into are: 1) how do the radiative feedback from the First Stars hosted in minihaloes affect the formation of early structures and subsequent star formation?; 2) is the recently pointed out effect of local modulation of the star formation in minihaloes due to differential supersonic drift velocities between baryons and dark matter important or not, both in the universe at large and the local one?; and 3) how does the metal enrichment and the transition from PopIII (metal-free) to PopII stars occur locally and how is this reflected in the metallicity distribution of the observed Local Group dwarf galaxies and globular clusters?. We will address these questions with series of state-of-the-art radiative transfer, hydrodynamics and radiative hydrodynamics simulations. Within our simulation set some simulations will use gaussian random noise initial conditions, while others will use initial conditions that are constrained in such a way that they faithfully reproduce our Local Group of galaxies and its large-scale environment at the present day.

Project Title: Microphysical Plasma Effects in Early Solar Systems

Project Leader: Dr. Oliver Gressel, Niels Bohr Institute, Niels Bohr International Academy, Copenhagen, Denmark

Resource Awarded: 9 225 000 core hours on ICHEC – Fionn-thin


  • Dr. Chao-Chin Yang – Lund University, Astronomy and Theoretical Physics, Lund, Sweden

As the nurseries of planetary systems, protoplanetary disks are of key interest to planet formation theory. Their dynamics and structure depend critically on the influence of magnetic fields that couple to tenuously ionized and low-density regions. As a consequence, microphysical effects have a strong influence on both the evolution of the magnetic field, and the ways in which the field affects the gas. With a combination of global and local MHD simulations, we will probe the effects of ambipolar diffusion and ohmic resistivity on the structure of the magnetic fields, and consequences for the evolution of the gas and the embedded solids that ultimately form planets. We will focus specifically on two particular aspects of early solar systems where these microphysical processes are critical: the role of magnetic fields in limiting accretion onto gas giant planets, and thermal processing of solids by magnetic dissipation heating. Gas giant planets, like Jupiter and Saturn as well as the observed exoplanet population of Hot Jupiters, are expected to form via gravitational accretion of surrounding gas onto a rocky core. Given the diversity of masses within the gas giant planets observed to date, it is an interesting question to pose whether there are generic mechanisms that determine the total amount of gas accreted. This mandates to develop a new and refined picture of the surrounding gaseous disk. Being comparatively cold and dense, the physical state of the disk plasma is dominated by external ionizing X-ray and cosmic-ray radiation, leading to a layered vertical structure – with turbulent, magnetized surface layers and a magnetically-decoupled midplane. This ‘dead-zone’ picture is further complicated by ambipolar diffusion, which is expected to dominate in the tenuous hot corona of the disk and within the lowdensity gap opened by the core. The complex interplay of non-ideal plasma dynamics and first-principle microphysics critically demands the use of a powerful computational approach using state-of-the-art resources. The accretion flow though a PPD, which powers the generation of magnetic fields in sufficiently ionized regions, brings to bear a massive reservoir of gravitational potential energy. For the disk to produce the observed accretion onto the young star, this energy must be dissipated in the disk flow. As the disk dynamo transfers energy into the magnetic field, this energy is available to be dissipated in regions of high electrical current – current sheets. At the high density midplane of the inner regions of the disk, the dominant microphysics of the current sheets is ohmic resistivity. We have recently demonstrated with high-resolution self-consistent MHD simulations that these current sheets can yield surprisingly large temperature variations. However, the numerical methods employed so far limit the practical regions of exploration both in maximum resolution, and by limiting the investigation to constant ohmic resistivity. We have separately shown, in simplified models, that when the temperature dependence of the resistivity is included in the dynamics of current sheets, they are subject to the ‘short-circuit instability’, which results in dramatically stronger and more localized heating. The primary objective in this context will be to investigate, for the first time in a self-consistent turbulent environment, the short-circuit instability of current sheets.

Project Title: Modelling the molecular content of high redshift galaxies (MoMoGal)

Project Leader: Prof. Dr. Cristiano Porciani,

Resource Awarded: 12 150 000 core hours on CYFRONET – Zeus BigMem and SURFSARA – Cartesius


  • Dr. Aaron Ludlow – University of Bonn, Argelander Institute for Astronomy, Bonn, Germany
  • Dr. Emilio Romano-Diaz – University of Bonn, Argelander Institute for Astronomy, Bonn, Germany
  • Matteo Tomassetti – University of Bonn, Argelander Institute for Astronomy, Bonn, Germany

Understanding the formation of disk galaxies is a outstanding problem in astrophysics. The formidable range of scales involved and the highly coupled, non-linear dynamics can only be addressed by numerical simulations run on massively-parallel supercomputers. These simulations combine gravity and fluid-dynamics solvers to follow the hierarchical assembly of dark-matter halos, the subsequent condensation of baryonic (and leptonic) material towards their centers, and the formation of star clusters in the densest regions. Small-scale phenomena that cannot be spatially resolved (e.g. star formation and energy return from the stellar clusters to the gas) are therefore treated phenomenologically. Recent steps forward include improved `sub-grid` modeling of the interstellar medium and star-formation, as well as increased spatial and temporal resolution made possible by the availability of better facilities and algorithms. Through the DECI-9 call we have been awarded supercomputing time to simulate the formation of a massive galaxies from the dawn of time up to redshift two, when the universe was 3.3 Gyr old. We here request to extend this study by simulating 20 objects in order to build an array of templates for high-redshift galaxies with different size, metallicity and environment. In our previous work, we developed a computational framework to track the abundance of molecular hydrogen and carbon monoxide during the galaxy formation process. In our models, star formation is regulated by the local abundance of molecular hydrogen rather than the total gas density, as is customary. This was motivated by recent observational evidence of a tight correlation between the star formation rate and the H2 abundance. The primary objective of this proposals is to test these models against a larger set of high-redshift galaxies, for which more observational data are available. Our proposed research programme will validate, or refute, recent claims that molecular-regulated star formation might explain the existence of `dark` dwarf galaxies that never reached the critical density threshold for molecule and star formation. Moreover, our results will be of paramount importance for interpreting recent observations of the interstellar medium made with the Herschel space observatory (in combination with other multiwavelength studies) and for designing observational campaigns with the ALMA array (which has been partially active since 2011 and will become fully operational in 2015). This project is part of a large ongoing programme titled `The conditions and impact of star formation` (SFB 956) and will benefit from the crosstalk between theorists and observers in the German ALMA Regional Center (Bochum-Bonn-Cologne).

Project Title: Simulations of cosmological boxes using a novel star formation and feedback model

Project Leader: Dr Giuseppe Murante, INAF – O.A. Trieste, Italy

Resource Awarded: 2 000 000 core hours on NIIF – NIIFI SC

We propose to carry out hydrodynamical simulations of cosmological volumes aimed at investigating galaxy formation and its interplay with IGM at different redshifts. We will use the TreePM-SPH code Gadget-3, in which we implemented MUPPI, our novel state-of-the-art sub-resolution model to describe star formation and stellar feedback. Our simulations will also include metal-dependent cooling, a detailed description of chemical evolution and an innovative and AGN feedback. Our approach is to use effective sub-resolution models in moderate resolution cosmological simulations, and sample the model parameter space with feasible computing resources. We already used MUPPI to simulate a periodic 25 Mpc cosmological volume, using four configurations of our sub-resolution model. In this proposal we plan to continue exploring the parameter space, running four more 25 Mpc boxes: (1) with an alternative estimation of the molecular gas fraction, (2) varying star formation efficiency and feedback strength with gas surface density, (3) switching on AGN feedback, (4) varying the stellar Initial Mass Function in quiet and bursty environments. We will then choose two configurations that best reproduce the observations, and use them to run two larger boxes of size of 50 Mpc. The aim of our work is twofold. First, we want to produce a sample of galaxies that gives the best possible reproduction of observational properties, to be used for detailed comparisons with observational data of galaxies and IGM at all accessible redshifts. Second, we want to understand how energetic feedback (from stars and AGN) shapes such properties in the course of galaxy evolution, so as to be able to interpret the successes/failures of our simulations as statements on the kind of feedback necessary to produce them. We will also investigate what are the observational signatures of different forms of feedback which can enable us to distinguish the astrophysical processes that give rise to feedback

Project Title: Planck LFI 2014 data analysis

Project Leader: Hannu Kurki-Suonio, University of Helsinki, Finland

Resource Awarded: 7 000 000 core hours on CSC – Sisu and CSC – Sisu_XC40


  • Andres Curto – Institute of Physics of Cantabria, Santander, Spain
  • Elina Keihänen – University of Helsinki, Finland
  • Reijo Keskitalo – University of California, Berkeley, USA
  • Kimmo Kiiveri – University of Helsinki, Finland
  • Theodore Kisner – Lawrence Berkeley National Laboratory, USA
  • Valtteri Lindholm – University of Helsinki, Finland
  • Luis Mendes – European Space Agency, SCIOPS, Madrid, Spain
  • Marina Migliaccio – University of Cambridge, UK
  • Paolo Natoli – University of Ferrara, Italy
  • Martin Reinecke – Max Planck Institute for Astrophysics, Garching, Germany
  • Matti Savelainen – University of Helsinki, Finland
  • Anna-Stiina Suur-Uski – University of Helsinki, Finland
  • Jussi Väliviita – University of Helsinki, Finland

Planck is a European Space Agency satellite mission, whose task is to map the structure of the cosmic microwave background (CMB) in unprecedented detail, surpassing the accuracy of previous missions, like the NASA WMAP. The cosmic microwave background is radiation from the Big Bang, and it shows us the structure of the early universe.
Planck constrains cosmological models and examines the birth of large-scale structure in the universe. It is thought that this structure originates from quantum fluctuations in the very early universe during a period of accelerated expansion called inflation, but Planck results are needed for a better understanding of this. The main scientific results from Planck are cosmological, but as a by-product, Planck will also yield all-sky maps of all the major sources of microwave to far-infrared emission, opening a broad expanse of other astrophysical topics to scrutiny. Planck was launched in May 2009. The first cosmological results were published in March 2013; there will be a second release in 2014; and a final third release with the final results from the mission in 2015. Planck carries two instruments, the Low-Frequency Instrument (LFI) and the High-Frequency Instrument (HFI), utilizing different technologies. It is important to map the sky at many frequencies to be able to separate the cosmic microwave background from the astrophysical foreground radiation. Two Planck data processing centres (DPCs) have been set up, one for each instrument, but the most resource-intensive tasks need to be done on supercomputers. Because of the weakness of the signal and the high accuracy desired, Planck data analysis is a complicated task, requiring sophisticated statistical methods to separate out the signal from instrument noise and systematic effects. Simulation work will dominate the computational load of Planck data analysis. Analysis pipelines will indeed be predominantly run on simulated rather than real data since Planck analysis codes either require simulations for selfcalibration and validation of results, or depend critically on simulations for the results themselves. The objective of this project is to carry out resource-intensive tasks that are needed as part of Planck LFI data analysis. These tasks include:
timeline-to-map Monte Carlo simulation of Planck data: thousands of realizations of instrument noise and cosmic microwave background signal; also simulations of astrophysical foreground radiation signal; the analysis of this data in parallel to the analysis of the real data from the sky
cosmological parameter estimation for a number of cosmological models, in particular those related to multi-field inflation.

Bio Sciences Projects

Project Title: Molecular simulations of transmembrane receptors in native-like membranes: focus on allosteric ligands

Project Leader: Agnieszka Kaczor, University of Eastern Finland, Kuopio, Finland

Resource Awarded: 7 000 000 core hours on CSC – Sisu_XC40 and EPCC – Archer


  • Damian Bartuzi – Medical University of Lublin, Poland
  • Ramon Guixa-Gonzalez – Barcelona Biomedical Research Park, Spain
  • Maria Marti-Solano – Barcelona Biomedical Research Park, Spain
  • Jana Selent – Barcelona Biomedical Research Park, Spain
  • Katarzyna Targowska-Duda – Medical University of Lublin, Poland

Human membrane proteins, in particular metabotropic and ionotropic transmembrane receptors, currently constitute the largest group of targets for drugs available on the market. In particular, G protein-coupled receptors (GPCRs) are drug targets for about 50% of recently launched drugs. However, state-of-the-art experimental procedures, able to characterize in depth both GPCR modulation in health and disease and the molecular mechanisms of drug action at these receptors, have provided a more nuanced picture than previously expected. Several aspects of GPCR function, which are currently being characterized, clarify some regulatory processes regarding these receptors and, at the same time, introduce novel levels of complexity which should be taken into consideration for rational drug design. Furthermore, nicotinic and glutamatergic ion channels are also very important drug targets.

Nowadays, molecular dynamics is an important computational tool to study transmembrane receptors in complex systems under native-like conditions at a molecular level. Molecular dynamics simulations can yield revealing information on receptor stability and on the mechanisms by which different modulators can modify the equilibrium between different receptor populations. The rapidly increasing computational resources, together with new crystal structure information on transmembrane receptors, will surely allow a deeper understanding on the basis of receptor transitions to be attained and help to guide the design of ligands stabilizing particular conformational states.

The project has the following general objectives: (1) Determination of the effect of orthosteric ligands, positive and negative allosteric modulators, water molecules and ions on GPCR functioning (activation, interactions with G proteins) and the gating of ion channels as well as design of compounds with more favorable properties; (2) Investigation of the process of GPCR oligomerization, determination of the most probable oligomerization interface and study of the effect of bivalent ligands on GPCR dimer functioning. All-atom molecular dynamics in native-like membranes will be applied to study: (1) the first events of the process of GPCR activation as well as the role of water and sodium ions in this process (cannabinoid system receptors, opioid receptors, dopamine receptors); (2) the modulation of selected GPCRs by positive and negative allosteric ligands; (3) the effect of CRIP1a protein on the stabilization of the inactive conformation of the CB1 receptor; (4) the interaction of D2 receptor bivalent ligands with the D2 receptor dimer; (5) the effect of competitive, uncompetitive and non-competitive antagonists of kainate ion channels on receptor functioning and gating; (6) the effect of positive and negative allosteric modulators on the functioning and gating of nicotinic ion channels, involving determination of the role of water molecules and selected ions. Coarse-grained molecular dynamics (CGMD) simulations will aim to study the assembly of CB1 and MOR opioid receptors in membranes of different compositions as well as gating of ion channels. Umbrella sampling will be used to study the process of ligand association and dissociation from selected transmembrane receptors. Normal mode analysis will be used to study the process of GPCR dimerization and the process of gating of glutamatergic and nicotinic ion channels. QM/MM simulations will be used to study the details of ligand-receptor interactions.

Project Title: Role of water and quinone dynamics in the catalytic mechanism of complex I

Project Leader: Ilpo Vattulainen, Tampere University of Technology, Finland

Resource Awarded: 14 000 000 core hours on PDC – Beskow and PDC – Lindgren


  • Tomasz Rog – Tampere University of Technology, Finland
  • Vivek Sharma – Tampere University of Technology, Finland

Life on earth is sustained by a constant input of energy. In all biological systems this energy is predominantly obtained from the hydrolysis of ‘high-energy’ phosphate bonds of adenosine triphosphate (ATP). This means that all biological systems require a constant supply of ATP through various mechanisms. In most organisms, ATP is mainly synthesized via a process called oxidative phosphorylation or cellular respiration – a process in which the energy in food is converted into ATP. Complex I or NADH:ubiquinone oxidoreductase acts as the first enzyme in the cellular respiration process of many organisms, including mitochondria and bacteria. It catalyzes a complex oxidoreduction reaction in which two electrons from NADH are supplied to quinone (or ubiquinone, UQ). This latter exergonic reaction is responsible for the pumping of protons across the biological membrane. The proton gradient thus formed across the membrane is then utilized in the synthesis of ATP by another molecular machine – ATP synthase. The current project aims to study the structure and dynamics of complex I using state-of-the-art atomistic molecular dynamics (MD) simulations. Complex I from mitochondria is a large protein weighing ca. 1 MDa and has been found to be associated with numerous mitochondrial and neuro-degenerative disorders. The detailed understanding of its molecular mechanism is thus necessary to develop novel therapeutics. Functional studies on complex I in the past were limited by its large size, its inherent complexity, and non-availability of structural data. However, the recently solved crystal structure of the entire complex I from Thermus thermophilus bacterium has provided a strong impetus to complex I research. The project proposed here is to our knowledge the first large-scale computational study to be performed on the entire structure of complex I. We perform long classical MD simulations on the complete structure of complex I in several different states. The simulations have potential to provide atomic level details of its molecular mechanism, which is crucial in understanding the basis of many pathologies associated with complex I.

Project Title: Computational studies of sodium channels

Project Leader: Carmen Domene, University of Oxford, UK

Resource Awarded: 561 000 core hours on CINECA – GALILEO and CINECA – PLX


  • Simone Furini – University of Siena, Department of Medical Surgery and Bioengineering, Siena, Italy

To understand fully the vast majority of chemical and biophysical processes, it is often necessary to examine their underlying free-energy behaviour. Ion conduction and selectivity in ion channels cannot be predicted reliably without the knowledge of the associated free-energy changes. Much of the statistical mechanical framework for calculating free-energy differences has been developed some time ago. How to apply this framework in computer simulations in an effective fashion remains, however, a very active area of research. In recent years, remarkable progress has been made in this field. However, while several techniques have been thoroughly tested in small systems, the complexity of most systems under study requires careful work. In addition, the time scales that can be currently simulated are only in the range of hundreds of nanoseconds for all-atom classical molecular dynamics. As the events that we would like to study in this project occur on a time scale much longer than the time scale accessible with conventional molecular dynamics, a combination of computational techniques that work at different length and time scales will be employed such as for example umbrella sampling and metadynamics. Clearly, ‘brute force’ simulations cannot cover the full range of length and time scales that are needed to understand the functioning of these proteins. In particular, this project aims at providing insight into the mechanism of ion permeation in mammalian sodium channels by modelling using a prokaryotic channel how the structure of the selectivity filter and the free-energy profile of permeating Ca++ ions are altered by changing the protonation state of a key residue.

Project Title: The optical spectra of photosynthetic pigment-protein complexes: A linear-scaling density functional theory study of the role of local environments, fluctuations and protein conformation in biological light harvesting

Project Leader: Dr. Nicholas Hine, Cambridge University, Cavendish Laboratory, UK

Resource Awarded: 4 590 000 core hours on EPCC – Blue Joule

The light reactions of photosynthesis occur in a diverse range of pigment-protein complexes (PPCs) which are individually tailored to perform photon capture, exciton transfer and charge separation. Remarkably, their internal efficiencies can often approach 100%, and the recent, and unexpected, observation of long-lasting quantum coherence in their energy transfer dynamics has led to the new suggestion that biomolecular quantum effects may be an important part of this efficiency. Of particular interest in the rapidly emerging field of `quantum effects in biological systems` is the relationship between optical quantum states and their protein environment, with several recent works proposing that a complex interplay between them, not only stabilises coherence under hot and noisy biological conditions, but may also provide the degrees of freedom required to harness or tune quantum effects for better functionality. The theoretical study of exciton transport through protein-pigment complexes depends critically on parameters derived from first principles quantum mechanical calculations. It is thought that the detailed structure of the protein matrix is set up to funnel energy highly efficiently through the complex. Yet, it is these long-ranged electrostatic interactions that make the theoretical problem so difficult. We have previously shown that system sizes of more than 2000 atoms are required to converge the optical transition energies. Density functional theory (DFT) is usually the method of choice for quantum mechanical simulations such as these. However, systems of this size set a challenge for traditional plane wave DFT codes, since their computational expense generally scales as the cube of the number of atoms. In this project, we propose to use ONETEP, a DFT code that scales linearly with the number of atoms, and which is efficiently parallelised and able to run on very large-scale HPC architectures. A recent advance implemented in ONETEP enables calculation of the response of the system to time-dependent optical excitations (TDDFT), thus improving the accuracy of excited state measurements whilst retaining favourable scaling to large system sizes and large numbers of cores. Furthermore, the pigment-protein complex is in constant motion, and little is known about whether these fluctuations in the environment may set up correlations between excitations on different pigments and, hence, aid exciton transport. We have recently used the computational software FRODA to generate an ensemble of structures that represent the long time-scale behaviour of the pigment-protein complex. This group of structures will be analysed with ONETEP to search for general theories that may explain how the motion of the protein is linked to highly efficient energy transport. In this way, lessons learned from light-harvesting in deep-sea bacteria will one day help in the design of artificial energy efficient photovoltaics.

Earth Sciences Projects

Project Title: Dynamical Model of Ice Transport and Evolution

Project Leader: DR Poul Christoffersen, University of Cambridge, UK

Resource Awarded: 2 100 000 core hours on CSC – Sisu and CSC – Sisu_XC40

Observations of rapid widespread mass loss from the Greenland Ice Sheet (GrIS) have raised concerns about its potential contribution to sea level under the changing climate of the coming century. Between half and two thirds of the GrIS’s 0.8 mm/yr contribution to sea level rise can be attributed to changes in the rate of iceberg calving at the GrIS’s marine terminating outlet glaciers. Unfortunately, due to the complexity of the processes involved and the potential feedback mechanisms, the link between climate and calving remains poorly understood. Recent empirical and theoretical evidence has highlighted submarine melting at the ice-ocean interface; changing basal lubrication; and the seasonal formation of a sikussak, a semi-rigid mixture of icebergs and sea-ice, as potential sources of calving variability. However, in order to assess which, if any, of the above exerts the primary control on calving glacier stability, a new generation of numerical models is required. In order to address this, European glaciologists and computer scientists working with the glaciological modelling suite Elmer/Ice formed the “Elmer Calving” collaboration, specifically focusing on developing and implementing calving glacier dynamics into the existing Elmer/Ice glaciological code repository. Store Glacier is a fast-flowing outlet glacier in the Uummannaq region of West Greenland, and has been the subject of many field investigations by various researchers over the past decade. In summer 2014, a multifaceted field project, led by the Scott Polar Research Institute (SPRI), will aim to determine characteristics of the basal sediment, crucial to understanding glacier dynamics, and undertake aerial surveys of the calving front in order to better understand the processes relevant to calving. The wealth of existing and on-going data collection makes Store Glacier an ideal candidate for a numerical modelling investigation. An on-going numerical modelling project based at SPRI, led by Dr Poul Christoffersen, aims to investigate various aspects of the dynamics of calving glaciers using Elmer/Ice. This project consists of three parallel and complementary streams respectively focusing on: the dynamic response of calving termini to the external and internal climate related processes described above; the rheology and dynamics of the sikussak; and a GrIS-wide investigation of basal properties using inverse methods. The computational requirements of this project are significant. In particular, the high spatial and temporal resolutions required of the 3D calving model, as well as the huge spatial extent of a full GrIS 3D model, means that our project would not be possible without a PRACE HPC allocation.

Project Title: Vegetation-feedback effect in the high-latitudes under climate warming – Impact on regional climate through atmospheric circulation

Project Leader: Prof. Deliang Chen, University of Gothenburg, Department of Earth Sciences, Gothenburg, Sweden

Resource Awarded: 500 000 core hours on EPCC – Archer


  • Dr. Tinghai Ou – University of Gothenburg, Department of Earth Sciences, Gothenburg, Sweden

High-latitudes have experienced pronounced climate warming in recent decades. Accompanying is overall enhancement in vegetation cover and greenness in the sub-Arctic tundra and boreal forest, which not only modifies the local energy and water balance, but also invokes a number of interactive processes in the climate system through the atmospheric circulation. These local and remote influences may enhance or weaken the original warming. Despite the importance of the vegetation feedback, however, understanding and quantitative estimation of the vegetation feedback effect remains deficient. This project aims at quantifying and better understanding the vegetation feedback effect in the highlatitudes under a doubling of CO2 by using a vegetation-climate coupled model. By performing a series of experiments for present and future climates, we will quantify the local and remote effects of the changed vegetation. Specifically, focus will be on the impacts on the Arctic Ocean and sea ice through atmospheric circulation, and on the high latitude climates with a focus on Northern Europe. Climate sensitivity is dictated by the feedbacks in the climate system. The vegetation-feedback is one of the less well understood feedbacks. Through this project we will learn more about how the vegetation interacts with the rest of the climate system, which will help improve our ability to understand how and why the global and regional climates will vary and change.

Engineering Projects

Project Title: Simulation of Adverse Pressure Gradient Turbulent Boundary Layers

Project Leader: Assistant Prof. Gungor Ayse, Istanbul Technical University, Faculty of Aeronautics and Astronautics, Istanbul, Turkey

Resource Awarded: 645 120 core hours on NIIF – NIIFI SC


  • Mr. Suleyman Karaca – Istanbul Technical University, Faculty of Aeronautics and Astronautics, Istanbul, Turkey
  • Dr. Mark Simens – Universidad Politécnica de Madrid, ETSI Aeronáuticos, Spain

The goal of this proposal is to investigate the statistical and dynamical properties of wall-bounded turbulence in adverse pressure gradient (APG) turbulent boundary layers (TBLs) using Direct Numerical Simulation (DNS). The target Reynolds numbers based on the momentum thickness is Re=500. The research program involves firstly the DNS of a non-equilibrium APG TBL at the verge of separation. This will require the careful application of the farfield boundary conditions to ensure we achieve the desired non-equilibrium state. Another important goal of this project is to obtain sufficiently well resolved data on an APG boundary layer at a sufficiently high Reynolds number to be able to tackle fundamental questions related to the scaling of this type of boundary layer flow. We will also be analysing the coherent structures in APG TBLs to clarify if they are similar to structures found in other wall-bounded flows or are similar to free shear layer structures.

Resource Awarded: 3 240 000 core hours on IPB – PARADOX

Project Title: Direct numerical simulation of finite size fibres in turbulent flow

Project Leader: Prof. Gustav Amberg, KTH, Department of Mechanics, Stockholm, Sweden

Resource Awarded: 6 250 000 core hours on CSC – Sisu and CSC – Sisu_XC40


  • Dr. Minh Do-Quang – KTH, Department of Mechanics, Stockholm, Sweden

This project is a continuation of the DNSFT project (DECI-10 / PRACE ‘s project), to further investigate the validation, optimisation and speed-up of the SLILAB code on a large scale, before applying for a tier- 0 system call. The goal of this project is to adapt a numerical model of the dynamical behaviour of finite-size fibres in high Reynolds number turbulent flow. The turbulent flow is modelled by an entropy lattice Boltzmann method and the interaction between fibres and carrier fluid is modelled through an external boundary force method (EBF). Direct contact and lubrication force models for fibre-fibre interactions and fibre-wall interaction are taken into account to allow for a full four-way interaction. This model will allow us to study the influence of wall effects and interaction effects on the turbulent flow. A code (named SLILAB) has been developed at Mechanics department, Royal institute of Technology, Sweden. The new version of SLILAB is now based on the Palabos library environment and offers a fully coupled model between fluid and solid particles. This code has been tested for a year in some HPC centers in Sweden and HECTOR in UK. It shows a good parallel efficiency. Due to the limitation of CPU hours we can get and the capacity of the computer we can access, the simulation we have tested before is for a small turbulent flow in a small channel size (friction Reynolds number=180 on a box 2cm height and 3cm width). To get better knowledge about the complicated phenomena at a scale that can be used to study the dynamic behaviour of fibres in a paper-making machine, for example in a head-box, or in a groove of a refiner, etc, we really need access to a large scale simulation resource as the ones provided by PRACE, to test and verify our code at the larger scales. Moreover, to get even better performance we also need the enabling help of PARCE experts for another 4 to 6 months.

Project Title: FENICS-HPC – High performance adaptive finite element methods for turbulent flow and multi-physics with industrial applications

Project Leader: Prof. Johan Hoffman, KTH, Numerical Analysis, Stockholm, Sweden

Resource Awarded: 1 250 000 core hours on PDC – Beskow and PDC – Lindgren


  • Dr. Luca Facciolo – Vattenfall AB, Safety Analysis, Stockholm, Sweden
  • Dr. Johan Jansson – KTH, Numerical Analysis, Stockholm, Sweden

The main scientific objectives of this proposal is to evaluate a new simulation technology for Computational fluid dynamics (CFD) and Fluid-structure interaction (FSI), applied to industrial problems of great interest for Vattenfall AB. The massive computational cost for resolving all turbulent scales in such problems makes Direct Numerical Simulation of the underlying Navier-Stokes equations impossible. Instead, various approaches based on partial resolution of the flow have been developed, such as Reynolds Averaged Navier-Stokes (RANS) or Large-Eddy Simulation (LES). For these methods new questions arise: what is the accuracy of the approximation, how fine scales have to be resolved, and what are the proper boundary conditions? To answer such questions, a number of challenges have to be addressed simultaneously in the fields of fluid mechanics, mathematics, numerical analysis and HPC. The main focus of the research at the Computational Technology Laboratory (CTL) is the development of high performance, parallel, adaptive algorithms and software for FEM modeling of turbulent flows and multi-physics, including FSI. The adaptive finite element method G2 and its implementation in the FEniCS-HPC software framework has been developed over the past 10 years for time-resolved simulations of turbulent flows and it works as an implicit LES method with a residual based sub-grid model that accounts for the unresolved scales. With mesh adaptivity based on `a posteriori` error estimates, efficient parallelization, and the use of unstructured meshes, Within our group, there are a number of projects in various applications areas, where the new adaptive algorithms are being used and developed. These areas include aerodynamics, aero-acoustics, biomedicine, geophysics and FSI. In the past 3 years, we have obtained significant results in the development of G2 and FEniCS-HPC. These include: a hybrid MPI+PGAS linear algebra backend, which enhanced the scalability for larger core counts; the successful computation of the flow past a full DLR-F11 aircraft and landing gear as part of benchmarking workshops organized by NASA and Boeing, where our adaptive results were specifically highlighted in the workshop summary by the organizers, and turbulent FSI computation of vibrating and contacting vocal folds. In 2013 we were granted the EU FP7 project EUNISON for the simulation of the human voice based on our framework. In this collaboration project with Vattenfall AB we will use our FEniCS-HPC framework to simulate fluid flow and FSI in pipe systems in nuclear power plants. Two examples are identified as being of high interest: mixing of water with temperature differences in a T-junction, and pressure wave generation and interaction in the pipe system. Simulation of the structure loads in these cases is highly demanding. The innovation potential of this proposal is to use a new adaptive finite element method coupled with supercomputing technology, to obtain a very high computational resolution, which for the first time will give engineers at Vattenfall the possibility to study these problems in detail, to evaluate the standard simulation software against high-resolution simulations and to study the effect of mesh resolution using the adaptive methodology.

Project Title: Assessment of the Wave Expansion Method for acoustics in engineering problems

Project Leader: Assistant Prof. Ciarán O’Reilly, KTH, Sweden

Resource Awarded: 3 750 000 core hours on EPCC – Archer, PDC – Beskow and PDC – Lindgren


  • Assistant Prof. Gunilla Efraimsson – KTH, Sweden
  • PhD student Romain Futrzynski – KTH, Sweden

The Wave Expansion Method (WEM) is a discrete numerical method to solve propagation equations and as such can be used to study the acoustic field emanating from sound sources. Knowledge of this field can lead, through better optimization, to noise reduction of e.g. vehicles, fans, ducts or engines. The WEM uses a discrete mesh to model the region where the sound propagation is to be studied, but it has several advantages over traditional methods like Finite Elements. The main advantage is that the WEM requires much less mesh-points to accurately resolve acoustic waves. This higher efficiency makes possible studies in domains larger than the sizes achievable with classical methods. In this project a WEM code will be improved to add the ability to solve for acoustic fields when a background flow is going through the domain. A flow field and acoustic sources are first computed with traditional CFD solver in the case of a plane’s landing gear. Then the code will be used to compute the acoustic field far away from the landing gear, and the results compared to both experimental measurements and other methods based on aero-acoustic analogies.

Project Title: Fully resolved simulations of fluid flows close to the vapor-liquid critical point

Project Leader: Dr. Rene Pecnik, Delft University of Technology, The Netherlands

Resource Awarded: 10 200 000 core hours on SURFSARA – Cartesius and VSB-TUO – Anselm


  • Prof.Dr. B.J. Boersma – Delft University of Technology, The Netherlands

Fluids at supercritical pressure, i.e at pressures above their vapor-liquid critical point, do not undergo distinct liquid to gas phase transitions. These fluids exhibit significant deviations from ideal behavior and their peculiar properties can be exploited in many industrial applications. In chemical industry fluids at supercritical pressure are used in extraction processes, such as desorption, drying, cleaning; formation of micro and nano particles using rapid expansion of supercritical solutions in pharmaceutical industry. Applications in the energy industry involve biodiesel production, working fluids in power cycles, refrigerants, enhanced oil recovery, fuel injection systems. In most of the aforementioned applications heat transfer under turbulent conditions plays a crucial role, whereby the physical mechanism and modeling requirements are currently not well understood.
During the continuous liquid-vapor transition the thermophysical properties of the fluid vary significantly within a narrow temperature range across the pseudo-critical temperature. The pseudo-critical temperature is defined as the temperature at which the specific heat at constant pressure attains its peak value. For example, the specific heat at constant pressure by a factor of 20 within a small temperature range and the density decreases by a factor of 4. These large thermophysical property variations alter the conventional behavior and statistical properties of turbulent flows and their corresponding turbulent heat transfer.
Research in the field of heat transfer at supercritical pressure has been active since the fifties to support the thermal design of fossil fuel power plants operating at supercritical pressures. The interest in this field regained momentum in the nineties owing to its potential to improve the thermal efficiency in modern nuclear plants. Several experiments were conducted during this period using water or CO2 flowing in heated vertical tubes at supercritical pressures to collect data for heat transfer distributions and wall temperatures. Most of the experiments were conducted in the turbulent regime with high Reynolds numbers. Results from these experiments, especially in upward flows, showed peculiar features of turbulent heat transfer to supercritical fluids such as heat transfer enhancement and deterioration; especially when the bulk temperature was less than the pseudo-critical temperature and the wall temperature was higher than the pseudo-critical temperature. Direct Numerical Simulation (DNS) can help to better understand the mechanisms involved in these heat transfer characteristics by obtaining insight in to the turbulent structures at supercritical pressures.

Project Title: Turbulence-combustion interaction in Syngas flame

Project Leader: Dr Francesco Battista, Universita di Roma la Sapienza, Dipartimento di Meccanica e Aeronautica, Italy

Resource Awarded: 2 000 000 core hours on CYFRONET – Zeus BigMem

The needed to decreases the pollutant emission for human health preservation and the production costs nowadays drives the use of fuels alternative to the classical fossil one. In particular hydrogen and hydrogen-based fuels are considered the proper fuels to this purpose. Among them the Syngas is considered the most promising especially to be employed in gas turbine for energy production or in reciprocating diesel engines. As a matter of fact Syngas is a mixture of hydrogen, carbon-monoxide, and small concentration of methane and carbon-dioxide and is mainly produced by steam reforming. Few studies have been addressed the Syngas flame interaction with turbulence, and, in literature, it does not exist a deep and organic analysis of the Syngas turbulent flame dynamics at different equivalent ratio or fuel composition. Aim of the present project is to fill this gap by producing a accurate database on the Syngas turbulent jet flame at different (four values) equivalent ratio, in particular only lean conditions will be addressed, and different fuel composition, namely hydrogen/carbon-monoxide mole ratio (five values). This will allow to deep understand the dynamics of the Syngas flames. As a matter of fact the pure hydrogen flame is extensively addressed in several studies in literature, in particular it has been seen that the high mass diffusivity of hydrogen is the main feature that affects its dynamics. It produces and enhances the so-called thermo-diffusive instability, producing the alternation of regions with intense chemical activity and of region characterized by the quenched flame. Preliminary simulations have been shown that the carbon-monoxide oxidation tends to mitigate the pure hydrogen flame instability and to avoid the local quenching characterizing the hydrogen flame. The simulation planned are expected to be helpful to understand the dynamics of the syngas flame, in particular addressing the opposite dynamics of the pure hydrogen flame and of the carbon-monoxide oxidation.

Project Title: Three-dimensional optimal active control of drag

Project Leader: Dr Xuerui Mao, University of Durham, UK

Resource Awarded: 6 750 000 core hours on CYFRONET – Zeus BigMem and EPCC – Archer

Two-dimensional optimal control of aerodynamic drag on bluff or streamlined bodies has been extensively studied. The obtained optimal control forcing was found to be no longer optimal for three-dimensional conditions, since large-scale three-dimensional structures, turbulence and the promising spanwise velocity control to reduce drag cannot be effectively taken into account in the absence of a third dimension. However three-dimensional optimal control has not been actively investigated owing to the high memory requirements to save the full historical data. In most of the work documented on three-dimensional control for drag reduction in solid body or boundary layer flows, the control forcing has been determined empirically or experimentally. This proposed project aims at developing three-dimensional optimal control of drag in flow around a NACA0012 aerofoil using a gradient-based adjoint method. Control, in the form of velocity boundary conditions of the Navier-Stokes equations, is considered as the perturbation of the uncontrolled flow, and is optimized for minimum drag. A checkpointing scheme, which is a tradeoff between the computational wall-time and the memory, is implemented to avoid the need to save the full developing history of the controlled flow. A novel algorithm is developed to calculate the optimal (in the linear sense) step length in the iterative optimization process. Even though the computed optimal control is distributed around the whole surface of the body, by adopting various norms to evaluate the magnitude of the control, a concentrated control forcing around limited segments of the body surface can be obtained. This concentrated forcing can be generated by a limited number of actuators in physical experiments. Two-dimensional optimal control of drag has been tested at low Reynolds number in collaborations between the Group at Durham University, Imperial College London and Monash University. In this project, we attempt to conduct three-dimensional optimal control at Reynolds number ~ 100,000 taking into account effects of outer streaks to near-wall streaks and subsequently friction drag observed most recently in a spanwise wall-oscillation control. Such an iterative optimization of three-dimensional control forcing requires state-of-the-art high-fidelity computations and cannot be simulated without a high amount of computer time. Access to the Tier-1 system would provide this capability which is essential for the project to go ahead.

Materials Science Projects

Project Title: Attosecond Control of Electron and Ion Dynamics

Project Leader: Prof. Matthieu Verstraete, University of Liege, Physics, Liege, Belgium

Resource Awarded: 5 000 000 core hours on EPCC – Archer


  • Dr. Micael Oliveira – Universidade de Coimbra, Center for Computational Physics, Coimbra, Portugal
  • Dr. Theodoros Papadopoulos – University of Liege, Physics, Liege, Belgium

We will investigate the real-time attosecond dynamics induced by ultra-short laser pulses in both organic and inorganic systems. Laser pulses are used to analyse the dynamics of molecules and surfaces. Ultrashort attosecond pulses can be used to direct the chemistry, the breaking, and the rearrangement of chemical bonds. Novel catalysis mechanisms, photosensitive reactions, and the preparation of states for quantum computing will all result from the mastery of attosecond laser physics.
In the ACEID PRACE project we will study the electronic response of small and medium sized molecules to attosecond pulses, using a real-time propagation scheme in the OCTOPUS code ( Within time dependent density functional theory (TDDFT) the interaction between electrons is boiled down to a classical interaction plus exchange correlation (xc) effects, which contain the details of the chemical bonding and dynamics. Here we will explore recent proposals of xc functionals, both in their ground state (adiabatic) formulations and with energy dependencies. The few benchmarks performed to date focus on few-electron or model systems, where DFT is at its most unfavorable. Comparing to quantum chemical reference points and established failures of adiabatic TDDFT we will determine the best xc approximations available for quantum dynamics of many-electron systems.
As a second step the ionic dynamics induced by the initial electronic state preparation will be studied. The dream of laser-chemists is to control the outcome of a chemical reaction (ionic motion through the breaking or re-arrangement of bonds) through the initial state the electrons are placed in. The coupling between the two (electronic and ionic) subsystems is complex, but naturally and implicitly contained in the real time dynamics scheme. The main limitation of this scheme is the calculation of forces, through the Ehrenfest method in the present implementation. Again through comparison with ionic dynamics of benchmark systems studied with quantum chemical methods (by a partner group and in literature) we will determine the limits of the Ehrenfest method and explore alternatives, such as, e.g., surface hopping or spawning methods.

Project Title: GraphEne oXide as a transpOrter of Drug molecUleS Project acronym

Project Leader: Dr. Biplab Sanyal, Uppsala University, Department of Physics and Astronomy, Uppsala, Sweden

Resource Awarded: 3 750 000 core hours on CASTORC – Cy-Tera and CYFRONET – Zeus BigMem

The motivation of this project is the application of theoretical tools to understand and predict the complex properties of graphene oxide, which belong to the frontier research topics of condensed matter physics nowadays. The project aims to explore the use of Graphene Oxide (GO) as a potential drug delivery target by accurate first principles computational studies. Modern medicines rely on targeted delivery of the drug molecules so that the curing effect of the drugs can be obtained by a very small concentration of the compound. Clearly, this objective will play an even more significant role in the medicine in future, which makes the research on nano-carriers of the drug molecules as important as the research of new drug molecules. This project aims to have a fundamental understanding of the interactions between drug molecules and GO as a nano-carrier of drugs and hence the identification of suitable drug molecules for this purpose. The theoretical works in this project will be complemented by the experimental synthesis and characterization done by experimental groups in Sweden and Italy.

Project Title: Morphology and charge transport properties of organic field-effect transistors

Project Leader: Dr. Francesco Mercuri, National Research Council (CNR), Institute of Molecular Sciences and Technologies, Italy

Resource Awarded: 2 500 000 core hours on CYFRONET – Zeus

The development of devices based on organic electronics, and in particular organic field-effect transistors (OFETs), is expected to make significant progresses the next future. However, intrinsic physico-chemical properties of the organic thin-film constituting the active layer of the device and complex fabrication processes result in a sensible limitation of performance and reliability. These limitations, in turn, hinder a clear understanding of the functioning of devices. Therefore, a scientific and technological breakthrough can only be achieved by the proper representation of phenomena related to the inherent complexity of devices. In this respect, simulations constitute a preferential tool of investigation in the field, being able to provide a detailed picture of phenomena at the nanoscale involved in the functioning of OFETs with atomistic resolution. Nevertheless, a realistic description of the systems under investigation can only be achieved by simulations performed on large model systems, often beyond the scope of standard computational approaches. These issues are particularly relevant in the assessment of structure-property relationships in nanoaggregates, where the role of quasi-crystalline structures, domains, grain boundaries and interfaces at the nanoscale determines the performance of devices. Accordingly, one of the major issues in the development of OFETs is the precise knowledge about the relationship between morphology of the organic layers and transport properties of devices. Namely, the morphology of gate dielectric and organic semiconducting layers and phenomena at the interface are among the major responsibles of efficient OFET functioning. The MoTOFET project focuses on the application of advanced computational techniques in the definition of structure-property relationships in materials for organic electronics through simulation of morphology and charge transport properties. To this end, large-scale molecular dynamics simulations will be coupled to electronic structure calculations. The approach proposed is expected to provide a realistic description of the phenomena involved in the interplay between morphology, dynamics and charge transport in OFETs. One of the key factors to achieve the objective of MoTOFET is the use of efficient computing facilities and infrastructures, as those provided by the DECI initiative, thus meeting the challenges posed by the computational approach of the project. The outcomes of MoTOFET constitute a major step forward in the definition of a simulation paradigm for the design and development of novel materials, architectures and processes for OFETs, thus contributing to the development of technological and industrial applications.

Project Title: Multifunctional two-dimensional and layered materials

Project Leader: Professor Dr Leonidas Tsetseris, National Technical University of Athens, Physics – Computational Materials Science, Athens, Greece

Resource Awarded: 903 168 core hours on ICM – Boreasz

In project Multi2D we used state-of-the-art DFT calculations to investigate the formation of stacks of graphene with other 2D materials, the chemisorption of small molecules on graphene nanoribbons, nanographenes, and silicene, the permeability of graphene and BN sheets with respect to protons, and the response of layered pnictides to external pressure.

Project Title: Large-scale atomistic simulation study of polymer-graphene nanocomposite

Project Leader: Prof. Vlasis Mavrantzas, University of Patras, Department of Chemical Engineering, Rio, Patras, Greece

Resource Awarded: 1 500 000 core hours on NIIF – SEGED

Polymer nanocomposites are materials in which nanoscopic organic or inorganic particles, typically 10-100 Å in at least one dimension are dispersed in a polymer matrix in order to improve the performance properties of the polymer. We examined the effects that are caused to the dynamical, structural and mechanical properties from the presence of graphene sheets (as nanoparticle) in a polymer matrix compared to the bulk material.

Project Title: Modelling of Metal Organic Framework Interfaces

Project Leader: Dr Ben Slater, University College London, Department of Chemistry, London, UK

Resource Awarded: 17 850 000 core hours on EPCC – Archer

Metal organic frameworks (MOFs) are an exciting, rapidly emerging class of materials, comprised of metal ions or clusters connected to other metal ions or clusters via molecules, giving 2 dimensional sheets or 3 dimensional solids. There is an almost unlimited combination of metal(s) and organic molecules that could form a MOF. Importantly, they are often porous, with empty channels and cages that can span a wide range of dimensions: from the size of small gas molecules (e.g. methane) to small proteins. This variety in composition and scale promises an enormous range of potential utility, including gas storage and delivery, molecular sieving and catalysis. Key to realising real-world applications for these materials is a detailed atomic-scale understanding of how the properties of MOFs can be predicted and rationalised. One of the most promising areas of active research concerns using composites of MOFs with established technological materials, such as complex metal oxides, to engineer new function or a modified response, such as the ability to facilitate a chemical reaction using visible light. An important step in understanding and controlling MOF composites is to be able elucidate how the MOF binds to a material, which requires knowledge of the interface structure. State of the art computational approaches offer the possibility of probing these interface structures and hence the results have the potential to direct synthetic materials scientists towards the most promising MOF composites.

Project Title: Strain in nano-scale piezoelectrics

Project Leader: Dr Anna Kimmel, National Physical Laboratory, Functional Materials, London, UK

Resource Awarded: 5 080 320 core hours on CSC – Sisu


  • Mr Oliver Gindele – University College London, London, UK

The project is a part of European Metrology Research Project Nanostrain* led by the UK’s National Physical Laboratory in close academic collaboration with University College London, UK and European experimental partners such as Bundesanstalt fuer Materialforschung undpruefung, Cesky Metrologicky Institut Brno, Physikalisch-Technische Bundesanstalt. Control of materials at the nanoscale through the application of strain opens up a wide range of innovative for new high technology products, e.g. next generation (beyond CMOS) transistors using strain control of channel resistance, strain enhancement of CMOS semiconductors, optical and plasmonic control in nanoparticles and nano-structured physical and bio-sensing. The characterisation of strain in nanoscale systems will drive innovation in these major technological areas, for example enabling the development of a Piezoelectric-Effect-Transistor (PET) – an innovation developed by collaborators IBM (York Town Heights, USA). Traditional semiconductor transistors are known to have reached their switching speed performance limit, and it was recently shown by IBM that piezoelectric coupling can modify conductivity in nanoscale devices to introduce a new paradigm in computing technology where electromechanical coupling replaces charge transport. This will enable major increases in speed with reductions in size and power consumption, accelerating the growth of the portable consumer electronics market. This project aims to provide a theoretical insight into the origin of nanostrain, role of defects at the interface between metallic substrate and nanoscale piezoelectric material. Our objective is to develop a series of physical models, which provide an insight into the origin of piezoelectric response at metal/piezoelectric interfaces. The outcome of this modelling project will support the European Metrology Research experimental programme by underpinning the experimental activities within this project. We are planning to study several piezoelectric materials, that belong to class of so-called solid-solution systems, such as Pt/Pb(ZrxTi1-x)O3 (Pt/PZT) and Pt/Pb(MgxNb1-x)O3-PbTiO3 (Pt/PMN-PT), known for their exceptional properties. We will develop a structure prediction technique adopted specifically for solid-solution systems. Using this technique we will model optimal compositions for the formation of morphotropic phase boundaries. Further, we study properties of morphotropic phase boundaries using large-scale simulations and semi-empirical potentials. The effect of strain, dopant cations on electronic structure, piezoelectric response and optical properties of these compounds will be studied using density functional theory. We will also perform modelling of the interfaces between metallic substrates (Pt) and piezoelectric materials (PZT, PMN-PT) in terms of chemical and electrostatic environments. We aim to study the piezoelectric response of these systems, characterise strain distribution across the interface, the stability and mobility of domain walls at and nearby the interface, and the role of defects in the functional properties of this system.

Project Title: First-principles modelling of radiation-resistant metallic interfaces

Project Leader: Dr. Roberto Iglesias, University of Oviedo, Physics, Oviedo, Spain

Resource Awarded: 6 300 000 core hours on ICHEC – Fionn-thin and WCSS – Supernova

Interfaces formed by immiscible and incoherent metals such as Cu/Nb or Cu/W have been proposed as promising candidate materials to be used as coatings of the first wall in the future nuclear fusion reactors due to their excellent resistance to irradiation-induced structural changes. Preliminary experimental and theoretical works have shown that point defects created in both metals by the passage of high-energy neutrons created in every fusion event, namely, vacancies and self-interstitial atoms, can move up to the interface, that acts as a perfect sink for their recombination. On the other hand, the He atoms, produced in great quantities after the decay of transmutation elements, as well as in the fusion reactions themselves, have been found to accumulate at the interfaces, from where it can be easily removed by processes such as outgassing, reducing the damage in the structural materials. Moreover, given that the foreseen preferential reaction in future fusion reactor technologies involves the two isotopes of hydrogen, i. e., deuterium and tritium, H atoms will also be implanted into first wall materials. The incoherent character of the heterointerfaces implies that a great number of atoms will be involved in the simulations of these systems, which meant that they were only accessible to the atomistic molecular dynamics (MD) technique until some years ago when more powerful computational resources have started to allow the use of codes working at the electronic scale. Furthermore, the fact that optimal interatomic potentials for ternary or more complicated systems are very hard to obtain goes against the routine use of MD for this kind of problems. For that reason, Density Functional Theory (DFT) appears as the adequate tool to create an energetic map of the interface. The energetically more favorable sites where the vacancies can be formed as well as the more stable positions for the adsorption of He/H atoms will be found. Later, He/H atoms can be combined with the vacancies at the first, second or third layer in order to study the trapping energy. The possible differences between fcc and bcc metals on both sides of the interface shall be specifically evaluated. In a final step, the migration processes of the different defects to and at the interface will be analyzed. The energy barriers and the diffusion coefficients among the different stable positions can be calculated using the Nudged Elastic Band (NEB) to minimum energy paths following the prescriptions of Transition State Theory (TST). The systematic approach proposed will contribute to understand the diffusion and solubility of light impurity atoms inside the pure metallic matrices building the interfaces. All the information obtained will be highly valuable for subsequent kinetic Monte Carlo (kMC) simulations that can be performed for larger systems in longer time scales. The DFT calculations hereby proposed require the use of High Performance Computing (HPC) infrastructures due to the huge amount of RAM memory and processors needed. This highly demanding modelling effort can be regarded as a pioneering attempt to tailor the stability and self-healing properties of nanoscale interfaces.

Project Title: Quantum Monte Carlo Studies of High-Pressure Solid Molecular Hydrogen

Project Leader: Prof. Matthew Foulkes, Imperial College London, UK

Resource Awarded: 10 000 000 core hours on EPCC – Archer and EPCC – Blue Joule

If hydrogen gas is compressed, it solidifies to form the simplest and most fundamental of all molecular crystals. As the pressure is increased further, the solid undergoes several structural phase transitions, but hydrogen atoms scatter X-rays so weakly that the new crystal structures are unknown. Eventually, at high enough pressure, hydrogen is predicted to become not only a metallic atomic solid (believed present in large quantities in the interiors of Saturn and Jupiter) but also a high-temperature superconductor and perhaps a superfluid. How does crystalline hydrogen progress from an insulating molecular solid to a metallic monatomic solid? This question was first asked in 1935 [1] and has preoccupied high-pressure physicists ever since [2], but remains unanswered. The crystal structures of the sequence of phases observed as the pressure increases have not been established, metallization has not been observed conclusively, and the mechanism of metallization remains controversial. Because of the lack of conclusive experimental data, theoretical and computational methods have a central role to play [3,4]. The main goal of this project is to understand the behaviour of solid hydrogen in the high-pressure, low-temperature regime by calculation and study of its phase diagram and infra-red spectrum. We have already shown [5-8] that there are substantial errors in the description of solid hydrogen provided by mean-field-like methods based on density functional theory (DFT), so we will use the diffusion quantum Monte Carlo (DMC) method instead. DMC is the most accurate many-body method available for calculating the electronic ground-state energy of bulk materials [9,10]. Direct DMC calculations of energy derivatives such as the pressure have always been problematic, but our new correlated-sampling and algorithmic differentiation methods promise to make this challenging task both practical and efficient. We will use Slater-Jastrow trial wave functions with backflow corrections, optimized using a variational procedure in which we minimize the variance of the energy. Because of the light mass of the protons, nuclear quantum effects play a significant role in the behaviour of high-pressure hydrogen and must lead to quantum melting at high enough pressure. We will calculate the proton zero-point energy utilizing phonon calculations with corrections for anharmonicity. We will also use density-functional perturbation theory to compute infra-red spectra for comparison with recent experimental results. [1] E. Wigner and H. B. Huntington, J. Chem. Phys. 3, 764 (1935) [2] H-K. Mao and R. J. Hemley, Rev. Mod. Phys. 66, 671 (1994) [3] K. A. Johnson and N. W. Ashcroft, Nature 403, 632 (2000) [4] C. J. Pickard and R. J. Needs, Nature Physics 3, 473 (2007) [5] S. Azadi and T. D. Kuhne, JETP Lett. 95, 509 (2012) [6] S. Azadi and W. M. C. Foulkes, Phys. Rev. B 88, 014115 (2013) [7] S. Azadi, W. M. C. Foulkes, and T. D. Kuhne, New J. Phys. 15, 113005 (2013) [8] S. Azadi, B. Monserrat, W. M. C. Foulkes, and R. J. Needs, submitted to Phys. Rev. Lett. (2013) [9] W. M. C. Foulkes, L. Mitas, R. J. Needs, and G. Rajagopal, Rev. Mod. Phys. 73, 33 (2001) [10] D. Alfe and M. J. Gillan, Phys. Rev. B 70, 161101(R) (20

Project Title: Theoretical Design of Novel Multiferroic Oxides

Project Leader: Prof. Philippe Ghosez, University of Liege, Physics, Liege, Belgium

Resource Awarded: 5 251 400 core hours on SURFSARA – Cartesius


  • Julien Varignon – University of Liege, Physics, Liege, Belgium

TheDeNoMO project aims at providing deeper insight on the different mechanisms involved in exotic multiferroic oxide superlattices from first-principles calculation. Such heterostructures appear to be of very high interest for the design of ideal magnetoelectric multiferroics in view of proposing realistic materials for technological spintronic applications.
Within our previous PRACE project TheoMoMuLaM, we have proposed to run first-principles calculations in order to identify new multifunctional systems presenting couplings between several degrees of freedom such as lattice modes, orbital orderings, electronic structure and magnetism. During the last 6 months of our present PRACE allocation on the Abel machine in Oslo, we were able to identify two main systems of very promising interest: vanadium based multilayered superlattices and charge ordered titanate based superlattices. Especially, we found a direct electric control of magnetic ordering from our density functional theory calculations; a very appealing property for future spintronic applications. Although we will take advantage of the remaining allocated PRACE time to further characterize these systems in the next months, this will be clearly insufficient to investigate in details the microscopic origin of such very complex responses is presently not yet understood, while it is of high importance to fully comprehend it in order to propose a design rule for reproducing and optimizing the phenomena. Thus, the present submission for a new PRACE access appears to be of crucial importance in order to pursue our exciting research on vanadium and titanate oxide superlattices.
For vanadium based multilayered structures, and their related bulk compounds, the microscopic origins of the electronic degrees of freedom and the orbital orderings have to be identified. Indeed, we have shown these two later parameters to be responsible for the main multiferroic character of vanadate layered structures. Moreover, vanadate ground states are known to develop a non-collinear antiferromagnetic structure with a net magnetization, presenting a well known temperature reversal of magnetization. Unfortunately, our preliminary results have been obtained using only a collinear approach to model the magnetic structure and time consuming non-collinear calculations are still to be performed. Going further, we will need to perform finite magnetic and electric field calculations in order to compute the magneoelectric coefficients.
In titanates based superlattices we have found that a ferromagnetic-ferroelectric ground state is possible, making such multilayered structures ideal multiferroic materials while the bulk parent compounds do not exhibit such a property. However, here again the origin of the polarization and its evolution with respect to the chemistry as well as the mechanism producing ferromagnetism are still to be clarified.

Plasma & Particle Physics Projects

Project Title: Fundamental QCD Green’s functions

Project Leader: Dr. Orlando Oliveira, Universidade de Coimbra, Center for Computational Physics, Coimbra, Portugal

Resource Awarded: 7 500 000 core hours on CSC – Sisu and CSC – Sisu_XC40

Our aim is to investigate the behaviour of fundamental QCD Green’s functions, both at zero and finite temperature, via lattice QCD simulations. We aim to look at the gluon, ghost, and quark propagators in the Landau gauge, both in the pure gauge theory and including dynamical fermions, using a large physical volume (La ~ 8 fm or larger), in combination with small lattice sizes (a ~ 0.1 fm or smaller), together with improved gauge and fermionic actions. Within this PRACE project our aim is to look at the gluon and ghost propagators, both in the pure gauge theory and including dynamical fermions, and the quark propagator. We will keep the fundamental propagators in a local disk at the University of Coimbra, after completion of the numerical intensive computations which requires HPC and is the main goal of this PRACE project. This will allow us to access other fundamental QCD correlation functions in the future. Hopefully, we will be able to contribute for a better understanding of non-perturbative features of QCD, like confinement, deconfinement and dynamical chiral symmetry breaking.

Project Title: ELSOC

Project Leader: Dr. Mihalis Mavridis, Aristotle University, Thessaloniki, Greece

Resource Awarded: 2 520 000 core hours on VSB-TUO – Anselm

Project Title: Micro-Structurally Faithful Finite Element Modelling For Fusion Energy

Project Leader: Dr Lee Margetts, University of Manchester, Manchester, UK

Resource Awarded: 7 425 000 core hours on EPCC – Blue Joule


  • Dr Paul Mummery – University of Manchester, Manchester, UK

The International Thermonuclear Experimental Reactor (ITER) project is an international nuclear fusion research and engineering project, which is currently building the world’s largest experimental tokamak nuclear fusion reactor at the Cadarache facility in the south of France. The ITER project aims to make the long-awaited transition from experimental studies of plasma physics to full-scale electricity-producing fusion power plants. The project is funded and run by seven member entities — the European Union (EU), India, Japan, China, Russia, South Korea and the United States. In this DECI-12 project, European supercomputing facilities will be used to carry out high resolution thermo-mechanical analyses of candidate materials which may be used to line the ITER tokamak. Previous use of PRACE resources by other scientific teams has focused on modelling the plasma. The proposed work is complementary to that effort and the current PRACE support could lead to a future research effort involving plasma-wall interaction.