Using a code that utilises the full knowledge of physics that dictates how waves propagate through the Earth, Dr Vadim Monteiller of CNRS has been using datasets gathered from the North Sea to create some of the most accurate images of the interior of the Earth to date, work that is now focusing on imaging a larger piece of the Earth in Japan to get information of the seismic hazards in that area
High-performance computing and computational seismology were made for each other. HPC feeds off big data and seismology is certainly rich in data. Put them together and a new world of possibilities for improving our understanding of geodynamics opens up.
This is particularly true of computational earthquake seismology. Analysing ground motion scenarios for potential seismic events like earthquakes relies on being able to accurately image geological structures at multiple scales and on using the information buried in seismic big data records. Vadim Monteiller of CNRS is one of many seismologists who are now using the power of Tier-0 systems to be able to capture realistic images of the interior of the Earth using models that are only possible through enormous datasets. This work is a valuable advance for physics-based probabilistic seismic hazard assessment.
Monteiller has been studying these possibilities in a project using data from the North Sea on the Marconi100 GPU cluster at Cineca in Italy. “Our
work involves looking at wave propagations,” he explains. “When seismic activity occurs, the waves interact with complex geological structures inside the Earth, which can then be recorded at the Earth’s surface. We use these records to image inside the Earth.
“We need to understand the structure of the Earth to see how earthquakes are related to those structures”
Vadim Monteiller
“It’s a little like medical imaging, where you create an image of the whole body by studying the waves that propagate inside the body. We are doing the same with the Earth. Small meters and sensors are placed inside the Earth to create datasets of seismic activity, and we use these records to create images of the inside of the Earth.”
A large dataset recorded using ocean-bottom seismometers in the North Sea above an oil field was used for Monteiller’s PRACE project. The aim was to image the geological structures where oil and gas are extracted and create a precise image of the area, which required enormous computational power.
Monteiller and his team have developed a code called SPECFEM3D to carry out simulations of wave propagation inside the Earth to develop a model that explains the data gathered from the seismometers. “We started with an initial guess, compared the simulation with the data, and then slightly altered the model over and over again in order to explain the data better and better,” he explains. “Tens of thousands of simulations were required to improve the model.”
3D Geological structures recovered by 3D viscoelastic seismic Full Waveform Inversion. The data are recorded by 131 Ocean Bottom Seismometers in the North Sea (black dots).The color palette represents the different geological layers.
SPECFEM3D was developed along with several partners at CNRS, Princeton University in the USA and ETH in Switzerland. First ported to GPUs 10 years ago, it was developed and optimised for the PRACE project by Monteiller along with teams from the Barcelona Supercomputing Centre and Cineca in Italy, who analysed and modified it to speed it up. The resulting code was used to simulate wave propagation inside the Earth using a finite element method with high polynomial interpolation that made it very precise.
“We had a speedup of a factor of 300 going from CPUs to GPUs,” says Monteiller. “It has made it possible to make precise images of the Earth, and since it is now a very fast and accurate code, we were also able to launch many simulations, especially with GPU clusters like Marconi100. This was the first time we were able to launch these simulations in such huge numbers.”
The development of SPECFEM3D was done as part of the larger ChEESE project, which aims to create new codes for solid earth physics that can harness the power of upcoming pre-exascale and exascale supercomputers in Europe. This project has proven a valuable exercise in proving the capabilities of the code. “This is the first time we have done this type of computation,” says Monteiller.
Detail of the different geological layers seen in vertical sections from the inverted 3D geological model (scale in kilometers)
Previously, codes used for this kind of imaging used approximations of the actual physics at play. This project enabled the researchers to use their full knowledge of the physics of wave propagations, taking into account the viscoelastic properties of the Earth, as well as anisotropic properties, in which seismic waves propagate at different speeds depending on the direction in which they travel due to the crystalline properties of certain minerals. While this drastically increased the computational cost, it showed that it was possible to include this level of detail in simulations.
Monteiller and his team are now working with the ChEESE project at a different scale. Instead of small-scale imaging like that used in this project on the North Sea, work is underway to image a larger piece of the Earth in Japan to get information of the seismic hazards in that area. “We need to understand the structure of the Earth there to see how earthquakes in the area are related to those structures,” he says. “Now with this code, we have the technique to image at this larger scale.
“So, this code, developed as part of ChEESE and used for the first time with a very big dataset in this PRACE project, is now being used for more geodynamic purposes,” concludes Monteiller.
This article was also published in PRACE Digest 2021.
More information:
http://www.lma.cnrs-mrs.fr/
https://cheese-coe.eu/pilot/seismic-tomography
https://github.com/geodynamics/specfem3d
Resources awarded:
This project was awarded 115 400 000 core hours on Marconi100, hosted by CINECA, Italy
Contact
For all questions about PRACE communications, promotional and press materials, social media, and publications: communication@prace-ri.eu