Statistics & KPIs

PRACE Statistics

The Council of PRACE has unanimously approved the publication of various PRACE user statistics. The tables and graphs below represent the awarded projects per country, and the awarded hours per country. PRACE will update these statistics periodically, adding information on each successive Call for Proposals as it becomes available.

Last update: February 2019

Data per Country

The graph to the left shows the following data per country:
  • Projects awarded % of total projects awarded (blue bars)
  • Resources awarded % of total resources (orange bars)

The graph includes data from the Early Access Call (EAC) as well as from PRACE Project Access Calls 1 through 17.

Awarded projects per country

To attribute a PRACE awarded project to a certain country, a “mixed metric” is used:

  • The nationality of the principal investigator (meaning the country of his/her research centre) counts for 30%
  • The countries of all the centres participating in the project count for 70%
  • Each centre counts for an equal share of the 70%. If two departments of the same university participate, they count as two centres.

An example

A project is led by a PI from Germany. There are 3 collaborating centres, one from Spain, one from United Kingdom and one from United States. The 4 participating countries are counted as follows:

Country Count Total
Germany 30% for the PI 0.25 of 70% for the collaboration 47.5%
Spain 0.25 of 70% for the collaboration 17.5%
UK 0.25 of 70% for the collaboration 17.5%
USA 0.25 of 70% for the collaboration 17.5%

These calculations are made for each project. The results per country are added up and divided by the total number of awarded projects.

Awarded core hours per country

The awarded core hours per country are counted according to the Linpack benchmark for each system. The following steps are taken to obtain this percentage:

1) The Linpack benchmark for each system is calculated via the equation:

2) Second, the value of the Linpack benchmark is multiplied with the total core hours awarded to the project on each system (this can be only one system or several):

3) The outcome of this calculation is then multiplied with the percentage for the system’s hosting country (obtained via the “mixed metric”) to arrive at the performance per system.

4) The performances in each project for each system are then added up and divided by the total awarded hours for all systems (grand total).

Please note that the PRACE Council has appointed a working group to develop a benchmark with a balanced set of Tier-0 applications, with real scientific user data. This new benchmark will replace the Linpack benchmark, which is application-naïve, does not represent scientific code performances and is subject to overclocking and accelerators.

Disclaimer and copyright

PRACE publishes the above statistics under the principle of transparency of its Peer Review Process. Re-publication of these statistics is allowed as long as the source is correctly referenced. PRACE cannot take responsibility for errors and omissions in quotations by third parties.

Contact

If you wish to obtain further information about these statistics or the PRACE Peer Review Process, please contact the PRACE Board of Directors via bod[at]prace-ri.eu.

PRACE Key Performance Indicators

Given the evolution of computational power in the PRACE portfolio, PRACE-related statistics are becoming increasingly important to highlight the impact of PRACE on HPC-based research, HPC know-how in Europe, and European industrial uptake of HPC.

In 2014, the PRACE Council approved a set of Key Performance Indicators (KPI) that facilitate the analysis and evaluation of PRACE achievements and successes.

PRACE’s impact on evolving research

Offer and demand of resources

Figure 1 shows the evolution of PRACE resources offered and requested in Project Access Calls. PRACE first provided HPC services in 2010 with contributions from German Tier-0 systems. France, Italy and Spain added their contributions gradually. This is reflected in the constant increase of HPC resources offered by PRACE to the scientific community until the 6th Call, where a stable regime was reached. The phasing out of the initial phase of PRACE (known as PRACE 1) started in the 10th Call, while the second phase of PRACE (known as PRACE 2) only started in Call 14. This is reflected in the valley from the 10th Call to the 13th Call. With PRACE 2, a substantial increase in the amount of available resources can be seen, thanks to the renewed contributions of all the original PRACE Hosting Members and Switzerland as a new Hosting Member. The stable regime has been reached quickly, and since the 14th Call an average of 1.9 billion core hours per Call has been offered. The 19th Call shows a high increase from this average, due to the inclusion of novel, more dense architectures into PRACE standard offer.

The demand for HPC resources has always exceeded the capacity of PRACE to provide them. The average oversubscription of PRACE Calls is 3:1, reaching a 5:1 ratio during the phasing out of PRACE 1. This constant interest in PRACE resources has been driving the periodic upgrades and additions of new systems that PRACE offers, specifically concerning the PRACE 2 programme.

Figure 1 – Core hours offered and requested in each PRACE Call

Number of projects

During the initial phase of PRACE (known as PRACE 1), the number of project applications received via PRACE Calls for Proposals for Project Access exhibited a clear overall upward trend. The phasing-in of PRACE 1 naturally incited an increase in demand for Tier-0 resources. This is particularly evident up to the 8th Call, with a large sustained increase between the 6th and 8th Call, followed by a slight decrease (Figure 2).

A downward trend of rejected projects below the scientific excellence threshold is noted, displaying maturity of submitted proposals, in which researchers put more effort into the quality of their proposals, as a reaction to increased competition. Moreover, the evolution reflects the positive outcomes of PRACE Preparatory Access Calls (including access type C) that enable prior technical support for application and scalability tests. Figure 2 also highlights an increase in rejected projects above the scientific threshold, particularly after the 6th Call. This is correlated with the increase in total applications.

During the phasing-out of PRACE 1, the number of available core hours dropped (Figure 1), and this decreased the demand, as researchers anticipated an even stronger competition for the remaining resources. This trend was mitigated in the 12th and 13th Call, when PRACE Hosting Members made additional core hours available during the preparation of the PRACE 2 programme, which started in the 14th Call.

With the start of PRACE 2, there has been an increase in the number of projects awarded, which combined with the increase of resources that started in the 14th Call (Figure 1) shows the clear success of the second phase of PRACE. In this second phase, the scientific objectives of PRACE have been updated towards an increase of the scope and excellence of the projects awarded. The minimum size of allocations has been increased three-fold, and the scientific threshold as well. The decrease in proposals submitted and the apparent decrease in their quality in Call 15th is a positive sign of the success of this scientific update.

Figure 2 –Total number of applications received, proposals above scientific excellence threshold, and projects awarded

Despite the competition, demand for PRACE resources remains high and all PRACE Calls are oversubscribed (Figure 1), indicating that scientists consider Tier-0 access an essential asset to their work. This was also underlined by the PRACE Scientific Steering Committee (SSC) during the preparation of PRACE 2.

Recurring users

PRACE also keeps track of the submission of Project Access proposals by recurrent Principal Investigators (PIs) (Figure 3). This KPI is created by checking for each Call if a PI is new to PRACE.

The ratio of first-time applicants is relatively high – roughly 50% of PIs who submitted to the two Project Access Calls in 2019 were recurrent applicants to a PRACE Call for Proposals for Project Access. This means that half of all project proposals are submitted by new users. This indicates that PRACE is continuously attracting new PIs, while remaining an essential support for existing users. The upward trend of the ratio of recurrence is visible, particularly from the 6th Call onwards, influenced by the downward trend on awarded projects, but recovered with the onset of PRACE 2.

Figure 3 – Ratio of new applicants and new awardees in each PRACE Call

International and transnational cooperation

Two-thirds (63%) of resources awarded under the Early Access Call through to the 15th Call are awarded to “foreign projects”. Foreign projects are defined as projects with Principal Investigators (PIs) from a different country (recorded as the country of the PI’s primary institution) than the machine on which the research is executed. The ratio of awarded foreign projects remains rather stable over time (Figure 4). This shows that the nationality of the PI’s institution does not influence the chances of a project being awarded.

It also demonstrates PRACE’s impact in the enhancement of European and International collaboration.

Figure 4 – Ratios of awarded ‘foreign’ projects and resources for awarded ‘foreign’ projects

Co-funding

PRACE awards are normally developed within larger scientific initiatives, where HPC resources are part of the needs of the project. PRACE asks Project Access awardees to declare such synergies.

On average, 75% of PRACE users have declared that their awards are complemented with EC, national or international funds (Figure 5). The major fraction corresponds to national projects, which is slowly showing a downward trend since the 10th Call. EC funding shows an increasing trend, coinciding with the implementation of the H2020 programme. International funding remains low, with 8% being the average contribution.

Figure 5 – Ratios of awarded projects with EC, National, and International support

PRACE’s impact on growing know-how in Europe

Since 2008, PRACE has been engaged in providing top-class education and training for computational scientists in Europe through the PRACE Training Centres (PTCs), the International HPC Summer School, and PRACE Seasonal Schools, with a clear increase of participants registered (Figure 6).

Six PTCs were first established, and these are located at:

  • Barcelona Supercomputing Centre (Spain)
  • CINECA – Consortio Interuniversitario (Italy)
  • CSC – IT Center for Science Ltd. (Finland)
  • EPCC at the University of Edinburgh (UK)
  • Gauss Centre for Supercomputing (Germany)
  • Maison de la Simulation (France)

After the rapid increase between 2010 and 2012, a plateau is evident since 2012. As this indicated that the maximum capacity of PRACE training offerings had been reached, four new PRACE Training Centers were opened in 2017, located at:

  • IT4Innovations (Czech Republic)
  • GRNET (Greece)
  • ICHEC (Ireland)
  • SURFSara (The Netherland)

PTC training events, Seasonal Schools and the International HPC Summer School are offered free of charge to eligible participants.

Figure 6 – Number of person-days registered at PRACE Training days between 2008 and 2019

Between August 2008 and December 2019, PRACE provided close to 52 000 participant-days of training through attendance-based courses, with an upward attendance trend. PRACE courses were attended by over 16 400 individuals. This shows the effectiveness of PRACE in attracting, training, and retaining competences.

In 2019 the number of participants attending PTCs courses was 2618 (359 with non-academia affiliation). 86% of participants attending PTCs trainings days have an academic affiliation, illustrating the impact of such event on research and scientific communities, in particular for early stage researchers and PhD students.

A clear difference of attendance is observed between the first and second semester of 2019. As observed in Figure 7, the total number of attendees registered in the first semester (Q1 and Q2) is significantly higher than during the second semester (Q3 and Q4). This indicates that the bulk of the training offered occurs in the first semester, with a notable drop in attendance during Q3 which corresponds with the summer and winter vacation periods.

Figure 7 – Number of person-days registered at PRACE Training days in 2019

PRACE’s impact on attracting the industrial sector

Industrial visitors of the PRACE booth at ISC and SC

The interest of industry in PRACE at high-level international events has remained high over the past years (Figure 8). The total number of industrial participants showing interest in PRACE during the two main HPC events (Supercomputing (SC) in the USA, and the International Supercomputing Conference (ISC) in Germany) between 2008 and 2019 was 2 058 individuals.

Figure 8 – Number of industrial attendees that made contact with the PRACE booth at ISC and SC; and related trend line

More than half of the companies that visited the PRACE booth at ISC’19 and SC19 were first-time visitors. This indicates that industrial interest in PRACE is growing on both sides of the Atlantic.

Industrial participants in PTCs

The average participation of industry in PTC trainings is 15.42% between 2012 and 2019 (12.9% in 2017). The increasing interest from industry in participating in HPC training is visible in Figure 9. Over 359 industrial participants were trained by PRACE in 2017. Eligible industrial participants enjoy the same service as academic trainees and can attend PTC courses free of charge.

Figure 9 – Industrial participation in PTCs training days, and related trend line

Non-academic use of PRACE HPC resources

PRACE opened its Calls for Proposals to non-academic applications mid-2012. This can take the form of a project led by a Principal Investigator coming from a private company, or a researcher from industry collaborating in an academia-led project. The number of applications with non-academic participation can be seen in Figure 10. Up to Call 18, Applications with a non-academic PI have competed for PRACE resources with applications with an academic PI. This was changed in Call 19 where an “Industry Access Pilot” which offered Principal Investigators from industry the possibility to apply for Single-year access to a special Industry Track which prioritised 10% of the total resources available. This will continue in future Calls and we expect an increasing non-academic use of PRACE HPC resources in future Calls.

Figure 10 – Industry participation in PRACE

Through the SHAPE programme, started in 2013, PRACE has received 73 applications so far, of which a total of 55 have been awarded both PRACE HPC resources and, more importantly, support and know-how of in the PRACE centres.