Statistics & KPIs

PRACE Statistics

The Council of PRACE has unanimously approved the publication of various PRACE user statistics. The tables and graphs below represent the awarded projects per country, and the awarded hours per country. PRACE will update these statistics periodically, adding information on each successive Call for Proposals as it becomes available.

Last update: February 2019

Data per Country

The graph to the left shows the following data per country:
  • Projects awarded % of total projects awarded (blue bars)
  • Resources awarded % of total resources (orange bars)

The graph includes data from the Early Access Call (EAC) as well as from PRACE Project Access Calls 1 through 17.

Awarded projects per country

To attribute a PRACE awarded project to a certain country, a “mixed metric” is used:

  • The nationality of the principal investigator (meaning the country of his/her research centre) counts for 30%
  • The countries of all the centres participating in the project count for 70%
  • Each centre counts for an equal share of the 70%. If two departments of the same university participate, they count as two centres.

An example

A project is led by a PI from Germany. There are 3 collaborating centres, one from Spain, one from United Kingdom and one from United States. The 4 participating countries are counted as follows:

Country Count Total
Germany 30% for the PI 0.25 of 70% for the collaboration 47.5%
Spain 0.25 of 70% for the collaboration 17.5%
UK 0.25 of 70% for the collaboration 17.5%
USA 0.25 of 70% for the collaboration 17.5%

These calculations are made for each project. The results per country are added up and divided by the total number of awarded projects.

Awarded core hours per country

The awarded core hours per country are counted according to the Linpack benchmark for each system. The following steps are taken to obtain this percentage:

1) The Linpack benchmark for each system is calculated via the equation:

2) Second, the value of the Linpack benchmark is multiplied with the total core hours awarded to the project on each system (this can be only one system or several):

3) The outcome of this calculation is then multiplied with the percentage for the system’s hosting country (obtained via the “mixed metric”) to arrive at the performance per system.

4) The performances in each project for each system are then added up and divided by the total awarded hours for all systems (grand total).

Please note that the PRACE Council has appointed a working group to develop a benchmark with a balanced set of Tier-0 applications, with real scientific user data. This new benchmark will replace the Linpack benchmark, which is application-naïve, does not represent scientific code performances and is subject to overclocking and accelerators.

Disclaimer and copyright

PRACE publishes the above statistics under the principle of transparency of its Peer Review Process. Re-publication of these statistics is allowed as long as the source is correctly referenced. PRACE cannot take responsibility for errors and omissions in quotations by third parties.

Contact

If you wish to obtain further information about these statistics or the PRACE Peer Review Process, please contact the PRACE Board of Directors via bod[at]prace-ri.eu.

PRACE Key Performance Indicators

Given the scale of the computational power in the PRACE portfolio, PRACE related statistics are becoming increasingly important to highlight the impact of PRACE on HPC based research, HPC know-how in Europe, and European Industry engagement in HPC.

In 2014, the PRACE Council approved a set of Key Performance Indicators (KPI) that facilitate the analysis and evaluation of PRACE achievements and successes.

PRACE’s impact on evolving research

Offer and demand of resources

Figure 1 shows the evolution of PRACE resources offered and requested in Project Access Calls. PRACE first provided HPC services in 2010 with contributions from German Tier-0 systems.France, Italy and Spain added their contributions gradually. This is reflected in the constant increase of HPC resources offered by PRACE to the scientific community until the 6 th Call, where a stable regime was reached. The phasing out of the initial phase of PRACE (known as PRACE 1) started in the 10th Call, while the second phase of PRACE (known as PRACE 2) only started in Call 14. This is reflected in the valley from the 10th Call to the 13th Call. With PRACE2, a substantial increase in the amount of available resources can be seen, thanks to the renewed contributions of all the original PRACE Hosting Members and Switzerland as a new hosting member. Since the 14th Call an average of 1.9 thousand million core hours has been offered to the HPC scientific community. The demand for HPC resources has always exceeded the capacity of PRACE to provide them. The average oversubscription of PRACE Calls is 3:1, reaching a 5:1 ratio during the phasing out of PRACE1. This constant interest in PRACE resources has been driving the periodic upgrades and additions of new systems that PRACE offers,specifically with regards to the PRACE 2 programme.

Number of projects

During the initial phase of PRACE (known as PRACE 1), the number of project applications received via PRACE Calls for Proposals for Project Access exhibited a clear overall upward trend. The phasing-in of PRACE 1 naturally incited an increase in demand for Tier-0 resources.This is particularly evident up to the 8th Call, with a large sustained increase between the 6th and 8th Call, followed by a slight decrease(Figure 2). A downward trend of rejected projects below the technical quality threshold is noted, displaying maturity of submitted proposals,in which researchers put more effort into the quality of their proposals, as a reaction to increased competition. Moreover, the evolution reflects the positive outcomes of PRACE Preparatory Access Calls (including access type C) that enable prior technical support for application and scalability tests. Figure 2 also highlights an increase in rejected projects above the scientific threshold,particularly after the 6th Call. This is correlated with the increase in total applications. During the phasing-out of PRACE 1, the number of available core hours dropped (Figure 1), and this decreased the demand, as researchers anticipated an even stronger competition for the remaining resources. This trend was mitigated in the 12th and 13th Call, when PRACE hosting members made additional core hours available during the preparation of the PRACE 2 programme, which started in the 14th Call. With the start of PRACE 2, there has been an increase in the number of projects awarded, which combined with the increase of resources that started in the 14th call (Figure 1) shows the clear success of the second phase of PRACE. In this second phase, the scientific objectives of PRACE have been updated towards an increase of the scope and excellence of the projects awarded. The minimum size of allocations has been increased three-fold, and the scientific threshold as well. The decrease in proposals submitted and the apparent decrease in their quality in the 15th Call is a positive sign of the success of this scientific update.
Despite the competition, demand for PRACE resources remains high and all PRACE calls are oversubscribed (Figure 1), indicating that scientists consider Tier-0 access an essential asset to their work.This was also underlined by the PRACE Scientific Steering Committee (SSC) during the preparation of PRACE 2 (see page 8 for the article by the Chair of the SSC)

Recurring users

PRACE also keeps track of the submission of Project Access proposals by recurrent Principal Investigators (PIs) (Figure 3). This KPI is created by checking for each call if a PI is new to PRACE. The ratio of first-time applicants is relatively high – roughly 30 per cent of PIs who submitted to the two Project Access Calls in 2017 were recurrent applicants to a PRACE Call for Proposals for Project Access. This means that more than two thirds of project proposals are submitted by new users. This indicates that PRACE is continuously attracting new PIs, while remaining an essential support for existing users. The upward trend of the ratio of recurrence is visible, particularly from the 6th Call onwards, influenced by the downward trend on awarded projects, but recovered with the onset of PRACE 2.

International and transnational cooperation

Two-thirds (63%) of resources awarded under the Early Access Call through to the 15th Call are awarded to “foreign projects”. Foreign projects are defined as projects with Principal Investigators (PIs)from a different country (recorded as the country of the PI’s primary institution ) than the machine on which the research is executed.The ratio of awarded foreign projects remains rather stable over time (Figure 4). This shows that the nationality of the PI’s institution does not influence the chances of a project being awarded, as another sign of the scientific excellence selection criterion of PRACE Project Access. It also demonstrates PRACE’s impact in the enhancement of European and International collaboration.

Co-funding

PRACE awards are normally developed within larger scientific initiatives,where HPC resources are part of the needs of the project. PRACE asks Project Access awardees to declare such synergies. On average, 75% of PRACE users have declared that their awards are complemented with EC, national or international funds (Figure 5). The major fraction corresponds to national projects, which has slowly been showing a downward trend since the 10th Call. EC funding shows an increasing trend, coinciding with the implementation of the H2020 programme. International funding remains low, with 8%being the average contribution.

PRACE’s impact on growing know-how in Europe

Since 2008, PRACE has been engaged in providing top-class education and training for computational scientists in Europe through the PRACE Training Centres (PATCs), the International HPC Summer School, and PRACE Seasonal Schools, with a clear increase of participants registered (Figure 6). Six PTCs were first established, and these are CSC – IT Center for Science Ltd. (Finland), Maison de la Simulation (France), Gauss Centre for Supercomputing (Germany), CINECA – Consortio Interuniversitario(Italy), Barcelona Supercomputing Centre (Spain), and EPCC at the University of Edinburgh (United Kingdom). After the rapid increase between 2010 and 2012, a plateau is evident since 2012. As this indicated that the maximum capacity of PRACE training offerings had been reached, four new PRACE Training Centres were opened in 2017 – in IT4Innovations (Czech Republic), GRNET (Greece), ICHEC (Ireland) and SURFSara(Netherlands). This acts as a second layer of training and education, below the first PTCs. PTC training events, Seasonal Schools and the International HPC Summer School are offered free of charge to eligible participants from academia and industry.

Between August 2008 and December 2017, PRACE provided 36 273 participant-days of training through attendance-based courses, with an upward attendance trend. PRACE courses were attended by over 11500 individuals. This shows the effectiveness of PRACE in attracting, training and retaining competences.

In 2017 the number of participants attending PRACE courses was 1858 (1 487 from academia and 371 from non-academia affiliation). More than 80% of participants attending trainings days have academic affiliation (1487), illustrating the impact of such event on research and scientific communities, in particular for early stage researchers and PhD students.

A clear difference of attendance is observed between the first and second semester of 2017. As observed in Figure 7, the total number of attendances registered in the first semester (Q1 and Q2)is significantly higher than during the second semester (Q3 and Q4). This indicates that the bulk of the training offered occurs in the first semester, with a notable drop in attendance during Q3 which corresponds with the summer vacation period.

PRACE’s impact on attracting the industrial sector

The interest of industry in PRACE at high-level international events has increased steadily over the past years (Figure 8). The total number of industrial participants showing interest in PRACE during the two main HPC events (Supercomputing (SC) in the USA, and the International Supercomputing Conference (ISC) in Germany) between 2008 and 2017 was 1 707 individuals.

More than half of the companies that visited the PRACE booth at ISC17and SC17 were first-time visitors. This indicates that industrial interest in PRACE is growing on both sides of the Atlantic.

Industrial participants in PTCs

The average participation of industry in PTC trainings is 15.68%between 2012 and 2017 (19.7% in 2017). The increasing interest fromindustry in participating in HPC training is visible in Figure 9. Over 360 industrial participants were trained by PRACE in 2017. Eligibleindustrial participants enjoy the same service as academic traineesand can attend PTC courses free of charge.

Industrial use of PRACE HPC resources

PRACE opened its Calls for Proposals to industrial applicants mid-2012. Industrial participation can take the form of a project led by a Principal Investigator coming from an industrial enterprise,or a researcher from industry collaborating in an academia-led project. The reduction and stabilisation of projects awarded after the 7th Call has had a strong impact on the number of projects awarded with industrial participants (Figure 10). In other words,industry suffers more from the competition for PRACE resources than academia. This trend is starting to change with the start of PRACE 2.

Regarding the SHAPE pilot, PRACE can report 10 success stories of SMEs from six different countries benefitting not only from PRACE HPC resources but, more importantly, from the know-how in the PRACE centres. The second SHAPE Call awarded 11 more projects, the third SHAPE Call another eight,the fourth SHAPE Call an additional four, and the fifth SHAPE Call (which opened in March 2017) another six.