Annual report 2022

During the Covid-19 pandemic, we made resources on the national supercomputers available for Norwegian Institute for Public Health to calculate infection spread and expected vaccine effects. As the pandemic faded away in 2022, also in terms of HPC, we could gradually free up more resources for other research and return to scientific production.

In 2022, we made no HPC hardware investments for the national e-infrastructure. However, during the year, the LUMI CPU quota (LUMI-C) was ready for production, while the GPU share of the machine received extensive pilot testing. At the end of the year, LUMI-G was made ready for production and welcomed by our GPU community.

Previous expansions of the HPC systems have provided sufficient CPU resources, yet GPU resources are in high demand. Even with the additional capacity from the LUMI GPUs, there will be a need for near-future expansions, near-future expansions will be needed.

In 2022, we commenced the process of acquiring Norway’s next supercomputer. Among other things, we have worked to gain insight into what type of resources it will be essential for Norwegian researchers to gain access to in the years ahead. We expect to complete the public procurement process by the end of 2023 and set the new HPC system in production during 2024.

 

The HPC service

Together with the national supercomputers, the national storage system NIRD forms the backbone of the national e-infrastructure for research and education in Norway. 

In 2022 we installed and tested the new national storage infrastructure for research data. The procurement was completed late the year before. Like its predecessor, the new storage system is named NIRD – the National Infrastructure for Research Data. The new NIRD consists of IBM technology and is the first national e-infrastructure system installed the space we rent at Lefdal Mine Datacenter. The new NIRD has a capacity of 32 petabyte (PB) and can be expanded to a whopping 70 PB if needed. In comparison, old NIRD had a capacity of 22 PB. The capacity will cover current and future needs for data storage under the paradigm of Open Science and FAIR data. NIRD was officially inaugurated in December 2022.  

Now, researchers in Norway can look forward to state-of-the-art storage services that support artificial intelligence, machine learning, data-intensive analysis and archiving.

The NIRD storage service

Data planning is essential to all researchers who work with digital scientific data. The easyDMP tool is designed to facilitate the data management process from the initial planning phase to storing, analysing, archiving, and finally sharing the research data at the end of your project. Presently, easyDMP features 1377 users and 1416 plans from 246 organisations/departments/institutions.  

EasyDMP is a step-by-step tool supporting several templates. In addition to templates following the Horizon2020 and Science Europe recommendations, easyDMP supports institution-specific templates with prefilled answers and easy-to-use guides.  

We collaborate with the EOSC Nordic project and the Research Data Alliance to keep improving the tool. In 2022, we used the easyDMP as a testbed to design and prototype the concept of machine-actionable DMP. As part of EOSC-Nordic, work was done on implementing support for importing and exporting DMPs that conform with the RDA machine-actionable DMP schema. Plans can now be exported and imported between tools, as long as they conform to the schema. Integration between easyDMP and the Data Stewardship Wizard also moved forward, and plans can be exported from Elixir’s Data Stewardship Wizard following the Research Data Alliance schema. The next step is to import the plan into easyDMP.


The easyDMP Service

The NIRD Service Platform is essential for many research environments consuming data on NIRD storage through services. In later years, the popularity of the Service Platform has grown as increasing data volumes are often more convenient to analyse and pre/post-process “locally”, i.e., directly on the storage infrastructure without moving data elsewhere. Furthermore, the possibility of spin-off community-specific portals to expose large datasets to external users allows sharing and reusing of data in line with the current policies and best practices in research. Services can run permanently or on-demand through the NIRD Toolkit, which has been improved during 2022.  

The NIRD Service Platform consists of two clusters with computing resources, orchestrated by a Kubernetes engine with a CPU/GPU architecture to enable services and computing environments to run data-intensive computing workflows, such as pre-/post-processing, visualisation and AI/ML analysis. In 2022 a new instance of the NIRD Service Platform was procured and installed in Lefdal Mine Datacenter to provision services on the new generation NIRD storage infrastructure. This instance will be put into production jointly with the new NIRD in 2023. In 2022, the average allocation on Kubernetes CPUs was 763 (on a total of 576 physical cores). The highest Kubernetes CPU allocation was 834 cores.


The NIRD Service Platform

The NIRD Toolkit is a Research Platform to easily spin off software on-demand on the NIRD Service Platform, thus avoiding cumbersome IT operations. With the NIRD Toolkit, computations on Spark, R-Studio, and several popular artificial intelligence algorithms are possible in a few clicks. In 2022 the authentication and authorization mechanisms were modified to make the access and administration of roles easier for all the end users, regardless of their affiliation. Furthermore, the security of the service has been increased together with the user-friendliness of the web-based front end, for a better user experience.

The current Research Data Archive came into service in June 2014 and has since stored 455 datasets, corresponding to almost 700 TiB in total (about 1.4 PiB, including replicas).    

The increasing FAIR and data management awareness of researchers has resulted in an increment in the number of datasets deposited during the last years (see Figure X and Figure Y below). The sharp rise observed in 2021 is remarkable and probably due to many projects ending their research cycles.    

In 2022, the Research Archive reached its capacity limits, and we had to temporarily close the archive of large-volume datasets due to capacity shortage. This is visible from the plateau observed in the data volume growth in 2022 (Figure X). However, the deposition of small datasets has never stopped and keeps increasing (Figure Y).

Bilde

Bilde

The data landscape in Norway has changed since 2014, and the need for new and more user-friendly functionalities is evident. This is why we launched a new Research Archive project in 2021. The project aims to re-design the concept for archiving within the Sigma2 scope. In 2022, the project designed the next-generation Research Data Archive’s architecture to comply with several identified use cases and use existent components and expertise. The roles of APIs and web services for integration with other systems and data sources are emphasised. Competencies and components will be procured, and the new system will be implemented in 2023.

Sensitive Data Services (TSD) is a service delivered by the University of Oslo and provided as a national services by Sigma2.

TSD offers a remote desktop solution for secure storage and high-performance computing on sensitive personal data. It is designed and set up to comply with the Norwegian regulations for individuals’ privacy. TSD has been a part of the national e-infrastructure since 2018. We own 1600 cores in the HPC cluster. We also own 1370 TB of the TSD disk which delivers storage for general use or HPC use.



The TSD Service

Basic user support provides a national helpdesk that deals with user requests such as issues relating to access, systems usage or software. Support tickets are handled by highly competent technical staff from NRIS. The number of cases handled in our basic user support service continued to increase in 2022.

Advanced User Support (AUS) is when we benefit from letting the experts within NRIS spend time going deeper into the researchers’ challenges. The effects are often of high impact, and the results are astonishing. 

The AUS activities of 2022 span the usual wide range of topics — from bioinformatics to astroparticle physics. Here we work to improve user experience by aiding with application interfaces and making shared input data more readily available. With the extensive GPU-resources becoming available last year, especially on LUMI, our team of GPU-experts in NRIS have been highly engaged in enabling and porting codes and workflows for accelerators. We wish to maximise the footprint of Norwegian projects on these external resources and thereby the return from participating in the EuroHPC Joint Undertaking and EU programmes such as DIGITAL EUROPE.

Additionally, interest has picked up for the AUS User Liaison collaborations. This framework allows experts to have a foot inside the NRIS activities while maintaining a close tie to specific application communities, major research infrastructures and centres of excellence.

The AUS Service

The interest for the Course Resources as a Service (CRaaS) continued to increase in 2022. Altogether, we ran 11 different projects and no less than 448 users accessed the service.

Activities have mainly been in the field of biology, physics, GPU programming, and training in the use of HPC. The average length of the courses is 41 days. For the most part, it is supercomputers Fram and Saga that are used from CRaaS, but we have also had some activities on the NIRD Toolkit.

Bilde

— Materials science has been transformed by the development of accurate prediction tools based on quantum mechanics (from first principles) and access to high-performance computing resources. The CRaaS resources make it possible to give students hands-on experience with a simulation environment that resembles those being used worldwide by professional researchers in academia, research institutes, and the industry. The teaching is thus relevant and realistic, which is made possible by CRaaS, says Ole Martin Løvvik, Chief Scientist at SINTEF and Adjunct Professor at UiO.

The CRaaS Service

Once a month we invite users to our open Question and Answer session. Here users meet NRIS Staff and can discuss problems, projects ideas, provide feedback and suggestions on how to improve the national services. Everyone is welcome to participate.

Some quotes from users who have attended NRIS training:

- Excellent course, Thank you! It will definitely give me more flexibility and independence.

- I have been user of Norwegian HPC facilities for over 15 years and of NIRD (formerly Norstore) since its beginning. I was therefore a bit concerned that the course would be too basic and not teach me anything new. But I was positively surprised about how many new things relevant to my work I learned during the course each day.

- Thank you very much for such a nice and comprehensive course. It exceeded my expectations. Usually, these types of courses are hard to follow when you have little prior knowledge - this was not the case at all this time. I am very happy and learned a lot.

• Company: Digifarm
• Sector: Agriculture

DigiFarm is one of Norway's leading agri-tech startups specialising in developing deep neural networks for automatically detecting field boundaries and sown areas based on high-resolution satellite data. The models are trained in several international regions, including Europe, South America, APAC (Thailand, Vietnam, and India), the USA and Canada. Training requires a large amount of image data as well as annotation data.

During the project, NCC provided Digifarm with access to HPC and performed workflow analysis to optimise the utilisation of HPC resources and graphic processing units (GPU). The project is still running, and the next step is to provide an API for HPC to allow their automation system to automatically dispatch, monitor and ensure correct workflows in the HPC environment.

Company: Zeabuz
Sector: Transportation

The Trondheim-based NTNU spin-off company Zeabuz launched its autonomous AI-controlled vessel during the autumn of 2022. Even in a limited ocean space such as Trondheimsfjorden, it is impossible to physically test all conceivable scenarios a ferry may encounter in daily operation. A digital twin of the vessel is therefore used to digitally test for and learn from anticipated, unforeseen, and unimaginable events.

Testing and machine learning like this requires a lot of HPC power and graphic processing units (GPUs). NCC has so far aided Zeabuz in identifying a need for an API for HPC to make the test system for the digital twin run efficiently and on demand on HPC systems. The test system is developed to test the Digital twin of the autonomous vessel. Efforts for the API in this project will be coordinated with the actions in the use case with Digifarm to ensure portability and compatibility. The use case is still ongoing.

Company: OceanTherm
Sector: Climate

OceanTherm develops technology that aims to mitigate the impacts of hurricanes. The high temperature of the surface water in tropical areas is likely a key condition for hurricanes to absorb energy and grow into the higher devastating classes, causing more damage.
Deeper down in the ocean the water temperature is cooler. OceanTherm develops technology that can lift cold water to the sea surface, cooling it and thus preventing hurricanes from building in strength.
The large-scale nature of the problem makes it helpful to deploy workflows that utilise HPC to answer questions related to system design, performance and, ultimately, operation and preparedness. Here, NCC and relevant research groups at SINTEF work together with OceanTherm to standardise workflows to realise a structured computational pipeline involving public data sources and at present three hierarchically nested computational modelling systems. All three systems rely heavily on HPC resources to produce timely and accurate results. The standardized workflow will enable documentation, reproduction, and high throughput to investigate a wide set of scenarios continuously.

Company: Sustainovate
Sector: Marine biomass
 
Sustainovate is a European consultancy company that works with sustainable marine business development and innovation. They work with everything from software and scientific research, to introducing partners to new technologies and ideas, and offering substantive expertise in acoustics and fisheries software.

NCC has provided Sustainovate with assistance and competence transfer when it comes to analysing large amounts of data. As statistical modelling of large datasets commenced, some challenges arose, and we are still working with Sustainovate to improve the data analysis processes.

Company: Norwegian Customs
Sector: Public

The Norwegian customs office keeps a careful record of goods that cross our national border. Their objective is to make it consistent, easy and efficient to follow customs rules and laws, to create a high-perceived risk of detection, and uncover activities falling outside of the rules and laws.

This use case is in its initial phase and the NCC´s role is to create proofs-of-concept for improving analysis of trends and outliers to detect anomalies and/or repetitive voids of rules with associated knowledge transfer and access to HPC resources.

The European Open Science Cloud´s (EOSC) ambition is to provide European researchers, innovators, companies and citizens with a federated and open multi-disciplinary environment where they can publish, find and reuse data, tools and services for research, innovation and educational purposes. EOSC supports the EU’s policy of Open Science and aims to give the EU a global leader in research data management and ensure that European scientists enjoy the full benefits of data-driven science.

Since 2017, several projects have been working to define and establish policies and infrastructure to form the EOSC. It is gradually becoming a reality for Norwegian and international users. Sigma2 and our partners through NRIS, have offered competencies and contributed to the EOSC since the first-generation projects (starting from EOSC-hub in 2018). Sigma2 supports the partnership of Norwegian institutions with the EOSC association by coordinating activities related to delivering services and infrastructure to the EOSC, possibly in synergies with activities happening in NeIC and under the EuroHPC Join Undertaking umbrella.

With HK-Dir, the Directorate for Higher Education and Skills in Norway, Sigma2 became a partner in the Skills4EOSC project in 2022. The project joins knowledge of national, regional, institutional and thematic Open Science (OS) and Data Competence Centres from 18 European countries to unify the current training landscape into a joint and trusted pan-European ecosystem. The goal is to accelerate the upskilling of European researchers and data professionals within FAIR and Open Data, intensive-data science and Scientific Data Management.

In 2020 Sigma2 and several international partners submitted a proposal to the H2020 INFRAEOSC-07a2 call. The project aims at supporting the implementation of EOSC with e-infrastructure resources for data services. The proposal was granted, and the Data Infrastructure Capacity for EOSC (DICE) project started in January 2021.

DICE is enabling a European storage and data management infrastructure for EOSC, providing generic services to store, find, access, and process data consistently and persistently while enriching the services to support sensitive data and long-term preservation. Through this project, Sigma2 offers storage and sensitive data service resources to the users coming from the EOSC portal, thus supporting cross-border collaboration between Norwegian and European scientists.

In 2023 Norway will offer 50 TiB of NIRD Storage to EOSC through DICE for users that will request and be granted access to the resources through the EOSC Virtual Access mechanism.