Development project: New Research Data Archive

We are happy to launch the new data research archive project. The project aims to design and develop Sigma2's next-generation Research Data Archive.

Map outline of Norway with icons associated with research and archiving on either side of the map.
The new Research Data Archive will be user-friendly, flexible and secure, while supporting the FAIR principles.

This project aims to design and develop Sigma2's next-generation Research Data Archive.

This project will develop the next-generation research data archive concept that best supports the Open Science paradigm. At the same time, it will ensure business confidentiality and personal privacy protection for data that require this.

The final solution will be flexible, interoperable and user-friendly by design. We strive to minimise overlapping and enhance the synergies and interoperability layer with existing services in the national and international data landscape.

The project will design, implement, and deploy the archive service. The new archive for non-sensitive data will be included in the NIRD ecosystem and will have NIRD as the backend.

Project background

The Sigma2 Research Data Archive provides long-term storage for datasets resulting from Norwegian Research. Each dataset is issued with a DOI from DataCite and kept for at least 10 years after publication. Currently, the archive has more than 750 TiB of archived data.

After operating the archive service for 5 years, we have uncovered the need for improving the service from a usability perspective. Also, the Open Science principles and the evolving needs of our explosion of data-era end-users reveal new functional requirements for enhanced interoperability, data protection and sharing, and support for data discovery.

Phases and timelines

The project is divided into 5 main phases separated by decision gates:

  1. Concept phase

We have followed the design thinking methodology in the concept phase to define problems/needs and reframe these in a human-centric way to focus on what is most important to the users.

  1. Planning phase

We have run a public procurement competition to appoint the enabler of the new CKAN-based Research Data Archive.

  1. Execution Phase

Current phase. The actual implementation of the new code is on-going. At the end of this phase the new service will be deployed on the NIRD ecosystem. This phase will be completed autumn 2024.

  1. Close-out Phase

The new system has been tested and has finally been launched. The old front end will be decommissioned. This phase will be completed by year-end.

  1. Benefit realisation phase

From January 2025, we will collect impressions from end-users regularly, to define and update roadmaps for the implementation of more FAIR features over the years to come.

Project organisation

The project follows the Scrum method, with sprint planning and review meetings every second week, and monthly sprint retrospect meetings. There are also scheduled technical and follow-up meetings between Datopian and Sigma2 Project managers every week. The work is primarily done online since the project resources are geographically widely spread.

Tore Aalberg is the Sigma2 Project Manager, while Daniela Popova is the Datopian Project manager. Project status is reported monthly to the project board.

The Sigma2 side of the project has a core-team of 5 persons, chosen according to their expertise and competence. The core-team is working together with Datopian in the development of the solution and is crucial in continuously testing elements. Datopian has a developer-team with 6 members.

A reference network with 10 participants provides comments from a user and stakeholder perspective and gives additional specialist views on solutions and methods. They will also participate in the acceptance testing with the Sigma2 core team.

About Datopian, our supplier

In November 2023, we signed a contract with Datopian AS to implement and deploy the next-generation national Research Data Archive.

The public procurement competition was conducted globally, and several CKAN enablers responded to the call for tender. Datopian AS stood out among them due to their extensive experience, broad competencies, close collaboration with the CKAN community, and open-source mindset.

Datopian employs an agile, SCRUM-based implementation methodology.

Status updates