In November, we signed a contract with Datopian AS to implement and deploy the next-generation national Research Data Archive.
The new Research Data Archive will allow the publishing and discovery of datasets of any size and format, support Open Access, facilitate the FAIRization of science in Norway, and will be based on the open-source data management platform CKAN.
The new service will replace our current Research Data Archive, which to date is the largest archive for research data in Norway with almost 1.000 TB of published data. This archive was set in production in 2014 and has since steadily increased to the current volume.
In recent years, we have identified a need to improve our service from a usability perspective. Additionally, the principles of Open Science and the evolving needs of our end-users in this era of data explosion have revealed new requirements for enhanced interoperability, data protection and sharing, and support for data discovery. This is why we initiated the Next Generation Research Data Archive project in late 2021. The project aims to design, procure, implement, and deploy the new service.
User-friendliness, modularity and interoperability as design principles.
We have conducted the concept work in participation with several Norwegian stakeholders. The driving force has been to restructure the service to allow easy interoperation with the services already present in the sector and to minimise the overlap and maximize the synergies.
The designing principles are based on concepts such as user-friendliness, modularised architecture, and interoperability by design. We have considered several possible data management platforms. We eventually chose CKAN, for, among other things, the open-source community behind it and the highly modular code architecture which ensures easy customisation, easy sustainability of the code and higher interoperability. CKAN is already adopted as a data archiving platform by several Norwegian institutions in research and public administration.
Who is Datopian AS?
The public procurement competition was conducted globally, and several CKAN enablers responded to the call for tender. Among them, Datopian AS stood out due to their extensive experience, broad competencies, close collaboration with the CKAN community, and open-source mindset. Once Datopian won the competition, an intensive dialogue began to translate our requirements into clear design and implementation strategies. The contract was signed in early November, after which the company initiated the first sprint. Datopian employs an agile, SCRUM-based implementation methodology and will continue working on implementing the new archive until the summer of 2024.
— We are thrilled to embark on this transformative journey with the Sigma2 AS team. We look forward to delivering a solution that not only meets but exceeds expectations. Together with Sigma2, we'll create an awesome solution that sets a new standard in research data management. Let's make this venture a resounding success! says Daniela Popova, Senior Project Manager at Datopian.
What are the next steps?
Throughout several sprints and reviews, Datopian, in collaboration with NRIS, will implement high-priority use cases selected during the project's concept phase step by step. These include uploading large volumes of data, customizing metadata schemas, and data streaming. The progress will be recorded and documented on the project webpage. Furthermore, the project will aim to further elaborate on the concept of an archiving solution for large, unstructured, sensitive data, in collaboration with key national stakeholders.
By autumn 2024, we will witness the deployment and production setting of the new Research Data Archive. It promises to be an exciting development.