By: Eric Lopatin
Since our inception, a main focus of UC3 has been to deliver high quality, reliable digital preservation services for the UC community. Currently, this takes the form of both consultative and community engagement work, as well as technical development to ensure our digital preservation repository, Merritt, remains durable and innovative.
For 2021, our digital preservation goals fit into three key themes: Community Engagement, Simplification, and Scaling. By working with these themes in mind, our goal is to promote the values at the core of many preservation systems and programs – values such as reliability, authenticity, integrity and sustainability. The crossroads of technology and policy are where these values play out, and we’re aiming to keep abreast of many of them while heading into 2021.
Community Engagement
Metrics and insight – Last year, our team laid the groundwork for more granular reporting on content held in Merritt. This work has already allowed us to provide reports for campuses that illustrate a variety of aspects related to their collections. This year, we’ll work on dashboards and data visualizations to provide for more insight to users into their collections.
UC system-wide digital preservation – Over the past two years, UC3 has participated in multiple phases of a system-wide Digital Preservation Strategy working group. The next phase of this effort will establish a systemwide leadership group and begin to construct a digital preservation training program across UC campuses. In 2021, our team will continue to participate in these efforts as they set the course for future projects across our campus community.
NDSA engagement – The National Digital Stewardship Alliance is a longstanding, community building organization which promotes discussion, learning and standards surrounding digital preservation. This year I’ll be co-chairing the NDSA Infrastructure Interest group with Leah Prescott at Georgetown University. We’ll be facilitating conversations surrounding preservation technologies and infrastructure, while also joining NDSA Leadership meetings to help apply input from interest group participants directly to activities the organization takes on throughout the year. Given the record attendance of NDSA’s recent DigiPres 2020 conference, I’m looking forward to helping build out future opportunities through which the larger preservation community can collaborate.
Simplification
Preservation Assurance – The overarching UC3 digital preservation strategy calls for creating and maintaining three copies of every object in Merritt, across two geographic regions with differing disaster threats, with at least one of those copies being less volatile in nearline storage. All Merritt collection content now adheres to this strategy, and we replicate new submissions to our cloud storage providers as it arrives.
New submissions – One of the most commonly used methods of adding new content to Merritt is uploading a manifest file that enables batch ingest of hundreds or thousands of objects that are pulled from a user’s on-prem storage. In 2021, we’re planning to simplify and automate the manifest creation process to assist users with this task, so they can be assured that all of their objects and object-level metadata will be handled correctly.
Common API – One API for use across the Merritt system has been a goal for quite a while, and we’re looking forward to making it a reality. In 2021, we will continue our work to design a common API for use by users and Merritt microservices alike. This will allow for submitting content, gaining insight into existing submissions, easier external systems integration, and of course access to individual files and object versions.
Scaling
Auto-scaling – The theme of Simplification goes hand-in-hand with Scaling. In our case, effectively scaling aspects of the Merritt system could be more aptly referred to as auto-scaling. In a recent blog post, we discussed how the team has been at work implementing a centralized parameter store with AWS Systems Manager to streamline Merritt microservice configuration.
Resilience – In 2021, our work will include simplifying the process of adding new hosts when needed (during periods of increased load on the system). Eventually our goal is to reach the point where this can happen without human intervention. And on the flip side, spinning down hosts when they are not needed will occur as well. Auto-scaling microservices in this sense promises to make the overall Merritt system more resilient, secure and cost effective.
In summary, 2021 promises to be a busy year for our digital preservation team at CDL. As always, feel free to contact me with any questions. I am happy to discuss any of these ideas and directions for 2021, along with others you may have in mind!
This blog is a part of the “A Peek Into 2021 for UC3″ series.
