Repository Ouroboros

This is observational. This is autobiographical. This melds my own experiences as Metadata Librarian and Digital Collections Librarian with those of dozens of metadataists I’ve known over the years. This is born of hearing the same existential crisis again and again. If you are somewhere on the ouroboros, you are not alone, you are not broken, and you are not hopelessly behind.

The library is going to adopt a new repository and you just got hired to make it happen. You may be fresh out of library school with a few metadata projects under your belt. Perhaps you did metadata work on a two-year digitization grant and are looking forward to getting out of the spreadsheet mines. Or maybe you worked in a similar job at your last institution—running a turnkey repository like DSpace, CONTENTdm, or BePress. Perhaps you’ve moved into the role at an angle, from something more traditional like cataloging.

Whatever the case, congrats on becoming a metadata librarian, digital collections librarian, digital archivist, or whatever they’ve decided to title this position no one quite understands! Looks like the institution has a couple developers on staff, so you’re going to adopt something more extensible than BePress—Repository Q. You’ve heard great things about Repository Q. A lot of tech questions and decisions are about to come your way, so you cram Agile/Scrum/XML/RDF/Linked Data/DublinCore/MODS/general design principles.

You do a lot of reading and go to presentations. You look at other repositories built using Q and its earlier cousins, P, J, and F. It’s exciting! Some of the top libraries in the country use Repository Q. You do outreach to stakeholders. You gather requirements. You do your best to make a whole lot of different needs work. You prioritize. You try to understand what the developers are asking. You try to explain what you need. You ask questions in the community, but it seems like everyone else is on a slightly older and different version of the software. Some major underlying pieces have changed. But how exciting! You get to be on the cutting edge.

You start preparing objects and data. You ask how you can add things in batches. You’re told it’s doable, but it doesn’t come out of the box, so you add it to the development roadmap. A couple years go by. You attend community groups. You do lots of testing. You keep meeting with the stakeholders. You remind them that you’re adapting this software to meet all kinds of different needs and you’re very close. Yes, that other person’s needs got prioritized first. No, that doesn’t come out of the box, but you’ve heard it’s being developed by another institution! You saw a demo at the last conference you went to. You send out a link to the presentation. Everyone loves it. It’s great that you’re adopting this software because community development means you’re going to get so many cool features automatically once you’ve adopted it.

You keep going to community meetings. You present on how your institution’s project is nearly done. You’re sitting in a tech talk where you only understand about a third of what’s being said. You start to get an uneasy sense about the really cool new feature you were expecting. What’s that they’re saying? Different direction? Some new product names you don’t recognize?

After the meeting, you check in with your dev coworkers. They’re really excited but also a bit stressed. Yes, you got the gist correctly. A group in the community has decided to take a completely new approach to how an underlying function is done. It’s going to allow for a lot of awesome stuff and make it much easier to implement that new feature. They’re already using it in production at Institution Y. One of the devs went to a half-day workshop on it yesterday and thinks it’ll be really transformative.

Of course, you won’t be able to adopt it quite yet. It’s much too close to your launch to completely change how you do that underlying function and would require shifting some major components. So you’re going to go ahead with Repository Q, but you should start learning now about Repository X in order to make sure you can migrate the data once X is more mainstream. You redo your schedule to attend sessions where you can learn more about X. You learn that X is going to bring you in line with some very cutting-edge data practices you’ve heard about elsewhere, but don’t quite understand yet.

You come back from the conference and report to your stakeholders. That new feature isn’t actually on the roadmap for this version of the software any more, but here’s all the cool new features you’ve learned about in those sessions on X. You do your best to manage expectations. It’s going to be a little while before you can move to X. Your software is still built on Q and all the data you’ve been preparing is intended for that. Anyway, the launch is coming up soon and how exciting is that?

You work very hard on preparing the data, putting it in, testing. You’re lucky, and you’ve got a pretty straightforward migration from some older system and a lot of new data to create and work with. You’ve heard stories from people doing much more difficult migrations. You watch the software’s Slack and show up to affiliate meetings online and in your local area. You notice that X has become the really new thing. Your AD for tech asks you about X. It seems like nobody is moving to Q any more. Of course, you’ve met a few other people who have recently adopted Q. You start talking about X migration with them.

You present on your brand new Q Repository at the next big community software meeting and everyone is very complimentary. Your developer colleagues have done a great job. Your stakeholders have found some great collections to highlight. Of course, it’s still new so it doesn’t support quite everything you’d like yet. That’s fine, one of the upsides of working on community software is you can find partners to develop this stuff together. You keep adding materials, meeting with stakeholders, and letting them know what’s next on the horizon.

Bad news. One of the developers reports back that four features on which you’d counted to make repository shine have been moved from Q-model to X-model. The community just put in two sprints to change a bunch of underlying code to fit with X and your coworker isn’t sure it would be really feasible to bring these back to Q. They work on some of the other changes you’ve identified, and you push those features to your “once we migrate to X” sheet. Since you presented on these features library-wide, you get a few questions about when they’re coming. You try to keep a positive attitude.

You’re pretty lucky, though. After about 2 years of having the repository up, you’ve put in some nice collections. You’re proud of it, even though you’ve got a mental list of all the features you’d like to see. You’ve had some rather rough encounters with coworkers whose collections can’t really go into the repository until after the X migration. You didn’t realize how much of your job would require expectations management and walking back things you had told people you’d be able to do. Mostly they’re not mad, just disappointed. But, you’re lucky. You’ve got some coworkers who love what you’ve been able to do.

Now that your repository has been up for a bit and the developers have had a chance to get their hands onto X, you start preparing for an X migration. You make some library-wide presentations about Repository X, why it’s a good idea to migrate to it, and what this will mean. Your AD has approved the work, as quite a few of their peers are interested in Repository X too. As you add more data, you continue to think about how you’ll migrate it and what you’ll be forced to change. You slow down a bit on adding data as you start to write out new data profiles for Repository X. You’re lucky, your developer coworkers are pretty good at data manipulation and they work with you on the crosswalks you’ve come up with. You start to get some copies of existing collections into your Test Repository X.

You bring the test repository to the next community meeting and present on your migration strategies and what you plan to do with these four great new features you’re getting. You start to hear about some even cooler new ideas and know exactly which stakeholders would love those. You come back to your library and present to stakeholders about how the Repository X migration is going. You explain that you’ll have to put a freeze on adding new collections while you work through the migration. Someone whose work got pushed to the end of the queue is pretty disappointed. Fortunately, this cooler new idea would be a bonus for their collection and you come up with a few really great examples which get everyone in the room excited again.

A few months go by and your move to the new Repository X instance is going pretty well. You’ve heard some real horror stories that make you grateful for your coworkers’ assistance and the knowledge you’ve picked up along the way. By now, you have a pretty decent idea of about two-thirds of what people are saying in community tech presentations. But you can’t help noticing a new term around the repository community. Someone’s gotten a big grant to work on Z. Five institutions are participating. Z will have all the features of X but be easier to manage. It’ll also have some rad new features and a better way of handling the underlying data.

You can’t help but notice that several of those new features are things which are on the Repository X roadmap. You take a deep breath and email the repository tech listserv. What does Project Z mean for Repository X. This time, you can only understand about 25% of the responses. You feel embarrassed. You’ve been part of this community for years, but sometimes it still all sounds like another language to you. You thank people for their replies, but don’t really know what it means.

You message one of your former developer coworkers who now works at an institution working on the Project Z grant. They’re a great person and always tell it to you straight. They confirm your suspicion. These features, including that one you introduced in the last stakeholder meeting, are all going away from the Repository X roadmap. Your former colleague notes that only one person is really maintaining stuff around Repository Q any more, so it’s good you’ve moved to Repository X, but advises you to stay on top of Z.

You hear more and more about Z and notice that nobody seems to be working on anything related to X any more. That one person who’d been maintaining Repository Q is keeping things from breaking in X too. Well, from breaking too often. Your new developer coworkers plan a couple sprints to fix some serious issues with X. They’re mostly familiar with Repository Z, because by the time they joined the community, that’s what most of the conversation was around. It looks like Repository X is pretty much dead. The team you work with knows how to maintain it and add a few little local tweaks which make your life easier. But it’s not getting those features you wanted. Of course, you don’t have as many people as the institutions working on Project Z.

You check in with that former coworker who’s on the Project Z grant. They suggest that you might apply for a job at their institution soon. They’re about to do a big migration from Repository J to Repository Z and they’ll be hiring people with your skills. Your CV looks great. You have 5 years of experience in the community. You know some people at the institution who’ve been very nice to you.

But you feel like a fraud. You feel so discouraged. You are sure everyone else is ahead of you. You do not yet see that you are just one more person riding the Repository Ouroboros.

an image of a sad and cute fox-like ouroboros eating its own tail

NB, This is often a very gendered situation. This does not mean that even male developers at well-resourced institutions do not experience similar crises, but I ask Who’s the One Left Saying Sorry to institutional stakeholders as the course continually changes?

Image: Fol. 279 of Codex Parisinus graecus 2327, wikimedia commons https://commons.wikimedia.org/wiki/File:Serpiente_alquimica.jpg

Further reading

The above is an affective and observational approach to the struggles of librarians working in repositories. The following articles provide scholarly perspectives on aspects of the work. Dohe’s “Care, Code, and Digital Libraries,” is the most directly parallel to this piece in its focus on repository development processes and roles. Kendrick studies morale on the whole. Salo’s writing deals more specifically with issues of scholarly communication and institutional repositories. My own survey on deposit rates and institutional repositories was conducted to provide context to those who might feel their own repository an outlier for having low deposit rates, but does not address issues of development process.

Kate Dohe. “Care, Code, and Digital Libraries: Embracing Critical Practice in Digital Library Communities.” In the Library with the Lead Pipe, February 20, 2019. http://www.inthelibrarywiththeleadpipe.org/2019/digital-libraries-critical-practice-in-communities/.

Kaetrena Davis Kendrick. “The low-morale experience of academic librarians: A phenomenological study.” Journal of Library Administration, 57, no. 8( 2017): 846-878. Retrieved from http://www.tandfonline.com/doi/full/10.1080/01930826.2017.1368325

Dorothea Salo. “Innkeeper at the Roach Motel.” Library Trends 57, no. 2 (2008): 98-123. http://digital.library.wisc.edu/1793/22088

Dorothea Salo, “How to Scuttle a Scholarly Communication Initiative.” Journal of Librarianship and Scholarly Communication, 1, no. 4 (2013): p.eP1075. DOI: http://doi.org/10.7710/2162-3309.1075

Ruth K. Tillman. “Where Are We Now? Survey on Rates of Faculty Self-Deposit in Institutional Repositories.” Journal of Librarianship and Scholarly Communication, 5, no. 1 (2017):, p.eP2203. DOI: http://doi.org/10.7710/2162-3309.2203