I’m working on the EESSI project and having had some conversations and questions on using named snapshots I was advised to post here.
We’re looking into the possibility of integrating the named snapshot feature as part of our version/revision workflow for EESSI, e.g., tag particular points in time for EESSI’s state with a DOI, including infrastructure code and available software.
The possibility of naming snapshots sounds super useful for this as we could point users to a given snapshot for a particular revision of EESSI and this seems to fit perfectly with how we are using CernVM-FS already. However, in the CernVM-FS documentation I see:
Named snapshots provide the ability to easily undo modifications and to preserve the state of the file system for the future. Nevertheless, named snapshots should not be used excessively. Less than 50 named snapshots are a good number of named snapshots in many cases.
Is this ~50 named snapshot maximum due to a technical consideration (maybe something really slows down or the storage requirements increase a lot) or is it something like “good practice” advice for what should in general be done with CernVM-FS?
Any clarification or tips would be super helpful. Thanks in advance!
Kenneth from EESSI raised this issue at the CVMFS coordination meeting on June 15. Named snapshots do result in increased space usage and increased snapshot time on Stratum 1s. A lot of people don’t really understand what they’re for and use them excessively so that’s why the documentation discourages their use. They require a client setting to move back to an older version and that’s usually not very convenient except on single-user clients for short periods of time. In general it’s probably better to leave old code that’s still desired visible at a path in the repository and don’t plan on backing up to old snapshots. Having said that, I think we concluded that the use cases that Kenneth had in mind for EESSI were probably reasonable.
I suppose that the threshold for when too many named snapshots become an issue is less of a hard limit but depends on each revision’s size and/or the differences between each revision, would that be the case?
I think indeed this feature can still be very useful for us, but we have to be careful with our revision policy to make sure we implement something sustainable in the long run.