for historic reasons, we operate CVMFS with an Alien cache on a shared filesystem in our HPC cluster.
According to the docs, there’s neither a CVMFS quota nor an automatic cleanup for this cache type.
The underlying filesystem would allow me to setup a quota on directories, so outside of CVMFS itself.
Would this work as a way to limit the cache usage by CVMFS? Or will it break, once it reaches the quota limit of the directory and receives a “no space left on device” error from the filesystem?
I would expect it to just receive “no space left on device” errors. Some other process would still need to clean up the alien cache.
I expect a cvmfs alien cache to cause a heavy load on your shared filesystem metadata servers, depending on how heavily your cvmfs is used when starting jobs on many nodes in parallel. Admins of HPC nodes with no local disk have had better success with creating a large file for each node on their shared filesystem, each containing a loopback filesystem mounted to just one node for that node’s cvmfs cache. That way the many metadata operations are moved to the nodes and each cvmfs client can manage the cleanup of their own cache.
Thanks, so I really need to implement some external cache cleanup
I found some infos in this forum for cleaning up the alien cache, I’ll have to see.
Regarding the performance:
The CVMFS usage has been light, so that was not (yet) an issue. I’ll keep your suggestion in mind, in case usage grows or we see thrashing on the filesystem.
