Move operation is extremely slow

[rptaylor]

Hello,

We have an older repository that we want to replace with a new repository, but users may still be accessing the path /cvmfs/oldrepo/content. So we want to move the ‘content’ directory out of the way so we can replace it with a symlink to the new repo:
/cvmfs/oldrepo/content → /cvmfs/newrepo/content

However a publish operation (on a remote publisher via GW) on the old repo to simply do ‘mv content whatever’ (so we can create a symlink named ‘content’) is extremely slow. There is practically no CPU usage, and the IO usage also appears low. The only discernable activity in /var/spool/cvmfs/oldrepo appeared to be a gradual growth in /var/spool/cvmfs/oldrepo/cache , so I wonder if it was trying to read the entire content directory first from the stratum 0 and then write that back to the new location on the stratum 0 which would be very inefficient.

OTOH, trying the same thing directly on the stratum 0 server quickly filled up the spool space - which is limited in size on the s0 because all publishing is normally done via the remote publisher node.

Is there anyway to do a fast and efficient move operation of a large directory of data?

Thanks.

[dwd]

I don’t know if it works through a gateway, but a template transaction may be what you want. That quickly duplicates a tree from one location to another by replicating the catalogs. Then you could remove the old tree, and hopefully that won’t take too long.