Files .cvmfswhite get truncated on version 2.7.4

Publisher / Stratum 0
1

Hi,

this is something that has started happening at our Stratum-0 after upgrading the service to version 2.7.4.
Sometimes, without any manual intervention, the files /path_to_CVMFS_UPSTREAM_STORAGE/repo/.cvmfswhitelist get truncated, missing the signature part. That prevents publishing from succeeding.

When it happens, we get something like this in our logs (anonymizing the repo name):

    Using auto tag 'generic-2021-02-21T09:50:25Z'
    failed to fetch repository manifest (8 - bad whitelist)
    Segmentation fault
    Synchronization failed

    Executed Command:
    cvmfs_swissknife sync -u /cvmfs/<repo name> -s /var/spool/cvmfs/<repo name>/scratch/current -c /var/spool/cvmfs/<repo name>/rdonly -t /var/spool/cvmfs/<repo name>/tmp -b 280626e31262af818cd30af0990ea2899ddad036 -r local,/srv/cvmfs/<repo name>/data/txn,/srv/cvmfs/<repo name> -w http://localhost/cvmfs/<repo name> -o /var/spool/cvmfs/<repo name>/tmp/manifest -e sha1 -Z default -C /etc/cvmfs/repositories.d/<repo name>/trusted_certs -N <repo name> -K /etc/cvmfs/keys/<repo name>.pub -L -D generic-2021-02-21T09:50:25Z -G 00 -f overlayfs -p -l 4194304 -a 8388608 -h 16777216

So far, quick and dirty solution has been to manually replaced them by a backup copy.

Is there a known issue in 2.7.4 causing this? If yes, is it fixed in 2.8.x?
Otherwise, is there a better solution that manually restoring the files from a backup copy?

Thanks a lot in advance.
Cheers,
Jose

2

Hi Jose,

The publication itself does not touch the whitelist, so I assume the problem is with your whitelist resigning procedure. How is this done at your site? Are you running a cron job that calls cvmfs_server resign?

Cheers,
Jakob

3

Hi Jakob,

that seems to be the issue. We had a set of script doing it on the old server, but they are missing on the new one.

Thanks a lot !!
Cheers,
Jose