I have some publishers running transactions in parallel on different paths of my repo, e.g.:
publisher 1 on my.cvmfs.repo/folder1/
publisher 2 on my.cvmfs.repo/folder2/
While there’s no problem in opening simultaneous transactions through the gateway, I sometimes obtain the following error while publishing:
Using auto tag 'generic-2022-10-07T14:14:04Z'
Warning: CAP_DAC_READ_SEARCH cannot be obtained. It's not in the process's permitted set.
Processing changes...
Waiting for upload of files before committing...
Committing file catalogs...
Wait for all uploads to finish
Exporting repository manifest
Lease end request - error reply: {"reason":"could not commit transaction: database is locked","status":"error"}
SessionContext: could not commit session. Aborting.
[ERROR] Failed to commit transaction.
Statistics stored at: /var/spool/cvmfs/herd.sw.common/stats.db
Synchronization failed
Executed Command:
cvmfs_swissknife sync -u /cvmfs/herd.sw.common -s /var/spool/cvmfs/herd.sw.common/scratch/current -c /var/spool/cvmfs/herd.sw.common/rdonly -t /var/spool/cvmfs/herd.sw.common/tmp -b c6c80f5c7300b50c3b4545658e702d0a7d9e732c -r gw,/srv/cvmfs/herd.sw.common/data/txn,http://gateway.131.154.96.113.myip.cloud.infn.it/api/v1 -w https://minio-herd.cloud.cnaf.infn.it/cvmfs/herd.sw.common -o /var/spool/cvmfs/herd.sw.common/tmp/manifest -e sha1 -Z default -C /etc/cvmfs/repositories.d/herd.sw.common/trusted_certs -N herd.sw.common -K /etc/cvmfs/keys/herd.sw.common.pub -L -D generic-2022-10-07T14:14:04Z -G 00 -H /etc/cvmfs/keys/herd.sw.common.gw -P /var/spool/cvmfs/herd.sw.common/session_token -f overlayfs -p -l 4194304 -a 8388608 -h 16777216 -i
I guess that while it’s possible to acquire two leases on different paths at the same time it’s not possible to publish at the same time, even on different paths. Is this correct? If yes, how can I check is a publish is ongoing before starting a new one?
That’s an interesting failure! It may actually a bug in the gateway’s handling of the SQlite database that keeps the leases. Would you mind opening an issue on Issues · cvmfs/cvmfs · GitHub?