Cvmfs_config reload stuck

Trying to test some manual configuration, after creating a file under /etc/cvmfs/domain.d/egi.eu.local, I am trying to reload a small testing repo to pick that up:

[root@host ~]# cvmfs_config reload test.egi.eu
Connecting to CernVM-FS loader... done
Entering maintenance mode
Draining out kernel caches (up to 60s)
Blocking new file system calls
Waiting for active file system calls
Saving inode tracker
Saving negative entry cache
Saving page cache entry tracker
Saving chunk tables
Saving inode generation
Saving fuse state
Saving open files table
Unloading Fuse module
Re-Loading Fuse module

It seems to get stuck on step Re-Loading Fuse module, and the client has become unresponsive. I cannot even ls /cvmfs/

What should I do now?

Context data:

[root@host ~]# uname -r
6.10.6-1.el8.elrepo.x86_64

[root@host ~]# rpm -qa | grep cvmfs
cvmfs-libs-2.11.5-1.el8.x86_64
cvmfs-2.11.5-1.el8.x86_64
cvmfs-config-egi-2.6-1.7.obs.el8.noarch
cvmfs-x509-helper-2.4-1.31.obs.el8.x86_64

Is it possible I messed up with the config filename, and it should have been .conf instead of .local?

Hi Jose,

both .conf and .local are valid, with .local taking overriding .conf files. See

https://cvmfs.readthedocs.io/en/stable/cpt-configure.html#structure-of-etc-cvmfs

What is in your /etc/cvmfs/domain.d/egi.eu.local ? And what does cvmfs_config showconfig output?

Cheers,
Valentin

[root@host ~]# cat /etc/cvmfs/domain.d/egi.eu.conf
CVMFS_SERVER_URL="http://cvmfs-stratum1-01.gridpp.rl.ac.uk/cvmfs/@fqrn@
CVMFS_KEYS_DIR=/etc/cvmfs/keys/egi.eu

[root@host ~]# cvmfs_config showconfig
[root@host ~]#

Seems like you are missing a closing " in:

CVMFS_SERVER_URL="http://cvmfs-stratum1-01.gridpp.rl.ac.uk/cvmfs/@fqrn@

I think we should actually do a bash -n <config_file> in chksetup - currently we are not catching errors like this - that would be a nice improvement, thanks for the report! :slight_smile:

Fixed the typo on a second host, but still having issues:

[root@host_2 domain.d]# cat egi.eu.conf
CVMFS_SERVER_URL="http://cvmfs-stratum1-01.gridpp.rl.ac.uk/cvmfs/@fqrn@"
CVMFS_KEYS_DIR=/etc/cvmfs/keys/egi.eu

[root@host_2 domain.d]# cvmfs_config reload
[root@host_2 domain.d]# cvmfs_config showconfig
[root@host_2 domain.d]# cvmfs_config reload test.egi.eu
Connecting to CernVM-FS loader... failed!

OK. Can you try to remount the repository with the config option CVMFS_DEBUGLOG=/tmp/deb.log set?

Should the server url be publicly accessible? Do you get an output from

curl http://cvmfs-stratum1-01.gridpp.rl.ac.uk/cvmfs/test.egi.eu/.cvmfspublished

?

Hi Valentin,

I am getting different behaviour, depending on the host.
Host 1:

[root@host_1]# export CVMFS_DEBUGLOG=/tmp/deb.log; cvmfs_config reload test.egi.eu
Connecting to CernVM-FS loader... done

It never finishes. Nothing on /tmp/

Host 2:

[root@host_2]# export CVMFS_DEBUGLOG=/tmp/deb.log; cvmfs_config reload test.egi.eu
Connecting to CernVM-FS loader... failed!

Nothing on /tmp/ either.

Answering your other question:

[10:25] wup22514 @ laptop : ~ $ curl http://cvmfs-stratum1-01.gridpp.rl.ac.uk/cvmfs/test.egi.eu/.cvmfspublished
Cb0d7b9e24fe371129451a6a1415c86f7adb7008a
B18432
Rd41d8cd98f00b204e9800998ecf8427e
D240
S16
Gyes
Ano
Ntest.egi.eu
X8861527c11de028635385e96a68e38cafbe503f4
H1f9fdee8d35e53493ddde8aaee682995ef0e93a7
T1726488188
M85e4f386da0d5d77c7f66cf2f58e10951f24b653
Y2fea59287322b43da834c354fce484c35a42dc69
--
43c9c57083e6000b586ad7ae3861a8c266df6c9c
�
 ebB�.o��Q��.��~{�dl2��W�0�Eu&ӿ5A��Ý@W��ʘ�=�G �E7%o�߂t%�L�n��*?4S�����j�cG��i?Q�������n�
�0��y���=���8
             �6s�!�n#G4j\
                         n�%�g��o������7}I��N�ʅ�/ʦ�ZE8}R;
                                                         �Ym��
��i|(#{=�&"�P�Xq&��������'劕���̆�ī�H�w+$Ī�����^�L�6[10:25] wup22514 @ laptop : ~ $

After fixing the typo, and rebooting the host, it now works.
I need to find a way to get it done without rebooting.

Question, is the command cvmfs_config showconfig - like that, with no arguments - expected to give any output? It doesn’t return anything.

Normally you should pass a repository name to cvmfs_config showconfig.

By default it shows the configuration of repositories listed in $CVMFS_REPOSITORIES, if you have set that.

Ah, I thought it would print the configuration for all repositories.
Anyways, it seems like all the issues were a consequence of the typo. Repeating the same action for another domain name worked fine.