Autofs timeout when no network connection

Hello,

at our institution we have installed CVMFS in a number of laptops. They mount /cvmfs (provided by our Stratum-1 servers in our LAN) via autofs, and this works without any trouble when they are connected to the internal network.

But we are having issues when the laptops are rebooted without a network connection. They obviously cannot reach the CVMFS servers, and in the logs I see entries like:

oct 04 14:59:28 xxx.iac.es systemd[1]: Starting Automounts filesystems on demand...
oct 04 14:59:38 xxx.iac.es automount[1873]: setautomntent: lookup(sss): setautomntent: Host is down
oct 04 14:59:49 xxx.iac.es automount[1873]: setautomntent: lookup(sss): setautomntent: Host is down
oct 04 15:00:01 xxx.iac.es automount[1873]: setautomntent: lookup(sss): setautomntent: Host is down
oct 04 15:00:13 xxx.iac.es automount[1873]: setautomntent: lookup(sss): setautomntent: Host is down
oct 04 15:00:26 xxx.iac.es automount[1873]: setautomntent: lookup(sss): setautomntent: Host is down
oct 04 15:00:39 xxx.iac.es automount[1873]: setautomntent: lookup(sss): setautomntent: Host is down
oct 04 15:00:41 xxx.iac.es automount[1873]: problem reading master map, maximum wait exceeded
oct 04 15:00:41 xxx.iac.es automount[1873]: /usr/sbin/automount: warning: could not read at least one map source after waiting, continuing ...
oct 04 15:00:41 xxx.iac.es systemd[1]: Started Automounts filesystems on demand.

It seems it takes around 70 seconds to timeout, but this seems to be giving issues in some situations together with GDM and the antivirus (we are still trying to find out what is really going on…)

Anyway, the thing is that I would like to play with that timeout, making it shorter or longer to see if things improve a bit that way, but despite adding the timeout option in the /etc/auto.master.d/cvmfs.autofs file, the time to start autofs when not online always seems to stay at 70 seconds. Is this hardcoded somewhere? Any way to change it?

Any pointers much appreciated.
Angel

Hi Angel,

this is very strange - I’m not sure why autofs would try to mount CVMFS during startup. Can you send me a bugreport tarball from

cvmfs_config bugreport

from one of the laptops?

Thanks,
Valentin

OK, I will try to get that report (the issue is that I can only have admin privileges in it when connected via ssh, so by then CVMFS is working fine, but I will try to get admin privileges locally, so that I can get the report while CVMFS is still not mounted).

I have no idea about how autofs does work internally, but from the logs, it looks as if the problem is just getting the master map, not really mounting CVMFS?

The bugreport gathers the autofs configuration and syslogs, so I think it’s fine to get the report from ssh.

Never mind, I managed to have admin rights while off-line, so I generated the bug report while still off-line.

I modified the master_wait parameter in the autofs.conf file, so that now autofs on startup timeouts much faster, but you still see a couple of:

setautomntent: lookup(sss): setautomntent: Host is down

As far as I can see I cannot attach files here in the forum. How can I send it to you?

Cheers,

Hello,

the timeout described by mount is the time until an unmount happens. this is the default of 5min
/etc/auto.cvmfs on /cvmfs type autofs (rw,relatime,fd=6,pgrp=2736831,timeout=300,minproto=5,maxproto=5,indirect,pipe_ino=69485686)

the 70 sec you are describing is seems more related to the following general config value of autofs: autofs.conf(5) - Linux manual page

negative_timeout
              Set the default timeout for caching failed key lookups (program default 60). If the  equivalent  command  line
              option is given it will override this setting.

can you try it if that changes something?

Not sure if I already tried the negative_timeout option, but what seems to work (and now is part of our installation script) is to modify the master_wait parameter.

I now run:

echo "MASTER_WAIT=1" >> /etc/default/autofs

and the waiting time goes from ~70 to ~10 seconds.

Maybe this is not the best option if the computer is trying to mount other drives via autofs when starting, but in our case we only mount the one used for CVMFS, and waiting for 10s is more than enough in our scenario. If the server cannot be reached in 10s, the laptop is either off-line or not connected to our VPN, so CVMFS will have to use its cache.

We are running this way so far with only a few laptops, but waiting time has been decreased substantially and there has been no issues reported.