We would like to build the cvmfs packages for opensuse tumbleweed since the suse rpms available are for much older suse products. They work, but it is not ideal and we worry things could break.
The first problem was that cvmfs-2.8.0.tar.gz unpacks to cvmfs-cvmfs-2.8.0, but the spec does not expect this. How do you actually build your Suse (or RH) packages?
So I move the unpacked dir to cvmfs-2.8.0 and a build started, but failed soon. After a few trivial fixes (zlib-devel, python2) I am stuck (after the actual build actually worked) at
…
[ 996s] vmfs-shrinkwrap-2.8.0-18.1.x86_64.rpm: directories not owned by a package:
[ 996s] - /usr/libexec/cvmfs
[ 996s] - /usr/libexec/cvmfs/shrinkwrap
[ 996s] cvmfs-server-2.8.0-18.1.x86_64.rpm: directories not owned by a package:
[ 996s] - /etc/cvmfs
[ 996s] - /var/spool/cvmfs
[ 996s] - /var/www
[ 996s] - /var/www/wsgi-scripts
[ 996s] - /var/www/wsgi-scripts/cvmfs-server
[ 996s] cvmfs-2.8.0-18.1.x86_64.rpm: directories not owned by a package:
[ 996s] - /etc/cvmfs
[ 996s] - /etc/cvmfs/default.d
[ 996s] - /usr/libexec/cvmfs
[ 996s] - /usr/libexec/cvmfs/authz
[ 996s] - /usr/libexec/cvmfs/cache
so the package definitions in the cvmfs-universal.spec seems broken?
I’ll try that when I am at home. Is there a reason why the spec in the tarball is broken?
For OBS opensuse tumbleweed (our target) you need to add BuildDepends: zlib-devel and fix the cvmfs_python=python2 setting in the spec by adding a test for suse_version.
The interesting part is the fuse kernel module crash though … we may have to add older kernels to OBS to survive …
Is there a reason why the spec in the tarball is broken?
It’s not terribly surprising if nobody has tried it before on OBS. OBS has its own way of doing things, and packaging often needs to be slightly adjusted to support it. The cvmfs team has their own cluster for compilations and it uses different conventions. It uses scripts in the ci directory. The Open Science Grid rebuilds the cvmfs packages for RHEL derivatives, but their system supports rebuilding from src rpm and that’s what they use. OBS doesn’t support rebuilding src rpms as far as I can see.
I had to take cvmfs-2.8.0.tar.gz from that repo to make it work with the spec from the tarball from the cvmfs web pages.
In addition the cvmfs-2.8.0.tar.gz from ecsft does not contain the “packaging” subdir, i.e. no spec files. Sigh. And the spec file used for the build is not on Index of /dist/cvmfs (or I was to dumb to find it). Sigh.
I haven’t tried cvmfs rpms before on OBS, but I did once get OBS to compile cvmfs for Debian, by adding a patch with a .dsc file. I’m trying it now with an older version of cvmfs for rpm: Show home:drdaved:cvmfs / cvmfs - openSUSE Build Service
I think they don’t do that in order to avoid a chicken-and-egg probem: a src.rpm is a product of the build, so it cannot be an input if you want to be able to bootstrap.
but it needed substantial changes to the spec in order to pass OBS sanity checks. It still has errors and inconsistencies, but they are below threshold and OBS allows me to publish the packages. Some of the errors need source changes so I can’t do that.
The cvmfs team should have a look at the spec I use to make sure I am not messing up something.
I would plead for inclusion of a working spec (for OBS and COPR) in the ecsft hosted tarball, or next to it, so there is a clean workflow from the cvmfs project home to package building.
It is exceedingly difficult to try to build for centos7 on OBS since one gets a minimal system w/o all CERN updates. I tried centos8 (stream) on OBS but that fails with not finding python2-setuptools and -devel and golang (“nothing provides …”). I have no idea why because these packages are in the “System” repo at least on lxplus8.
To be honest, for CERN centos7 (or 8) (as it is really not the same as pure centos7/8 with the ton of updates in it) CERN should host a public building platform a la OBS or COPR, or leverage OBS or COPR to publish their stack of packages there. That would go a very long way towards packages with clean spec files and eventual inclusion in fedora or t-weed for some basic system-level packages.
What do the CERN updates provide? I build cvmfs no problem on the OSG koji builders and those are based on vanilla CentOS 7 & 8. The OSG koji does support EPEL, and I was able to get pretty far on my OBS build by including the Fedora:EPEL:7/standard repository. Everything built for me, it just had errors that there were extra unpackaged files at the end of the build.
I too ran into the problem of no python2 packages on CentOS 8 with and without stream and also have no idea why those are missing … ah, this comment thread on the OBS CentOS-8 project explains that extra project config is required for optional CentOS-8 modules.
ok, I tried to copy from your project setup the centos8 stream config … that resolves the python2-* stuff but it still deosn’t find golang. But on lxplus8 golang is from
Repository : @System
From repo : AppStream
[skluth@lxplus803 ~] yum module provides golang
Last metadata expiration check: 0:50:53 ago on Fri Mar 5 18:55:12 2021.
golang-1.14.12-1.module_el8.3.0+605+410c5674.x86_64
Module : go-toolset:rhel8:8030020201222185934:13702366:x86_64
Profiles :
Repo : AppStream
Summary : Go
[skluth@lxplus803 ~] yum module list go-toolset
Last metadata expiration check: 0:51:05 ago on Fri Mar 5 18:55:12 2021.
CentOS-8 - Appstream [HEAD]
Name Stream Profiles Summary
go-toolset rhel8 [d][e] common [d] Go
It turns one has to set in “Project” this line
ExpandFlags: module:“go-toolset”-rhel8
to enable the module. I had a lucky guess at the quotes to overcome the first “-” …
Wait, I’m confused. Where exactly did you set that ExpandFlags? I don’t see it with quotes in your Project Config. I haven’t had success yet in getting my Project Config to resolve the python2-* or anything.
first a centos8 build ran then it didn’t so I played with the quotes and eventually it ran without them … no idea what OBS is doing behind the scenes … but its done now