distutils was deprecated in Python 3.10 (PEP 632) and removed in
Python 3.12. The `python3-distutils` package no longer exists in
Debian Trixie (Python 3.13) or Ubuntu 24.04+ (Python 3.12).
The only runtime reference was in `debian/ceph-mgr.requires`, already
cleaned up by 3fb3f892aa. This override is now dead code, hence no
installed file declares a runtime dependency on `distutils`, so
`dh_python3` never resolves it. Removing it prevents a latent
uninstallable-dependency bug if `distutils` were accidentally
reintroduced in a `.requires` file.
Fixes: https://tracker.ceph.com/issues/75901
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
as the modules are now being loaded onto the main interpreter (see
https://github.com/ceph/ceph/pull/66244), the
cherrypy is getting hit with an issue where its global state is being
affecting all the modules updating the cherrypy config simultaneously in
the same tree.
So i am adding a CherryPyMgr which manages all the independent servers
that will be created across all modules. This CherryPyMgr will create
its own server instances by utilizing cherrypy's WSGI Server and
eliminates the global state sharing. Each module or app can create their
own tree and start an adapter which will open an independent server for
that app.
- also added a method to update the config in place so CORS urls can be
configured without restarting servers.
Fixes: https://tracker.ceph.com/issues/74643, https://tracker.ceph.com/issues/74543, https://tracker.ceph.com/issues/74980
Signed-off-by: Nizamudeen A <nia@redhat.com>
The `BaseController` auto-imports the packaged `mgr/dashboard/controllers/smb.py`
file, which in turn wants to import `smb.enums` etc. which is part of the `smb`
package which is missing from `debian/ceph-mgr-modules-core.install`, thus
missing in the package. The missing module causes an exception
`ModuleNotFoundError: No module named 'smb'` on mgr instances when running a
ceph tentacle cluster installed from debian packages.
See: https://tracker.ceph.com/issues/74268
Signed-off-by: Roland Sommer <rol@ndsommer.de>
Replace 'setup.py install' with 'pip install --use-pep517' to fix
Cython compilation failures and eliminate deprecation warnings.
Problem Statement:
The build process for Cython modules involves preprocessing .pyx files
(e.g., generating rbd_processed.pyx from rbd.pyx) and then cythonizing
with specific compiler_directives. The previous approach using separate
'setup.py build' and 'setup.py install' commands caused this failure:
```
Error compiling Cython file:
------------------------------------------------------------
...
"""
name = cstr(name, 'name')
cdef:
rados_ioctx_t _ioctx = convert_ioctx(ioctx)
char *_name = name
librbd_progress_fn_t _prog_cb = &no_op_progress_callback
^
------------------------------------------------------------
rbd_processed.pyx:781:44: Cannot assign type 'int (*)(uint64_t, uint64_t, void *) except? -1' to 'librbd_progress_fn_t'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (uint64_t, uint64_t, void *) except? -1'.
```
This occurs because:
1. 'setup.py build build_ext' successfully preprocesses and cythonizes
with compiler_directives from setup.py's cythonize() call
2. 'setup.py install' internally triggers a rebuild that:
- Regenerates the preprocessed .pyx files
- Re-runs cythonize() through Cython.Distutils.build_ext
- Does NOT apply the compiler_directives from setup.py
- Fails on the regenerated files missing required directives
New Options Explained:
`--use-pep517`:
Addresses deprecation warning:
```
DEPRECATION: Building 'rados' using the legacy setup.py bdist_wheel
mechanism, which will be removed in a future version. pip 25.3 will
enforce this behaviour change.
```
Uses the modern PEP 517 build backend which:
- Performs a single build pass with all compiler_directives applied
- Prevents the implicit rebuild that caused CompileError
- Future-proofs against pip 25.3+ which will require this
`--no-build-isolation`:
Ensures that environment variables set by CMake are respected:
- CC, LDSHARED (compiler toolchain)
- CPPFLAGS, LDFLAGS (compilation flags)
- CYTHON_BUILD_DIR, CEPH_LIBDIR (build paths)
Without this flag, pip would create an isolated build environment
that ignores these critical build settings.
`--no-deps`:
Prevents pip from attempting to install Python dependencies listed
in setup.py's install_requires. All dependencies are managed by
CMake and the distribution's package manager, not pip.
`--ignore-installed`:
Addresses installation error when DESTDIR is set:
```
ERROR: Could not install packages due to an OSError: [Errno 13]
Permission denied: '/usr/lib/python3/dist-packages/rados-2.0.0.egg-info'
OSError: [Errno 18] Invalid cross-device link:
'/usr/lib/python3/dist-packages/rados-2.0.0.egg-info' -> '/tmp/pip-uninstall-...'
```
This error occurs because pip detects an existing system installation
and tries to uninstall it before installing to DESTDIR. With
--ignore-installed, pip skips the uninstall step and directly installs
to the DESTDIR staging directory, which is the correct behavior for
packaging.
Removed Options:
`--install-layout=deb`:
This Debian-specific patch to 'setup.py install' is no longer needed.
Modern pip automatically detects the distribution and uses the correct
layout (dist-packages on Debian, site-packages on RPM distros).
`--single-version-externally-managed`:
This option was specific to 'setup.py install' to prevent egg
installation. With pip, this is handled automatically.
`--record /dev/null`:
No longer needed as pip manages installation records internally.
`egg_info --egg-base`:
Not needed with pip as metadata is generated automatically during
the build process.
Not added option:
`--root-user-action=ignore`: not added
In this change, we installing a python module using pip with
`fakeroot` before packaging it. But pip warned:
```
Error: WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behavior with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
```
But we use fakeroot on purpose, this option could have been added to
silence this warning. But it is not available in all supported pip
versions. see
2e1112a814
New environmental variable:
`DEB_PYTHON_INSTALL_LAYOUT=deb` is conditionally applied when packaging
for debian-derivative distributions. As pip does not support
`--install-layout` option. Since debian patches pip so it installs Python
modules into /usr/local/lib instead of /usr/lib where debian dh_install
helper looks for the content to be packaged, so we have to enforce the
debian layout using the environmental variable.
Working Directory Change:
Changed from `CMAKE_CURRENT_SOURCE_DIR` to `CMAKE_CURRENT_BINARY_DIR` to
keep pip's temporary files and logs in the build directory rather than
polluting the source tree.
Additional Dependencies:
Since the build process uses pip and creates a wheel distribution,
we need to add `pip` and `wheel` Python modules as build dependencies.
Python moduels packaging:
- with `--use-pep517`, pip creates .dist-info directoires as per PEP-517
instead of .egg-info, so we need to package the new metadata directory.
Future Improvements
We considered implementing a custom `build_templates` command or using
setuptools' `sub_commands` mechanism to avoid regenerating `*_processed.pyx`
files on every build (tracking dependencies via file modification times or
hash-based checks). However, to keep `setup.py` simple and maintainable,
we've deferred this optimization for future work. The current solution
using `pip install --use-pep517` ensures correct builds without additional
complexity.
This solution works correctly for both Debian and RPM packaging workflows,
both of which use DESTDIR-based staged installations.
Fixes: 719b749846
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
Test scripts like qa/tasks/cephfs/mount.py expect the ip command to be
available in the container environment. Without it, tests fail with:
```
/bin/bash: line 1: ip: command not found
File "/ceph/qa/tasks/cephfs/mount.py", line 96, in cleanup_stale_netnses_and_bridge
p = remote.run(args=['ip', 'netns', 'list'],
...
teuthology.exceptions.CommandFailedError: Command failed with status 127: 'ip netns list'
```
Add iproute2 to the debian package build dependencies when the
<pkg.ceph.check> build profile is enabled. This ensures the package is
available during container-based builds, since buildcontainer-setup.sh
→ script/run-make.sh → install-deps.sh → debian/control → generated
dependency package chain respects build profiles configured via
`FOR_MAKE_CHECK` and `WITH_CRIMSON` environment variables set in
Dockerfile.build.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
With https://github.com/ceph/ceph/pull/65782 merged, upgrading ceph-osd
would need to replace the previous ceph-osd existing on the machine.
Otherwise, we won't be able to symlink the newly installed package:
```
2025-12-05T21:09:20.472 INFO:teuthology.orchestra.run.smithi077.stdout:
Installing : ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6
24/87
2025-12-05T21:09:20.478 INFO:teuthology.orchestra.run.smithi077.stdout:
Running scriptlet: ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6
24/87
2025-12-05T21:09:20.479
INFO:teuthology.orchestra.run.smithi077.stdout:failed to link
/usr/bin/ceph-osd -> /etc/alternatives/ceph-osd: /usr/bin/ceph-osd
exists and it is not a symlink
```
Note: debian/control ceph-osd-classic already had Replace and Breaks:
- Breaks is replaced with Conflicts to not allow coexistence.
- Release version is bumped up to be relevant for latest main
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
cephadm.postinst can fail if cephadm was originally installed using a
version that didn't configure a home directory for the user at all.
Newer versions do configure a home directory (as either `/home/cephadm`
or `/var/lib/cephadm`) so if that is configured then nothing needs to be
done. But if the user was created with no home directory then one needs
to be added for the configure step to succeed.
Fixes: https://tracker.ceph.com/issues/72083
commit 90bc036924 added home directories
for new cephadm users created, but didn't add home directories to
cephadm users that already existed.
Signed-off-by: Elliot Courant <me@elliotcourant.dev>
Commit 0985e201 added `from packaging import version` to
ceph_volume/util/encryption.py. On Debian and its derivatives, the
packaging module is provided by the python3-packaging package.
Add python3-packaging to ceph-volume's runtime dependencies to ensure
this import is satisfied.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
debian: include rgw-gap-list manpage and rgw-policy-check in ceph-common
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@ibm.com>
Previously, we called "invoke-rc.d ceph stop" in postrm scripts to
support sysvinit-based installations. However, this fails on systemd-
based systems, which are now the default on modern Debian distributions.
When invoke-rc.d detects systemd as the init system, it delegates to
systemctl, converting "invoke-rc.d ceph stop" to "systemctl stop
ceph.service". Since Ceph provides ceph.target and template units
(ceph-osd@.service, ceph-mon@.service, etc.) rather than a monolithic
ceph.service, this command always fails with exit code 5
(LSB EXIT_NOTINSTALLED).
This failure prevents the auto-generated cleanup sections added by
debhelper from executing properly, which can leave the system in an
inconsistent state during package removal.
Changes:
- Remove the invoke-rc.d call entirely. Systemd will handle service
cleanup through its own mechanisms when the package is removed.
- Remove redundant "exit 0" statement (the script exits successfully
by default, and "set -e" is no longer needed without commands that
might fail).
- Remove vim modeline comment, as it's unnecessary for a 3-line script.
- Eventually remove this 3-line script stanza, as this is exactly what
debhelper provides us.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
a37b5b5bde added ceph-osd-crimson as dependency using OR requirement.
Once ceph-osd-classic dependency is satisfied, ceph-osd-crimson package must be added manually.
In order to not add crimson packages manually (See "Containerfile"
removed lines) - mark osd-crimson as dependency if with_crimson is
enabled for rpm and if pkg.ceph.crimson build profile is enabled for debian.
With this change, users won't be required to obtain ceph-osd-crimson
package manually when relevant. Similarly to ceph-osd-classic being
a dependency for ceph-osd.
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Since commit 9b1d524839 ("debian: mark "crimson" specific deps with
"pkg.ceph.crimson""), crimson-specific build dependencies have been
gated by the Build-Profiles: <pkg.ceph.crimson> tag. However,
debian/rules was never updated to pass -DWITH_CRIMSON=ON when this
build profile is active.
This causes builds with the crimson profile enabled to fail during
dh_install, as the crimson-osd binary is never built but the install
file tries to package it:
Failed to copy 'usr/bin/crimson-osd': No such file or directory
dh_install: error: debian/ceph-crimson-osd.install returned exit code 127
Fix this by checking for pkg.ceph.crimson in DEB_BUILD_PROFILES and
enabling the CMake option accordingly, following the same pattern used
for pkg.ceph.arrow.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
Previously, ceph-osd packaging had two mutually exclusive flavors that
could only be built one at a time: one with classic OSD and another
with crimson OSD. Both provided /usr/bin/ceph-osd, making them
impossible to coexist and confusing from a user perspective.
This commit restructures the packaging to enable both implementations
to coexist on the same system:
- ceph-osd: Contains shared components (systemd units, sysctl configs,
common executables like ceph-erasure-code-tool) and depends on exactly
one OSD implementation
- ceph-osd-classic: Contains the classic OSD implementation binary and
classic-specific tools
- ceph-osd-crimson: Contains the crimson OSD implementation binary and
crimson-specific tools
The two implementation packages install different sets of file, so they
don't conflict with each other anymore, and both depend on ceph-osd for
shared resources.
Changes:
Debian packaging:
- Revert e5f00d2f
- Add ceph-osd-crimson package
- Add Recommends: ceph-osd-classic to prefer classic on upgrades
- Add Replaces/Breaks for smooth upgrades from old monolithic package
- Create separate .install files for crimson and classic osd packages
Enforce exact version matching using ${binary:Version}
RPM packaging:
- Use rich dependencies for OR requirement (classic or crimson)
- Add Recommends: ceph-osd-classic for upgrade preference
Upgrade behavior:
Users upgrading from older versions will automatically get
ceph-osd-classic due to the Recommends directive, maintaining
backward compatibility. Users can explicitly choose crimson by
installing ceph-osd-crimson, which will coexist with classic.
Switching between implementations is supported via standard package
operations, with the alternatives system ensuring /usr/bin/ceph-osd
always points to the active implementation.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
Configure the Debian build to use CEPHADM_BUNDLED_DEPENDENCIES=deb,
which instructs the cephadm build script to bundle dependencies from
system-installed Debian packages instead of downloading from PyPI.
This change addresses build failures in restricted network environments
where Debian build tools do not permit internet access. By leveraging
the Debian package support added in commit 9378a2988e1, the build now
uses python3-markupsafe, python3-jinja2, and python3-yaml packages
that are already installed as build dependencies.
This approach mirrors the existing RPM packaging workflow, ensuring
consistent behavior across different distribution package formats.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
In commit e8460cbd, we introduced the "pkg.ceph.arrow" build profile to
support building with system Arrow packages. However, neither Debian nor
Ubuntu currently ships Arrow packages.
Since WITH_RADOSGW_SELECT_PARQUET is always enabled in debian/rules,
Arrow support is required for all builds. When the pkg.ceph.arrow profile
is not selected, the build uses vendored Arrow. With the recent change to
use AUTO mode for xsimd detection, Arrow will attempt to find system xsimd
>= 9.0.1. Adding libxsimd-dev as a build dependency ensures it's available
for Arrow to detect and use, reducing build time on supported distributions.
On distributions with insufficient xsimd versions (< 9.0.1), Arrow will
automatically fall back to its bundled version.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
The rgw-policy-check tool was added to RPM packaging in commit 28864311
but was not included in Debian packaging, creating an inconsistency
between the two package formats.
Include rgw-policy-check in the ceph-common Debian package to match the
RPM packaging and ensure feature parity across distribution formats.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
The rgw-gap-list manpage was added to RPM packaging in commit 8d0ec766
but was omitted from the Debian package. Add it to ceph-common to align
with the RPM packaging, where both the executable and manpage are
included together.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
The arm64-only module uadk needs numa.h to build; nothing else
ensures it's available. Make it an unconditional ceph build
dependency on behalf of the arm64 build.
Fixes: https://tracker.ceph.com/issues/72594
Signed-off-by: Dan Mick <dan.mick@redhat.com>
cephadm.postinst fails on ubuntu 24.04, because the cephadm user it
creates ends up with a `/nonexistent` home directory:
> mkdir: cannot create directory '/nonexistent/.ssh': No such file or directory
on ubuntu 22.04, this user's home directory is `/home/cephadm`
the `adduser` man page for 24.04 includes this new text:
> If no home directory is specified, the default home directory for a new system user is /nonexistent. This directory should never exist on any Debian system, and adduser will never create it automatically.
commit 90bc036924 had previously added
`--home /home/cephadm`, but that was later removed by commit
c5a316cfd5 to support non-default home
directories. the latter commit's author recommends use of
`--home /var/lib/cephadm` instead
Fixes: https://tracker.ceph.com/issues/72083
Signed-off-by: Casey Bodley <cbodley@redhat.com>
deb/mgr: remove deprecated distutils from ceph-mgr.requires
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Ubuntu changed the systemd unit directory location between releases:
- Jammy (22.04): /lib/systemd/system
- Noble (24.04): /usr/lib/systemd/system
To maintain compatibility across both versions, update .install files
to use brace expansion pattern {usr/,}lib/systemd/system/<service>.
This pattern works because dh_install uses bsd_glob() with GLOB_CSH
flags, which expands braces and matches files in both locations
depending on where CMakeLists.txt actually installed them.
Fixes installation issues when building packages on Noble while
maintaining backward compatibility with Jammy builds.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Back in 2020 the '/etc/mime.types' file was split out into it's own
leaner 'media-types' Debian package [0], the previous 'mime-support'
was kept as transitional package to handle upgrades, but it did its
job and isn't included in Debian Trixie repos anymore.
So accept both, the new 'media-types' package or the legacy
'mime-support' one as valid dependency for radosgw, this ensures one
can install radosgw on newer Debian based releases while keeping it
also working for older ones.
[0]: 4d6ff19de5
Fixes: https://tracker.ceph.com/issues/71545
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
This patch removes the assumption that user home directories always live under /home. If the user has set home directories to live under a different directory by changing /etc/adduser.conf, this will honor that setting.
Signed-off-by: Shawn Edwards <shedwards@nvidia.com>
Fixes https://tracker.ceph.com/issues/70366
- adds support for ISV
- using jmespath expression for fetching roles from payload
- added roles_path parameter in sso enable command as optional argument
- modified roles mapper for sso login
Signed-off-by: Afreen Misbah <afreen@ibm.com>
* refs/pull/58376/head:
Temporarily change the libcephfs dependencies
proxy: Add the design document
proxy: Add the proxy to the deb builds
proxy: Add the proxy to the rpm builds
Initial version of the libcephfs proxy
Reviewed-by: Sachin Prabhu <sp@spui.uk>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
