| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |\
| |
| |
| |
| | |
cephadm: add runtime python minor version to RPM query macro
Reviewed-by: John Mulligan <jmulligan@redhat.com>
|
| | |
| |
| |
| |
| |
| |
| | |
when multiple versions of python are installed, the `python3dist` macro
produces a multiline string causing later rpm query commands to fail.
Signed-off-by: Michael Fritch <mfritch@suse.com>
|
| | |
| |
| |
| | |
Signed-off-by: John Mulligan <jmulligan@redhat.com>
|
| | |
| |
| |
| | |
Signed-off-by: John Mulligan <jmulligan@redhat.com>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Add a pyyaml zipapp dependency based on installing from wheels but
stripping those wheels of compiled (shared object) extensions. This is
the only way I could get it to work because the pyproject.toml file in
pyyaml requires Cython and having Cython present enables the extension
in setup.py even though some setup.py options exist to disable building
the C/Cython based extension.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Previously, the code assumed that all dependencies can be cleanly
installed from source with native-code compilers disabled. This worked
ok for MarkupSafe because it falls back to pure-python if it can't
compile the C extension code. Unfortunately, not all packages that can
fall back to pure python do so cleanly in the build. As a first step
to resolving that, make the current behavior an explicitly selected mode
and permit installing from wheels in the future.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
|
| |/
|
|
|
|
|
|
| |
Add a new InstallSpec type to build.py and rework the requirements lists
in terms of the InstallSpec. The new type will be able to encode more
per-dependency behavior than the previous approach.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
|
| |\
| |
| |
| |
| | |
cephadm: expand support for dependencies bundled into the zipapp
Reviewed-by: Adam King <adking@redhat.com>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Add additional modes for how to handle bundled dependencies.
Add a mode to disable bundled dependencies entirely.
Add a mode to source dependencies from installed RPM packages. This mode
will likely be the basis for some distributions of ceph were the exact
version of the package can be used via a controlled build environment.
Add the capability to store metadata about the bundled dependencies
in the zipapp. Move all the metadata created by build.py to a synthetic
package `_cephadmmeta` that gets created when building the zipapp.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
|
| |/
|
|
|
|
|
|
|
|
| |
Work around an encoding/locale issue when the dashboard tests are run
(ubuntu 20.04).
The build.py changes brought in a9d1c62ca86 were validated for package
builds, teuthology, and other CI jobs but a different error was masking
this failure in the dashboard ci job.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Unfortunately, a single simple call to pip does not work on all the
distributions that ceph is built on. In particular, Ubuntu 20.04 and
Ubuntu 22.04 come with pip versions that can not correctly handle
disabling wheels and installing Jinja2 (it tries to use the markupsafe
dependency before it is installed). This can be worked around by using a
virtual env and updating pip before proceeding. However, this is not
enough because CentOS/RHEL 8 uses python 3.6 and there is no version of
pip that supports 3.6 that we can update to that is new enough to fix
the issue with disabling wheels. The workaround in this case is to
install each dependency one at a time through multiple calls to pip.
Because of this extra complexity is it simpler to eschew the use of a
requirements.txt file in build.py entirely. Thus the zipapp is built
using build.py only. Requirements files for cephadm are for setting up
the tox environments *only*.
For completeness a new option is added that gives the caller control
over when build.py uses a virtualenv or not. Thus the build.py script
requires at least one of: a working pip that handles disabling wheels;
or, a virtualenv (venv) and the ability to update to a working version
of pip. If the list of distros ceph supports (and the python versions
they use) ever becomes simpler/newer some of this complexity could be
removed from the build.py script.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
|
| |
|
|
|
|
|
|
|
|
| |
We can not rely on any particular python version (py 3.6+ is supported)
and can not assume any particular architecture. So using wheels
based on the build system is pointless. Installing binary .so files
compiled from C/C++ similarly so. Attempt to block the behaviors
when adding dependencies to the zipapp.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
|
| |
|
|
|
|
| |
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Pair-programmed-with: Adam King <adking@redhat.com>
Co-authored-by: Adam King <adking@redhat.com>
|
| |\
| |
| |
| |
| | |
Support cephadm "binary" versioning
Reviewed-by: Adam King <adking@redhat.com>
|
| | |
| |
| |
| |
| |
| |
| |
| | |
Pass --set-version-var=KEY=VALUE cli options to the script and it
will create an embedded "_version.py" file in the zipapp containing
the passed parameters.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Python provides the compileall module to explicitly create pyc files
from py files. If we byte-compile the content of the cephadm zipapp
we get a small speed boost when running the application.
In a not-very-scientific benchmark I found that running with pyc files
almost halved the time to run the help command 50 times.
```
$ time for _ in {0..50}; do /tmp/cephadm-nobc -h >/dev/null; done
real 0m9.486s
user 0m8.547s
sys 0m0.893s
$ time for _ in {0..50}; do /tmp/cephadm-bc -h >/dev/null; done
real 0m4.634s
user 0m3.992s
sys 0m0.618s
```
I ran the above a few times on my laptop and the numbers are pretty
consistent.
One thing to note is that zipapp doesn't seem to understand the current
`__pycache__` approach to storing the bytecode files so we have to set
the `legacy` argument for compileall.compile_dir to true. Since
__pycache__ dirs mostly exist to allow multiple bytecode files for
different python versions to coexist, and a zipapp is read-only this
should not be a major issue. Tangentially related, we lose out on the
speedup if you run the zipapp with a version of python other than the
one the zipapp was built with but it continues to function.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
|
|
|
As discussed in a ceph orch meeting, the build.sh script was deemed
"unpythonic". This script is a first attempt to do it more pythonic
with fewer explicit version checks.
This change minimizes the existing build.sh to simply call build.py.
We can completely eliminate build.sh at a later time if desired.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
|
