From 06bed38931846749dba71ecbada5e3fe64b0dae2 Mon Sep 17 00:00:00 2001 From: John Mulligan Date: Thu, 11 May 2023 14:20:41 -0400 Subject: cephadm: call compile_dir to byte compile zipapp py files Python provides the compileall module to explicitly create pyc files from py files. If we byte-compile the content of the cephadm zipapp we get a small speed boost when running the application. In a not-very-scientific benchmark I found that running with pyc files almost halved the time to run the help command 50 times. ``` $ time for _ in {0..50}; do /tmp/cephadm-nobc -h >/dev/null; done real 0m9.486s user 0m8.547s sys 0m0.893s $ time for _ in {0..50}; do /tmp/cephadm-bc -h >/dev/null; done real 0m4.634s user 0m3.992s sys 0m0.618s ``` I ran the above a few times on my laptop and the numbers are pretty consistent. One thing to note is that zipapp doesn't seem to understand the current `__pycache__` approach to storing the bytecode files so we have to set the `legacy` argument for compileall.compile_dir to true. Since __pycache__ dirs mostly exist to allow multiple bytecode files for different python versions to coexist, and a zipapp is read-only this should not be a major issue. Tangentially related, we lose out on the speedup if you run the zipapp with a version of python other than the one the zipapp was built with but it continues to function. Signed-off-by: John Mulligan --- src/cephadm/build.py | 9 +++++++++ 1 file changed, 9 insertions(+) (limited to 'src/cephadm/build.py') diff --git a/src/cephadm/build.py b/src/cephadm/build.py index 4e97f5d3757..39c93ce3b30 100755 --- a/src/cephadm/build.py +++ b/src/cephadm/build.py @@ -6,6 +6,7 @@ # of python to build with? Even with the intermediate cmake layer? import argparse +import compileall import logging import os import pathlib @@ -67,6 +68,14 @@ def _build(dest, src): def _compile(dest, tempdir): """Compile the zipapp.""" + log.info("Byte-compiling py to pyc") + compileall.compile_dir( + tempdir, + maxlevels=16, + legacy=True, + quiet=1, + workers=0, + ) # TODO we could explicitly pass a python version here log.info("Constructing the zipapp file") try: -- cgit v1.2.3