summaryrefslogtreecommitdiffstats
path: root/ceph-erasure-code-corpus (unfollow)
Commit message (Collapse)AuthorFilesLines
2024-01-08osd/OSDMap: minor formatting nitSamuel Just1-2/+1
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-01-08crush: add msr rule types and CrushWrapper infrastructureSamuel Just6-4/+49
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-01-08src/crush: add a rule type enumSamuel Just2-5/+10
We're going to add more rule types that don't map directly onto pool types. Signed-off-by: Samuel Just <sjust@redhat.com>
2024-01-08OSDMonitor: update require_osd_release for crush featuresSamuel Just1-0/+6
The existing implementation checks that the currently live OSDs support the pending crush features, but we don't actually check that require_osd_release is new enough. Signed-off-by: Samuel Just <sjust@redhat.com>
2024-01-08make-dist: don't use --continue option for wgetCasey Bodley1-1/+1
the boost jfrog mirror is broken and returns an HTML error page instead of the archive. the file size of this page is 11534 bytes when download_from() retries the download from download.ceph.com, the -c option tells it to resume the download of the existing file. the resulting boost_1_82_0.tar.bz2 ends up with the correct total file size of 121325129 bytes, but the first 11534 bytes still correspond to the HTML from jfrog. that causes the sha256sum mismatch remove the -c option so that wget fetches the archive in its entirety Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-01-08doc/radosgw: edit admin.rst - quotaZac Dover1-65/+104
Edit the "Quota Management" section (all of it) in doc/radosgw/admin.rst. Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-01-08win32_deps_build.sh: change Boost URLLucian Petrut1-1/+1
The Boost download URL is broken again [1], we'll switch to a working mirror. [1] https://github.com/boostorg/boost/issues/842 Fixes: https://tracker.ceph.com/issues/63952 Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2024-01-05doc: update rgw admin api req params for get user infoAli Maredia1-0/+10
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2024-01-05osd/scrub: avoid "over clearing" queued_or_active flagRonen Friedman5-11/+23
If two StartScrub messages are received in quick succession, the earlier one might clear the queued_or_active flag as it fails for being from an old interval. When that happens - a 3'rd scrub request will actually be allowed to go through, while the scrubber is still handling the second one. Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-05osd/scrub: manage queue registration lifetime in the FSMRonen Friedman7-32/+23
As the state of 'being registered in the OSDs scrub queue' corresponds to the PrimaryActive FSM state. Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-05doc/radosrgw: edit admin.rstZac Dover1-14/+25
Edit "Add / Remove Admin Capabilities" in doc/radosgw/admin.rst. Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-01-05crimson/os/seastore/cache: return false explicitly if delta is not replayedMyoungwon Oh1-1/+1
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2024-01-05crimson/os/seastore: check correct crc for inplace-rewritten extents after ↵Myoungwon Oh8-26/+56
replay is done Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2024-01-05test/crimson/seastore: fix overflow issue and make len > 0 when set_contents ↵Myoungwon Oh1-9/+11
is called set_contents causes the overflow at times because alloc_extent is allowed to use uint32_t. Specifically, in random_writes case, PADDING_SIZE is 256<<10, whereas set_contents's len is uint16_t. Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2024-01-05crimson/os/seastore/epm: detect transaction confict during rbm write using OOLMyoungwon Oh1-7/+9
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2024-01-05crimson/os/seastore: introduce modified_region in DATA_BLOCK to keep track ↵Myoungwon Oh7-39/+40
of modified region It has a limitation to keep track of the modified region using the existing deltas because we can not get the correct region in two cases: 1) a case where replay is done and 2) duplicate_for_write. This commit introduces modified region to solve the problem. Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2024-01-05crimson/os/seastore: write only modified region when rewriting the extent in ↵Myoungwon Oh4-5/+55
an inplace manner Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2024-01-05test/crimson/seastore: add a random overwrite test to test_objec_date_handlermyoungwon oh2-3/+46
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2024-01-05crimson/os/seastore/cache: reset version to 1 if inplace write occurs before ↵Myoungwon Oh1-0/+14
in prepare_record Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2024-01-04rgwlc: even current object versions have a unique instanceMatt Benjamin1-3/+3
Fixes: https://tracker.ceph.com/issues/63546 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2024-01-04rgwlc: additional lc notify cleanupsMatt Benjamin1-16/+26
Follow the same formula to build up obj_state and version_id in all call sites. Resolves: rhbz#2163667 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2024-01-04crimson/osd: Keep track of modified_rangesMatan Breizman4-28/+108
* `modifies_ranges` interval_set is added to osd_op_params_t * keep track of modified_ranges while executing relevant ops * Add `osd_op_params` parameter to `PGBackend::remove()`. Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04crimson/osd/replicated_recovery_backend: handle_pull_response() - ↵Matan Breizman2-2/+28
recalc_subsets() Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04mson/osd/replicated_recovery_backend: Introduce prep_push_to_repli…Matan Breizman2-6/+56
…ca() Intelligently push an object to a replica. make use of existing clones/heads and dup data ranges where possible. Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04crimson/osd/replicated_recovery_backend: seperate prep_push_to_replica and ↵Matan Breizman2-11/+28
prep_push Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04rename prep_push to prep_push_to_replicaMatan Breizman2-7/+3
dismiss head_ctx Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04crimson/osd/replicated_recovery_backend: prepare_pull to support ↵Matan Breizman2-12/+43
calc_clone_subsets Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04crimson/osd/object_metadata_helper: Introduce calc_*_subsetsMatan Breizman6-0/+510
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-03doc/rados/operations: document `ceph balancer status detail`Laura Flores2-0/+10
Document change in https://github.com/ceph/ceph/pull/54801 Signed-off-by: Laura Flores <lflores@ibm.com>
2024-01-03cepahdm: add some specific tests for daemon id systemd name generationJohn Mulligan1-0/+34
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-03doc/radosgw: edit "Add/Remove a Key"Zac Dover1-21/+49
Edit the section "Add/Remove a Key" in doc/radosgw/admin.rst. Each operation (e.g. "Adding an S3 key pair for a user", "Removing an S3 key pair for a user") now has its own subsection. This increased granularity should make it easier in the future to link to each of these specific operations, if needed. Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-01-03doc/releases: edit reef.rstZac Dover1-7/+6
Make minor corrections to doc/releases/reef.rst. These corrections were suggested by Anthony D'Atri in https://github.com/ceph/ceph/pull/55049. Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-01-03rgw/notifications/lc: handle publish_commit() failures as warningsYuval Lifshitz1-12/+11
Fixes: https://tracker.ceph.com/issues/63859 Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2024-01-02PendingReleaseNotes: add release note for 62338Samuel Just1-0/+8
See https://tracker.ceph.com/issues/62338 and 2fc5486e. Signed-off-by: Samuel Just <sjust@redhat.com>
2024-01-02cehphadm: disable restarting init containers serviceJohn Mulligan1-2/+6
Disable the Restart= line in the init containers systemd template. It is commented out as there are probably more appropriate workarounds to use in the future but this should fix testing on centos8 based systems. Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02cephadm: add stop and poststop commands to init container serviceJohn Mulligan1-0/+2
Now that there is a more robust init_containers.run script, we can use the new stop and poststop subcommands from the systemd unit. Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02cephadm: convert init containers script to use a templateJohn Mulligan5-53/+84
Convert the init containers run script to be based on a template, like the sidecar run scripts are. The new script is loosely based on the sidecars run script but only does actions in batches - logically iterating over each init container configured. Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02cephadm: move init container script generation to a functionJohn Mulligan1-4/+13
Move the generation of the init container run script to a small function fixing a missing `set -e` along the way. This isolates the logic of generating this run script. Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02cephadm: call functions instead of executing rm in rm_clusterJohn Mulligan1-6/+11
Convert a bunch of invocations of rm via a subprocess to function calls. This should make it easier (or possible?) to test the function in the unit test framework as well as possibly saving a few resources. Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02cephadm: remove a non-functional line in _rm_clusterJohn Mulligan1-2/+0
Remove a line from _rm_cluster that has no effect. The line uses call_throws to execute an `rm -rf` command. The argument contains asterisk chars that indicate that the file(s)/dir(s) to be created are expected to match the given pattern. However, globs do not work in this context in contemporary versions of cephadm. To double check I added the following temporary unit test: ``` def test_does_it_glob(tmp_path): from cephadmlib.call_wrappers import call_throws d1 = (tmp_path / 'dir1') d1.mkdir() fns = ['f1.txt', 'f2.txt', 'f3.txt', 'f4'] for fn in fns: with open(d1/fn, "w") as fh: fh.write("xxx") assert d1.exists() for fn in fns: assert (d1 / fn).exists() ctx = FakeContext() call_throws(ctx, ['rm', '-rf', f'{d1}/*.txt']) print('files:', os.listdir(d1)) assert d1.exists() for fn in fns: if fn.endswith('.txt'): assert not (d1 / fn).exists() else: assert (d1 / fn).exists() ``` If globs worked in this context this test would have passed. It does not. I confirmed that the current implementation of call/call_wrapper does not execute the command in a shell context. I wondered if it was possible that an earlier version of cephadm did execute this command in a shell context and some changes along the way changed the behavior. I tracked the origin of the line back to 16ebc620349f6e7c9afa6b992c85900f56fcfca3 the first change to implement rm-cluster. In this commit the code uses subprocess.check_output directly. I am familiar with check_output and unless `shell=True` is passed this function doesn't execute the args in a shell context. The `shell=True` argument is not passed to check_output. This means that the very first implementation of this line suffered from the same issue - it would have no effect on any files except one named with actual asterisk (`*`) characters. While I'm sure there was a good intention behind this line, the fact that it persisted in the code so long in a non-functional state and no one noticed in both production and qa testing makes me feel that it can be safely removed with no negative effect. Removing the line simplifies the code and avoids needing to spend effort unit-testing or manually checking a fix. Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02cephadm: add a unit test to check sidecar removalJohn Mulligan1-0/+44
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02cephadm: update rm cluster to remove sidecar/init-ctr servicesJohn Mulligan1-6/+20
A previous patch updated the rm-daemon functionality to remove sidecar and init-container systemd services. This patch does the same for the rm-cluster codepath. Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02cephadm: update _rm_cluster to call terminate_serviceJohn Mulligan2-18/+23
The terminate_service function was added to encapsulate the act of terminating a systemd service. The rm-deamon handler was updated to use this function previously; this commit updates rm-cluster function(s) to do the same. This needed a bunch of test updates do to how the tests were mocking commands. Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02cephadm: add support to remove sidecar/init-ctr systemd unitsJohn Mulligan1-7/+18
Add support to the command_rm_daemon function to remove systemd services and corresponding unit files for any sidecars or init-containers associated with a primary daemon. Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02cephadm: replace some calls to rm with fs function callsJohn Mulligan1-1/+1
This should be a little cleaner, perhaps a tiny performance improvement, and most importantly work better with the virtual-fs provided for the unit tests and make this function more testable in the future. Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02cephadm: minor reorgs to command_rm_daemonJohn Mulligan1-9/+18
Make a few small chanes to command_rm_daemon to clean it up in preparation for future changes. Move the construction of a DaemonIdentity earlier and use the service name derived directly from it if possible. I didn't remove the old approach of looking up a service name in the listing because I don't know if doing so would break old/adopted services and there's no unit test coverage for that. Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02cephadm: add a terminate_service systemd functionJohn Mulligan1-0/+18
Add a function that encapsulates the common 3-step process found in cephadm of stopping, resetting errors, and disabling a service. This will be used to handle new services to stop as well as cleaning up some places where the three steps are performed inline. Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02cephadm: add sidecars_from_dropin functionJohn Mulligan1-0/+27
Add a sidecars_from_dropin function that takes a minimal PathInfo object and returns a PathInfo populated with sidecars information based on reading the content of the dropin file. This will be useful when we need information on configured sidecars but are not on the deploy path. Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02cephadm: add method for parsing a sub identity from a service nameJohn Mulligan2-1/+107
This will become handy later when we need DaemonSubIdentity objects and are not on the deploy path. Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02cephadm: add unlink_file functionJohn Mulligan1-0/+18
Add the unlink_file function to file_utils. This fills in a gap between python 3.6 and features provided in pathlib.Path in later versions of python. Adds an option to ignore all errors for good measure. Signed-off-by: John Mulligan <jmulligan@redhat.com>