summaryrefslogtreecommitdiffstats
path: root/src/librados (follow)
Commit message (Collapse)AuthorAgeFilesLines
* librados/asio: document requirement on IoCtx's lifetimeCasey Bodley2024-11-191-0/+15
| | | | Signed-off-by: Casey Bodley <cbodley@redhat.com>
* librados/asio: forward asio cancellations to AioCompletion::cancel()Casey Bodley2024-11-191-6/+51
| | | | | | | | | | | | if the completion handler has a cancellation slot connected, construct a cancellation handler for it that calls AioCompletion::cancel() read operations can support more cancellation types than writes. see the table of cancellation types and their associated requirements/guarantees: https://www.boost.org/doc/libs/1_85_0/doc/html/boost_asio/overview/core/cancellation.html#boost_asio.overview.core.cancellation.t0 Signed-off-by: Casey Bodley <cbodley@redhat.com>
* librados: expose op cancellation through AioCompletion::cancel()Casey Bodley2024-11-191-0/+8
| | | | Signed-off-by: Casey Bodley <cbodley@redhat.com>
* librados: dont duplicate class names in member definitionsCasey Bodley2024-10-281-22/+21
| | | | Signed-off-by: Casey Bodley <cbodley@redhat.com>
* librados/asio: add version_t to completion signaturesCasey Bodley2024-09-261-20/+24
| | | | | | | | IoCtx::aio_operate() doesn't update IoCtx::get_last_version(). to make the resulting version_t available to the caller, we have to read it out of the AioCompletionImpl and return it to the caller Signed-off-by: Casey Bodley <cbodley@redhat.com>
* Merge pull request #55592 from cbodley/wip-boost-asio-spawnCasey Bodley2024-05-211-64/+75
|\ | | | | | | | | | | rgw: switch back to boost::asio for spawn() and yield_context Reviewed-by: Adam Emerson <aemerson@redhat.com> Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
| * librados/asio: functions use async_initiateCasey Bodley2024-05-131-64/+75
| | | | | | | | Signed-off-by: Casey Bodley <cbodley@redhat.com>
* | cmake: targets against legacy-option-headers when appropriateKefu Chai2024-05-081-1/+2
|/ | | | | | | | | | | | | | | | | | | | | | | | legacy-option-headers provides the headers included by `src/common/options/legacy_config_opts.h`. these headers are generated from corresponding .yaml.in file. because these headers file are generated at build time, we need to construct the file-level dependencies between the targets using them and these headers, otherwise the compiler could fail to compile the tree if any of the headers are not generated yet, when compiling .cc file which (indirectly) includes it. in order to address this, in this change, we 1. search for all .cc files which have `#include "common/config.h"` in it 2. and find out the targets building the .cc file, then 3. add `legacy-option-headers` to its linkage using CMake this should partially address the above race condition we've been running into on slow build hosts. because we have not audited the .h files including `common/config.h`, this change should be considered a partial fix. Signed-off-by: Kefu Chai <tchaikov@gmail.com>
* Merge pull request #55348 from chenyuanrun/fix-io-ctx-impl-flagsYuri Weinstein2024-03-211-1/+1
|\ | | | | | | | | | | librados: use CEPH_OSD_FLAG_FULL_FORCE for IoCtxImpl::remove Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com> Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
| * librados: use CEPH_OSD_FLAG_FULL_FORCE for IoCtxImpl::removeChen Yuanrun2024-03-141-1/+1
| | | | | | | | | | | | | | | | librados::OPERATION_FULL_FORCE should be translated to CEPH_OSD_FLAG_FULL_FORCE before calling IoCtxImpl::remove(). Fixes: https://tracker.ceph.com/issues/64558 Signed-off-by: Chen Yuanrun <chen-yuanrun@foxmail.com>
* | Merge pull request #56064 from idryomov/wip-63770Ilya Dryomov2024-03-141-15/+25
|\ \ | | | | | | | | | | | | | | | librbd: account for discards that truncate in ObjectListSnapsRequest Reviewed-by: Mykola Golub <mgolub@suse.com> Reviewed-by: Ramana Raja <rraja@redhat.com>
| * | librados/snap_set_diff: ignore truncates above size at startIlya Dryomov2024-03-081-15/+25
| |/ | | | | | | | | | | | | | | | | | | | | Because currently calc_snap_set_diff() only ever appends to the running diff, an excessive (either too large or completely bogus) zero extent is reported in cases where an object is first expanded (with a snapshot taken at that point) and then truncated but still above the size of the object as of the starting snapshot. Fixes: https://tracker.ceph.com/issues/63770 Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
* / tracer/osd/librados/build/rgw: rgw and osd end2end tracing using opentelemetryOmri Zeneva2024-02-064-10/+30
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | * build: add opentelemetry to cmake system crimson targets that uses Message.cc/h are built before opentelemetry (o-tel), so we need to build o-tel eralier so we also add the library to the include path earlier this shoud work for WITH_JAEGER flag both the ON/OFF cases, and for librados where the compilation flag is ignored * msg/tracer: add o-tel trace to Messages with decode/encode function in tracer.h some files that uses Message.cc/h just need the encode/decode functions and not all others functions. some crimson targets does not link with ceph_context (common) which is required for tracer.cc file. so we just need to include that functions * librados: Add opentelemtry trace param for aio_operate and operate methods in order to propagate the trace info I added the otel-trace as an extra param. in some places, there already was a blkin trace info, and since it is not used in other places we can safely change it to o-tel trace info. this will be done in another commit, so the cleanup of blkin trace will be in a dedicated commit * osd: use the o-tel trace of the msg as a parent span of the osd trace if there is a valid span in the msg, we will add this op to the request trace, otherwise it will start a new trace for the OSD op * rgw: pass put obj trace info to librados in order to make it possible, I saved the trace info inside the sal::Object, so we can use it later when writing the object to rados it could be used also later for read ops. note the trace field of req_state is initalized only in rgw_process, so it's also required in librgw request flow * prevent breaking channges to kSize. make sure that changes between components built with different versions of OTEL do not break message compatibility Signed-off-by: Omri Zeneva <ozeneva@redhat.com>
* Merge pull request #54644 from idryomov/wip-63607Ilya Dryomov2023-12-283-8/+21
|\ | | | | | | | | | | librados: make querying pools for selfmanaged snaps reliable Reviewed-by: Ramana Raja <rraja@redhat.com> Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
| * librados: make querying pools for selfmanaged snaps reliableIlya Dryomov2023-12-043-8/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If get_pool_is_selfmanaged_snaps_mode() is invoked on a fresh RADOS client instance that still lacks an osdmap, it returns false, same as for "this pool is not in selfmanaged snaps mode". The same happens if the pool in question doesn't exist since the signature doesn't allow to return an error. The motivation for this API was to prevent users from running "rados cppool" on a pool with unmanaged snapshots and deleting the original thinking that they have a full copy. Unfortunately, it's exactly "rados cppool" that fell into this trap, so no warning is printed and --yes-i-really-mean-it flag isn't enforced. Fixes: https://tracker.ceph.com/issues/63607 Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
* | Merge pull request #52495 from adamemerson/wip-neorados-learning-from-experienceCasey Bodley2023-12-182-5/+12
|\ \ | | | | | | | | | | | | | | | neorados: Learning from experience Reviewed-by: Ilya Dryomov <idryomov@gmail.com> Reviewed-by: Casey Bodley <cbodley@redhat.com>
| * | neorados: Use `asio::any_completion_handler`Adam C. Emerson2023-12-061-4/+10
| | | | | | | | | | | | | | | | | | | | | | | | As we'd like to reduce (and eliminate) internal Ceph dependencies to the extent possible, now that Boost.Asio has a type-erased handler type, let's use it. Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
| * | osdc: replace asio::io_context::strand with asio::strand<>Casey Bodley2023-12-061-1/+2
| | | | | | | | | | | | Signed-off-by: Casey Bodley <cbodley@redhat.com>
* | | librbd: diff-iterate shouldn't ever report "new hole" against a holeIlya Dryomov2023-12-111-10/+15
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If an object doesn't exist in both start and end versions but there is an intermediate snapshot which contains it (i.e. the object is written to and captured at some point but then discarded prior to or in the end version), diff-iterate reports "new hole" -- callback is invoked with exists=false. This occurs both on the slow list_snaps path and in fast-diff mode. Despite going all the way back to the introduction of diff-iterate in commit 0296c7cdae91 ("librbd: implement diff_iterate"), this behavior is wrong and contradicts diff-iterate API documentation added in commit a69532e86450 ("librbd: document diff_iterate in header") in the same series: If the source snapshot name is NULL, we interpret that as the beginning of time and return all allocated regions of the image. It also triggered an assert added in commit c680531e070a ("librbd: change diff_iterate interface to be more C-friendly") in the same series. Unfortunately, commit f1f6407221a0 ("test_librbd: add diff_iterate test including discard"), also part of the same series, added a test which expected the wrong behavior. Very confusing! A year later, a different manifestation of this bug was fixed in commit 9a1ab95176fe ("rbd: Fix rbd diff for non-existent objects"), but the fix only covered the case where calc_snap_set_diff() goes past the end snap ID while processing clones. The case where it runs out of clones to process before reaching the end snap ID remained mishandled. A year after that, commit 3ccc3bb4bd35 ("librbd: diff_iterate needs to handle holes in parent images") dropped the assert mentioned above and this bug got enshrined in the newly introduced fast-diff mode. Finally, a few years later, deep-copy actually started relying on this bug in commit e5a21e904142 ("librbd: deep-copy image copy state machine skips clean objects"). This necessitates bifurcation in DiffRequest because deep-copy wants the "has this object been touched" semantics, which is different from diff-iterate (and also potentially much more expensive to produce!). This commit brings a minimal update to TestMockObjectMapDiffRequest tests and DiffIterateTest.DiffIterateDiscard. Coverage is expanded in the following commits. Fixes: https://tracker.ceph.com/issues/53897 Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
* / librados/snap_set_diff: set end_size only if end object existsIlya Dryomov2023-12-021-1/+1
|/ | | | | | | | | | | Since commit 73f50a13109f ("rbd-mirror: use generalized deep copy for image sync"), the only user of calc_snap_set_diff() immediately unsets end_size otherwise. calc_snap_set_diff() semantics are clearer if end_size is set together with end_exists and clone_end_snap_id. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
* librados: add rados_aio_write_op_operate2() for high-resolution timestampsCasey Bodley2023-05-231-0/+25
| | | | | | expose the aio version of rados_write_op_operate2() for mtimes in timespec form Signed-off-by: Casey Bodley <cbodley@redhat.com>
* librados: rados_write_op_operate() uses ObjectOperationImpl for mtimeCasey Bodley2023-05-231-14/+8
| | | | Signed-off-by: Casey Bodley <cbodley@redhat.com>
* librados: rados_aio_write_op_operate() propagates mtimeCasey Bodley2023-05-231-2/+8
| | | | Signed-off-by: Casey Bodley <cbodley@redhat.com>
* librados: use ObjectOperationImpl for rados_write_op_tCasey Bodley2023-05-233-51/+72
| | | | | | | | | the c++ api uses ObjectOperationImpl to wrap ObjectOperation with additional storage for an optional mtime. the c api now reuses ObjectOperationImpl for its write operations only - the mtime isn't needed for read ops Signed-off-by: Casey Bodley <cbodley@redhat.com>
* librados: aio_operate() supports ObjectWriteOperation::mtime2()Casey Bodley2023-05-234-11/+13
| | | | | | | | | copy the optional mtime logic from operate(), so that mtimes set on the ObjectOperation are propagated through aio_operate() as well Fixes: https://tracker.ceph.com/issues/61349 Signed-off-by: Casey Bodley <cbodley@redhat.com>
* Revert "tracer/osd/librados/build/rgw: rgw and osd end2end tracing using ↵Yuval Lifshitz2023-04-304-13/+13
| | | | | | | | opentelemetry" This reverts commit 5d4c5e65be94a346e62b9f8ac63241779c4c7a6a. Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
* tracer/osd/librados/build/rgw: rgw and osd end2end tracing using opentelemetryOmri Zeneva2023-03-194-13/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | * build: add opentelemetry to cmake system crimson targets that uses Message.cc/h are built before opentelemetry (o-tel), so we need to build o-tel eralier so we also add the library to the include path earlier this shoud work for WITH_JAEGER flag both the ON/OFF cases, and for librados where the compilation flag is ignored * msg/tracer: add o-tel trace to Messages with decode/encode function in tracer.h some files that uses Message.cc/h just need the encode/decode functions and not all others functions. some crimson targets does not link with ceph_context (common) which is required for tracer.cc file. so we just need to include that functions * librados: Add opentelemtry trace param for aio_operate and operate methods in order to propagate the trace info I added the otel-trace as an extra param. in some places, there already was a blkin trace info, and since it is not used in other places we can safely change it to o-tel trace info. this will be done in another commit, so the cleanup of blkin trace will be in a dedicated commit * osd: use the o-tel trace of the msg as a parent span of the osd trace if there is a valid span in the msg, we will add this op to the request trace, otherwise it will start a new trace for the OSD op * rgw: pass put obj trace info to librados in order to make it possible, I saved the trace info inside the sal::Object, so we can use it later when writing the object to rados it could be used also later for read ops. note the trace field of req_state is initalized only in rgw_process, so it's also required in librgw request flow Signed-off-by: Omri Zeneva <ozeneva@redhat.com>
* librados/asio: use boost::system::generic_category on WindowsLucian Petrut2023-02-201-6/+15
| | | | | | | | | | | | | | | | | | | | | | | librados uses "errno.h" error codes. On Windows, boost::system::system_category refers to errors from winerror.h. That being considered, we'll add a platform check and use boost::system::generic_category on Windows. This ensures that error code comparisons work as expected. This issue was brought up by the following test assertion failure: ft2.4: ceph_test_rados_api_asio.AsioRados.AsyncWriteCallbacktesttools. testresult.real._StringException: details: {{{ /ceph/src/test/librados/asio.cc:148 Expected equality of these values: boost::system::errc::read_only_file_system Which is: 30 ec Which is: system:30 }}} Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
* Merge pull request #46823 from lightmelodies/librados-c-api-stat2Kefu Chai2022-09-061-0/+42
|\ | | | | | | | | librados: export stat2 in C API Reviewed-by: Kefu Chai <tchaikov@gmail.com>
| * librados: export stat2 in C APIWei Wang2022-06-251-0/+42
| | | | | | | | Signed-off-by: Wei Wang <lightmelodies@outlook.com>
* | librados, os: drop WRITE_{EQ,CMP}_OPERATORS_3()Kefu Chai2022-08-041-4/+2
| | | | | | | | | | | | the default-generated comparison operators can fulfill our needs. Signed-off-by: Kefu Chai <tchaikov@gmail.com>
* | librados: rados_ioctx_destroy check for initialized ioctxNitzanMordhai2022-07-191-1/+3
|/ | | | | | | | If rados_ioctx_destroy called with un-initilized ioctx we will hit segmented fault checking ioctx before calling put will prevent that. Fixes: https://tracker.ceph.com/issues/55001 Signed-off-by: Nitzan Mordechai <nmordec@redhat.com>
* client: Inode::hold_caps_until is time from monotonic clock now.Neeraj Pratap Singh2022-05-271-1/+1
| | | | | | | Inode::hold_caps_until storing time from ceph::coarse_mono_clock now. Fixes: https://tracker.ceph.com/issues/52982 Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
* Merge pull request #45468 from idryomov/wip-pool-reverse-lookup-osdmapYuri Weinstein2022-03-232-2/+2
|\ | | | | | | | | librados: check latest osdmap on ENOENT in pool_reverse_lookup() Reviewed-by: Josh Durgin <jdurgin@redhat.com>
| * librados: check latest osdmap on ENOENT in pool_reverse_lookup()Ilya Dryomov2022-03-162-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | Avoid spurious ENOENT errors from rados_pool_reverse_lookup() and Rados::pool_reverse_lookup(). This makes lookup by id consistent with lookup by name: the latter has been checking latest osdmap since commit 7e5669b11b14 ("rados: we need to get the latest osdmap when pool does not exists"). Fixes: https://tracker.ceph.com/issues/54593 Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
* | Merge pull request #45419 from tchaikov/wip-cxx20-fixes-coreKefu Chai2022-03-184-13/+13
|\ \ | | | | | | | | | | | | | | | osd, common: C++20 related fixes Reviewed-by: Casey Bodley <cbodley@redhat.com> Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
| * | librados: stop using c++20 keyword 'requires'Casey Bodley2022-03-174-13/+13
| |/ | | | | | | Signed-off-by: Casey Bodley <cbodley@redhat.com>
* / librados: add ability to pass a truncate_size/seq to sparse_readJeff Layton2022-03-071-2/+4
|/ | | | | Fixes: http://tracker.ceph.com/issues/54280 Signed-off-by: Jeff Layton <jlayton@redhat.com>
* librbd: propagate CEPH_OSD_FLAG_FULL_TRY from IoCtx to IOContextIlya Dryomov2021-09-171-0/+5
| | | | | | | | We use neorados on the I/O path so data_io_context needs to have the same setting as data_ctx. Fixes: https://tracker.ceph.com/issues/52648 Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
* librados/IoCtxImpl: apply extra_op_flags to watches and notifiesIlya Dryomov2021-09-171-4/+8
| | | | | | | | | Prior to e468fc0d3b8c ("osdc/Objecter: remove pool_full_try interface"), librados::IoCtx set_pool_full_try() affected watches and notifies. Resurrect that to allow "rbd rm" to work when the metadata pool is full. Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
* librados/IoCtxImpl: preserve extra_op_flags when dupingIlya Dryomov2021-09-171-0/+1
| | | | | | | | | | extra_op_flags doesn't survive librados::IoCtx dup() which means that the effect librados::IoCtx set_pool_full_try() is lost. These contexts are duped in numerous places, especially in librbd. This is the first step to fix the full cluster handling which got broken in e468fc0d3b8c ("osdc/Objecter: remove pool_full_try interface"). Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
* common: modify 'main()s' to use new argv_to_vec() signatureRonen Friedman2021-08-191-2/+1
| | | | | | | A followup to PR #42820 that modified argv_to_vec() signature (for style and performance). Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
* librados: build without "using namespace std"Kefu Chai2021-08-137-35/+50
| | | | | | | | | | * add "std::" prefix in headers * add "using" declarations in .cc files. so we don't rely on "using namespace std" in one or more included headers. Signed-off-by: Kefu Chai <kchai@redhat.com>
* librados/librados_c: check for both symversBoris Ranto2021-08-051-1/+7
| | | | | | | | We should check if either asm symver or attribute symver is defined and not assume that attribute symver implies asm symver. Fixes: https://tracker.ceph.com/issues/40060 Signed-off-by: Boris Ranto <branto@redhat.com>
* librados/librados_c: Use symver attribute if availableBoris Ranto2021-08-051-0/+10
| | | | | Fixes: https://tracker.ceph.com/issues/40060 Signed-off-by: Boris Ranto <branto@redhat.com>
* rados: Update symver defsBoris Ranto2021-08-051-3/+3
| | | | | | | | The gcc compiler now supports symver attribute. We should update the symvers to be able to support LTO. Fixes: https://tracker.ceph.com/issues/40060 Signed-off-by: Boris Ranto <branto@redhat.com>
* rados: Fix function orderingBoris Ranto2021-08-051-4/+6
| | | | | | | | The build fails because the function definitions are out of order. This also reuses the LIBRADOS_C_API_DEFAULT_F macro for function definitions. Fixes: https://tracker.ceph.com/issues/40060 Signed-off-by: Boris Ranto <branto@redhat.com>
* Merge PR #42041 into masterSage Weil2021-07-141-2/+2
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * refs/pull/42041/head: mgr/restful: ignore min/max_size test/crush: drop min/max_size refs qa/workunits/mon/pool_ops: remove test for min/max_size check qa: scrub a few remaining mentions of ruleset qa/standalone/mon/osd-*: fix tests PendingReleaseNotes: note min/max_size removal mgr/dashboard: remove max/min_size and ruleset mon/OSDMonitor: fix calls to CrushTester crush: eliminate min_size and max_size test/cli/crushtool: reunumber rulesets in test maps crushtool: require min/max or num-rep for --test crush: remove last traces of 'ruleset' test/cli/crushtool: use 'id' instead of 'ruleset' in crush inputs crushtool: take --min-rep and --max-rep explicitly crush/CrushTester: drop --ruleset doc: scrub 'ruleset' from docs src/erasure-code: rule, not ruleset mon/OSDMonitor: remove check_crush_rule() callers mon/OSDMonitor: rule, not ruleset crushtool: remove check for overlapped ruels crush/CrushWrapper: get_osd_pool_default_crush_replicated_ruleset -> rule crush: remove find_rule() mon/OSDMonitor: use pool's crush rule directly osd/OSDMap: drop checks for ruleset == ruleid osd/OSDMap: use pool's crush rule_id directly mon/PGMap: use pool's crush_rule directly mon/OSDMonitor: remove crush ruleset->rule rewrite Reviewed-by: Ernesto Puerta <epuertat@redhat.com> Reviewed-by: Avan Thakkar <athakkar@redhat.com>
| * crush: remove last traces of 'ruleset'Sage Weil2021-06-291-2/+2
| | | | | | | | | | | | | | | | | | | | Remove last traces of rulesets from crush. Mostly this means removing the checks that rule masks don't overlap, since no such rules should exist at this point, and the compiler no longer allows them. Scrub all mention of 'ruleset' from other code. Signed-off-by: Sage Weil <sage@newdream.net>
* | mgr,mon: s/boost::optional/std::optional/Kefu Chai2021-06-291-1/+1
|/ | | | | | | since the encoding schemes of boost::optional and std::optional are identical, they can be used interchangeably. Signed-off-by: Kefu Chai <kchai@redhat.com>
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-19 03:38:51 +0100