summaryrefslogtreecommitdiffstats
path: root/src/kv (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Merge pull request #60278 from rzarzynski/wip-os-fastomapiterYuri Weinstein2025-01-133-2/+116
|\ | | | | | | | | | | | | | | | | | | | | | | os, osd: bring the lightweight OMAP iteration Reviewed-by: Casey Bodley <cbodley@redhat.com> Reviewed-by: Matan Breizman <Matan.Brz@gmail.com> Reviewed-by: Mark Kogan <mkogan@redhat.com> Reviewed-by: Adam Kupczyk <akupczyk@redhat.com> Reviewed-by: Samuel Just <sjust@redhat.com>
| * kv: avoid memcpy around key() in OMAP iterator of KeyValueDBRadoslaw Zarzynski2024-12-173-2/+86
| | | | | | | | Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
| * kv: avoid memcpy in OMAP iterator of KeyValueDBRadoslaw Zarzynski2024-12-173-0/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ``` - 63.07% _ZN12PrimaryLogPG19prepare_transactionEPNS_9OpContextE ▒ - 63.06% _ZN12PrimaryLogPG10do_osd_opsEPNS_9OpContextERSt6vectorI5OSDOpSaIS3_EE ▒ - 20.19% _ZN9BlueStore16OmapIteratorImpl4nextEv ▒ - 12.21% _ZN14CFIteratorImpl4nextEv ▒ + 10.56% _ZN7rocksdb6DBIter4NextEv ▒ 1.02% _ZN7rocksdb18ArenaWrappedDBIter4NextEv ▒ + 3.11% clock_gettime@@GLIBC_2.17 ▒ + 2.44% _ZN9BlueStore11log_latencyEPKciRKNSt6chrono8durationImSt5ratioILl1ELl1000000000EEEEdS1_i ▒ 0.78% pthread_rwlock_rdlock@plt ▒ 0.69% pthread_rwlock_unlock@plt ▒ - 14.28% _ZN9BlueStore16OmapIteratorImpl5valueEv ▒ - 11.60% _ZN14CFIteratorImpl5valueEv ▒ - 11.41% _ZL13to_bufferlistN7rocksdb5SliceE ▒ - 10.50% _ZN4ceph6buffer7v15_2_03ptrC1EPKcj ▒ - _ZN4ceph6buffer7v15_2_04copyEPKcj ▒ - 10.01% _ZN4ceph6buffer7v15_2_014create_alignedEjj ▒ - _ZN4ceph6buffer7v15_2_025create_aligned_in_mempoolEjji ▒ 5.27% _ZN7mempool6pool_t12adjust_countEll ▒ + 3.72% tc_posix_memalign ▒ 0.54% _ZN4ceph6buffer7v15_2_04list6appendEONS1_3ptrE ▒ 1.25% pthread_rwlock_rdlock@plt ▒ 0.90% pthread_rwlock_unlock@plt ``` Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
* | kv: add a pair of informative getters to KeyValueDB::TransactionImplIgor Fedotov2024-09-242-0/+12
|/ | | | Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
* Merge pull request #58728 from ifed01/wip-ifed-ret-error-kv-statsIgor Fedotov2024-09-171-2/+7
|\ | | | | | | | | kv/rocksdb: return error for dump_objectstore_kv_stats asok command Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>
| * kv/rocksdb: return error for dump_objectstore_kv_stats asok commandIgor Fedotov2024-07-221-2/+7
| | | | | | | | | | | | if relevant config settings result in no output. Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
* | kv/RocksDBStore: add kv_stats function to print all CF infowanglinke2024-07-231-7/+20
|/ | | | | | | | | | | - Added the function of printing all cf's rocksdb level information. - Currently, only the default cf information is printed. - Print all cf information to facilitate observation of the level to which rocksdb is written. - The main purpose is to observe the usage of db/ssd. co-author: Jrchyang Yu <yuzhiqiang_yewu@cmss.chinamobile.com> Signed-off-by: Wang Linke <wanglinke_yewu@cmss.chinamobile.com>
* kv/RocksDBStore: add more perf counters to track compactions.Igor Fedotov2024-05-082-3/+12
| | | | Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
* os/bluestore: enable async manual compactionsIgor Fedotov2024-05-081-0/+2
| | | | Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
* kv/RocksDBStore: Configure compact-on-deletion for all CFsJoshua Baergen2024-02-251-8/+8
| | | | | | | | | | | | | update_column_family_options() is called only for non-default CFs, whereas load_rocksdb_options() sets options for all CFs, including default. This isn't really a problem for new installs, where the default CF isn't used for very much, but for upgrades where resharding has not yet happened, the bulk of rocksdb data is still in the default CF and so it's important that it also gets compact-on-deletion configured, if desired. Fixes: https://tracker.ceph.com/issues/64511 Signed-off-by: Joshua Baergen <jbaergen@digitalocean.com>
* kv:resolve three complication warningscuiming_yewu2023-10-101-0/+3
| | | | | | | | | | | | | | | Resolve three warning issues in the derived classes overriding base class virtual functions with mismatched parameter lists at lines 69, 72, and 74 in src/kv/rocksdb_cache/ShardedCache.h. with ../src/kv/rocksdb_cache/ShardedCache.h:74:16: warning: by 'virtual bool rocksdb_cache::ShardedCache::Release(rocksdb::Cache::Handle*, bool)' [-Woverloaded-virtual] virtual bool Release(rocksdb::Cache::Handle* handle, bool force_erase = false) override; ^~~~~~~ ../src/kv/rocksdb_cache/ShardedCache.h:72:35: warning: by 'virtual rocksdb::Cache::Handle* rocksdb_cache::ShardedCache::Lookup(const rocksdb::Slice&, rocksdb::Statistics*)' [-Woverloaded-virtual] virtual rocksdb::Cache::Handle* Lookup(const rocksdb::Slice& key, rocksdb::Statistics* stats) override; ^~~~~~ ../src/kv/rocksdb_cache/ShardedCache.h:69:27: warning: by 'virtual rocksdb::Status rocksdb_cache::ShardedCache::Insert(const rocksdb::Slice&, void*, size_t, rocksdb::Cache::DeleterFn, rocksdb::Cache::Handle**, rocksdb::Cache::Priority)' [-Woverloaded-virtual] virtual rocksdb::Status Insert(const rocksdb::Slice& key, void* value, size_t charge, ^~~~~~ Signed-off-by: cuiming <cuiming_yewu@cmss.chinamobile.com>
* Merge pull request #49748 from ifed01/wip-ifed-bound-rm-range-keysYuri Weinstein2023-03-232-17/+34
|\ | | | | | | | | kv/RocksDBStore: use bounded iterators in rm_range_keys Reviewed-by: Cory Snyder <csnyder@iland.com>
| * kv/RocksDBStore: use bounded iterators in rm_range_keysIgor Fedotov2023-03-102-17/+34
| | | | | | | | | | Fixes: https://tracker.ceph.com/issues/58463 Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
* | Merge pull request #49963 from ifed01/wip-ifed-fix-prefixed-kv-iteratorYuri Weinstein2023-03-103-7/+18
|\ \ | |/ |/| | | | | | | | | kv/RocksDBStore: don't use real wholespace iterator for prefixed access Reviewed-by: Adam Kupczyk <akupczyk@redhat.com> Reviewed-by: Cory Snyder <csnyder@iland.com>
| * kv/RocksDBStore: don't use real wholespace iterator for prefixed accessIgor Fedotov2023-02-012-3/+14
| | | | | | | | | | | | | | | | We can bound to default CF when here are no matching CF for a specified prefix. No need to use real wholespace iterator running over every CF. Hence we might benefit from not iterating over large but useless CFs, e.g. OMAP related ones. Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
| * kv/RocksDBStore: avoid redundant prefix lookup.Igor Fedotov2023-02-012-4/+4
| | | | | | | | Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
* | Merge pull request #49870 from ifed01/wip-ifed-rm-range-threshold-on-the-flyYuri Weinstein2023-03-092-8/+27
|\ \ | | | | | | | | | | | | | | | kv/rocksdbstore: apply rocksdb_delete_range_threshold on the fly Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
| * | kv/rocksdbstore: apply rocksdb_delete_range_threshold on the flyIgor Fedotov2023-01-252-8/+27
| |/ | | | | | | | | | | | | Plus more effective rm_range_keys implementation when this parameter is set to 0 Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
* / common: Add labeled perf countersAli Maredia2023-02-231-1/+1
|/ | | | | | | | | | | | | | | | | Add the ability to dump labeled perf counters for a daemon. Labeled perf counters are stored in a CephContext's PerfCountersCollection. Labeled and unlabeled perf counters are dumped to the admin socket via `counters dump` command. The schema for labeled and unlabeled perf counters are dumped to the admin socket via `counters schema` command. This commit includes docs and additional unit tests Signed-off-by: Ali Maredia <amaredia@redhat.com>
* Merge pull request #47221 from markhpc/wip-faster-rocksdbMark Nelson2022-11-281-0/+9
|\ | | | | [WIP] kv/RocksDBStore: Improved RocksDB Settings and Tombstone behavior
| * kv/RocksDBStore: Add CompactOnDeletion supportMark Nelson2022-07-221-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | This commit adds support to compact column families when a certain number of tombstone entries have been observed within a certain sliding window during iteration. It only helps when itereating over entries already in SST files and not when iterating over ranges in memtables. Likely we will still need to provide a mechanism to flush memtables and compact column families once a certain number of rmkey or rm_range_key calls are made. Signed-off-by: Mark Nelson <mnelson@redhat.com>
* | osd, kv: add debugs along the pg_log_t::dups paths.Radoslaw Zarzynski2022-08-091-0/+7
| | | | | | | | Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
* | Merge pull request #36282 from rzarzynski/wip-bl-drop-cloneYuri Weinstein2022-07-224-888/+0
|\ \ | |/ |/| | | | | | | | | | | | | | | common/bl, kv, tests: drop MemDB and simplify buffer::ptr and buffer::raw Reviewed-by: Kefu Chai <kchai@redhat.com> Reviewed-by: Ilya Dryomov <idryomov@redhat.com> Reviewed-by: Neha Ojha <nojha@redhat.com>
| * kv: drop the MemDB backend.Radoslaw Zarzynski2022-05-234-888/+0
| | | | | | | | | | | | | | | | | | It's an experimental feature since 2016. It doesn't seem to be used anywhere nor actively maintained. However, it's the sole major user of `ceph::buffer::ptr::clone()` which could have been dropped otherwise. Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
* | rocksdb: build with rocksdb-7.y.zKaleb S. KEITHLEY2022-05-245-10/+7
|/ | | | | | | | RocksDB 7, specifically 7.2.2 has landed in Fedora 37/rawhide. https://tracker.ceph.com/issues/55730 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* kv/RocksDBStore: Remove feature to make WholeSpaceIterator based on bounded ↵Adam Kupczyk2022-04-304-25/+9
| | | | | | | | | | | | | | | iterator Iterator-bounding feature is introduced to make RocksDB iterators limited, so they would less likely traverse over tombstones. This is used when listing keys in fixed range, for example OMAPS for specific object. It is problematic when extending this logic to WholeSpaceIterator, since prefix must be taken into account. Fixes: https://tracker.ceph.com/issues/55444 Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
* kv/RocksDBStore: simplify RocksDBStore::get_cf_handle(string, IteratorBounds)Cory Snyder2022-04-211-13/+10
| | | | | | | | Adds a precondition to RocksDBStore::get_cf_handle(string, IteratorBounds) to avoid duplicating logic of the only caller (RocksDBStore::get_iterator). Assertions will fail if preconditions are not met. Signed-off-by: Cory Snyder <csnyder@iland.com>
* bluestore: add config option to allow rocksdb iterator bounds to be disabledCory Snyder2022-04-212-16/+22
| | | | | | | Add osd_rocksdb_iterator_bounds_enabled config option to allow rocksdb iterator bounds to be disabled. Also includes minor refactoring to shorten code associated with IteratorBounds initialization in bluestore. Signed-off-by: Cory Snyder <csnyder@iland.com>
* bluestore: set upper and lower bounds on rocksdb omap iteratorsCory Snyder2022-04-184-34/+143
| | | | | | | | | | | | Limits RocksDB omap Seek operations to the relevant key range of the object's omap. This prevents RocksDB from unnecessarily iterating over delete range tombstones in irrelevant omap CF shards. Avoids extreme performance degradation commonly caused by tombstones generated from RGW bucket resharding cleanup. Also prefer CFIteratorImpl over ShardMergeIteratorImpl when we can determine that all keys within specified IteratorBounds must be in a single CF. Fixes: https://tracker.ceph.com/issues/55324 Signed-off-by: Cory Snyder <csnyder@iland.com>
* common/PriorityCache: Implement Cache Age BinningMark Nelson2022-01-113-9/+146
| | | | Signed-off-by: Mark Nelson <mnelson@redhat.com>
* build: Remove LevelDB supportAdam C. Emerson2021-11-015-887/+3
| | | | | | | | | | | | Whereas new LevelDB packages are build with -fno-rtti and break our attempts to compile against them, and whereas LevelDB has been deprecated for some time, with the Ceph team expending great effort to migrating OSDs off of it, let it be therefore removed. Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
* Merge pull request #36733 from majianpeng/rocksdb-remove-redundant-perfcountYuri Weinstein2021-09-302-5/+0
|\ | | | | | | | | kv/RocksdbStore: remove perfcounter l_rocksdb_gets. Reviewed-by: Josh Durgin <jdurgin@redhat.com>
| * kv/RocksdbStore: remove perfcounter l_rocksdb_gets.Jianpeng Ma2020-08-202-5/+0
| | | | | | | | | | | | Perfcounter l_rocksdb_get_latency contain l_rocksdb_gets. Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
* | kv/rocksdb_cache: implement methods required by rocksdb v6.22.1Kefu Chai2021-09-014-9/+76
| | | | | | | | | | | | rocksdb v6.22.1 added couple pure methods, so let's implement them. Signed-off-by: Kefu Chai <kchai@redhat.com>
* | kv/rocksdb_cache: mark Shard() constKefu Chai2021-09-011-1/+1
| | | | | | | | | | | | it does not mutate anything, so mark it `const`. Signed-off-by: Kefu Chai <kchai@redhat.com>
* | kv/rocksdb_cache: define DeleterFn function pointer typeKefu Chai2021-09-014-6/+8
| | | | | | | | | | | | this paves the road to rocksdb v6.22.1 compatible implementation Signed-off-by: Kefu Chai <kchai@redhat.com>
* | kv/rocksdb_cache: reorder ShardedCache methods declarationsKefu Chai2021-09-011-11/+10
| | | | | | | | | | | | | | in the exact order in which rocksdb::Cache declare its public virtual methods. Signed-off-by: Kefu Chai <kchai@redhat.com>
* | kv/rocksdb_cache: drop ROCKSDB_PRIsztKefu Chai2021-09-012-2/+1
| | | | | | | | | | | | | | | | | | | | | | ROCKSDB_PRIszt is a macro for "zu", which is in turn use for printing an (unsigned) size_t variable. there is no point having it in the header file or define a macro for it, as %zu is standard compliant, and we don't get any advantage by using a macro for it. Signed-off-by: Kefu Chai <kchai@redhat.com>
* | kv: build without "using namespace std"Kefu Chai2021-08-132-14/+14
| | | | | | | | | | | | | | | | | | | | * add "std::" prefix in headers * add "using" declarations in .cc files. so we don't rely on "using namespace std" in one or more included headers. Signed-off-by: Kefu Chai <kchai@redhat.com>
* | kv/RocksDBStore: Add handling of block_cache option for reshardingAdam Kupczyk2021-08-052-119/+148
| | | | | | | | | | | | | | | | | | Synchronized all situations when we initialize DB to include handling of block_cache option. Lack of it prevented ability to reshard into specification that we have as default. Fixes: https://tracker.ceph.com/issues/51445 Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
* | kv/RocksDBStore: enrich debug messageSatoru Takeuchi2021-07-271-0/+1
| | | | | | | | | | | | | | It's better to print why ListColumnFamilies() failed. Signed-off-by: Toshikuni Fukaya <toshiq2@gmail.com> Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
* | Merge pull request #42485 from tchaikov/wip-strtollKefu Chai2021-07-271-3/+3
|\ \ | | | | | | | | | | | | common/strtol: replace `const char*` with `std::string_view` Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
| * | *: drop strict_iecstrtoll(const char *str,..)Kefu Chai2021-07-261-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | replace strict_iecstrtoll(const char *str,..) with strict_iecstrtoll(std::string_view, ..). which is more convenient. and both of them share the same implementation: strict_iec_cast<uint64_t>(str, err); so they are interchangeable. Signed-off-by: Kefu Chai <kchai@redhat.com>
* | | kv/RocksDBStore: always initialize local variableKefu Chai2021-07-241-1/+1
|/ / | | | | | | | | | | | | | | | | | | silences following warning from GCC: ../src/kv/RocksDBStore.cc:386:7: warning: ‘r’ may be used uninitialized in this function [-Wmaybe-uninitialized] 386 | if (r < 0) { | ^~ Signed-off-by: Kefu Chai <kchai@redhat.com>
* | *: always include <filesystem>Kefu Chai2021-06-092-16/+8
| | | | | | | | | | | | | | | | since there is no need to be compatible with GCC older than GCC-8, so there is no need to use <experimental/filesystem> as an alternative of <filesystem> anymore. Signed-off-by: Kefu Chai <kchai@redhat.com>
* | tools/kvstore-tool: implement 'dissect' commandIgor Fedotov2021-02-121-3/+13
| | | | | | | | | | | | | | This command shows information on key/value counts and lengths for KV DB. Signed-off-by: Igor Fedotov <ifedotov@suse.com>
* | kv/KeyValueHistogram: make Bluestore's DBHistogram class reusable.Igor Fedotov2021-02-103-0/+117
| | | | | | | | Signed-off-by: Igor Fedotov <ifedotov@suse.com>
* | Merge pull request #38962 from aclamk/wip-aclamk-bluestore-tool-reshard-fixNeha Ojha2021-01-251-5/+18
|\ \ | | | | | | | | | | | | Fix ceph-bluestore-tool reshard command Reviewed-by: Josh Durgin <jdurgin@redhat.com>
| * | kv/RocksDBStore: Fix ceph-bluestore-tool reshard commandAdam Kupczyk2021-01-191-5/+18
| | | | | | | | | | | | | | | | | | | | | Fix ceph-bluestore-tool reshard command. Add conditions that allow to continue resharding that was previously terminated. Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
* | | tools/bluestore: Add command 'show-sharding' to ceph-bluestore-toolAdam Kupczyk2021-01-192-1/+20
|/ / | | | | | | | | | | Add command 'show-sharding' to ceph-bluestore-tool. Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-30 20:34:55 +0100