summaryrefslogtreecommitdiffstats
path: root/src/perfglue (follow)
Commit message (Collapse)AuthorAgeFilesLines
* [CHERRY-PICKED FOR TESTING ONLY] crimson: Enable tcmalloc when using seastarMark Nelson2023-06-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | classic-osds have always caused significant memory fragmentation when using the libc memory allocator due to the way that Ceph tends to utilize memory. In recent testing, crimson-osd was found to use 25-27GB of RAM with the stock 3GB bluestore cache settings (osd_memory_target is only used when tcmalloc is available). Upon further testing, it was found that the classic OSD is even worse, using between 32-33GB of RAM after a 5 minute 4K sequential write test when using libc malloc. The good news is that it appears that crimson-osd is able to use tcmalloc for alienstore without significant modification. Better still, it drastically reduces memory usage. In the same test that resulted in 25GB RSS memory usage for crimson-osd with libc malloc, a tcmalloc linked version took around 9GB (with an 8GB osd_memory_target). Since we do not yet (afaik) expose classic OSD debugging in crimson it is tough to tell why we are still a little over, but it's clear that for alienstore we are going to need to use tcmalloc as we do in classic. Signed-off-by: Mark Nelson <mnelson@redhat.com> (cherry picked from commit 9f31ad7b7388996307808e26722260aed29d4ea1)
* Merge PR #41338 into masterPatrick Donnelly2021-05-201-2/+2
|\ | | | | | | | | | | | | | | * refs/pull/41338/head: mds: reset code after cpu_profiler cpu_profiler: fix asok command crash Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
| * cpu_profiler: fix asok command crashliu shi2021-05-141-2/+2
| | | | | | | | | | fixes: https://tracker.ceph.com/issues/50814 Signed-off-by: liu shi <liu.shi@navercorp.com>
* | librbd: do not use ceph::allocator<>Kefu Chai2021-05-141-1/+1
|/ | | | | | | | | | | | | | | | | | | ceph::allocator was originally introduced to address https://tracker.ceph.com/issues/39703, so that if tcmalloc < 2.6.2 detected, the homebrew allocator is used. otherwise std::allocator would use the aligned_alloc() function which was not implemented by tcmalloc < 2.6.2. but the fix in tcmalloc implementing aligned_alloc() was introduced in https://github.com/gperftools/gperftools/commit/d406f228. that fix was included since gperftools 2.6.2. - CentOS/RHEL8 includes gperftools 2.7, - fedora 33 includes gperftools 2.8, - ubuntu/focal comes with libgoogle-perftools-dev 2.7 - ubuntu/bionic ships libgoogle-perftools-dev 2.5. but since we detect libtcmalloc 2.6.2, when building on bionic, JeMalloc or libc allocator is used as fallback. Signed-off-by: Kefu Chai <kchai@redhat.com>
* perfglue: Build ceph-osd without using namespace declarations in headersAdam C. Emerson2020-04-064-4/+4
| | | | | | | This is part of a series of commits to clean up using namespace at top level in headers. Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
* mon: Build ceph-mon without using namespace declarations in headersAdam C. Emerson2020-03-222-2/+2
| | | | | | | This is part of a series of commits to clean up using namespace at top level in headers. Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
* crimson:disable heap profiler for seastarChunmei Liu2020-02-281-1/+1
| | | | | | | alien bluestore will link with heap profiler, but seastar not support heap profiler with seastar defaut allocator, so disable it. Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
* osd,mon,mds: support get|set tcmalloc release rateZengran Zhang2019-02-123-0/+30
| | | | Signed-off-by: Zengran Zhang <zhangzengran@sangfor.com.cn>
* cmake,make-dist: revert "build gperftools if WITH_STATIC_LIBSTDCXX"Kefu Chai2018-08-061-7/+1
| | | | | | | | | | as the higher version of libstdc++ is backward compatible with the lower ones. so there is no need to statically link against C++ libraries. they can always use the libstdc++ ships with the distro. This reverts commit a6c73b6ac1d81cc9f467aa38e5afb146d5dea6c2 Signed-off-by: Kefu Chai <kchai@redhat.com>
* cmake,make-dist: build gperftools if WITH_STATIC_LIBSTDCXXKefu Chai2018-07-291-1/+7
| | | | | | | | | | we could create a mini project to build a shared library, and use try_compile() to test if the found gperftools is compiled with -fPIC. but as we are targeting mostly xenial when enabling WITH_STATIC_LIBSTDCXX, and google-perftools on xenial by default is built without -fPIC. so let's keep it simple. Signed-off-by: Kefu Chai <kchai@redhat.com>
* cmake: make cpu_profiler a library targetKefu Chai2018-07-261-0/+13
| | | | Signed-off-by: Kefu Chai <kchai@redhat.com>
* cmake: make heap_profiler a library targetKefu Chai2018-07-261-0/+9
| | | | Signed-off-by: Kefu Chai <kchai@redhat.com>
* common,rbd,rgw,osd: extract config values into ConfigValuesKefu Chai2018-07-101-8/+8
| | | | | | | | | | | | | | | | | | | | | | | | | this change introduce three classes: ConfigValues, ConfigProxy and ConfigReader. in seastar port of OSD, each CPU shard will hold its own reference of configuration, and upon changes of settings, each shard will be updated with the new setting in async. so this forces us to be able to keep two set of configuration at the same time. so we need to extract the changeable part of md_config_t out. so we can replace the old one with new one on demand, and let different shards share the same unchanged part, amon the other things, the Options map and the lookup tables. that's why we need ConfigValues. we will add a policy template for this class, so we can specialize for Seastar implementation to allow different ConfigProxy instances to point md_config_impl<> to different ConfigValues. because the observer interface is still using md_config_t, to minimise the impact of this change, handle_conf_change() and handle_subsys_change() are not changed. but as it accepts a `const md_config_t`, which cannot be used to create/reference the ConfigProxy holding it, we need to introduce ConfigReader for reading the updated setting from md_config_t in a simpler way, without exposing the internal "values" member variable. Signed-off-by: Kefu Chai <kchai@redhat.com>
* perfglue: Silence truncation warningAdam C. Emerson2018-05-091-0/+9
| | | | Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
* dout: Use dout_contextAdam C. Emerson2016-12-221-0/+2
| | | | | | | As a transition for g_ceph_context removal, make all dout macros depends on a local macro. Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
* remove autotoolsSage Weil2016-09-071-35/+0
| | | | Signed-off-by: Sage Weil <sage@redhat.com>
* cmake: cleanup Findgperftools.cmakeKefu Chai2016-08-112-19/+3
| | | | | | | | | | | | | | | | * remove Findtcmalloc.cmake, use Findgperftools.cmake instead. * tcmalloc is packaged in gperftools-dev, so we can check them in a single place. * fix the check of heap-profiler.h, we should enable the check as long as tcmalloc is enabled or profiler is enabled. * only check gperftools headers in "include/gperftools", and do not "include/google" anymore. as we only support the distros which shipping recent gperftools-dev package with "include/gperftools". and "google/*.h" are deprecated. * set ALLOC_LIBS with GPERFTOOLS_TCMALLOC_LIBRARY, so we can link against tcmalloc with full path. Signed-off-by: Kefu Chai <kchai@redhat.com>
* osd/: make heap properties settable via admin socketSamuel Just2016-01-203-0/+15
| | | | Signed-off-by: Samuel Just <sjust@redhat.com>
* osd/: make heap properties gettable from admin_socketSamuel Just2016-01-203-0/+15
| | | | Signed-off-by: Samuel Just <sjust@redhat.com>
* Merge pull request #4008 from SUSE/wip-tcmalloc-conditionalsSamuel Just2015-05-012-1/+21
|\ | | | | | | | | Conditional-compile against minimal tcmalloc. Reviewed-by: Loic Dachary <ldachary@redhat.com>
| * Conditional-compile against minimal tcmalloc.Thorsten Behrens2015-03-302-1/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Certain older systems (SLE11 in this case) do not provide the full tcmalloc functionality, due to e.g. incomplete libunwind pieces. Use --with-tcmalloc-minimal to enable the cut-down version. Here's how the various mem allocator switches interact now: --with-jemalloc: overrides --with-tcmalloc & --with-tcmalloc-minimal --with-tcmalloc-minimal: overrides --with-tcmalloc --with-tcmalloc: the default. use --without-tcmalloc to disable Signed-off-by: Thorsten Behrens <tbehrens@suse.com>
* | Dencoder should never be built with tcmallocBoris Ranto2015-04-141-0/+5
|/ | | | | | | | | The patch adds disabled perfglue stubs to DENCODER sources in order to avoid tcmalloc-enabled ceph-dencoder builds. Refs: #10691 Signed-off-by: Boris Ranto <branto@redhat.com>
* heap_profiler: support new gperftools header locationsKen Dreyer2014-12-032-4/+28
| | | | | | | | | The google/ headers location has been deprecated as of gperftools 2.0. As of gperftools 2.2rc, the google/ headers will now give deprecation warnings, and they will probably disappear in a future gperftools update. Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
* perfglue: profiler stats need more than 1024 bytesLoic Dachary2014-10-081-6/+8
| | | | | | | With a 1024 bytes buffer to display the stats, the end is truncated. Use an array on the stack instead of leaking the buffer. Signed-off-by: Loic Dachary <loic-201408@dachary.org>
* Merge remote-tracking branch 'gh/next'Sage Weil2013-09-231-5/+5
|\
| * perfglue/heap_profiler.cc: expect args as first element on cmd vectorJoao Eduardo Luis2013-09-201-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We used to pass 'heap' as the first element of the cmd vector when handling commands. We haven't been doing so for a while now, so we needed to fix this. Not expecting 'heap' also makes sense, considering that what we need to know when we reach this function is what command we should handle, and we should not care what the caller calls us when handling his business. Fixes: #6361 Backport: dumpling Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
* | make: add tmalloc lib dep in tcmalloc guardNoah Watkins2013-09-131-1/+1
| | | | | | | | | | | | Fixes --without-tcmalloc on boxes without libtcmalloc. Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
* | automake cleanup: implementing non-recursive makeRoald J. van Loon2013-09-081-0/+23
|/ | | | | | | | - Enabling subdir objects - Created a Makefile-env.am with basic automake init - Created .am files per subdir, included from src/Makefile.am Signed-off-by: Roald J. van Loon <roaldvanloon@gmail.com>
* cpu_profiler: drop start, stop commandsSage Weil2012-08-171-18/+2
| | | | | | These don't appear to work. Setting CPUPROFILE=path on startup does. Signed-off-by: Sage Weil <sage@inktank.com>
* heap_profiler: return result via ostreamSage Weil2012-08-164-19/+18
| | | | Signed-off-by: Sage Weil <sage@inktank.com>
* cpu_profiler: outout to ostream instead of clogSage Weil2012-08-163-22/+15
| | | | | | | This let's the 'ceph tell osd.N cpu_profiler ...' command return the result to the caller. Signed-off-by: Sage Weil <sage@inktank.com>
* logclient: not a dispatcherSage Weil2012-06-061-0/+2
| | | | | | | | Let MonClient and Monitor handle delivery of messages. This puts them in control and lets them trigger sending of more messages when we have a bunch queued. Signed-off-by: Sage Weil <sage@inktank.com>
* src/perfglue/heap_profiler.cc: fix snprintfColin Patrick McCabe2011-08-171-1/+1
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* Fix bug in turning off logging. Remove log_dir.Colin Patrick McCabe2011-08-161-3/+18
| | | | | | | | | | | | Fix a bug that made it impossible to turn off logging to a file. Remove the old "log_dir" setting. It has been deprecated for a long time, and it adds a lot of complexity to the code. heap_profiler: use the directory that log_file is in to construct a profiler output file name. Don't use log_dir. Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* heap_profiler: if log_dir is empty, don't try and log to root dir!Greg Farnum2011-06-101-1/+2
| | | | | | | If log_dir was empty then the prefix would be of the form /mon.a That isn't good, so fill it in so it writes to the current dir. Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
* g_conf: change to pointerColin Patrick McCabe2011-06-071-6/+6
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* Change some global config options to env varsColin Patrick McCabe2011-04-191-25/+9
| | | | | | | | | | | | Options that are inherently global, like malloc settings, and also inherently debugging or profiling settings should be environment variables. tcmalloc_profiler_run, profiler_allocation_interval, profiler_highwater_interval, and buffer_track_alloc fall into this category. Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* config: make md_config_t.name a value, not ptrColin Patrick McCabe2011-04-151-2/+2
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* config: use std::string in md_config_tColin Patrick McCabe2011-03-301-2/+2
| | | | | | | | | | | | | | Use std::string to represent md_config_t strings. This makes memory management a lot easier and should fix some leaks. "No value" is now represented by an empty string, whereas before some places were using empty strings and some were using NULL. config.cc: Fix a minor decode bug. In pid_file.cc, copy the pid_file using snprintf, since strncpy does not always NULL-terminate. Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* Replace g_conf.name and g_conf.id with entitynameColin Patrick McCabe2011-03-081-6/+9
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* tcmalloc: warn if you try and dump without the profiler running.Greg Farnum2011-03-031-5/+9
| | | | Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
* tcmalloc: create perfglue handle_command functionality.Greg Farnum2011-03-033-1/+44
| | | | | | | Switch the OSD and MDS to use it, instead of handling the calls on their own. Note that this is a command interface change! Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
* tcmalloc: switch the interface.Greg Farnum2011-03-023-0/+149
| | | | | | | | | | | | | | | Previously, we used function pointers. Fun for me to learn about, icky to actually have! Now we use our own wrapper functions with two implementations -- one for with tcmalloc and one without. Make those programs which are tcmalloc-aware build with the appropriate implementation source at compile-time, but leave the wrapper function stubs in no matter what. While we're at it, implement two of the "MallocExtension" calls in the OSD. Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
* profiler: move perftools glue into perfglue/Colin Patrick McCabe2011-03-013-0/+112
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-18 20:06:17 +0100