rgw: D3N Cache changes for Upstream

Upstreaming / rebase of #24500

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

rgw: change io_ctx pool per storage class

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: Changing free() to delete()

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

rgw: Addressing review comments

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

rgw: Fixing seg fault

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

Moving CacheRequest out of librados

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

rgw: cache initialization fix

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

rgw: fix missing spawn.hpp compile errors

resolves compilation errors similar to:
```
[ 15%] Building CXX object src/tools/immutable_object_cache/CMakeFiles/ceph_immutable_object_cache_lib.dir/CacheController.cc.o
In file included from /home/jenkins-build/build/workspace/ceph-pull-requests/src/rgw/rgw_common.h:31,
                 from /home/jenkins-build/build/workspace/ceph-pull-requests/src/rgw/rgw_rados.h:17,
                 from /home/jenkins-build/build/workspace/ceph-pull-requests/src/librados/IoCtxImpl.h:30,
                 from /home/jenkins-build/build/workspace/ceph-pull-requests/src/librados/RadosClient.h:35,
                 from /home/jenkins-build/build/workspace/ceph-pull-requests/src/neorados/RADOSImpl.h:27,
                 from /home/jenkins-build/build/workspace/ceph-pull-requests/src/neorados/RADOS.cc:37:
/home/jenkins-build/build/workspace/ceph-pull-requests/src/common/async/yield_context.h:31:10: fatal error: spawn/spawn.hpp: No such file or directory
   31 | #include <spawn/spawn.hpp>
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.
src/neorados/CMakeFiles/neorados_api_obj.dir/build.make:62: recipe for target 'src/neorados/CMakeFiles/neorados_api_obj.dir/RADOS.cc.o' failed
make[3]: *** [src/neorados/CMakeFiles/neorados_api_obj.dir/RADOS.cc.o] Error 1
```

Signed-off-by: Mark Kogan <mkogan@redhat.com>

Resolving merge conflict

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

Removing all logs and unnecessary comments

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

Cache Read and Write working

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

Initial Commit L1 Cache

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

post re-rebase merge, update fixes

Signed-off-by: Mark Kogan <mkogan@redhat.com>

fixup! post re-rebase merge, update fixes

rgw: continuation of rgwcache branch rebase

Signed-off-by: Mark Kogan <mkogan@redhat.com>

RGW: DataCache: post merge fixes

Signed-off-by: Mark Kogan <mkogan@redhat.com>

fixes of segmentation fault caused by oid

Signed-off-by: E. Ugur Kaynar <ukaynar@bu.edu>

rgw: fixes for segmentation faults and configuration processing

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: data cache first commit

Signed-off-by: Mania Abdi <mania.abdi287@gmail.com>

rgw: cleanup addressing PR comments

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: cleanup addressing PR comments, continuation.

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: pr cleanup addressing second review round

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: Addressing review comments, removing all D3N code from librados

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

rgw: for compilation err from removal of mydout() helper

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rge: addressing review comments

rgw: move d3n datacache into separate files
rgw: 1st part of datacache rebranding to d3n

fix forward declaration compile err (only with clang):
../src/rgw/rgw_cache.h:396:4: error: member access into incomplete type 'struct get_obj_data'
  d->data_lock.lock();
   ^
../src/rgw/rgw_cache.h:365:8: note: forward declaration of 'get_obj_data'
struct get_obj_data;
       ^

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: addressing review comments, datacache rebranding to d3n cache

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: Cleaning up unused D3N cache code

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

rgw: cont. cleaning up of rgw_obj_data()

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: Removing redundant code, fix for multipart S3 objects

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

rgw: fix: incorrect content len on multipart get

in s3tests_boto3.functional.test_s3:test_multipart_copy_versioned
when d3n cache is disabled

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: fix segfault reading from cache

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: fix segfault in multisite

sync on secondary site

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: fix segfault in multisite teuthology tests, cont.

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: Adding drain to wait for all AIO reads to complete

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>

rgw: fix for using read() by liabio or posix io per config

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: improve persistent data cache directory handling

on start create the persistent datacache directory if necessary
and add an option to evict it's content if already exists

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: fix possible segfault during eviction

Signed-off-by: Mark Kogan <mkogan@redhat.com>

Co-authored-by: Mania Abdi <mania.abdi287@gmail.com>
Co-authored-by: E. Ugur Kaynar <ukaynar@bu.edu>
Co-authored-by: Aishwarya Mathuria <amathuri@redhat.com>
Co-authored-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: addressing latest review comments

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: add debug logs for cache in/out flow

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: move the L2 cache functionality to separate PR

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: addressing review comments

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: address java_s3tests teuthology issues

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: do not handle compressed objects fro now

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: l2 cleanup and log fixups + post dpp

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: thread dpp thru get_obj_iterate_cb() and related

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: bypass reading versioned objects from cache

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: cleanup and fix s3tests

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: allow to enable cache only on beast

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: delete the content of the cache directory on rgw start

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: re-enable d3n cache with civetweb frontend

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: rebase post zipper 10

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: address teuthoogy valgrind leaks detected

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: mitigating valgrind leaks

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: remove rgw_d3n_l1_libaio_read option

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: wip segfault fix

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: mitigate libaio SIGEV_THREAD cb race

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: change libaio signaling mechanism

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: wip cont. libaio cb thread race

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: wip libaio cb thread race

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: libaio cleanups and edge case handling fixes

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: narrow the libaio locking scope

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: add libaio req ordering mechanism

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: fix lock regression

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: addressing reviwe comments and cleasnup

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: libaio locks cleanup

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: refactor libaio abstraction to share the ioc implementation

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: addressing latest review comments and cleanup

Signed-off-by: Mark Kogan <mkogan@redhat.com>

rgw: d3n: address review comments, cont.

Signed-off-by: Mark Kogan <mkogan@redhat.com>
Co-authored-by: Mania Abdi <mania.abdi287@gmail.com>
Co-authored-by: E. Ugur Kaynar <ukaynar@bu.edu>
Co-authored-by: Aishwarya Mathuria <amathuri@redhat.com>
Co-authored-by: Ali Maredia <amaredia@redhat.com>
Co-authored-by: Feng Hualong <hualong.feng@intel.com>

1 files changed, 148 insertions, 0 deletions

diff --git a/src/rgw/rgw_d3n_cacherequest.h b/src/rgw/rgw_d3n_cacherequest.h
new file mode 100644
index 00000000000..1825d0cb0ff
--- /dev/null
+++ b/src/rgw/rgw_d3n_cacherequest.h
@@ -0,0 +1,148 @@
+// -*- mode:C++; tab-width:8; c-basic-offset:2; indent-tabs-mode:t -*-
+// vim: ts=8 sw=2 smarttab ft=cpp
+
+#ifndef RGW_CACHEREQUEST_H
+#define RGW_CACHEREQUEST_H
+
+#include <fcntl.h>
+#include <stdlib.h>
+#include <aio.h>
+
+#include "include/rados/librados.hpp"
+#include "include/Context.h"
+#include "common/async/completion.h"
+
+#include <errno.h>
+#include "common/error_code.h"
+#include "common/errno.h"
+
+#include "rgw_aio.h"
+#include "rgw_cache.h"
+
+
+struct D3nGetObjData {
+  std::mutex d3n_lock;
+};
+
+struct D3nL1CacheRequest {
+  ~D3nL1CacheRequest() {
+    lsubdout(g_ceph_context, rgw_datacache, 30) << "D3nDataCache: " << __func__ << "(): Read From Cache, comlete" << dendl;
+  }
+
+  // unique_ptr with custom deleter for struct aiocb
+  struct libaio_aiocb_deleter {
+    void operator()(struct aiocb* c) {
+      if(c->aio_fildes > 0) {
+        if( ::close(c->aio_fildes) != 0) {
+          lsubdout(g_ceph_context, rgw_datacache, 2) << "D3nDataCache: " << __func__ << "(): Error - can't close file, errno=" << -errno << dendl;
+        }
+      }
+      delete c;
+    }
+  };
+
+  using unique_aio_cb_ptr = std::unique_ptr<struct aiocb, libaio_aiocb_deleter>;
+
+  struct AsyncFileReadOp {
+    bufferlist result;
+    unique_aio_cb_ptr aio_cb;
+    using Signature = void(boost::system::error_code, bufferlist);
+    using Completion = ceph::async::Completion<Signature, AsyncFileReadOp>;
+
+    int init(const DoutPrefixProvider *dpp, const std::string& file_path, off_t read_ofs, off_t read_len, void* arg) {
+      ldpp_dout(dpp, 20) << "D3nDataCache: " << __func__ << "(): file_path=" << file_path << dendl;
+      aio_cb.reset(new struct aiocb);
+      memset(aio_cb.get(), 0, sizeof(struct aiocb));
+      aio_cb->aio_fildes = TEMP_FAILURE_RETRY(::open(file_path.c_str(), O_RDONLY|O_CLOEXEC|O_BINARY));
+      if(aio_cb->aio_fildes < 0) {
+        int err = errno;
+        ldpp_dout(dpp, 1) << "ERROR: D3nDataCache: " << __func__ << "(): can't open " << file_path << " : " << cpp_strerror(err) << dendl;
+        return -err;
+      }
+      if (g_conf()->rgw_d3n_l1_fadvise != POSIX_FADV_NORMAL)
+        posix_fadvise(aio_cb->aio_fildes, 0, 0, g_conf()->rgw_d3n_l1_fadvise);
+
+      bufferptr bp(read_len);
+      aio_cb->aio_buf = bp.c_str();
+      result.append(std::move(bp));
+
+      aio_cb->aio_nbytes = read_len;
+      aio_cb->aio_offset = read_ofs;
+      aio_cb->aio_sigevent.sigev_notify = SIGEV_THREAD;
+      aio_cb->aio_sigevent.sigev_notify_function = libaio_cb_aio_dispatch;
+      aio_cb->aio_sigevent.sigev_notify_attributes = nullptr;
+      aio_cb->aio_sigevent.sigev_value.sival_ptr = arg;
+
+      return 0;
+    }
+
+    static void libaio_cb_aio_dispatch(sigval_t sigval) {
+      lsubdout(g_ceph_context, rgw_datacache, 20) << "D3nDataCache: " << __func__ << "()" << dendl;
+      auto p = std::unique_ptr<Completion>{static_cast<Completion*>(sigval.sival_ptr)};
+      auto op = std::move(p->user_data);
+      const int ret = -aio_error(op.aio_cb.get());
+      boost::system::error_code ec;
+      if (ret < 0) {
+          ec.assign(-ret, boost::system::system_category());
+      }
+
+      ceph::async::dispatch(std::move(p), ec, std::move(op.result));
+    }
+
+    template <typename Executor1, typename CompletionHandler>
+    static auto create(const Executor1& ex1, CompletionHandler&& handler) {
+      auto p = Completion::create(ex1, std::move(handler));
+      return p;
+    }
+  };
+
+  template <typename ExecutionContext, typename CompletionToken>
+  auto async_read(const DoutPrefixProvider *dpp, ExecutionContext& ctx, const std::string& file_path,
+                  off_t read_ofs, off_t read_len, CompletionToken&& token) {
+    using Op = AsyncFileReadOp;
+    using Signature = typename Op::Signature;
+    boost::asio::async_completion<CompletionToken, Signature> init(token);
+    auto p = Op::create(ctx.get_executor(), init.completion_handler);
+    auto& op = p->user_data;
+
+    ldpp_dout(dpp, 20) << "D3nDataCache: " << __func__ << "(): file_path=" << file_path << dendl;
+    int ret = op.init(dpp, file_path, read_ofs, read_len, p.get());
+    if(0 == ret) {
+      ret = ::aio_read(op.aio_cb.get());
+    }
+    ldpp_dout(dpp, 20) << "D3nDataCache: " << __func__ << "(): ::aio_read(), ret=" << ret << dendl;
+    if(ret < 0) {
+      auto ec = boost::system::error_code{-ret, boost::system::system_category()};
+      ceph::async::post(std::move(p), ec, bufferlist{});
+    } else {
+      (void)p.release();
+    }
+    return init.result.get();
+  }
+
+  struct d3n_libaio_handler {
+    rgw::Aio* throttle = nullptr;
+    rgw::AioResult& r;
+    // read callback
+    void operator()(boost::system::error_code ec, bufferlist bl) const {
+      r.result = -ec.value();
+      r.data = std::move(bl);
+      throttle->put(r);
+    }
+  };
+
+  void file_aio_read_abstract(const DoutPrefixProvider *dpp, boost::asio::io_context& context, spawn::yield_context yield,
+                              std::string& file_path, off_t read_ofs, off_t read_len,
+                              rgw::Aio* aio, rgw::AioResult& r) {
+    using namespace boost::asio;
+    async_completion<spawn::yield_context, void()> init(yield);
+    auto ex = get_associated_executor(init.completion_handler);
+
+    auto& ref = r.obj.get_ref();
+    ldpp_dout(dpp, 20) << "D3nDataCache: " << __func__ << "(): oid=" << ref.obj.oid << dendl;
+    async_read(dpp, context, file_path+"/"+ref.obj.oid, read_ofs, read_len, bind_executor(ex, d3n_libaio_handler{aio, r}));
+  }
+
+};
+
+#endif