summaryrefslogtreecommitdiffstats
path: root/src/rgw/rgw_rest.cc (follow)
Commit message (Collapse)AuthorAgeFilesLines
* rgw: cumulatively fix 6 AWS SigV4 request failure casesMatt Benjamin2024-02-131-18/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These changes address checksum header identification and signing algorithm selection, including checksum trailer verification for signed- and unsigned-payload cases. These changes address all the actual S3 request failures I have so far been able to reproduce, with and without content checksums and/or new trailing checksum headers, and with and without SSL. Fixes: https://tracker.ceph.com/issues/63153 Specifically, it fixes the request failures that motivated the initial tracker filing. It extracts but does not validate new client content checksums if present. Validation and management of new S3 content-checksum headers will follow in a subsequent change. Signed-off-by: Matt Benjamin <mbenjamin@redhat.com> squashed commits: * wip chunk meta parsing--seem to have first AWSv4ComplMulti::ChunkMeta::create_next sort of parsing * use constexpr sarlen(...) for static array lengths throughout rgw_auth_s3.cc * link AWSv4CompleMulti::ChunkMeta to its enclosing completer * capture original content-length header before AWSv4ComplMulti overwrites it * mostly extract the trailer * fix misordered content-length, experiment w/exbuf * save leftover bytes between calls to AWSv4ComplMulti::recv_chunk() * propagate data_offset_in_stream from AWSv4ComplMulti::recv_chunk() * clean up trailer section extract * trailer section cleanup and introduce extract_helper * unrolled checksum extract--fixup * fix sv_trailer end pos, and cleanup * add proplist interface to rgw::auth::Completer and AWSv4ComplMulti * spliterate trailers * check completer props * redefine prop_map to point into already-allocated trailer_vec * hax: thread a counter onto AWSv4ComplMulti recv_body() and recv_chunk path * fix apparent bug where due to reads less than chunk_size induce a final, zero-length read that was skipped before forcing recognition of the last chunk in the stream * check only for a trailing checksum named in x-amz-trailer * don't try to match signatures when no signature provided (because streaming unsigned) * oops, fix content_length decl * fix recognition of next chunk envelope in unsigned aws-chunk case * clean up AWSv4CompMulti flags and correctly detect aws unsigned chunked * rework checksum-trailer extraction and introduce AWSv4ComplMulti::calc_v4_trailing_signature * thread const struct req_state* into AWSv4ComplMulti * large cleanup of trailer parsing, no regression * fix trailer signature calculation--checks * correctly generate final chunk hmac * typo in comment * verify trailing signature when expected (using expected final chunk signature) * move trailer_vec back onto recv_body()'s stack * remove strange completer comment * remove last_frag (now points into parsing_buf) * remove implied dependency on content_length * move trailer recognition to AWSv4ComplMulti::complete() * remove now-unused is_last_chunk() predicate * remove unused ChunkMeta::completer * responses to review comments * when trailer is sig expected, fail (only) if none present or if it does not match calculated * remove stale parse_content_length(...) decl * remove now-unused AWSv4ComplMulti::content_length * fix extract_helper end search position as in mut_extract_helper * change "\n" reserve term in get_canon_amz_hdrs() part of the sum (review) and initialize length to 0 * remove debugging code Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
* rgw: fix spelling errorsJosh Soref2023-11-271-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * access * acquire * across * algorithm * alternatively * ambiguous * attributes * authenticated * automatically * based * between * beware * bootstrap * bucket * cache * character * check * cleanup * combinations * committed * comparison * compatibility * concurrent * conditions * conjunction * connections * considered * contains * correct * creates * crypto * dependencies * dequeued * dequeuing * derivative * described * determine * deterministic * detritus * discrepancy * display * either way * elements * entirely * entries * error * everything * expected * explicitly * extend * false * fclose * feature * following * foreign * format * forward * functionality * functions * generated * getting * gracefully * happened * hashicorp * healthcheck * hierarchy * however * http * identify * idleness * ignored * incremental * information * initialization * length * locally * macros * malformed * means * mechanism * multipart * multiple * must * namespace * necessary * nonexistent * notifications * occurred * official * outputs * outputted * overridden * ownership * parallelism * partition * passthrough * pause * penguin * perform * permission * populated * position * preferably * prefixes * presumably * previous * print * quadruple * reference * regardless * registry * renewed * request * requires * reshard * resource * result to * satisfy * secret * separate * separator * setting * shouldn't * skipping * some * somehow * something * specific * struct * succeeded * success * successful * successfully * syntax * testing * truncated * unambiguously * unittest * unmodified * unnecessary * valid_action * verifying * versioning * visible * whatever * whenever Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
* rgw/swift: convert underscores to hyphens in swift user metadata namesJane Zhu2023-11-231-2/+2
| | | | Signed-off-by: Juan Zhu <jzhu4@dev-10-34-20-139.pw1.bcc.bloomberg.com>
* rgw: fix the Content-Length in response header is inconsistent with response ↵mengxiangrui2023-07-261-0/+1
| | | | | | | | | | | | | | body size when rgw returns default html error page in static website The default html error page as response body should be built completely include three ending html symbols(/ul, /body and /html) before rgw computes Content-Length in response header. The Content-Length in response header will be consistent with response body size. Client can get complete page. Fixes: https://tracker.ceph.com/issues/52363 (cherry picked from commit cbeaef9fbe96c930fc10b793773e7ecd137fde81) Co-authored-by: aicun hu <huaicun@chinatelecom.cn> Co-authored-by: yupeng chen <chenyupeng@chinatelecom.cn> Signed-off-by: xiangrui meng <mengxr@chinatelecom.cn>
* rgw: use string_view to parse Accept headerCasey Bodley2023-05-161-9/+7
| | | | | | | | avoid copying the header into a separate buffer to do comparisons Fixes: https://tracker.ceph.com/issues/59490 Signed-off-by: Casey Bodley <cbodley@redhat.com>
* Merge pull request #50507 from cbodley/wip-rgw-api-zeroCasey Bodley2023-05-041-5/+9
|\ | | | | | | | | | | rgw/rest: add 'zero' rest api Reviewed-by: Daniel Gryniewicz <dang@redhat.com> Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
| * rgw/rest: don't require s->formatterCasey Bodley2023-03-131-5/+9
| | | | | | | | Signed-off-by: Casey Bodley <cbodley@redhat.com>
* | rgw: add default Server response headerCasey Bodley2023-03-131-1/+8
|/ | | | | | | | | if a specific rgw_service_provider_name is not configured, add a default Server response header that includes the ceph release name: Server: Ceph Object Gateway (reef) Signed-off-by: Casey Bodley <cbodley@redhat.com>
* rgw: rgw_op.h doesn't include rgw_torrent.hCasey Bodley2023-03-021-3/+0
| | | | Signed-off-by: Casey Bodley <cbodley@redhat.com>
* rgw/torrent: PutObj uses the filter to write RGW_ATTR_TORRENTCasey Bodley2023-03-021-13/+0
| | | | Signed-off-by: Casey Bodley <cbodley@redhat.com>
* rgw: refactor service handlersYuval Lifshitz2023-01-251-0/+3
| | | | | | | seperate between the different non-bucket handler operations: iam, sts, sns (topic) and non bucket s3 ops Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
* Merge pull request #48554 from giubacc/rgw_dns_name_multivalueCasey Bodley2022-12-091-2/+6
|\ | | | | | | | | rgw: add multivalue support to rgw_dns_name config option Reviewed-by: Casey Bodley <cbodley@redhat.com>
| * rgw: add multivalue support to rgw_dns_name config optionGiuseppe Baccini2022-10-211-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | rgw_dns_name configuration option has extended to define multiple domain values. This option is now interpreted as a comma separated list of DNS names. Example: rgw_dns_name = cname.domain.com,cname2.domain2.com, cname3.domain3.com Signed-off-by: Giuseppe Baccini <giuseppe.baccini@suse.com>
* | RGW - Zipper - Rename rgw::sal::Store to rgw::sal::DriverDaniel Gryniewicz2022-12-081-5/+5
|/ | | | | | | | | | | | All along, we've had an overload of the term "store". It was the base class of the entrypoint, and it was the lowest layer in a stack. This renames the base class to Driver. So, the primary elements of zipper that to work are now Drivers, and they come in two different flavors: Filters live in the middle of the stack, and do not need to implement every API, but pass through instead. And Stores live a the bottom of the stack, and must implement the entire API. Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
* rgw/s3: align output format with client requestMatt Benjamin2022-08-161-14/+1
| | | | | | | | | | | | | | | In general, e.g., in the S3 ListBucket response, the output Formatter is already of the type requested in the HTTP Accept header. Therefore, while there are still many instances where the of Formatter is assumed to be XML, it appears necessarily safe and seemingly correct to extend this assumption to end_header(...), which sends the response content-type. Fixes: https://tracker.ceph.com/issues/55680 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
* rgw/rest: replace RGW_FORMAT_ macros with an enum classMatt Benjamin2022-08-161-17/+17
| | | | | | | This is a pure cleanup. The method to print an RGWFormat object as a MIME type is now called to_mime_type(). Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
* rgw: remove unnecessary "struct" from req_state declarationsJ. Eric Ivancich2022-06-221-43/+43
| | | | | | | The "struct" is superfluous and makes it harder for those looking for the definition. Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
* RGW: Zipper - zonegroup, placement tierDaniel Gryniewicz2022-04-201-3/+6
| | | | | | | | | Add ZoneGroup and PlacementTier to the Zipper API. This cleaned up a lot of issues in the Module branch. In addition, add cloud transition. Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
* rgw: Add rgw rate limiting per user and per bucketOr Friedmann2022-01-051-1/+25
| | | | | | Add rgw rate limiting per user and per bucket Signed-off-by: Or Friedmann <ofriedma@redhat.com>
* RGW Zipper - API CleanupsDaniel Gryniewicz2021-11-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | During the documentation pass for the Zipper API, a number of cleanups were found: APIs that should be slightly different, or that were unused entirely. This is a rollup commit of all those cleanups. - move get_multipart_upload() to Bucket - remove unused defer_gc - move create_bucket() into User - rename get_bucket_info() to load_bucket() to match load_user() - Remove read_bucket_stats() The codepaths using read_bucket_stats() used CLS data types, and the function is confusingly named. Load the ent in load_bucket(), and use an alternative data structure to get size stats for the bucket. - rename get_bucket_stats to read_stats - Remove remove_metadata() from API - remove copy_obj_data() from API - rename get_obj_layout to dump_obj_layout - use SAL range_to_ofs Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
* rgw/rest: dump_owner() takes name by const refCasey Bodley2021-10-291-1/+1
| | | | Signed-off-by: Casey Bodley <cbodley@redhat.com>
* rgw/rest: dump_time() takes real_time by valueCasey Bodley2021-10-291-2/+2
| | | | Signed-off-by: Casey Bodley <cbodley@redhat.com>
* rgw/rgw_rest: do not set unused variableKefu Chai2021-08-161-2/+1
| | | | | | | | | | | | | `len` is used to track the total length of received data, but we already collect the data in `bl`, so we can always check `bl` for the total length. this change also silences following warning from Clang: ../src/rgw/rgw_rest.cc:1477:21: warning: variable 'len' set but not used [-Wunused-but-set-variable] int read_len = 0, len = 0; ^ 1 warning generated. Signed-off-by: Kefu Chai <kchai@redhat.com>
* rgw: build without "using namespace std"Kefu Chai2021-08-131-0/+2
| | | | | | | | | | * add "std::" prefix in headers * add "using" declarations in .cc files. so we don't rely on "using namespace std" in one or more included headers. Signed-off-by: Kefu Chai <kchai@redhat.com>
* RGW - Zipper - MultipartUploadDaniel Gryniewicz2021-07-281-2/+8
| | | | | | Create a MultipartUpload object in the Zipper API. Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
* rgw: add DPP's to logging for most opsAli Maredia2021-05-041-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | This commit contains the following major changes: - Most log output lines for major ops now have DoutPrefixProviders. - Create new macro ldpp_subdout. This macro is meant as a replacement for lsubdout for headerfiles that do not define the rgw subsys. - Changes to RGWObjManifest begin and end iterators. - Make it so that rgw_dencoder.cc should only contain the logic related to encoding. Also add dpp to ldouts and lderr's already using req_state and replace sync_env/env->dpp's with dpp's Signed-off-by: Ali Maredia <amaredia@redhat.com> Signed-off-by: Kalpesh Pandya <kapandya@redhat.com> Signed-off-by: Casey Bodley <cbodley@redhat.com>
* RGW Zipper - move attrs into UserDaniel Gryniewicz2021-04-281-3/+2
| | | | Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
* RGW Zipper - Fix style violationsDaniel Gryniewicz2021-04-151-1/+1
| | | | | | Change type *var; to type* var; Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
* RGW Zipper - The Great RenameDaniel Gryniewicz2021-04-151-2/+2
| | | | | | | Rename RGWFoo to Foo. It's all in the rgw::sal namespace anyway, so the RGW is redundant. Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
* rgw: handle aws4 completion when reading all op dataYehuda Sadeh2021-03-111-6/+6
| | | | | | and add a few ops to the list of ops that complete aws4 signature. Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
* RGW - Zipper 10: Pull The ThreadDaniel Gryniewicz2021-03-021-4/+4
| | | | | | | | This commit changes the RGWStoreManager to return a RGWStore* rather than a RGWRadosStore*. This is the thread that unravels the rest of the Zipper work, removing hard-coded uses of the RGWRados* classes. Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
* rgw: Add DoutPrefixProvider to create_bucket loggingAli Maredia2021-01-271-26/+26
| | | | | | | | | | | | | This commit hits all log messages for a create bucket request and ensures each has a DoutPrefixProvider. This commit gets dpp's all the way down to rgw/services up way though to rgw_op.cc, rgw_rest.cc files, rgw_admin.cc, etc. This commit also removes store as dpp in rgw_sal.h Signed-off-by: Ali Maredia <amaredia@redhat.com>
* rgw: No null_yield in RGWOpAdam C. Emerson2020-11-131-20/+20
| | | | Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
* rgw: RGWRESTOp no longer tracks separate error codeCasey Bodley2020-09-101-1/+1
| | | | | | | | | | | RGWOp_Bucket_Remove::execute() was storing failures from bucket->remove_bucket() in op_ret, but left http_ret=0 so we responded to the client with '200 OK' to avoid bugs like this, remove the extra http_ret variable and only use the op_ret from RGWOp Signed-off-by: Casey Bodley <cbodley@redhat.com>
* Zipper - Split RadosStore from base SALDaniel Gryniewicz2020-09-081-0/+1
| | | | Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
* Zipper ObjectDaniel Gryniewicz2020-07-171-6/+6
| | | | | | | | This pass modifies RGW to use RGWObject and RGWRadosObject. Also improves RGWBucket usage. This converts many of the APIs to unique_ptr. Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
* Merge pull request #33228 from galsalomon66/s3select_integration_into_RGW_2Matt Benjamin2020-06-251-0/+1
|\ | | | | rgw: add s3select support
| * adding s3select sub-module; integrating sub-module into RGW; current commit ↵gal salomon2020-06-231-0/+1
| | | | | | | | | | | | is able to run s3-select queries on CSV-s3object; Signed-off-by: gal salomon <gal.salomon@gmail.com>
* | rgw: Replace boost::string_ref/view with std::string_viewPrateek Sachan2020-06-181-11/+11
|/ | | | | | | This commit replaces boost's string_ref and string_view cpp's in built std::string_view Fixes: https://tracker.ceph.com/issues/39619 Signed-off-by: Prateek Sachan <psachan@cs.iitr.ac.in>
* rgw: append user policies to existing policiesAbhishek Lekshmanan2020-03-111-7/+4
| | | | | | Drop emptiness check and then appending to the policy Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
* Merge pull request #30778 from joke-lee/rgw-ipv6Casey Bodley2020-01-271-3/+11
|\ | | | | | | | | | | rgw: fix SignatureDoesNotMatch when use ipv6 address in s3 client Reviewed-By: Casey Bodley <cbodley@redhat.com> Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
| * rgw: fix SignatureDoesNotMatch when use ipv6 address in s3 clientyuliyang2019-10-081-3/+11
| | | | | | | | | | | | fix: https://tracker.ceph.com/issues/42218 Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>
* | Zipper - RGWUserDaniel Gryniewicz2020-01-171-7/+7
| | | | | | | | Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
* | rgw: support encoding-type param for list bucket multipartsAbhishek Lekshmanan2019-10-181-0/+10
|/ | | | | | | | url encode keys, upload ids & prefixes based on encoding-type, reject invalid encoding types at the param parsing phase Fixes: https://tracker.ceph.com/issues/42372 Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
* rgw: fix dns name comparison for virtual hostingCasey Bodley2019-09-061-2/+3
| | | | | | Fixes: https://tracker.ceph.com/issues/41692 Signed-off-by: Casey Bodley <cbodley@redhat.com>
* RGW - Make sure editor settings are correct for all filesDaniel Gryniewicz2019-08-191-1/+1
| | | | Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
* Merge pull request #29380 from theanalyst/rgw-dns-name-fixesCasey Bodley2019-08-161-3/+3
|\ | | | | | | | | rgw: make dns hostnames matching case insensitive Reviewed-by: Casey Bodley <cbodley@redhat.com>
| * rgw: make dns hostnames matching case insensitiveAbhishek Lekshmanan2019-07-291-3/+3
| | | | | | | | | | | | | | | | | | Currently when parsing host in requests, we try to case match against supplied hostnames set, which violates dns hostnames being case insensitive. Do a case insensitive comparision instead Fixes: http://tracker.ceph.com/issues/40995 Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
* | Project Zipper Part 1 - Framework and RGWRadosStoreDaniel Gryniewicz2019-08-151-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | This is the first part of Project Zipper, the Store Abstraction Layer. It introduces the basic framework, and wraps RGWRados in RGWRadosStore. The goal over the next few weeks is to do the same for user, bucket, and object. This will make most of the remaining users of RGWRados wrapped in SAL classes, allowing it to be completely absorbed into the private RGWRadosStore. This will also expose all the APIs that need to be pusheg up to higher layers in the SAL. Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
* | rgw: fixes following rebaseYehuda Sadeh2019-07-291-1/+1
| | | | | | | | Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-19 10:08:24 +0100