summaryrefslogtreecommitdiffstats
path: root/convert.c (unfollow)
Commit message (Collapse)AuthorFilesLines
2022-10-12bundle-uri: create "key=value" line parsingÆvar Arnfjörð Bjarmason2-1/+38
When advertising a bundle list over Git's protocol v2, we will use packet lines. Each line will be of the form "key=value" representing a bundle list. Connect the API necessary for Git's transport to the key-value pair parsing created in the previous change. We are not currently implementing this protocol v2 functionality, but instead preparing to expose this parsing to be unit-testable. Co-authored-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-12bundle-uri: create base key-value pair parsingDerrick Stolee5-1/+104
There will be two primary ways to advertise a bundle list: as a list of packet lines in Git's protocol v2 and as a config file served from a bundle URI. Both of these fundamentally use a list of key-value pairs. We will use the same set of key-value pairs across these formats. Create a new bundle_list_update() method that is currently unusued, but will be used in the next change. It inspects each key to see if it is understood and then applies it to the given bundle_list. Here are the keys that we teach Git to understand: * bundle.version: This value should be an integer. Git currently understands only version 1 and will ignore the list if the version is any other value. This version can be increased in the future if we need to add new keys that Git should not ignore. We can add new "heuristic" keys without incrementing the version. * bundle.mode: This value should be one of "all" or "any". If this mode is not understood, then Git will ignore the list. This mode indicates whether Git needs all of the bundle list items to make a complete view of the content or if any single item is sufficient. The rest of the keys use a bundle identifier "<id>" as part of the key name. Keys using the same "<id>" describe a single bundle list item. * bundle.<id>.uri: This stores the URI of the bundle item. This currently is expected to be an absolute URI, but will be relaxed to be a relative URI in the future. While parsing, return an error if a URI key is repeated, since we can make that restriction with bundle lists. Make the git_parse_int() method global so we can parse the integer version value carefully. Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-12bundle-uri: create bundle_list struct and helpersDerrick Stolee2-0/+116
It will likely be rare where a user uses a single bundle URI and expects that URI to point to a bundle. Instead, that URI will likely be a list of bundles provided in some format. Alternatively, the Git server could advertise a list of bundles. In anticipation of these two ways of advertising multiple bundles, create a data structure that represents such a list. This will be populated using a common API, but for now focus on what data can be represented. Each list contains a number of remote_bundle_info structs. These contain an 'id' that is used to uniquely identify them in the list, and also a 'uri' that contains the location of its data. Finally, there is a strbuf containing the filename used when Git downloads the contents to disk. The list itself stores these remote_bundle_info structs in a hashtable using 'id' as the key. The order of the structs in the input is considered unimportant, but future modifications to the format and these data structures will place ordering possibilities on the set. The list also has a few "global" properties, including the version (used when parsing the list) and the mode. The mode is one of these two options: 1. BUNDLE_MODE_ALL: all listed URIs are intended to be combined together. The client should download all of the advertised data to have a complete copy of the data. 2. BUNDLE_MODE_ANY: any one listed item is sufficient to have a complete copy of the data. The client can choose arbitrarily from these options. In the future, the client may use pings to find the closest URI among geodistributed replicas, or use some other heuristic information added to the format. This API is currently unused, but will soon be expanded with parsing logic and then be consumed by the bundle URI download logic. Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-12bundle-uri: use plain string in find_temp_filename()Derrick Stolee1-12/+16
The find_temp_filename() method was created in 53a50892be2 (bundle-uri: create basic file-copy logic, 2022-08-09) and uses odb_mkstemp() to create a temporary filename. The odb_mkstemp() method uses a strbuf in its interface, but we do not need to continue carrying a strbuf throughout the bundle URI code. Convert the find_temp_filename() method to use a 'char *' and modify its only caller. This makes sense that we don't actually need to modify this filename directly later, so using a strbuf is overkill. This change will simplify the data structure for tracking a bundle list to use plain strings instead of strbufs. Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-24clone: warn on failure to repo_init()Derrick Stolee1-2/+3
The --bundle-uri option was added in 55568919616 (clone: add --bundle-uri option, 2022-08-09), but this also introduced a call to repo_init() whose return value was ignored. Fix that ignored value by warning that the bundle URI process could not continue if it failed. Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-18The fourteenth batchJunio C Hamano1-0/+24
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-15The thirteenth batchJunio C Hamano1-0/+12
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-12The twelfth batchJunio C Hamano1-0/+4
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-11rev-list: support human-readable output for `--disk-usage`Li Linchao3-4/+57
The '--disk-usage' option for git-rev-list was introduced in 16950f8384 (rev-list: add --disk-usage option for calculating disk usage, 2021-02-09). This is very useful for people inspect their git repo's objects usage infomation, but the resulting number is quit hard for a human to read. Teach git rev-list to output a human readable result when using '--disk-usage'. Signed-off-by: Li Linchao <lilinchao@oschina.cn> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-11Git 2.37.2v2.37.2Junio C Hamano2-1/+25
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10fsck: downgrade tree badFilemode to "info"Jeff King2-1/+18
The previous commit un-broke the "badFileMode" check; before then it was literally testing nothing. And as far as I can tell, it has been so since the very initial version of fsck. The current severity of "badFileMode" is just "warning". But in the --strict mode used by transfer.fsckObjects, that is elevated to an error. This will potentially cause hassle for users, because historical objects with bad modes will suddenly start causing pushes to many server operators to be rejected. At the same time, these bogus modes aren't actually a big risk. Because we canonicalize them everywhere besides fsck, they can't cause too much mischief in the real world. The worst thing you can do is end up with two almost-identical trees that have different hashes but are interpreted the same. That will generally cause things to be inefficient rather than wrong, and is a bug somebody working on a Git implementation would want to fix, but probably not worth inconveniencing users by refusing to push or fetch. So let's downgrade this to "info" by default, which is our setting for "mention this when fscking, but don't ever reject, even under strict mode". If somebody really wants to be paranoid, they can still adjust the level using config. Suggested-by: Xavier Morel <xavier.morel@masklinn.net> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10fsck: actually detect bad file modes in treesJeff King2-1/+15
We use the normal tree_desc code to iterate over trees in fsck, meaning we only see the canonicalized modes it returns. And hence we'd never see anything unexpected, since it will coerce literally any garbage into one of our normal and accepted modes. We can use the new RAW_MODES flag to see the real modes, and then use the existing code to actually analyze them. The existing code is written as allow-known-good, so there's not much point in testing a variety of breakages. The one tested here should be S_IFREG but with nonsense permissions. Do note that the error-reporting here isn't great. We don't mention the specific bad mode, but just that the tree has one or more broken modes. But when you go to look at it with "git ls-tree", we'll report the canonicalized mode! This isn't ideal, but given that this should come up rarely, and that any number of other tree corruptions might force you into looking at the binary bytes via "cat-file", it's not the end of the world. And it's something we can improve on top later if we choose. Reported-by: Xavier Morel <xavier.morel@masklinn.net> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10tree-walk: add a mechanism for getting non-canonicalized modesJeff King4-9/+19
When using init_tree_desc() and tree_entry() to iterate over a tree, we always canonicalize the modes coming out of the tree. This is a good thing to prevent bugs or oddities in normal code paths, but it's counter-productive for tools like fsck that want to see the exact contents. We can address this by adding an option to avoid the extra canonicalization. A few notes on the implementation: - I've attached the new option to the tree_desc struct itself. The actual code change is in decode_tree_entry(), which is in turn called by the public update_tree_entry(), tree_entry(), and init_tree_desc() functions, plus their "gently" counterparts. By letting it ride along in the struct, we can avoid changing the signature of those functions, which are called many times. Plus it's conceptually simpler: you really want a particular iteration of a tree to be "raw" or not, rather than individual calls. - We still have to set the new option somewhere. The struct is initialized by init_tree_desc(). I added the new flags field only to the "gently" version. That avoids disturbing the much more numerous non-gentle callers, and it makes sense that anybody being careful about looking at raw modes would also be careful about bogus trees (i.e., the caller will be something like fsck in the first place). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10clone: --bundle-uri cannot be combined with --depthDerrick Stolee3-1/+13
A previous change added the '--bundle-uri' option, but did not check if the --depth parameter was included. Since bundles are not compatible with shallow clones, provide an error message to the user who is attempting this combination. I am leaving this as its own change, separate from the one that implements '--bundle-uri', because this is more of an advisory for the user. There is nothing wrong with bootstrapping with bundles and then fetching a shallow clone. However, that is likely going to involve too much work for the client _and_ the server. The client will download all of this bundle information containing the full history of the repository only to ignore most of it. The server will get a shallow fetch request, but with a list of haves that might cause a more painful computation of that shallow pack-file. Reviewed-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10bundle-uri: add support for http(s):// and file://Derrick Stolee2-3/+112
The previous change created the 'git clone --bundle-uri=<uri>' option. Currently, <uri> must be a filename. Update copy_uri_to_file() to first inspect the URI for an HTTP(S) prefix and use git-remote-https as the way to download the data at that URI. Otherwise, check to see if file:// is present and modify the prefix accordingly. Reviewed-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10clone: add --bundle-uri optionDerrick Stolee3-0/+57
Cloning a remote repository is one of the most expensive operations in Git. The server can spend a lot of CPU time generating a pack-file for the client's request. The amount of data can clog the network for a long time, and the Git protocol is not resumable. For users with poor network connections or are located far away from the origin server, this can be especially painful. Add a new '--bundle-uri' option to 'git clone' to bootstrap a clone from a bundle. If the user is aware of a bundle server, then they can tell Git to bootstrap the new repository with these bundles before fetching the remaining objects from the origin server. Reviewed-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10bundle-uri: create basic file-copy logicDerrick Stolee4-0/+121
Before implementing a way to fetch bundles into a repository, create the basic logic. Assume that the URI is actually a file path. Future logic will make this more careful to other protocols. For now, we also only succeed if the content at the URI is a bundle file, not a bundle list. Bundle lists will be implemented in a future change. Note that the discovery of a temporary filename is slightly racy because the odb_mkstemp() relies on the temporary file not existing. With the current implementation being limited to file copies, we could replace the copy_file() with copy_fd(). The tricky part comes in future changes that send the filename to 'git remote-https' and its 'get' capability. At that point, we need the file descriptor closed _and_ the file unlinked. If we were to keep the file descriptor open for the sake of normal file copies, then we would pollute the rest of the code for little benefit. This is especially the case because we expect that most bundle URI use will be based on HTTPS instead of file copies. Reviewed-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10remote-curl: add 'get' capabilityDerrick Stolee3-0/+74
A future change will want a way to download a file over HTTP(S) using the simplest of download mechanisms. We do not want to assume that the server on the other side understands anything about the Git protocol but could be a simple static web server. Create the new 'get' capability for the remote helpers which advertises that the 'get' command is avalable. A caller can send a line containing 'get <url> <path>' to download the file at <url> into the file at <path>. Reviewed-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10bundle-uri: add example bundle organizationDerrick Stolee1-0/+105
The previous change introduced the bundle URI design document. It creates a flexible set of options that allow bundle providers many ways to organize Git object data and speed up clones and fetches. It is particularly important that we have flexibility so we can apply future advancements as new ideas for efficiently organizing Git data are discovered. However, the design document does not provide even an example of how bundles could be organized, and that makes it difficult to envision how the feature should work at the end of the implementation plan. Add a section that details how a bundle provider could work, including using the Git server advertisement for multiple geo-distributed servers. This organization is based on the GVFS Cache Servers which have successfully used similar ideas to provide fast object access and reduced server load for very large repositories. Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10docs: document bundle URI standardDerrick Stolee2-0/+469
Introduce the idea of bundle URIs to the Git codebase through an aspirational design document. This document includes the full design intended to include the feature in its fully-implemented form. This will take several steps as detailed in the Implementation Plan section. By committing this document now, it can be used to motivate changes necessary to reach these final goals. The design can still be altered as new information is discovered. Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10mergetools: vimdiff: simplify tabfirstFelipe Contreras1-25/+19
If we wrap the tabdo command there's no need for a separate command call. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Reviewed-by: Fernando Ramos <greenfoo@u92.eu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10mergetools: vimdiff: fix single window layoutsFelipe Contreras1-12/+8
Layouts with a single window other than "MERGED" do not work (e.g. "LOCAL" or "MERGED+LOCAL"). This is because as the documentation of bufdo says: The last buffer (or where an error occurred) becomes the current buffer. And we do always do bufdo the end. Additionally, we do it only once, when it should be per tab. Fix this by doing it once per tab right after it's created and before any buffer is switched. Cc: Fernando Ramos <greenfoo@u92.eu> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Reviewed-by: Fernando Ramos <greenfoo@u92.eu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10mergetools: vimdiff: rework tab logicFelipe Contreras1-28/+22
If we treat tabs especially, the logic becomes much simpler. Cc: Fernando Ramos <greenfoo@u92.eu> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Reviewed-by: Fernando Ramos <greenfoo@u92.eu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10mergetools: vimdiff: fix for diffoptFelipe Contreras1-18/+18
When diffopt has hiddenoff set and there's only one window (as is the case in the single window mode) the diff mode is turned off. We don't want that, so turn that option off. Cc: Fernando Ramos <greenfoo@u92.eu> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Reviewed-by: Fernando Ramos <greenfoo@u92.eu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10mergetools: vimdiff: silence annoying messagesFelipe Contreras1-2/+2
When using the single window mode we are greeted with the following warning: "./content_LOCAL_8975" 6L, 28B "./content_BASE_8975" 6 lines, 29 bytes "./content_REMOTE_8975" 6 lines, 29 bytes "content" 16 lines, 115 bytes Press ENTER or type command to continue every time. Silence that. Suggested-by: Fernando Ramos <greenfoo@u92.eu> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Reviewed-by: Fernando Ramos <greenfoo@u92.eu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10mergetools: vimdiff: make vimdiff3 actually workFelipe Contreras1-18/+18
When vimdiff3 was added in 7c147b77d3 (mergetools: add vimdiff3 mode, 2014-04-20), the description made clear the intention: It's similar to the default, except that the other windows are hidden. This ensures that removed/added colors are still visible on the main merge window, but the other windows not visible. However, in 0041797449 (vimdiff: new implementation with layout support, 2022-03-30) this was broken by generating a command that never creates windows, and therefore vim never shows the diff. The layout support implementation broke the whole purpose of vimdiff3, and simply shows MERGED, which is no different from simply opening the file with vim. In order to show the diff, the windows need to be created first, and then when they are hidden the diff remains (if hidenoff isn't set), but by setting the `hidden` option the initial buffers are marked as hidden thus making the feature work. Suggested-by: Fernando Ramos <greenfoo@u92.eu> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Reviewed-by: Fernando Ramos <greenfoo@u92.eu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10mergetools: vimdiff: fix commentFelipe Contreras1-2/+2
The name of the variable is wrong, and it can be set to anything, like 1. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Reviewed-by: Fernando Ramos <greenfoo@u92.eu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10doc add: renormalize is not idempotent for CRCRLFPhilip Oakley1-1/+3
Bug report https://lore.kernel.org/git/AM0PR02MB56357CC96B702244F3271014E8DC9@AM0PR02MB5635.eurprd02.prod.outlook.com/ noted that a file containing /r/r/n needed renormalising twice. This is by design. Lone CR characters, not paired with an LF, are left unchanged. Note this limitation of the "clean" filter in the documentation. Renormalize was introduced at 9472935d81e (add: introduce "--renormalize", Torsten Bögershausen, 2017-11-16) Signed-off-by: Philip Oakley <philipoakley@iee.email> Reviewed-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-08rm: integrate with sparse-indexShaoxuan Yuan3-3/+20
Enable the sparse index within the `git-rm` command. The `p2000` tests demonstrate a ~92% execution time reduction for 'git rm' using a sparse index. Test HEAD~1 HEAD -------------------------------------------------------------------------- 2000.74: git rm ... (full-v3) 0.41(0.37+0.05) 0.43(0.36+0.07) +4.9% 2000.75: git rm ... (full-v4) 0.38(0.34+0.05) 0.39(0.35+0.05) +2.6% 2000.76: git rm ... (sparse-v3) 0.57(0.56+0.01) 0.05(0.05+0.00) -91.2% 2000.77: git rm ... (sparse-v4) 0.57(0.55+0.02) 0.03(0.03+0.00) -94.7% ---- Also, normalize a behavioral difference of `git-rm` under sparse-index. See related discussion [1]. `git-rm` a sparse-directory entry within a sparse-index enabled repo behaves differently from a sparse directory within a sparse-checkout enabled repo. For example, in a sparse-index repo, where 'folder1' is a sparse-directory entry, `git rm -r --sparse folder1` provides this: rm 'folder1/' Whereas in a sparse-checkout repo *without* sparse-index, doing so provides this: rm 'folder1/0/0/0' rm 'folder1/0/1' rm 'folder1/a' Because `git rm` a sparse-directory entry does not need to expand the index, therefore we should accept the current behavior, which is faster than "expand the sparse-directory entry to match the sparse-checkout situation". Modify a previous test so such difference is not considered as an error. [1] https://github.com/ffyuanda/git/pull/6#discussion_r934861398 Helped-by: Victoria Dye <vdye@github.com> Helped-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Shaoxuan Yuan <shaoxuan.yuan02@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-08rm: expand the index only when necessaryShaoxuan Yuan2-4/+28
Remove the `ensure_full_index()` method so `git-rm` does not always expand the index when the expansion is unnecessary, i.e. when <pathspec> does not have any possibilities to match anything outside of sparse-checkout definition. Expand the index when the <pathspec> needs an expanded index, i.e. the <pathspec> contains wildcard that may need a full-index or the <pathspec> is simply outside of sparse-checkout definition. Notice that the test 'rm pathspec expands index when necessary' in t1092 *is* testing this code change behavior, though it will be marked as 'test_expect_success' only in the next patch, where we officially mark `command_requires_full_index = 0`, so the index does not expand unless we tell it to do so. Notice that because we also want `ensure_full_index` to record the stdout and stderr from Git command, a corresponding modification is also included in this patch. The reason we want the "sparse-index-out" and "sparse-index-err", is that we need to make sure there is no error from Git command itself, so we can rely on the `test_region` result and determine if the index is expanded or not. Helped-by: Victoria Dye <vdye@github.com> Helped-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Shaoxuan Yuan <shaoxuan.yuan02@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-08pathspec.h: move pathspec_needs_expanded_index() from reset.c to hereShaoxuan Yuan3-83/+102
Method pathspec_needs_expanded_index() in reset.c from 4d1cfc1351 (reset: make --mixed sparse-aware, 2021-11-29) is reusable when we need to verify if the index needs to be expanded when the command is utilizing a pathspec rather than a literal path. Move it to pathspec.h for reusability. Add a few items to the function so it can better serve its purpose as a standalone public function: * Add a check in front so if the index is not sparse, return early since no expansion is needed. * It now takes an arbitrary 'struct index_state' pointer instead of using `the_index` and `active_cache`. * Add documentation to the function. Helped-by: Victoria Dye <vdye@github.com> Helped-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Shaoxuan Yuan <shaoxuan.yuan02@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-08t1092: add tests for `git-rm`Shaoxuan Yuan1-0/+57
Add tests for `git-rm`, make sure it behaves as expected when <pathspec> is both inside or outside of sparse-checkout definition. Helped-by: Victoria Dye <vdye@github.com> Helped-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Shaoxuan Yuan <shaoxuan.yuan02@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-08unpack-trees: unpack new trees as sparse directoriesVictoria Dye2-10/+113
If 'unpack_single_entry()' is unpacking a new directory tree (that is, one not already present in the index) into a sparse index, unpack the tree as a sparse directory rather than traversing its contents and unpacking each file individually. This helps keep the sparse index as collapsed as possible in cases such as 'git reset --hard' restoring a outside-of-cone directory removed with 'git rm -r --sparse'. Without this patch, 'unpack_single_entry()' will only unpack a directory into the index as a sparse directory (rather than traversing into it and unpacking its files one-by-one) if an entry with the same name already exists in the index. This patch allows sparse directory unpacking without a matching index entry when the following conditions are met: 1. the directory's path is outside the sparse cone, and 2. there are no children of the directory in the index If a directory meets these requirements (as determined by 'is_new_sparse_dir()'), 'unpack_single_entry()' unpacks the sparse directory index entry and propagates the decision back up to 'unpack_callback()' to prevent unnecessary tree traversal into the unpacked directory. Reported-by: Shaoxuan Yuan <shaoxuan.yuan02@gmail.com> Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-08cache.h: create 'index_name_pos_sparse()'Victoria Dye2-0/+14
Add 'index_name_pos_sparse()', which behaves the same as 'index_name_pos()', except that it does not expand a sparse index to search for an entry inside a sparse directory. 'index_entry_exists()' was originally implemented in 20ec2d034c (reset: make sparse-aware (except --mixed), 2021-11-29) as an alternative to 'index_name_pos()' to allow callers to search for an index entry without expanding a sparse index. However, that particular use case only required knowing whether the requested entry existed, so 'index_entry_exists()' does not return the index positioning information provided by 'index_name_pos()'. This patch implements 'index_name_pos_sparse()' to accommodate callers that need the positioning information of 'index_name_pos()', but do not want to expand the index. Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-08oneway_diff: handle removed sparse directoriesVictoria Dye1-0/+5
Update 'do_oneway_diff()' to perform a 'diff_tree_oid()' on removed sparse directories, as it does for added or modified sparse directories (see 9eb00af562 (diff-lib: handle index diffs with sparse dirs, 2021-07-14)). At the moment, this update is unreachable code because 'unpack_trees()' (currently the only way 'oneway_diff()' can be called, via 'diff_cache()') will always traverse trees down to the individual removed files of a deleted sparse directory. A subsequent patch will change this to better preserve a sparse index in other uses of 'unpack_tree()', e.g. 'git reset --hard'. However, making that change without this patch would result in (among other issues) 'git status' printing only the name of a deleted sparse directory, not its contents. To avoid introducing that bug, 'do_oneway_diff()' is updated before modifying 'unpack_trees()'. Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-08checkout: fix nested sparse directory diff in sparse indexVictoria Dye2-0/+9
Add the 'recursive' diff flag to the local changes reporting done by 'git checkout' in 'show_local_changes()'. Without the flag enabled, unexpanded sparse directories will not be recursed into to report the diff of each file's contents, resulting in the reported local changes including "modified" sparse directories. The same issue was found and fixed for 'git status' in 2c521b0e49 (status: fix nested sparse directory diff in sparse index, 2022-03-01) Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-08The eleventh batchJunio C Hamano1-0/+20
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-08mingw: handle a file owned by the Administrators group correctlyJohannes Schindelin1-0/+10
When an Administrator creates a file or directory, the created file/directory is owned not by the Administrator SID, but by the _Administrators Group_ SID. The reason is that users with administrator privileges usually run in unprivileged ("non-elevated") mode, and their user SID does not change when running in elevated mode. This is is relevant e.g. when running a GitHub workflow on a build agent, which runs in elevated mode: cloning a Git repository in a script step will cause the worktree to be owned by the Administrators Group SID, for example. Let's handle this case as following: if the current user is an administrator, Git should consider a worktree owned by the Administrators Group as if it were owned by said user. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-08mingw: be more informative when ownership check fails on FAT32Johannes Schindelin1-1/+24
The FAT file system has no concept of ACLs. Therefore, it cannot store any ownership information anyway, and the `GetNamedSecurityInfoW()` call pretends that everything is owned "by the world". Let's special-case that scenario and tell the user what's going on. This addresses https://github.com/git-for-windows/git/issues/3886 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-08mingw: provide details about unsafe directories' ownershipJohannes Schindelin1-0/+24
When Git refuses to use an existing repository because it is owned by someone else than the current user, it can be a bit tricky on Windows to figure out what is going on. Let's help with that by providing more detailed information. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-08setup: prepare for more detailed "dubious ownership" messagesJohannes Schindelin4-13/+21
When verifying the ownership of the Git directory, we sometimes would like to say a bit more about it, e.g. when using a platform-dependent code path (think: Windows has the permission model that is so different from Unix'), but only when it is a appropriate to actually say something. To allow for that, collect that information and hand it back to the caller (whose responsibility it is to show it or not). Note: We do not actually fill in any platform-dependent information yet, this commit just adds the infrastructure to be able to do so. Based-on-an-idea-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-08setup: fix some formattingJohannes Schindelin1-4/+5
In preparation for touching code that was introduced in 3b0bf2704980 (setup: tighten ownership checks post CVE-2022-24765, 2022-05-10) and that was formatted differently than preferred in the Git project, fix the indentation before actually modifying the code. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-06The tenth batchJunio C Hamano1-8/+11
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-06Downmerge a bit more for 2.37.xJunio C Hamano1-0/+17
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-05hook API: don't segfault on strbuf_addf() to NULL "out"Ævar Arnfjörð Bjarmason2-7/+26
Fix a logic error in a082345372e (hook API: fix v2.36.0 regression: hooks should be connected to a TTY, 2022-06-07). When it started using the "ungroup" API added in fd3aaf53f71 (run-command: add an "ungroup" option to run_process_parallel(), 2022-06-07) it should have made the same sort of change that fd3aaf53f71 itself made in "t/helper/test-run-command.c". The correct way to emit this "Couldn't start" output with "ungroup" would be: fprintf(stderr, _("Couldn't start hook '%s'\n"), hook_path); But we should instead remove the emitting of this output. As the added test shows we already emit output when we can't run the child. The "cannot run" output here is emitted by run-command.c's child_err_spew(). So the addition of the "Couldn't start hook" output here in 96e7225b310 (hook: add 'run' subcommand, 2021-12-22) was always redundant. For the pre-commit hook we'll now emit exactly the same output as we did before f443246b9f2 (commit: convert {pre-commit,prepare-commit-msg} hook to hook.h, 2021-12-22) (and likewise for others). We could at this point add this to the pick_next_hook() callbacks in hook.c: assert(!out); assert(!*pp_task_cb); And this to notify_start_failure() and notify_hook_finished() (in the latter case the parameter is called "pp_task_cp"): assert(!out); assert(!pp_task_cb); But let's leave any such instrumentation for some eventual cleanup of the "ungroup" API. Reported-by: Ilya K <me@0upti.me> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Reviewed-by: Emily Shaffer <emilyshaffer@google.com> Reviewed-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-04docs: move http-protocol docs to man section 5Ævar Arnfjörð Bjarmason6-11/+33
Continue the move of existing Documentation/technical/* protocol and file-format documentation into our main documentation space by moving the http-protocol.txt documentation over. I'm renaming it to "protocol-http" to be consistent with other things in the new gitformat-protocol-* namespace. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-04docs: move cruft pack docs to gitformat-packÆvar Arnfjörð Bjarmason3-124/+126
Integrate the cruft packs documentation initially added in 3d89a8c1180 (Documentation/technical: add cruft-packs.txt, 2022-05-20) to the newly created "gitformat-pack" documentation. Like the "bitmap-format" added before it in 0d4455a3ab0 (documentation: add documentation for the bitmap format, 2013-11-14) the "cruft-packs" were documented in their own file. As the diff move detection will show there is no change to "Documentation/technical/cruft-packs.txt" here except to move it, and to "indent" the existing sections by adding an extra "=" to them. We could similarly convert the "bitmap-format.txt", but let's leave it for now due to a conflict with the in-flight ac/bitmap-lookup-table series. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-04docs: move pack format docs to man section 5Ævar Arnfjörð Bjarmason15-22/+68
Continue the move of existing Documentation/technical/* protocol and file-format documentation into our main documentation space by moving the various documentation pertaining to the *.pack format and related files, and updating things that refer to it to link to the new location. By moving these we can properly link from the newly created gitformat-commit-graph to a gitformat-chunk-format page. Integrating "Documentation/technical/bitmap-format.txt" and "Documentation/technical/cruft-packs.txt" might logically be part of this change, but as those cover parts of the wider "pack format" (including associated files) that's documented outside of "Documentation/technical/pack-format.txt" let's leave those for now, subsequent commit(s) will address those. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-04docs: move signature docs to man section 5Ævar Arnfjörð Bjarmason3-4/+20
Continue the move of existing Documentation/technical/* protocol and file-format documentation into our main documentation space by moving the signature format documentation. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-04docs: move index format docs to man section 5Ævar Arnfjörð Bjarmason3-2/+21
Continue the move of existing Documentation/technical/* protocol and file-format documentation into our main documentation space by moving the index format documentation. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>