Web lists-archives.com

Re: test suite: why does git add large_file create a pack, rather than an object?

hi Junio,
On 01/04/2019 11:47, Junio C Hamano wrote:
Philip Oakley <philipoakley@xxxxxxx> writes:

At the moment I'm using an extended _test_ case that starts by adding
a ~5.1Gb file and then using verify-pack, which aborts with an error.

         dd if=/dev/zero of=file bs=1M count=5100 &&
         git config core.compression 0 &&
         git config core.looseCompression 0 &&
         git add file &&
         git verify-pack -s .git/objects/pack/*.pack &&
         git fsck --verbose --strict --full &&

If however I simple execute the commands from the GfW bash, the added
file is stored as a blob object, rather than a pack.

I'm at a loss to understand the reason for the change in behaviour
[store file as pack, vs store as object] between running the code as a
test script and at the terminal. What am I missing?
To which test are you adding the above piece?  Perhaps one of those
that configures core.bigfilethreashold?
The test script (t-large-files-on-winows.sh: [1] below) was specific to this debugging.

I didn't set core.bigfilethreshold - Is that done (or unset) by the test setup at all?

It does prompt me to check that all the bigfilethreshold checks are actually size_t, rather than a simple 'long'/uInt which would only be 32bits on Windows and potentially a downcast comparison, resulting in mistaken bigfile actions because of the modulo 2^32 action.

So when I run the test script [1] on Windows I get my error from verify-pack, and the trash directory contains a single pack file. I tried doing the commands singly on a fresh repo, but that time found that the add/verify produced a blob object (rather than a pack with one object), so it got me wondering if I was testing like for like.

When I tried using gdb at the add stage, with a break point, I got a back trace [2], and when run to completion it had the loose object, so I was confused. (my fixup code is at [3])

Does the test_expect_success clear all the local/user config variable to sandbox the tests?


[1] https://github.com/PhilipOakley/git/blob/size_t2/t/t-large-files-on-windows.sh


   test_description='test large file handling on windows'
   . ./test-lib.sh

   test_expect_success SIZE_T_IS_64BIT 'blah blubb' '

   	test-tool zlib-compile-flags >zlibFlags.txt &&
   	dd if=/dev/zero of=file bs=1M count=5100 &&
   	git config core.compression 0 &&
   	git config core.looseCompression 0 &&
   	gdb git  &&
   	git verify-pack -s .git/objects/pack/*.pack &&
   	git fsck --verbose --strict --full &&
   	git commit -m msg file &&
   	git verify-pack -s .git/objects/pack/*.pack &&
   	git log --stat &&
   	git fsck --verbose --strict --full &&
   	git repack -a -f &&
   	git verify-pack -s .git/objects/pack/*.pack &&
   	git verify-pack -v .git/objects/pack/*.pack &&
   	git gc


[2] Thread 1 hit Breakpoint 1, git_deflate (strm=0x138dac0, flush=0) at zlib.c:235
235                     zlib_pre_call(strm);
(gdb) bt
#0  git_deflate (strm=0x138dac0, flush=0) at zlib.c:235
#1  0x00000000005f4d11 in write_loose_object (oid=0x7ff4fda90070,
    hdr=0x138f560 "blob 5347737600", hdrlen=16, buf=0x7fff0000,
    len=5347737600, mtime=0) at sha1-file.c:1707
#2  0x00000000005f50db in write_object_file (buf=0x7fff0000, len=5347737600,
    type=0x6e3e9c <__ac_HASH_UPPER+20> "blob", oid=0x7ff4fda90070)
    at sha1-file.c:1779
#3  0x00000000005f5696 in index_mem (istate=0x78a3c0 <the_index>,
    oid=0x7ff4fda90070, buf=0x7fff0000, size=5347737600, type=OBJ_BLOB,
    path=0x13b6c2c "file", flags=1) at sha1-file.c:1901
#4  0x00000000005f5aca in index_core (istate=0x78a3c0 <the_index>,
    oid=0x7ff4fda90070, fd=4, size=5347737600, type=OBJ_BLOB,
    path=0x13b6c2c "file", flags=1) at sha1-file.c:1975
#5  0x00000000005f5c5d in index_fd (istate=0x78a3c0 <the_index>,
    oid=0x7ff4fda90070, fd=4, st=0x138f860, type=OBJ_BLOB,
    path=0x13b6c2c "file", flags=1) at sha1-file.c:2019
#6  0x00000000005f5d9a in index_path (istate=0x78a3c0 <the_index>,
    oid=0x7ff4fda90070, path=0x13b6c2c "file", st=0x138f860, flags=1)
    at sha1-file.c:2040
#7  0x000000000059e818 in add_to_index (istate=0x78a3c0 <the_index>,
    path=0x13b6c2c "file", st=0x138f860, flags=0) at read-cache.c:763
#8  0x000000000059ea04 in add_file_to_index (istate=0x78a3c0 <the_index>,
    path=0x13b6c2c "file", flags=0) at read-cache.c:796
#9  0x0000000000404b73 in add_files (dir=0x138f9c0, flags=0)
    at builtin/add.c:378
#10 0x0000000000405286 in cmd_add (argc=1, argv=0x1390658, prefix=0x0)
    at builtin/add.c:534
#11 0x0000000000402e86 in run_builtin (p=0x68a040 <commands>, argc=2,
    argv=0x1390650) at git.c:422
#12 0x000000000040326a in handle_builtin (argc=2, argv=0x1390650) at git.c:654
#13 0x00000000004034ac in run_argv (argcp=0x138fd90, argv=0x138fd38)
    at git.c:708
#14 0x00000000004038ca in cmd_main (argc=2, argv=0x1390650) at git.c:830
#15 0x00000000004c2c5f in main (argc=3, argv=0x1390648) at common-main.c:45

[3] https://github.com/PhilipOakley/git/tree/size_t2