Web lists-archives.com

Re: [RFC][PATCH] index-pack: add testcases found using AFL




On 10/03/2017 16:15, Vegard Nossum wrote:
I've used AFL to generate a corpus of pack files that maximises the edge
coverage for 'git index-pack'.

This is a supplement to (and not a replacement for) the regular test cases
where we know exactly what each test is checking for. These testcases are
more useful for avoiding regressions in edge cases or as a starting point
for future fuzzing efforts.

To see the output of running 'git index-pack' on each file, you can do
something like this:

  make -C t GIT_TEST_OPTS="--run=34 --verbose" t5300-pack-object.sh

I observe the following coverage changes (for t5300 only):

  path                  old%  new%    pp
  ----------------------------------------
  builtin/index-pack.c  74.3  76.6   2.3
  pack-write.c          79.8  80.4    .6
  patch-delta.c         67.4  81.4  14.0
  usage.c               26.6  35.5   8.9
  wrapper.c             42.0  46.1   4.1
  zlib.c                58.7  64.1   5.4

And if you add this simple patch on top (sorry, I didn't think of it
until after I'd sent the previous e-mail):

diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh
index 19e02ffc2..db705ba5c 100755
--- a/t/t5300-pack-object.sh
+++ b/t/t5300-pack-object.sh
@@ -425,8 +425,10 @@ test_expect_success 'index-pack <pack> works in non-repo' '
 test_expect_success 'index-pack edge coverage' '
        for pack in "$TEST_DIRECTORY"/t5300/*.pack
        do
-               rm -rf "${pack%.pack}.idx" &&
-               test_might_fail git index-pack $pack
+               rm -rf "${pack%.pack}.idx" tmp.pack tmp.idx &&
+               test_might_fail git index-pack $pack &&
+               test_might_fail git index-pack --strict $pack &&
+ test_might_fail git index-pack --stdin --fix-thin tmp.pack < $pack
        done
 '


you get this change to the coverage profile instead:

path                  old%  new%    pp
----------------------------------------

alloc.c               58.1  67.4   9.3
builtin/index-pack.c  74.3  80.7   6.4
commit.c              13.9  17.4   3.5
date.c                 3.5   4.2    .7
fsck.c                15.7  33.7  18.0
object.c              56.0  58.7   2.7
pack-write.c          79.8  81.4   1.6
patch-delta.c         67.4  81.4  14.0
path.c                31.6  32.1    .5
sha1_file.c           48.9  49.6    .7
tag.c                  3.7  16.8  13.1
tree.c                36.6  37.5    .9
usage.c               26.6  35.5   8.9
wrapper.c             42.0  46.1   4.1
zlib.c                58.7  64.1   5.4

Of course, it's likely some of those gains can be found in other
testcases outside t5300 -- also, coverage isn't everything. Still seems
like a nice gain with very little effort.


Vegard