Web lists-archives.com

Fetching new refs gets progressively slower

I have a weird issue where fetching a large number of refs will start off with lines like these:

* [new ref]               refs/pull/10000/head -> origin/pr/10000
* [new ref]               refs/pull/10001/head -> origin/pr/10001

going fairly fast, and then progressively getting slower and slower. By the time git is working on 40 thousandth such ref, it seems like it is only handling about 3-5 such “new ref”s.

These are the steps I used to reproduce:

$ git clone git@xxxxxxxxxx:rust-lang/rust
$ # edit .git/config to add $ # `fetch = +refs/pull/*/head:refs/remotes/origin/pr/*` under origin remote
$ git fetch

I tried this on three distinct file systems: zfs, ext4 and tmpfs, and on two distinct systems (both with different SSDs in it). Both exhibit the approximately the same behaviour. Both on a fairly recent version of Linux.

Here’s some timings:

System 1 (ext4) 97% (1047.74 real, 700.11 kernel, 319.89 user); 599476k resident
System 1 (tmpf) 97% (963.78 real, 647.51 kernel, 292.77 user); 600052k resident
System 1 (zfs) 98% (2116.66 real, 1715.86 kernel, 370.32 user); 531232k resident
System 2 (ext4) 97% (1036.56 real, 710.54 kernel, 300.81 user); 602160k resident

Git version is same on both systems: 2.13.1

I did not investigate the issue more throughoutly, but I suspect that git will end up doing something resembling listing the contents of the directory of refs for each “new ref” it is creating.