Re: [PATCH v3] coccicheck: process every source file at once
- Date: Sat, 6 Oct 2018 10:42:57 +0200
- From: René Scharfe <l.s.r@xxxxxx>
- Subject: Re: [PATCH v3] coccicheck: process every source file at once
Am 05.10.2018 um 21:00 schrieb Jeff King:
> On Fri, Oct 05, 2018 at 08:50:50PM +0200, SZEDER Gábor wrote:
>> On Fri, Oct 05, 2018 at 12:59:01PM -0400, Jeff King wrote:
>>> On Fri, Oct 05, 2018 at 04:53:35PM +0000, Keller, Jacob E wrote:
>>>>> Are we OK with saying 1.3-1.8GB is necessary to run coccicheck? That
>>>>> doesn't feel like an exorbitant request for a developer-only tool these
>>>>> days, but I have noticed some people on the list tend to have lousier
>>>>> machines than I do. ;)
>>>> It's probably not worth trying to make this more complicated and scale
>>>> up how many files we do at once based on the amount of available
>>>> memory on the system...
>>> Yeah, that sounds too complicated. At most I'd give a Makefile knob to
>>> say "spatch in batches of $(N)". But I'd prefer to avoid even that
>>> complexity if we can.
>> But perhaps one more if-else, e.g.:
>> if test -n "$(COCCICHECK_ALL_AT_ONCE)"; then \
>> <all at once from Jacob>
>> <old for loop>
>> would be an acceptable compromise? Dunno.
> That's OK, too, assuming people would actually want to use it. I'm also
> OK shipping this (with the "make -j" fix you suggested) and seeing if
> anybody actually complains. I assume there are only a handful of people
> running coccicheck in the first place.
FWIW, my development environment is a virtual machine with 1200MB RAM
and 900MB swap space. coccicheck takes almost eight minutes
sequentially, and four and a half minutes with -j4.
Unsurprisingly, it fails after almost three minutes with the patch,
reporting that it ran out of memory. With 2900MB it fails after almost
two minutes, with 3000MB it succeeds after a good two minutes.
time(1) says (for -j1):
433.30user 36.17system 7:49.84elapsed 99%CPU (0avgtext+0avgdata 108212maxresident)k
192inputs+1512outputs (0major+16409056minor)pagefaults 0swaps
129.74user 2.06system 2:13.27elapsed 98%CPU (0avgtext+0avgdata 1884568maxresident)k
236896inputs+1096outputs (795major+462129minor)pagefaults 0swaps
So with the patch it's more than three times faster, but needs more
than seventeen times more memory. And I need a bigger VM. :-/