Web lists-archives.com

Re: [PATCH 2/2] test-lib: exhaustively insert non-alnum ASCII into the TRASH_DIRECTORY name

On Mon, Apr 10, 2017 at 08:19:40PM +0200, Joachim Durchholz wrote:

> > I very much disagree with that. Git's test operate under a set of
> > assumptions, and if you violate those assumptions, then the failures are
> > not meaningful.
> In that case the tests do not validate that git can properly work with
> special characters.
> That's a pretty big coverage gap.

That's not necessarily true either; there may be specific tests that
create exotic paths and check them. My point is that the outcome depends
on that paths. So you cannot just take a test which runs "git clone -s"
and expect it to work both with and without paths with newlines. You
need two tests, because there are two different outcomes, depending on
the test environment.

So if you're proposing to write a bunch of new tests that check the
proper behavior under various conditions, go for it. But I don't think
running the entire existing test suite with exotic paths tells you much.
A failure might be a bug, or it might be that the thing is untestable
given the environment.

> > Sure, and I'd encourage people who are interested to dig through the
> > results and see if they can find a real problem. I looked at several and
> > didn't find anything that wasn't an example of the "test assumptions"
> > thing above.
> Don't assume that there's no risk just because you didn't find anything.

I'm not assuming that at all. Didn't I say somebody would need to dig
into all of these to find out the real answer? I'm only arguing that
blindly adding this feature to the test suite has no value. It's the
digging that has value, and you do not even need to modify the test
suite to do it (you can just use --root).

I've been trying to invite you to do that digging, if it's something you
care about.