Web lists-archives.com

Re: [PATCH 2/3] t: teach 'test_must_fail' to save the command's stderr to a file

On Fri, Feb 09, 2018 at 03:42:34AM +0100, SZEDER Gábor wrote:

> To check that a git command fails and to inspect its error message we
> usually execute a command like this throughout our test suite:
>   test_must_fail git command --option 2>output.err
> Note that this command doesn't limit the redirection to the git
> command, but it redirects the standard error of the 'test_must_fail'
> helper function as well.  This is wrong for several reasons:
>   - If that git command were to succeed or die for an unexpected
>     reason e.g. signal, then 'test_must_fail's helpful error message
>     would end up in the given file instead of on the terminal or in
>     't/test-results/$T.out', when the test is run with '-v' or
>     '--verbose-log', respectively.
>   - If the test is run with '-x', the trace of executed commands in
>     'test_must_fail' will go to stderr as well (except for more recent
>     Bash versions supporting $BASH_XTRACEFD), and then in turn will
>     end up in the given file.

I have to admit that I'm slightly negative on this approach, just

  1. It requires every caller to do something special, rather than just
     using normal redirection. And the feature isn't as powerful as
     redirection. E.g., some callers do:

       test_must_fail foo >output 2>&1


       test_must_fail stderr=output foo >output

     is not quite right (stdout and stderr outputs may clobber each
     other, because they have independent position pointers).

  2. The "-x" problems aren't specific to test_must_fail at all. They're
     a general issue with shell functions.

I'm not entirely happy with saying "if you want to use -x, please use
bash". But given that it actually solves the problems everywhere with no
further effort, is it really that bad a solution?

For the error messages from test_must_fail, could we go in the same
direction, and send them to descriptor 4 rather than 2? We've already
staked out descriptor 4 as something magical that must be left alone
(see 9be795fb). If we can rely on that, then it becomes a convenient way
for functions to make sure their output is going to the script's stderr.