Web lists-archives.com

aren't unreliable tests worse than none? (Re: Help requested: Packages which FTBFS randomly)




On Mon, Feb 20, 2017 at 11:45:00AM +0100, Vincent Bernat wrote:
> Time is a limited resource and we need to set our priorities. Having
> test suites that work 100% of the time with constrained resources is not
> a goal I find worthy of the time I can spend on Debian.

While I agree with Niels that it would be very worthwhile to be able to define
ressource requirements for a package to build (and thus know I have to life
with some packages having trouble sometimes) I find it *very* strange to be
content with test suites which randomly fail.

How do you know an error in a testsuite is a non-critical one which can be
ignored? *Especialy* if you have flaky tests, how can you be sure (or even
guesstimate) a test failure is harmless one to ignore and not a critical one
which needs acting upon???

I *really* don't get why people advocate keeping unreliable tests enabled in
releases.


-- 
cheers,
	Holger

Attachment: signature.asc
Description: Digital signature