Web lists-archives.com

Re: Fuzzing KDE software

Le mer. 6 mars 2019 à 20:36, Albert Astals Cid <aacid@xxxxxxx> a écrit :
El dimecres, 6 de març de 2019, a les 20:20:03 CET, alcinos va escriure:
> Hello,
> As some of you might have heard, we have been rewriting significantly some
> key components of Kdenlive over the past couple years, in an effort to
> improve stability, testability, and separation of concerns between the view
> and the model.
> As part of this effort, on top of traditional unit-tests (none existed
> before the rewrite), I have been looking at fuzzing as a way to push the
> code to its limit. For those who are not familiar with it, fuzzing is a
> testing technique that essentially consists in generating a lot of random
> inputs and feeding them to a target program in an attempt to generate
> unintended behaviours (memoryleak, crash,...). This technique has an
> excellent track record of finding bugs and vulnerabilites, even in well
> tested softwares (see eg. https://llvm.org/docs/LibFuzzer.html#trophies)
> In the case of Kdenlive, we generate a lot of operations that the user
> could execute on the timeline, and simulate them in the model, to check for
> robustness over unexpected action sequences.
> Note that this kind of testing doesn't explicitly test for correctness (it
> doesn't check whether a clip is correctly moved after a move operation for
> example), but try to get the software to crash, if there is a way to make
> that happen. Also, we have some internal checking mechanisms that
> periodically ensure that the model is in a sane state, and the invariants
> are respected. So if things go wrong with respect to that, it will also be
> caught by the fuzzer.
> This approach has already led to the discovery of several bugs in kdenlive,
> although the parts covered by the fuzzer are still fairly limited.
> Are any other projects in KDE using that technique? In order for it to be
> useful, it pretty much needs to be run continuously (I've been running it a
> bit on my laptop, but it's not very practical). Some companies offer such
> service for free for OSS, under some conditions: see for eg:
> https://github.com/google/oss-fuzz and https://fuzzbuzz.io/pricing,
> although I'm not sure whether Kdenlive would meet the requirements for
> either of those. Is there any way we could do this inside the KDE
> infrastructure instead?

We (KDE Security team, but here basically just me) are running kcodecs and kimageformats in oss-fuzz.

I can help you with the setup if you want.
I'm worried that Kdenlive may not meet oss-fuzz's requirements: "a significant user base and/or be critical to the global IT infrastructure" (the latter is definitely false, the former is subjective).
If we do meet the requirements, any help setting this up would be greatly appreciated. If I understand correctly, since we already have a fuzz-target in the repo, the only thing we need is to have the project build in the docker image?

To Boudewijn Rempt: if you want some technical details about how it's done for Kdenlive, feel free to ask! I guess a similar approach could be taken for Krita, if the architecture permits it.
The code is located here: https://invent.kde.org/kde/kdenlive/tree/refactoring_timeline/fuzzer



> Ideally, this are the requirements for such a
> service:
>    - Continuous running
>    - Git hook to get the latest code
>    - Automatic bug deduplication and reporting
>    - (optional) Coverage report to ensure the fuzzing is effective.
> Google open-sourced most of its tooling for fuzzing at scale on a cluster,
> so there is no need to start from scratch.
> What are your thoughts?
> Best regards,
> Nicolas Carion