Web lists-archives.com

Re: Fuzzing KDE software




El dijous, 7 de març de 2019, a les 1:56:47 CET, alcinos va escriure:
> Le mer. 6 mars 2019 à 20:36, Albert Astals Cid <aacid@xxxxxxx> a écrit :
> 
> > El dimecres, 6 de març de 2019, a les 20:20:03 CET, alcinos va escriure:
> > > Hello,
> > >
> > > As some of you might have heard, we have been rewriting significantly
> > some
> > > key components of Kdenlive over the past couple years, in an effort to
> > > improve stability, testability, and separation of concerns between the
> > view
> > > and the model.
> > >
> > > As part of this effort, on top of traditional unit-tests (none existed
> > > before the rewrite), I have been looking at fuzzing as a way to push the
> > > code to its limit. For those who are not familiar with it, fuzzing is a
> > > testing technique that essentially consists in generating a lot of random
> > > inputs and feeding them to a target program in an attempt to generate
> > > unintended behaviours (memoryleak, crash,...). This technique has an
> > > excellent track record of finding bugs and vulnerabilites, even in well
> > > tested softwares (see eg. https://llvm.org/docs/LibFuzzer.html#trophies)
> > >
> > > In the case of Kdenlive, we generate a lot of operations that the user
> > > could execute on the timeline, and simulate them in the model, to check
> > for
> > > robustness over unexpected action sequences.
> > > Note that this kind of testing doesn't explicitly test for correctness
> > (it
> > > doesn't check whether a clip is correctly moved after a move operation
> > for
> > > example), but try to get the software to crash, if there is a way to make
> > > that happen. Also, we have some internal checking mechanisms that
> > > periodically ensure that the model is in a sane state, and the invariants
> > > are respected. So if things go wrong with respect to that, it will also
> > be
> > > caught by the fuzzer.
> > >
> > > This approach has already led to the discovery of several bugs in
> > kdenlive,
> > > although the parts covered by the fuzzer are still fairly limited.
> > >
> > > Are any other projects in KDE using that technique? In order for it to be
> > > useful, it pretty much needs to be run continuously (I've been running
> > it a
> > > bit on my laptop, but it's not very practical). Some companies offer such
> > > service for free for OSS, under some conditions: see for eg:
> > > https://github.com/google/oss-fuzz and https://fuzzbuzz.io/pricing,
> > > although I'm not sure whether Kdenlive would meet the requirements for
> > > either of those. Is there any way we could do this inside the KDE
> > > infrastructure instead?
> >
> > We (KDE Security team, but here basically just me) are running kcodecs and
> > kimageformats in oss-fuzz.
> >
> > I can help you with the setup if you want.
> >
> I'm worried that Kdenlive may not meet oss-fuzz's requirements: "a
> significant user base and/or be critical to the global IT infrastructure"
> (the latter is definitely false, the former is subjective).
> If we do meet the requirements, any help setting this up would be greatly
> appreciated. If I understand correctly, since we already have a fuzz-target
> in the repo, the only thing we need is to have the project build in the
> docker image?

Yes, make sure it builds and runs on their docker system, it's going to be tricky since you need to compile everything statically (not sure if you already do that).

And honestly, kdenlive has a significant user base if you ask me, so don't pre-disqualify yourself, if they disagree, let them say it, not you.

Cheers,
  Albert

> 
> To Boudewijn Rempt: if you want some technical details about how it's done
> for Kdenlive, feel free to ask! I guess a similar approach could be taken
> for Krita, if the architecture permits it.
> The code is located here:
> https://invent.kde.org/kde/kdenlive/tree/refactoring_timeline/fuzzer
> 
> Cheers,
> Nicolas
> 
> 
> > Cheers,
> >   Albert
> 
> 
> > > Ideally, this are the requirements for such a
> > > service:
> > >
> > >    - Continuous running
> > >    - Git hook to get the latest code
> > >    - Automatic bug deduplication and reporting
> > >    - (optional) Coverage report to ensure the fuzzing is effective.
> > >
> > > Google open-sourced most of its tooling for fuzzing at scale on a
> > cluster,
> > > so there is no need to start from scratch.
> > >
> > > What are your thoughts?
> > >
> > > Best regards,
> > > Nicolas Carion
> > >
> >
> >
> >
> >
> >
>