08 Jul 2017 · 1 min read

Fuzzing libVLC - Improvements

I have been making some few specific improvement to demux fuzz target and also working on the decoder target side by side which isn’t done yet. Some general improvements that I have made for all fuzz targets is abstracting away and initializing the parent <code>libvlc_instance_t</code> object in <code>LLVMFuzzerInitialize</code> which obviously has some performance improvements as it isn’t created and destroyed on every run of the <code>LLVMFuzzerTestOneInput</code>. <h4 id="fuzzer-common-c"><a class="zola-anchor" href="#fuzzer-common-c" aria-label="Anchor link for: fuzzer-common-c">🔗</a> fuzzer_common.c</h4> <pre class="giallo" style="color: #657B83; background-color: #FDF6E3;" ><code data-lang="c">#include <assert.h> #include "fuzzer.h" static libvlc_instance_t *p_libvlc; extern int LLVMFuzzerInitialize(int *argc, char ***argv) { setenv("VLC_PLUGIN_PATH", "../../modules", 1); p_libvlc = libvlc_new(0, NULL); assert(p_libvlc != NULL); return 0; } extern int LLVMFuzzerTestOneInput(const uint8_t *buf, size_t len) { return FuzzerTestOneInput(buf, len, p_libvlc); } void FuzzerCleanup(void) { libvlc_release(p_libvlc); }</code></pre> Another benefit of this abstraction is that our fuzz targets are now completely agnostic to the fuzzing engine we use which has been <a rel="noopener nofollow noreferrer external" target="_blank" href="http://llvm.org/docs/LibFuzzer.html#startup-initialization">libFuzzer</a> until now. We can now use the exact same target with <a rel="noopener nofollow noreferrer external" target="_blank" href="http://lcamtuf.coredump.cx/afl/">AFL</a> by writing a driver, since AFL expects a main function, here is an <a rel="noopener nofollow noreferrer external" target="_blank" href="https://github.com/llvm-mirror/compiler-rt/blob/58d43607862096aeb32d72173911c9df244a30f1/lib/fuzzer/afl/afl_driver.cpp">example</a> driver. <h3 id="dictionaries"><a class="zola-anchor" href="#dictionaries" aria-label="Anchor link for: dictionaries">🔗</a> Dictionaries</h3> At this point the demux fuzzer had found a few more bugs but it had kind of reached a saturation point in terms of the line coverage it had and it just wasn’t increasing any more significantly with new sample input. So I decided instead of adding sample files of every possible video file format with different types of chunks and atoms that VLC supports, to create a keywords dictionary of magic values of the various file formats and feed it to the demux target. Libfuzzer has in-built support for dictionary files and can much more efficiently create structured inputs with it. libFuzzer and AFL have the same syntax of dictionary files and the AFL project even has some excellent dictionaries for <a rel="noopener nofollow noreferrer external" target="_blank" href="https://github.com/mcarpenter/afl/blob/master/dictionaries/jpeg.dict">JPEG</a>, <a rel="noopener nofollow noreferrer external" target="_blank" href="https://github.com/mcarpenter/afl/blob/master/dictionaries/png.dict">PNG</a> and <a rel="noopener nofollow noreferrer external" target="_blank" href="https://github.com/mcarpenter/afl/blob/master/dictionaries/gif.dict">GIF</a> file formats. And you guessed it VLC can actually “Play” those files as well since <code>ffmpeg</code> provides their codec as well. I’m still working on the dictionary and I’ll maybe post it once it’s completed or if it’s ever merged into the VLC codebase.