Make the funfuzz Python package able to be run in the Docker, by porting funfuzz.sh

[nth10sd]

Here's the port of funfuzz.sh to Orion with the following output

$ docker run -it --rm mozillasecurity/funfuzz:latest
bash: ccache: command not found
$ python3 -u -m funfuzz --version
Platform details: Linux 8fe850dce50a 4.15.0-43-generic #46-Ubuntu SMP Thu Dec 6 14:45:28 UTC 2018 x86_64 x86_64
hg info: Mercurial Distributed SCM (version 4.8.1)
git info: git version 2.17.1
Python version: 3.6.7
Number of cores visible to OS: 4
Free space (GB): 59.11
Corefile size (soft limit, hard limit) is: (-1, -1)
Usage: __main__.py [options]

__main__.py: error: no such option: --version
$

As can be seen, funfuzz is installed and can be run after the docker initialises. There is a bunch of issues to be looked into prior to deployment though, e.g. why ccache is not found/installed even though we clearly import from FuzzOS. Any ideas?

Thanks a lot for your help earlier this month! Other priorities are coming up so actual deployment may be pushed off some number of quarters.

[posidron]

Cool, will look at it soon. Regarding ccache. It's not installed on FuzzOS, it is only temporary installed for the rr recipe but then deleted. You can install it the same way as in rr.sh.

[nth10sd]

Notes to self:

cd orion
make lint  # Install prereqs via requirements.txt in orion repo first
cd services/funfuzz
docker build -t mozillasecurity/funfuzz .
docker run -t mozillasecurity/funfuzz -it --rm

And something about docker run -it --rm mozillasecurity/funfuzz:latest bash -li too.

[nth10sd]

Travis shows a test failure at:

/snip
Setting up tree (1.6.0-1) ...
Setting up pass (1.4.2-3) ...
++grep -Po '"tag_name": "\K.*?(?=")'
./scripts/ci-docker-login.sh: line 5: GH_TOKEN: unbound variable
+LATEST_VERSION=

but this PR does not touch scripts/ci-docker-login.sh...

make lint passes though.

[nth10sd]

Another note to self: This may be useful in the future to get core files to be generated in docker containers/images:

https://stackoverflow.com/questions/28335614/how-to-generate-core-file-in-docker-container

[posidron]

Building PRs is now fixed. I restarted the task. The failure is now on your end. =) Travis secrets are not allowed in PRs, hence we skip now the login process for PRs - which is not needed anyway in order to just build.

Regarding core dumps, see also the comment link here: https://github.com/MozillaSecurity/orion/blob/master/base/fuzzos/recipes/fuzzos.sh#L22

[posidron]

Let's fix those first and then do another review.

[posidron]

LD_LIBRARY_PATH I am not sure this could cause problems if it is defined globally rather than in a script where it is to be used.

[nth10sd]

True, this needs to be added whenever a user chooses to get into the funfuzz container. Any ideas?

[posidron]

Hm, you could it revert back to the .bashrc file but the .bashrc won't load if you run the container in non-interactive mode, meaning without -i. To get around that you would need to add -e "BASH_ENV=/home/worker/.bashrc" to the run command.

Keep it there for now, let's see what would break and then make the changes. If the container is only supposed to run a certain CMD (the fuzzer / harness etc) and a user would want to have a shell, he would need to manually make the adjustments.

[posidron]

You could also add those things to some sort of init script.
Example:

Dockerfile

CMD = [funfuzz.sh]

funfuzz.sh

export bash envs
./run_fuzzer

[nth10sd]

Most of the issues have been addressed. Let's land this first and we can follow up as needed.