Added Dockerfile

[Manouchehri]

Had some issues with cmake on Xenial, so switching to bionic (next LTS) seemed to be the route of least resistance.

[Manouchehri]

Using Prebuilt Image

docker run -it --rm thawsystems/retdec

Building Docker Image

git clone https://github.com/Manouchehri/retdec
docker build -t retdec .
docker run -it --rm retdec

[aidanhs]

How about removing the maintainer (since it'll be maintained by repo contributors if it ends up in the repo) and doing COPY rather than cloning?

[Manouchehri]

@aidanhs Hmm, doing COPY gets weird fast since we cannot assume that the submodules have been cloned (otherwise it will breaks most automated Docker build systems).

Could we merge in this now and revisit it once #14 is solved?

[cquick97]

Pull i386 Ubuntu image instead of x64, since only 32-bit is supported.

FROM i386/ubuntu:bionic

"Supported architectures (32b only): Intel x86, ARM, MIPS, PIC32, and PowerPC"

[Manouchehri]

@cquick97 That's for target binaries, not the host platform?

[cquick97]

Sure, but if 64-bit isn't required, why not just stick to 32-bit? I noticed that I couldn't compile a test binary within the container with gcc -m32, as there were architecture dependency problems. Of course you can just compile elsewhere and analyze within the container, but for simple testing I think having i386 dependencies would be better.

I'm halfway through an i386 build right now, so we'll see if that fixes anything. Or maybe I'm just crazy/dumb and you can ignore my nonsense.

[Manouchehri]

You can install a 32-bit compiler if needed.

docker run -it --rm --user root thawsystems/retdec
apt-get install gcc-multilib
su - retdec
export PATH=${PATH}:/home/retdec/retdec-install/bin
echo 'void main() { printf("Dave says hello.\n"); }' > test.c
gcc -m32 test.c
decompile.sh a.out

I don't think changing the host is a good idea, going back to 32-bit Linux distributions is a nightmare. Other RE tools like IDA Pro, Binary Ninja, etc are already 64-bit only, I think it's an acceptable requirement.

[cquick97]

That's probably a better idea. Disregard :)

[breznak]

👏 for the docker image, instead of full-dependencies installation.
Also, imho the64bit host is more future proof, as the 64bit decompilation support would hopefully be added soon? #9

[aidanhs]

@Manouchehri the main thing I wasn't keen on was cloning the whole repo graph twice - maybe COPY and then git submodule update --init --recursive? Still not perfect (because it discards changes people might have made to change the commit of a submodule) but at least you can make changes in this repo and rebuild with those changes, which is one of the use-cases of a Dockerfile.

I'm just making suggestions while waiting for official feedback :)

[s3rvac]

@breznak I am not sure if I understood your question correctly, but the future support for decompilations of 64b binaries (#9) will be independent from the OS that is used to build and run RetDec. That is, once there is support for decompilations of 64b files, you will be able to use both 32b/64b RetDec builds.

[breznak]

@s3rvac my idea was that on 64 you can run both 64/32bit apps, if you were to develop, debug something in the image. But basically disregard that, as it boils down to

I don't think changing the host is a good idea, going back to 32-bit Linux distributions is a nightmare. Other RE tools like IDA Pro, Binary Ninja, etc are already 64-bit only, I think it's an acceptable requirement.

[breznak]

Also, Alpine linux is a very popular docker container OS for its low footprint,
https://hub.docker.com/_/alpine/

It may be more work to set all the things up (once), but then Alpine could be a lighter alternative than a full-blown Ubuntu. What do you think?

[Manouchehri]

@breznak While Alpine is lighter, I prefer to use Ubuntu as it's more similar to the platform developers will be using and testing with.

[Manouchehri]

@aidanhs You do not need to clone any of the submodules before building the image. See the commands listed in #3 (comment).

[aidanhs]

@Manouchehri my point is that if you initialise submodules, you can COPY in the current working directory.

People can use your Dockerfile until they actually want to make and test a change, at which point they need to either edit it to use my proposed solution (COPY in the repo) or push every single thing they want to test to their own repo. I think COPY is a better solution, and following that with RUN git submodule update --init --recursive addresses your comment at #3 (comment).

Part of the benefit of Dockerfiles is to ease development, which is what I'm shooting for.

[Manouchehri]

The problem is COPY only works if the submodules are initialized. Breaking expected build behavior isn't a good idea.

My suggestion is to make a separate Dockerfile for intended your development use case. This one is meant for users to get started quickly with using RetDec.

[HugoKlepsch]

@Manouchehri I must agree with @aidanhs here. If you clone the repo while building the image, that Dockerfile could never be used to test local changes. That means you can never test to see if the Docker image works without commiting and pushing the change!

I also don't think that running git submodule update --init --recursive is a good idea, because that would forget any changes made to submodule versions.

At my work we also use Docker. Our general flow is to have the Dockerfile build an image from the local repository. This allows us to test local changes. Then when we want to publish a new version of the image, our CI system fully checks out the master branch and builds the new image.

If you are concerned users won't fully clone the repository, could we not add a README.md that explains the image creation process?

[HugoKlepsch]

Thanks for your contribution! 😊
Some minor feedback...

[HugoKlepsch]

Is there some way we can auto-detect the number of processors available?

[Manouchehri]

It could be done with a little bash, but we should probably try to improve the normal build process instead of diverging and hacking about in a Dockerfile.

[breznak]

I don't know if this should be multi-platform, but on linux you can go
make -j$(nproc)
https://unix.stackexchange.com/questions/208568/how-to-determine-the-maximum-number-to-pass-to-make-j-option

[s3rvac]

I would also suggest using make -j$(nproc) because otherwise, only a single core will be utilized for build. Or by setting MAKEFLAGS="-j $(nproc)".

[lilianmoraru]

-j(if the project is huge, it will spawn sometimes to many jobs) and -j $(nproc) are both good - you need coreutils for that, but I would expect this to be dragged with something like build-essential or already be there.

[Manouchehri]

Done!

[HugoKlepsch]

I would also use --no-install-recommends

[Manouchehri]

Thanks. I'll eventually split this into a multi-stage so the build tools won't need to be in the production image.

[HugoKlepsch]

I would prefer to use COPY.
Could it be optional somehow? Maybe two Dockerfiles: one for local changes and one from upstream master?

[Manouchehri]

That's what I suggested.

Since it's not my use case, I'll let someone else create a "development" Dockerfile that they like.

[Manouchehri]

@s3rvac Any feedback from the Avast team? =)

[s3rvac]

Shouldn't we install it into /home/retdec/install? The additional retdec- prefix seems redundant to me. Or is there a motivation behind the current path?

[Manouchehri]

The reason I use retdec-install is because if the end user adds more things to /home/retdec/, it can be a bit ambiguous what an install/ folder is referring to.

Not opposed to doing it either way though, which do you want me to use? =)

[aidanhs]

Ok - I get that having two dockerfiles might be useful. But in that case, I'd like to suggest renaming the file in this pr to Dockerfile.master or Dockerfile.prod or something, so when I create a PR for a COPY dockerfile it will be the default (as it's the one that people will most likely want to use if they already have the repo cloned).

On that note, it'd be cool to have some instructions for using this particular Dockerfile - for example, you will (when this PR is merged) be able to just curl -sSL https://raw.githubusercontent.com/avast-tl/retdec/master/Dockerfile.prod | docker build - to get the latest build without cloning.

[Manouchehri]

@aidanhs Dockerfiles must be named Dockerfile or otherwise they cannot be built on platforms like Docker Hub..

Once a functional Dockerfile like you're describing is written, we can switch to that. At the moment it's nonexistence, so it makes little sense to make this PR harder to use.

[HugoKlepsch]

@Manouchehri In that case then the path it is under must make it obvious it is for publishing only, not development. ie. '/productionDockerfile/Dockerfile' or the like

[Manouchehri]

Sure, but why do that if there's currently only one Dockerfile to begin with? If another Dockerfile is written, it should be added to a folder at that point. We're debating folder structure for files that don't exist.

[HugoKlepsch]

Honestly if this patch were complete it would have all common use case Dockerfiles. (in our case, production and development)

It's better to not have to change readme's, build scripts etc later when the other common use case Dockerfile is added.

[Manouchehri]

@HugoKlepsch Okay, go do that yourself then.

[aidanhs]

Dockerfiles must be named Dockerfile or otherwise they cannot be built on platforms like Docker Hub..

FWIW, this isn't true (though I think it used to be the case) - you can see that https://hub.docker.com/r/rustlang/crater/builds/bnftxfvygyjsakpah66cvyh/ uses the docker/Dockerfile.mini file from https://github.com/rust-lang-nursery/crater/tree/b3168a8/docker

[HugoKlepsch]

If that is the case, maybe we can have a Dockerfile and a Dockerfile.dev. I'm working on such a development Dockerfile in my off time now.