Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

oss-fuzz installation, stuck at step 2 #24

Closed
swonlek opened this issue Jun 25, 2018 · 10 comments
Closed

oss-fuzz installation, stuck at step 2 #24

swonlek opened this issue Jun 25, 2018 · 10 comments

Comments

@swonlek
Copy link

swonlek commented Jun 25, 2018

Hi, I am trying to install the oss-fuzz before installing aflgo and im doing this behind corporate firewall. I keep getting the error:

E: Unable to locate package libc6-dev
E: Unable to locate package binutils
E: Unable to locate package libgcc-5-dev

when the /infra/base-images/base-clang/Dockerfile is running. I poked around google and most suggestions include doing sudo apt-get update, sudo apt-get upgrade, adding repositories like multiverse etc etc, and i've tried them but this error is still thrown.
I've also tried editing the Dockerfile to run RUN apt-get install -y libc6-dev binutils libgcc-5-dev, but the error message:

Step 3/12 : RUN sudo apt-get install -y libc6-dev binutils libgcc-5-dev
 ---> Running in fa2d4767bc4c
/bin/sh: 1: sudo: not found
The command '/bin/sh -c sudo apt-get install -y libc6-dev binutils libgcc-5-dev' returned a non-zero code: 127

then appears. Help!! Not sure if its the corporate firewall preventing it from downloading the packages (which, last time I checked, were the latest versions already as of 25 June 2018)

Below is what comes up in the terminal when entering infra/base-images/all.sh:

+ docker build --pull -t gcr.io/oss-fuzz-base/base-image infra/base-images/base-image
Sending build context to Docker daemon   2.56kB
Step 1/9 : FROM ubuntu:16.04
16.04: Pulling from library/ubuntu
Digest: sha256:b050c1822d37a4463c01ceda24d0fc4c679b0dd3c43e742730e2884d3c582e3a
Status: Image is up to date for ubuntu:16.04
 ---> 5e8b97a2a082
Step 2/9 : MAINTAINER mike.aizatsky@gmail.com
 ---> Using cache
 ---> ce3911a754ea
Step 3/9 : ENV DEBIAN_FRONTEND noninteractive
 ---> Using cache
 ---> ae05540f823a
Step 4/9 : RUN apt-get update && apt-get upgrade -y && apt-get autoremove -y
 ---> Using cache
 ---> 9dbd6c3bdefc
Step 5/9 : ENV OUT /out
 ---> Using cache
 ---> 498e0b22b8b3
Step 6/9 : ENV SRC /src
 ---> Using cache
 ---> 9a6be30850ec
Step 7/9 : ENV WORK /work
 ---> Using cache
 ---> 0ca61162fa17
Step 8/9 : ENV PATH "$PATH:/out"
 ---> Using cache
 ---> 5e933374ad11
Step 9/9 : RUN mkdir -p $OUT $SRC $WORK && chmod a+rwx $OUT $SRC $WORK
 ---> Using cache
 ---> 85e3704aafae
Successfully built 85e3704aafae
Successfully tagged gcr.io/oss-fuzz-base/base-image:latest
+ docker build -t gcr.io/oss-fuzz-base/base-clang infra/base-images/base-clang
Sending build context to Docker daemon  6.656kB
Step 1/12 : FROM gcr.io/oss-fuzz-base/base-image
 ---> 85e3704aafae
Step 2/12 : MAINTAINER mike.aizatsky@gmail.com
 ---> Using cache
 ---> 15da6853474d
Step 3/12 : RUN apt-get install -y libc6-dev binutils libgcc-5-dev
 ---> Running in 0c969a46d227
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package libc6-dev
E: Unable to locate package binutils
E: Unable to locate package libgcc-5-dev
The command '/bin/sh -c apt-get install -y libc6-dev binutils libgcc-5-dev' returned a non-zero code: 100```
@mboehme
Copy link
Collaborator

mboehme commented Jun 25, 2018

Can you try and delete all docker images related to OSS-Fuzz and run infra/base-images/all.sh again?

@swonlek
Copy link
Author

swonlek commented Jun 26, 2018 

uia65408@igl1534u:~/oss-fuzz$ docker system prune -a
WARNING! This will remove:
	- all stopped containers
	- all volumes not used by at least one container
	- all networks not used by at least one container
	- all images without at least one container associated to them
Are you sure you want to continue? [y/N] y
Deleted Containers:
d3ae819ebe9ab966fdb4d7d431a03e6ca6081c05cd25c8182cb9b92272b93a32

Deleted Images:
untagged: ubuntu:16.04
untagged: ubuntu@sha256:b050c1822d37a4463c01ceda24d0fc4c679b0dd3c43e742730e2884d3c582e3a
deleted: sha256:1e3cd592ea3b4dabc9be763cf8174bd32b08b85d0a5961566d107f3fe62a4a04
untagged: gcr.io/oss-fuzz-base/base-image:latest
deleted: sha256:d647bdf01c5e2d308c23777b2f34c97c2e7e6e813a9a1d959cbf4fd708d7db88
deleted: sha256:4a195e93863915e69b0c4ee674124b95d000b547bff8aaa918e3ea6247be4be4
deleted: sha256:8da8b30bb69aa6ed4a908f789311ecde6295e3aa99ad9c4bd3380e65975ac728
deleted: sha256:6a41aaf8a2884ae51e513e1954fad658436df9643d9a10f1660f044e4f8279c7
deleted: sha256:d374f8438a6b8b4536bd52d55d01cf796cff69e4f394f06b96dcb55607ce192b
deleted: sha256:7cab5a2664f98bdc14be1c2b49aa0d12809121873352aea53fabfcd775eef81e
deleted: sha256:f9dbea605d7d5ad5343c22d4808a2e1667f504cafff1e5469f562d9bd3d19e74
deleted: sha256:fc2621359b1681fbfa8d35a17ec1e2896e46fd1652b6c27ca47ed8591a223073
deleted: sha256:a857e155305f8e94386fb7ee653f009231e1dea1f742c63eb453490d466ab44a
deleted: sha256:35208b9db62b195a7dcd19acd7a51413d54aafe906260865d7ad3f28fef6ae68
deleted: sha256:5e8b97a2a0820b10338bd91674249a94679e4568fd1183ea46acff63b9883e9c
deleted: sha256:ef572e1ba2ecca900f0ec3db00e997de12dd380ce3e360b5813fd75920232359
deleted: sha256:98fc4d5421178c7be7d5718d2d44abba8053dc5c712e51658fe5b872675b4f7a
deleted: sha256:7b2cc05dfd889e28234f8831c80ac20cf299d5bbebbbac013f8f7d2b7abc0d65
deleted: sha256:6b0187d1cdff63eb5966ac72bf4ccd96150586c1409eb858bb98783f02018ee7
deleted: sha256:644879075e24394efef8a7dddefbc133aad42002df6223cacf98bd1e3d5ddde2

Total reclaimed space: 113.9MB
uia65408@igl1534u:~/oss-fuzz$ sudo service docker restart
uia65408@igl1534u:~/oss-fuzz$ infra/base-images/all.sh
+ docker build --pull -t gcr.io/oss-fuzz-base/base-image infra/base-images/base-image
Sending build context to Docker daemon   2.56kB
Step 1/9 : FROM ubuntu:16.04
16.04: Pulling from library/ubuntu
b234f539f7a1: Pull complete 
55172d420b43: Pull complete 
5ba5bbeb6b91: Pull complete 
43ae2841ad7a: Pull complete 
f6c9c6de4190: Pull complete 
Digest: sha256:b050c1822d37a4463c01ceda24d0fc4c679b0dd3c43e742730e2884d3c582e3a
Status: Downloaded newer image for ubuntu:16.04
 ---> 5e8b97a2a082
Step 2/9 : MAINTAINER mike.aizatsky@gmail.com
 ---> Running in 3ebf0465e661
 ---> ff8b80cde5ab
Removing intermediate container 3ebf0465e661
Step 3/9 : ENV DEBIAN_FRONTEND noninteractive
 ---> Running in 5375e9fa92a4
 ---> 50e4cb6d5f93
Removing intermediate container 5375e9fa92a4
Step 4/9 : RUN apt-get update && apt-get upgrade -y && apt-get autoremove -y
 ---> Running in 0684d56717f1
Err:1 http://archive.ubuntu.com/ubuntu xenial InRelease
  Could not connect to archive.ubuntu.com:80 (91.189.88.149), connection timed out [IP: 91.189.88.149 80]
Err:2 http://archive.ubuntu.com/ubuntu xenial-updates InRelease
  Unable to connect to archive.ubuntu.com:http: [IP: 91.189.88.149 80]
Err:3 http://archive.ubuntu.com/ubuntu xenial-backports InRelease
  Unable to connect to archive.ubuntu.com:http: [IP: 91.189.88.149 80]
Err:4 http://security.ubuntu.com/ubuntu xenial-security InRelease
  Could not connect to security.ubuntu.com:80 (91.189.88.149), connection timed out [IP: 91.189.88.149 80]
Reading package lists...

W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/xenial/InRelease  Could not connect to archive.ubuntu.com:80 (91.189.88.149), connection timed out [IP: 91.189.88.149 80]
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/xenial-updates/InRelease  Unable to connect to archive.ubuntu.com:http: [IP: 91.189.88.149 80]
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/xenial-backports/InRelease  Unable to connect to archive.ubuntu.com:http: [IP: 91.189.88.149 80]
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/xenial-security/InRelease  Could not connect to security.ubuntu.com:80 (91.189.88.149), connection timed out [IP: 91.189.88.149 80]
W: Some index files failed to download. They have been ignored, or old ones used instead.

Reading package lists...
Building dependency tree...
Reading state information...
Calculating upgrade...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Reading package lists...
Building dependency tree...
Reading state information...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
 ---> 21cd9df072ee
Removing intermediate container 0684d56717f1
Step 5/9 : ENV OUT /out
 ---> Running in 352758f0b711
 ---> 25c75c42b16a
Removing intermediate container 352758f0b711
Step 6/9 : ENV SRC /src
 ---> Running in e51462ac5719
 ---> 0ed53b6fbdce
Removing intermediate container e51462ac5719
Step 7/9 : ENV WORK /work
 ---> Running in 2802f1f926f9
 ---> 32fe791c35ba
Removing intermediate container 2802f1f926f9
Step 8/9 : ENV PATH "$PATH:/out"
 ---> Running in 7bc919ad3c4b
 ---> 7dacd311c00f
Removing intermediate container 7bc919ad3c4b
Step 9/9 : RUN mkdir -p $OUT $SRC $WORK && chmod a+rwx $OUT $SRC $WORK
 ---> Running in 67e2fa0c685d
 ---> 1de87ceb4be4
Removing intermediate container 67e2fa0c685d
Successfully built 1de87ceb4be4
Successfully tagged gcr.io/oss-fuzz-base/base-image:latest
+ docker build -t gcr.io/oss-fuzz-base/base-clang infra/base-images/base-clang
Sending build context to Docker daemon  6.656kB
Step 1/12 : FROM gcr.io/oss-fuzz-base/base-image
 ---> 1de87ceb4be4
Step 2/12 : MAINTAINER mike.aizatsky@gmail.com
 ---> Running in f997b1de80e0
 ---> 4e4429089e38
Removing intermediate container f997b1de80e0
Step 3/12 : RUN apt-get install -y libc6-dev binutils libgcc-5-dev
 ---> Running in 74380308ce38
Reading package lists...
Building dependency tree...
Reading state information...

E: Unable to locate package libc6-dev
E: Unable to locate package binutils
E: Unable to locate package libgcc-5-dev
The command '/bin/sh -c apt-get install -y libc6-dev binutils libgcc-5-dev' returned a non-zero code: 100

Not sure if what I did was right, but I don't think it fixed the problem.

@mboehme
Copy link
Collaborator

mboehme commented Jun 26, 2018

It seems apt-get update is not running properly in Step 4/9.

Can you try any of these solutions: https://askubuntu.com/questions/91543/apt-get-update-fails-to-fetch-files-temporary-failure-resolving-error

@mboehme
Copy link
Collaborator

mboehme commented Jun 26, 2018

This one is Docker-specific: https://stackoverflow.com/questions/24832972/docker-apt-get-update-fails

@swonlek
Copy link
Author

swonlek commented Jul 9, 2018

The connection timed out error is probably caused by being behind a proxy:
https://askubuntu.com/questions/1051739/unable-to-apt-get-update-through-dockerfile-could-not-connect-to-archive-ubuntu

@swonlek swonlek closed this as completed Jul 9, 2018
@swonlek swonlek reopened this Jul 11, 2018
@swonlek
Copy link
Author

swonlek commented Jul 11, 2018

I'm still stuck at this step haha. I feel like the problem is more with docker. as the apt update from the dockerfiles keeps showing that repositories have no release file. So far i haven't found a solution that works for me yet. So, now I want to ask instead, is there a way to install this whole thing without using docker? That would be much much much appreciated

@yxliang01
Copy link

@swonlek I have seen few people installed and run aflgo with and without docker. So, it should be fine as long as you have required dependencies installed (e.g. llvm 4.0).

@mboehme
Copy link
Collaborator

mboehme commented Jul 11, 2018

This seems like an OSS-Fuzz problem. It is definitely not an AFLGo problem.
You can try syncing our OSS-Fuzz fork with upstream: https://github.com/google/oss-fuzz/
You might need to resolve some conflicts, since OSSFuzz has very frequent commits.
Let me know how it goes.

More info: https://help.github.com/articles/syncing-a-fork/

@swonlek
Copy link
Author

swonlek commented Jul 12, 2018

Regardless of docker, we definitely need oss-fuzz to use aflgo right? I might not have time to try out your suggestion right now, but I will definitely give it a shot when my schedule's more free

@mboehme
Copy link
Collaborator

mboehme commented Jul 12, 2018

No. You can simply follow the instructions in the AFLGo readme to set it up without Docker or OSS-Fuzz.

@mboehme mboehme closed this as completed Jul 12, 2018
@z1w z1w mentioned this issue Jul 27, 2018
Closed
@fuzzer123 fuzzer123 mentioned this issue Apr 20, 2018
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mboehme @yxliang01 @swonlek