Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking bug: enabling DFT-based fuzzing #503

Closed
Dor1s opened this issue May 29, 2019 · 4 comments

Comments

@Dor1s
Copy link
Collaborator

commented May 29, 2019

There are three main components:

  1. Build / test support in OSS-Fuzz: this is tracked in google/oss-fuzz#1632

  2. Job support on ClusterFuzz side: we don't need a new job for DFT-based fuzzing, but we do need a separate build. This creates a problem: how do I enable a build on OSS-Fuzz side (project.yaml is the obvious answer), but do not create any additional jobs on CF side?

  3. Strategy support on ClusterFuzz: might get too complicated, given that we need to download a separate build for that.

@Dor1s

This comment has been minimized.

Copy link
Collaborator Author

commented May 29, 2019

While writing this, I came up to another solution:

  1. Set up DataFlow build in a way that it would have both ASan and DFSan binaries in the same archive.

  2. Create a new Job for DataFlow config. Use it as an ASan job, except two minor differences:
    A) always use collect_data_flow option
    B) always use fork mode (LF currently crashes when collect_data_flow is used without fork mode)

Besides cleaner set up and less confusing strategy logic, this will also guarantee that we're using exactly the same source code for both ASan and DFSan binaries. I don't know whether this is a blocker, but it's definitely not beneficial to have mismatch in the corresponding ASan and DFSan binaries.

I'm really favoring this approach now. Will think more and pitch it to you in the meeting today :)

@Dor1s Dor1s changed the title Tracking bug: enabling DataFlow strategy Tracking bug: enabling DFT-based fuzzing May 29, 2019

@Dor1s

This comment has been minimized.

Copy link
Collaborator Author

commented May 30, 2019

Thanks team for the discussion today. Now I'm leaning towards a new strategy again. I'll take a closer look at the code and try to predict / draft the changes necessary for enabling that.

Dor1s added a commit that referenced this issue Jun 9, 2019

Dor1s added a commit that referenced this issue Jun 13, 2019

Dor1s added a commit that referenced this issue Jun 18, 2019

Dor1s added a commit that referenced this issue Jun 18, 2019

Dor1s added a commit that referenced this issue Jun 18, 2019

Dor1s added a commit that referenced this issue Jun 18, 2019

@Dor1s

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 18, 2019

The new functionality is broken into three CLs:

  1. #580: Add support for dataflow builds to the OSS-Fuzz setup handler.

  2. #582: Add AuxiliaryBuild class for extra builds like dataflow.

  3. #583: Add dataflow tracing strategy to libFuzzer launcher.

Dor1s added a commit that referenced this issue Jun 20, 2019

Add support for dataflow builds to the OSS-Fuzz setup handler (#503). (
…#580)

* Add support for dataflow builds to the OSS-Fuzz setup handler (#503)

* Apply to x64 builds only.

Dor1s added a commit that referenced this issue Jun 26, 2019

Dor1s added a commit that referenced this issue Jun 27, 2019

Add support for auxiliary builds and use it for dataflow builds (#503)…
…. (#582)

* Add AuxiliaryBuild class for extra builds like dataflow (#503).

* Get rid of the AuxiliaryBuild class, use RegularBuild instead.

* fix a test in untrusted_runner

* Revert the change in mock_unpack_build.

* Refactor setup_trunk_build and re-use it + other review comments.

* update the tests, revert some unnecessary changes

* address review comments, use self.env_prefix

Dor1s added a commit that referenced this issue Jun 27, 2019

Dor1s added a commit that referenced this issue Jun 27, 2019

Dor1s added a commit that referenced this issue Jun 28, 2019

Dor1s added a commit that referenced this issue Jun 28, 2019

Add dataflow tracing strategy and refactor strategy parsing code (#503)…
…. (#583)

* Add dataflow tracing strategy to libFuzzer launcher (#503).

* Address review feedback, use new strategy code, refactor strategy parsing for stats

Dor1s added a commit that referenced this issue Jun 28, 2019

Dor1s added a commit that referenced this issue Jun 28, 2019

Dor1s added a commit that referenced this issue Jun 29, 2019

Dor1s added a commit that referenced this issue Jul 1, 2019

Ensure that fuzz target exists in DFSan build, log_warn otherwise (#503
…). (#625)

* Ensure that fuzz target exists in DFSan build, log_warn otherwise (#503).

* Log error instead of warning and extend the message
@Dor1s

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 2, 2019

The strategy is enabled and seems to be running fine. I'm closing this. The documentation is being tracked as #558.

@Dor1s Dor1s closed this Jul 2, 2019

Dor1s added a commit that referenced this issue Jul 8, 2019

Dor1s added a commit that referenced this issue Jul 8, 2019

Disable corpus subset when DFT is used, also extend strategy_pool API (
…#503). (#659)

* Disable corpus subset when DFT is used, also extend strategy_pool API (#503).

* Pass strategy_tuple to the strategy pool remove method.

* Address review comment, do not remove strategy from the pool
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.