Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix incorrect imports #1758

Merged
merged 3 commits into from
Dec 23, 2023
Merged

Fix incorrect imports #1758

merged 3 commits into from
Dec 23, 2023

Conversation

wtdcode
Copy link
Contributor

@wtdcode wtdcode commented Dec 23, 2023

According to https://doc.rust-lang.org/core/simd/trait.SimdOrd.html , the correct import seems core::simd::SimdOrd.

0.11.2 breaks several downstream project buildings. Currently, I'm setting =0.11.1 as workaround.

@wtdcode
Copy link
Contributor Author

wtdcode commented Dec 23, 2023

cc @tokatoka

@tokatoka
Copy link
Member

it got updated early this month
https://doc.rust-lang.org/nightly/core/simd/cmp/trait.SimdOrd.html
can you make sure you are using the latest nightly?

@wtdcode
Copy link
Contributor Author

wtdcode commented Dec 23, 2023

it got updated early this month https://doc.rust-lang.org/nightly/core/simd/cmp/trait.SimdOrd.html can you make sure you are using the latest nightly?

I'm not sure if I understand cargo toolchains correctly. If my project use a stable toolchain while my dependencies use nightly, the code should be compiled with my stable toolchain?

Just found a quick pointer: fuzzland/ityfuzz#412

See logs: https://github.com/fuzzland/ityfuzz/actions/runs/7304832850/job/19907590056#step:8:416

@wtdcode
Copy link
Contributor Author

wtdcode commented Dec 23, 2023

Even it got updated, using core::simd::prelude::... should work for both versions according to the document links above.

@wtdcode wtdcode closed this Dec 23, 2023
@wtdcode wtdcode reopened this Dec 23, 2023
@tokatoka
Copy link
Member

I'm not sure if I understand cargo toolchains correctly. If my project use a stable toolchain while my dependencies use nightly, the code should be compiled with my stable toolchain?

this is strange. if you are using stable, this simd stuff won't be compiled
because it's guarded by #[rustversion::nightly]

@tokatoka
Copy link
Member

Even it got updated, using core::simd::prelude::... should work for both versions according to the document links above.

ok

@wtdcode
Copy link
Contributor Author

wtdcode commented Dec 23, 2023

I'm not sure if I understand cargo toolchains correctly. If my project use a stable toolchain while my dependencies use nightly, the code should be compiled with my stable toolchain?

this is strange. if you are using stable, this simd stuff won't be compiled because it's guarded by #[rustversion::nightly]

Oh I see the actual problem. My project (and ityfuzz above) uses a slightly outdated nightly toolchain.

But anyway, the slight change in this PR should make it compatible with a wider range of nightly toolchains.

@wtdcode
Copy link
Contributor Author

wtdcode commented Dec 23, 2023

I could give a minimal example:

.
├── Cargo.lock
├── Cargo.toml
├── Dockerfile
├── rust-toolchain.toml
├── src
│   └── main.rs

Then main.rs:

use libafl::feedbacks::ConstFeedback;
fn main() {
    let f = ConstFeedback::True;
    println!("Hello, world {}!", libafl_bolts::current_nanos());
}

Cargo.toml:

[package]
name = "hello"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
libafl = "=0.11.2"
libafl_bolts = "=0.11.2"

rust-toolchain.toml

[toolchain]
channel = "nightly-2023-11-01"

Dockerfile

FROM rust:bookworm

WORKDIR /work

COPY Cargo.toml .
COPY src ./src
COPY rust-toolchain.toml .

RUN cargo build --release

Then docker build . gives:

30.00 error[E0432]: unresolved import `core::simd::cmp`
30.00  --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/libafl-0.11.2/src/feedbacks/map.rs:8:17
30.00   |
30.00 8 | use core::simd::cmp::SimdOrd;
30.00   |                 ^^^ could not find `cmp` in `simd`
30.00 
30.64 error[E0599]: no associated item named `LEN` found for struct `Simd` in the current scope
30.64    --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/libafl-0.11.2/src/feedbacks/map.rs:550:40
30.64     |
30.64 550 |         let steps = size / VectorType::LEN;
30.64     |                                        ^^^ associated item not found in `Simd<u8, 16>`
30.64 
30.64 error[E0599]: no associated item named `LEN` found for struct `Simd` in the current scope
30.64    --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/libafl-0.11.2/src/feedbacks/map.rs:551:39
30.64     |
30.64 551 |         let left = size % VectorType::LEN;
30.64     |                                       ^^^ associated item not found in `Simd<u8, 16>`
30.64 
30.64 error[E0599]: no associated item named `LEN` found for struct `Simd` in the current scope
30.64    --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/libafl-0.11.2/src/feedbacks/map.rs:556:44
30.64     |
30.64 556 |                 let i = step * VectorType::LEN;
30.64     |                                            ^^^ associated item not found in `Simd<u8, 16>`
30.64 
30.64 error[E0599]: no method named `simd_max` found for struct `Simd` in the current scope
30.64    --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/libafl-0.11.2/src/feedbacks/map.rs:560:26
30.64     |
30.64 560 |                 if items.simd_max(history) != history {
30.64     |                          ^^^^^^^^ method not found in `Simd<u8, 16>`
30.64    --> /rustc/9d83ac217957eece2189eccf4a7232caec7232ee/library/core/src/../../portable-simd/crates/core_simd/src/ord.rs:28:8
30.64     |
30.64     = note: the method is available for `Simd<u8, 16>` here
30.64     |
30.64     = help: items from traits can only be used if the trait is in scope
30.64 help: the following trait is implemented but not in scope; perhaps add a `use` for it:
30.64     |
30.64 3   + use core::simd::SimdOrd;
30.64     |
30.64 
30.64 error[E0599]: no associated item named `LEN` found for struct `Simd` in the current scope
30.64    --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/libafl-0.11.2/src/feedbacks/map.rs:563:54
30.64     |
30.64 563 |                         for j in i..(i + VectorType::LEN) {
30.64     |                                                      ^^^ associated item not found in `Simd<u8, 16>`
30.64 
30.64 error[E0599]: no associated item named `LEN` found for struct `Simd` in the current scope
30.64    --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/libafl-0.11.2/src/feedbacks/map.rs:584:44
30.64     |
30.64 584 |                 let i = step * VectorType::LEN;
30.64     |                                            ^^^ associated item not found in `Simd<u8, 16>`
30.64 
30.64 error[E0599]: no method named `simd_max` found for struct `Simd` in the current scope
30.64    --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/libafl-0.11.2/src/feedbacks/map.rs:588:26
30.64     |
30.64 588 |                 if items.simd_max(history) != history {
30.64     |                          ^^^^^^^^ method not found in `Simd<u8, 16>`
30.64    --> /rustc/9d83ac217957eece2189eccf4a7232caec7232ee/library/core/src/../../portable-simd/crates/core_simd/src/ord.rs:28:8
30.64     |
30.64     = note: the method is available for `Simd<u8, 16>` here
30.64     |
30.64     = help: items from traits can only be used if the trait is in scope
30.64 help: the following trait is implemented but not in scope; perhaps add a `use` for it:
30.64     |
30.64 3   + use core::simd::SimdOrd;
30.64     |
30.64 
31.10 Some errors have detailed explanations: E0432, E0599.
31.10 For more information about an error, try `rustc --explain E0432`.
31.12 error: could not compile `libafl` (lib) due to 8 previous errors
------
Dockerfile:9
--------------------
   7 |     COPY rust-toolchain.toml .
   8 |     
   9 | >>> RUN cargo build --release
--------------------
ERROR: failed to solve: process "/bin/sh -c cargo build --release" did not complete successfully: exit code: 101

@tokatoka
Copy link
Member

https://github.com/AFLplusplus/LibAFL/blob/main/Dockerfile#L24

to fix ci
can you change this file to use llvm 16?

@wtdcode
Copy link
Contributor Author

wtdcode commented Dec 23, 2023

https://github.com/AFLplusplus/LibAFL/blob/main/Dockerfile#L24

to fix ci can you change this file to use llvm 16?

This looks like a bug in LLVM side.

e.g.:

mio@lazymbp16 /tmp> curl -I https://apt.llvm.org/bookworm
HTTP/2 404
server: nginx/1.18.0 (Ubuntu)
content-type: text/html
accept-ranges: bytes
date: Sat, 23 Dec 2023 14:05:40 GMT
via: 1.1 varnish
age: 1481
x-served-by: cache-bur-kbur8200167-BUR
x-cache: HIT
x-cache-hits: 1
x-timer: S1703340340.205043,VS0,VE0
strict-transport-security: max-age=300
content-length: 162

mio@lazymbp16 /tmp> curl -I https://apt.llvm.org/bookworm/
HTTP/2 200
server: nginx/1.18.0 (Ubuntu)
content-type: text/html
accept-ranges: bytes
date: Sat, 23 Dec 2023 14:05:42 GMT
via: 1.1 varnish
age: 1446
x-served-by: cache-bur-kbur8200081-BUR
x-cache: HIT
x-cache-hits: 1
x-timer: S1703340342.390855,VS0,VE1
strict-transport-security: max-age=300
content-length: 592

@tokatoka tokatoka merged commit ea5ea34 into AFLplusplus:main Dec 23, 2023
14 of 15 checks passed
@domenukk
Copy link
Member

@tokatoka @wtdcode this broke CI!

0.980 + exit 2
0.980 Distribution 'debian' in version '12 (bookworm)' is not supported by this script.
------
Dockerfile:24
--------------------
  22 |     # Install clang 11, common build tools
  23 |     RUN apt update && apt install -y build-essential gdb git wget python3-venv ninja-build lsb-release software-properties-common gnupg
  24 | >>> RUN wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh && ./llvm.sh 16
  25 |     
  26 |     # Copy a dummy.rs and Cargo.toml first, so that dependencies are cached
--------------------
ERROR: failed to solve: process "/bin/sh -c wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh && ./llvm.sh 16" did not complete successfully: exit code: 2

Guess we got to update the docker?

@wtdcode
Copy link
Contributor Author

wtdcode commented Dec 23, 2023

@domenukk This is LLVM website issue and I have sent it to them.

@tokatoka
Copy link
Member

fixed now

rmalmain pushed a commit to rmalmain/LibAFL that referenced this pull request Jan 2, 2024
rmalmain added a commit that referenced this pull request Mar 22, 2024
* Added paging filtering.
Reworked address range filtering to fit with new generic code.

* Fix: renamed remaining QemuInstrumentationFilter instances.

* Renamed sync breakpoint to sync exit.

* Split emu in systemmode.rs / usermode.rs for specific code.
EmuExitHandler implementation.

* sync_backdoor.rs removal.
Formatting.

* Updated `bindgen` and `which`.
Adapting code to work with update.

* fix: reconfigure cleanly if prior configure was interrupted abruptly.

* Enable sanitizers in QEMU during debug.

* Added target-usable files.

* Added breakpoint structure.

* Adapted other files to work with ExitHandler.

* Adapted existing fuzzer to work with new exit handler.

* fix: use get to avoid crashes.

* Updated README to indicate cargo-make should be installed.

* Added QEMU internal exit handler.

* Adapted qemu_systemmode example with new exit handler.

* Fixed fuzzers to work with new exit handler.

* Trying to fix CI (#1739)

* test

* dummy

* dummy

* Added new examples.

* Forgot to add build scripts.

* format

* format

* clang-format

* python emulator adaptation.

* fixed python bindings.

* clippy fixes.

* python bindings.

* fix qemu_sugar.

* fix fuzzbench.

* fixed import issues.

* misc fixes.

* renamed crate.

* Updated x86_64 stub bindings.

* Fixed static naming.

* binding fmt

* clippy

* clippy

* Removed useless return statement.

* removed advice to install cargo-make in individual repositories.

* symcc_update (#1749)

* Remove unused create_anymap_for_trait macro (fixes #1719) (#1752)

* Fix `as_object` UB discussed in #1748 (#1751)

* Fix as_object UB discussed in #1748

* More cleanup, more less UB

* Fix fixes

* Added uninit_on_shmem api

* clippy

* fmt

* trying to fix fuzzers, libfuzzer wrapper

* Add OwnedRefMit::owned constructor, libfuzzer fix

* Some more fixes

* Add BacktaceObserver::owned fn

* fmt

* more fmt

* Ignore SigPipe by default (#1741)

* Ignore SigPipe by default

* Fix no_std

* fmt

* Fix incorrect imports (#1758)

* Fix incorrect imports

https://doc.rust-lang.org/core/simd/trait.SimdOrd.html

* Fix

* Try fix ci

* Documentation fixes (#1761)

* Documentation fixes

* Fix InProcessExecutor url

* Update all urls to latest

* Miri ignores for M1 regex (#1762)

* Enabling DrCov on Windows (#1765)

* Enabling DrCov for Windows

* Dedup common code in scheduler (#1702)

* dedup common code in scheduler

* del eco

* fixing

* fix

* replace `Emulator::new_empty` by `Emulator::get` calls outside of `emu.rs` for safety. (#1763)

* Add mute_inprocess_target fn, SimpleFdLogger::set_logger, and more (#1754)

* Add mute_inprocess_target fn, SimpleFdLogger::set_logger, set_error_print_panic_hook

* Trying to fix #1753

* typo

* More fix

* Fix test?

* more testcase fixes

* Fix: renamed remaining QemuInstrumentationFilter instances.

* Split emu in systemmode.rs / usermode.rs for specific code.
EmuExitHandler implementation.

* format

* format

* format

* Replace sync_exit with sync_backdoor.

* Rework command system.

* fix bad import.

* format.

* cargo fmt

* disable af-xdp as well to avoid linking errors.

* End of merging.

* format.

* Adaptation for usermode.

* format.

* injection support.

* usermode fixes.
format.

* clippy

* clippy + format

* Do not unwrap emu + format.

* fix: entry_point breakpoint

* inital commit.

* clippy

* tests

* clippy

* adapt example

* systemmode.

* renaming

* fmt

* fix lints.

* more lint fix.

* even more lint fixes.

* always more lint fixes.

* lint fix.

* allow unused qualifications for crate when it could be confusing.

* Still lint fixes.

* Lint fixes on generated code.

* Some lint fixes.

* merge continue.

* renamed modules as well.

* fixing merge.

* systemmode compiling.

* fmt

* fix early emulator drop.

* fmt

* fix cast to c_void of the wrong object.

* Added global enum for snapshot managers.
Some renaming.

* move things around.

* WIP: generic inclusion of QEMU Executor in exit handler.

* * Moved extern calls to `libafl_qemu_sys`
* Replaced old `Emulator` by `Qemu` and only kept C functions wrappers
* Now `Emulator` is for higher-level interactions with QEMU. Kept old functions for compatibility calling to `Qemu` functions
* A direct side effect of this slit is the removal of the `IsEmuExitHandler` trait dependency added in many parts of the code.
* Removed old dirty casting for `QemuExecutor` helpers and used the brand-new access to `QemuExecutorState` instead.
* Minor changes to `Qemu` and `Emulator` `get` methods for cleaner getters.

* Add missing `Qemu` function.

* Updated `qemu_systemmode` example.

* Adapted QEMU fuzzers + renaming.

* Fixed python.

* fix libafl_sugar with new implementation.

* fix dangling RefCell.
adapt new examples.
TODO: merge `libafl_systemmode.*` examples.

* clippy.

* fix more fuzzers.

* clippy.

* Implement `HasInstrumentationFilter` generically.
Updated `StdInstrumentationFilter` accordingly.

* Renamed breakpoint functions for QEMU.
`qemu.run()` handling.

* Removed OnceCell / RefCell in signature.
more explicit `MmapPerms` method names.

* minor code refactoring

* Emulator::run_handle refactoring

* deprecated Emulator functions calling directly to QEMU functions.

* IsSnapshotManager -> SnapshotManager

* IsEmuExitHandler -> EmuExitHandler + fmt

* Generic register when it makes sense.

* reverted IsSnapshotManager -> SnapshotManager because of a collision.

* fix syntax + clippy

* fmt

---------

Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: lazymio <mio@lazym.io>
Co-authored-by: Bet4 <0xbet4@gmail.com>
Co-authored-by: mkravchik <mkravchik@hotmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants