-
Notifications
You must be signed in to change notification settings - Fork 157
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
extern crate loading #1362
Merged
Merged
extern crate loading #1362
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
philberty
force-pushed
the
phil/metadata-output
branch
9 times, most recently
from
July 8, 2022 18:34
29c6c3d
to
e3b3a33
Compare
philberty
force-pushed
the
phil/metadata-output
branch
from
July 14, 2022 12:23
e3b3a33
to
da3d1c4
Compare
It seems as though mac does not like dumping data into a custom section. This might need us to just always export a seperate file instead. |
philberty
force-pushed
the
phil/metadata-output
branch
from
July 15, 2022 15:06
0b1d80e
to
87a265b
Compare
This allows AST dump of attributes on Traits this code needs to be applied for all attribute dumps. Traits also have an implicit Self generic param this ensure we ignore this during an AST dump.
Extern crates statements to tell the front-end to look for another library. The mechanism here is heavily inspired from gccgo, so when we compile a library for example we invoke: gccrs -g -O2 -frust-crate=mylib -c src/lib.rs -o src/mylib.o All going well this object file will now contain extra data inside .rust-export section inside the object file which will be preserved inside archives and shared objects. When we have another application which uses this library 'mylib'. extern crate mylib; use mylib::foo; fn main() { foo(); } We compile using: gcc -g -O2 -frust-crate=test -c src/main.rs -o src/main.o When the extern crate line is hit the front-end will look for mylib.o, libmylib.a, mylib.rox. If it finds a raw object file it will read the .rust-export section directly from the object for the public metadata such as public functions, types constants etc. If it fails to find an object it might find .rox which is the objdump of the .rust-export to a raw file, it might even find libmylib.a and read the export directly out of the archive file reusing code from gccgo to do so. The full compiler pipeline is reused here, so the metatadata is actually just real rust code. The benifit here is that Rust supports exporting, macros and generics so this requires the name-resolution and type info all to be generated and inserted into the apropriate context classes. Since the metadata is real rust code it means we can reuse the full pipeline to generate the code as nessecary. So for the simple case of a public struct we simply emit the AST dump of this struct directly into the metadata. If its a non-generic public function we emit and extern rust abi block for that function. If its a trait we can simply emit the trait with the public memebers. Generics are more complicated since we need to emit the function fully for it to be compiled correctly this still needs tests to be added. The hardest part is non generic impl blocks which is still a WIP. To finally link the two crates together you run: gcc -g -O2 -o rust-program.exe src/main.o src/mylib.o Fixes: 1169
This interface will allow us to write the metadata directly to a file. MacOs does not support metadata embeded directly into a .section of object files and archives. This allows us to seperate this mechanism so that we can follow rustc and write this directly to a seperate file. This patch adds two new options to control metadata outputs -frust-embed-metadata this toggles to embed the metadata into .rust_export section of the target asm output -frust-metadata-output= specifies the path to directly write the metadata to file We need these options as embeding the metadata does not seem to be supported for all platforms like MacOs so writing directly to file is a platform agnostic solution. There is an implicit naming convetion to metadata export files. They need to be of crate_name.rox. This patch adds some simple validation by checking the basename of the specified path.
philberty
force-pushed
the
phil/metadata-output
branch
from
July 15, 2022 16:06
87a265b
to
2d1378f
Compare
bors r+ |
Build succeeded: |
tschwinge
added a commit
that referenced
this pull request
Jul 20, 2022
The recent commit 418aef1 "Support extern crate" (merged in commit fd2bd65 "Merge #1362"), added 'gcc/testsuite/rust/link/link.exp', which I understand to be a hacked up variant of the LTO multi-file testing harness (not reviewed in detail). Since that commit, there are issues in parallel testing, so that, for example with: $ make -j12 check-rust RUNTESTFLAGS='--target_board=unix\{,-m32,-mx32\}' ..., the first variant does test as expected: @@ -6490,9 +6490,31 @@ PASS: rust/execute/torture/trait9.rs -Os at line 13 (test for warnings, line PASS: rust/execute/torture/trait9.rs -Os (test for excess errors) PASS: rust/execute/torture/trait9.rs -Os execution test PASS: rust/execute/torture/trait9.rs -Os output pattern test +Running /home/thomas/tmp/source/gcc/rust/build/trace-slim/source-gcc/gcc/testsuite/rust/link/link.exp ... +PASS: rust/link/generic_function_0.rs -O0 (test for excess errors) +[...] +PASS: rust/link/trait_import_0.rs -Os (test for excess errors) === rust Summary for unix === -# of expected passes 6445 +# of expected passes 6466 # of expected failures 31 ..., but we then get undesired changes for the second and following variants: Running target unix/-m32 Running /home/thomas/tmp/source/gcc/rust/build/trace-slim/source-gcc/gcc/testsuite/rust/compile/compile.exp ... PASS: rust/compile/abi-options1.rs at line 2 (test for errors, line 1) PASS: rust/compile/abi-options1.rs at line 7 (test for errors, line 6) PASS: rust/compile/abi-options1.rs (test for excess errors) -PASS: rust/compile/array3.rs at line 2 (test for warnings, line 1) -PASS: rust/compile/array3.rs (test for excess errors) -PASS: rust/compile/array_empty_list.rs at line 3 (test for errors, line 2) -PASS: rust/compile/array_empty_list.rs (test for excess errors) -PASS: rust/compile/arrays1.rs (test for errors, line 3) -PASS: rust/compile/arrays1.rs (test for excess errors) PASS: rust/compile/arrays2.rs at line 4 (test for errors, line 3) PASS: rust/compile/arrays2.rs (test for excess errors) PASS: rust/compile/attr-mismatch-crate-name.rs (test for excess errors) @@ -8355,110 +8371,215 @@ PASS: rust/compile/torture/generics17.rs -O3 -g (test for excess errors) PASS: rust/compile/torture/generics17.rs -Os at line 18 (test for warnings, line 17) PASS: rust/compile/torture/generics17.rs -Os (test for excess errors) PASS: rust/compile/torture/generics18.rs -O0 at line 19 (test for warnings, line 18) +PASS: rust/compile/torture/generics18.rs -O0 at line 19 (test for warnings, line 18) +PASS: rust/compile/torture/generics18.rs -O0 at line 5 (test for warnings, line 4) PASS: rust/compile/torture/generics18.rs -O0 at line 5 (test for warnings, line 4) PASS: rust/compile/torture/generics18.rs -O0 (test for excess errors) +PASS: rust/compile/torture/generics18.rs -O0 (test for excess errors) ..., and so on. This has been determined to be due to repetitive 'runtest_file_p' calls (it's a mess...), thusly fixed (some would call it a "hack", no doubt), and verified that, for example: $ make -j12 check-rust RUNTESTFLAGS='--target_board=unix\{,-m32,-mx32\} link.exp=simple_function_0.rs' ... only runs the one test as indicated.
bors bot
added a commit
that referenced
this pull request
Jul 20, 2022
1404: 'gcc/testsuite/rust/link/link.exp': Fix 'runtest_file_p' usage r=philberty a=tschwinge The recent commit 418aef1 "Support extern crate" (merged in commit fd2bd65 "Merge #1362"), added 'gcc/testsuite/rust/link/link.exp', which I understand to be a hacked up variant of the LTO multi-file testing harness (not reviewed in detail). Since that commit, there are issues in parallel testing, so that, for example with: $ make -j12 check-rust RUNTESTFLAGS='--target_board=unix\{,-m32,-mx32\}' ..., the first variant does test as expected: `@@` -6490,9 +6490,31 `@@` PASS: rust/execute/torture/trait9.rs -Os at line 13 (test for warnings, line PASS: rust/execute/torture/trait9.rs -Os (test for excess errors) PASS: rust/execute/torture/trait9.rs -Os execution test PASS: rust/execute/torture/trait9.rs -Os output pattern test +Running /home/thomas/tmp/source/gcc/rust/build/trace-slim/source-gcc/gcc/testsuite/rust/link/link.exp ... +PASS: rust/link/generic_function_0.rs -O0 (test for excess errors) +[...] +PASS: rust/link/trait_import_0.rs -Os (test for excess errors) === rust Summary for unix === -# of expected passes 6445 +# of expected passes 6466 # of expected failures 31 ..., but we then get undesired changes for the second and following variants: Running target unix/-m32 Running /home/thomas/tmp/source/gcc/rust/build/trace-slim/source-gcc/gcc/testsuite/rust/compile/compile.exp ... PASS: rust/compile/abi-options1.rs at line 2 (test for errors, line 1) PASS: rust/compile/abi-options1.rs at line 7 (test for errors, line 6) PASS: rust/compile/abi-options1.rs (test for excess errors) -PASS: rust/compile/array3.rs at line 2 (test for warnings, line 1) -PASS: rust/compile/array3.rs (test for excess errors) -PASS: rust/compile/array_empty_list.rs at line 3 (test for errors, line 2) -PASS: rust/compile/array_empty_list.rs (test for excess errors) -PASS: rust/compile/arrays1.rs (test for errors, line 3) -PASS: rust/compile/arrays1.rs (test for excess errors) PASS: rust/compile/arrays2.rs at line 4 (test for errors, line 3) PASS: rust/compile/arrays2.rs (test for excess errors) PASS: rust/compile/attr-mismatch-crate-name.rs (test for excess errors) `@@` -8355,110 +8371,215 `@@` PASS: rust/compile/torture/generics17.rs -O3 -g (test for excess errors) PASS: rust/compile/torture/generics17.rs -Os at line 18 (test for warnings, line 17) PASS: rust/compile/torture/generics17.rs -Os (test for excess errors) PASS: rust/compile/torture/generics18.rs -O0 at line 19 (test for warnings, line 18) +PASS: rust/compile/torture/generics18.rs -O0 at line 19 (test for warnings, line 18) +PASS: rust/compile/torture/generics18.rs -O0 at line 5 (test for warnings, line 4) PASS: rust/compile/torture/generics18.rs -O0 at line 5 (test for warnings, line 4) PASS: rust/compile/torture/generics18.rs -O0 (test for excess errors) +PASS: rust/compile/torture/generics18.rs -O0 (test for excess errors) ..., and so on. This has been determined to be due to repetitive 'runtest_file_p' calls (it's a mess...), thusly fixed (some would call it a "hack", no doubt), and verified that, for example: $ make -j12 check-rust RUNTESTFLAGS='--target_board=unix\{,-m32,-mx32\} link.exp=simple_function_0.rs' ... only runs the one test as indicated. Co-authored-by: Thomas Schwinge <thomas@codesourcery.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
WIP still trying to write dejagnu to enable automated tests
Extern crates statements to tell the front-end to look for another library.
The mechanism here is heavily inspired from gccgo, so when we compile a
library for example we invoke:
All going well this object file will now contain extra data inside
.rust-export section inside the object file which will be preserved inside
archives and shared objects. When we have another application which uses
this library 'mylib'.
We compile using:
When the extern crate line is hit the front-end will look for mylib.o,
libmylib.a, mylib.rox. If it finds a raw object file it will read the
.rust-export section directly from the object for the public metadata
such as public functions, types constants etc. If it fails to find an
object it might find .rox which is the objdump of the .rust-export to a
raw file, it might even find libmylib.a and read the export directly out
of the archive file reusing code from gccgo to do so.
The full compiler pipeline is reused here, so the metatadata is actually
just real rust code. The benifit here is that Rust supports exporting,
macros and generics so this requires the name-resolution and type info
all to be generated and inserted into the apropriate context classes. Since
the metadata is real rust code it means we can reuse the full pipeline to
generate the code as nessecary. So for the simple case of a public struct
we simply emit the AST dump of this struct directly into the metadata. If
its a non-generic public function we emit and extern rust abi block for
that function. If its a trait we can simply emit the trait with the public
memebers. Generics are more complicated since we need to emit the function
fully for it to be compiled correctly this still needs tests to be added.
The hardest part is non generic impl blocks which is still a WIP.
To finally link the two crates together you run:
Fixes: 1169