Support for compiling to binary object #3232

dkm · 2022-01-04T21:03:29Z

Support for compiling until assembler, before invoking the linker (eg. gcc -c).
Basic support for displaying relocation information.

Currently only used in c/c++/d/ada/rust and may/will propagate to other lang/compilers later.

fixes #3222

Signed-off-by: Marc Poulhiès dkm@kataplop.net

dkm · 2022-01-04T21:06:20Z

This is still WIP and mostly of PoC.
Currently able to compile to binary object and match relocations and display (crudely) them.

Still have to figure out the best way to select this mode (vs current binary, execution, ...) and how to nicely display reloc info.

dkm · 2022-01-05T14:59:12Z

Relocation parsing fixed, no particular handling and reusing the text field of the asm lines.

Also, the checkboxes are greyed for incompatible choices (binaryobj vs binary/exec):

dkm · 2022-01-05T15:01:44Z

I should probably find how to display relocation after the opcode hex value instead of before.

dkm · 2022-01-23T10:11:15Z

Updated few config files, should do some more tests. Screenshot showing what to expect.

dkm · 2022-01-23T10:37:31Z

I'll probably add some filters to hide/show relocations.

partouf · 2022-04-09T23:42:21Z

So far I like where this code is going. What do you want it to feature next?

dkm · 2022-05-03T07:10:27Z

So far I like where this code is going. What do you want it to feature next?

Sorry for being (very) silent lately, have been busy with other things. I think the prototype is mostly OK, but I would like to:

check for better parsing of relocation info from the objdump dump. Can't say I really understand how the asm layout is being done currently. I hacked something that looks ok-ish, but would like it to be "correct"
check it interacts correctly with existing stuff. Don't want to discover after the facts that it counter intuitive when X or Y is used
better test with different compilers/targets
(opt) only checked with C/C++/Ada, but could be enabled with other languages (at least Rust, but there needs to be an objdump available and defined).

I'm planning on reviving it, it's next in my todo list. It's not a dead PR :)

RubenRBS · 2022-11-09T01:51:55Z

Do let us know if we can help with anything!

partouf · 2023-01-08T22:03:28Z

The objdump/parser changes will require changes to https://github.com/compiler-explorer/asm-parser
If you have some examples of in -> out for unittesting that will probably go a long way

mattgodbolt

Some early comments: will take a look properly tomorrow! Thanks so much for this!!

lib/base-compiler.ts

lib/compilers/ada.ts

lib/compilers/rust.ts

lib/parsers/asm-parser.ts

static/panes/compiler.ts

dkm · 2023-01-09T06:50:21Z

THanks for the review, will respin this later tonight!

lib/base-compiler.ts

dkm · 2023-01-09T21:39:55Z

lib/parsers/asm-parser.ts

+                const relocdata = match.groups.relocdata;
+                // value/addend matched but not used yet.
+                const match_value = relocdata.match(this.relocDataSymNameRe);
+                asm.push({


@partouf does that look the correct way to have the relocation displayed? I remember that when starting this I thought of having a dedicated structure for relocations... But then kept things simple like this.

I think so. You could add the label/symbolname that's used (with https://github.com/compiler-explorer/compiler-explorer/blob/main/types/asmresult/asmresult.interfaces.ts#L31), but since it probably doesn't actually exist, it wouldn't be used anyway. So I think it's good enough.

Ok! So will replicate this in asm-parser

dkm · 2023-01-10T21:44:35Z

The objdump/parser changes will require changes to https://github.com/compiler-explorer/asm-parser If you have some examples of in -> out for unittesting that will probably go a long way

Hopefully done in compiler-explorer/asm-parser#31

mattgodbolt

LGTM! Pending the merge and build of the asm-parser too (which I'm not sure how to do)

mattgodbolt · 2023-01-11T02:25:15Z

docs/API.md

@@ -65,6 +65,7 @@ To specify a compilation request as a JSON document, post it as the appropriate
        },
        "filters": {
             "binary": false,
+             "binaryObject": false,


Thanks 👍🏻

partouf · 2023-01-11T15:21:34Z

LGTM! Pending the merge and build of the asm-parser too (which I'm not sure how to do)

https://github.com/compiler-explorer/infra/blob/main/bin/yaml/tools.yaml#L66

I believe it's merge the PR in asm-parser repo - wait for the workflow to build the new version, then on admin do ce_install install --force asm-parser
and then restart the instances to take effect (new version + environment refresh)

partouf · 2023-01-11T15:23:10Z

Oh, I'm wrong, it might also require making a release here https://github.com/compiler-explorer/asm-parser/releases

With the merge of #3232, we need to distinguish "compile to binary object" (compiler + assembler) and "compile to binary" (compiler + assembler + linker). Renaming "compile to binary" as "link to binary" is a more correct in this context and less confusing. We don't rename the internal field (kept as 'binary') because it's part of the API and we can't break it. Signed-off-by: Marc Poulhiès <dkm@kataplop.net>

partouf · 2023-01-11T21:13:27Z

this is now live

Condition was badly changed in #3232 (compiling to binary objects)

dkm marked this pull request as draft January 4, 2022 21:03

github-actions bot added the ui label Jan 4, 2022

github-actions bot added the lang-rust label Jan 5, 2022

This was referenced Jan 5, 2022

[BUG]: Missing GCC output with std::atomic and "compile to binary" #3221

Closed

Demangler breaks assembly parser #3235

Closed

github-actions bot added the lang-ada label Jan 6, 2022

dkm force-pushed the dkm/compile_binary_obj branch 2 times, most recently from 4f13edd to faf233e Compare January 20, 2022 21:49

github-actions bot added the documentation label Jan 20, 2022

dkm force-pushed the dkm/compile_binary_obj branch from faf233e to c186cc1 Compare January 20, 2022 21:51

github-actions bot added the lang-c label Jan 23, 2022

dkm force-pushed the dkm/compile_binary_obj branch from 7abfa55 to 2a5b892 Compare January 25, 2022 21:42

dkm force-pushed the dkm/compile_binary_obj branch from 2a5b892 to 4444d65 Compare February 23, 2022 07:49

dkm force-pushed the dkm/compile_binary_obj branch from 4444d65 to a9f189a Compare April 9, 2022 09:01

dkm force-pushed the dkm/compile_binary_obj branch from a9f189a to 1d15d60 Compare May 11, 2022 19:54

dkm force-pushed the dkm/compile_binary_obj branch from 1d15d60 to d9f6848 Compare August 22, 2022 18:15

github-actions bot removed the lang-rust label Aug 22, 2022

dkm force-pushed the dkm/compile_binary_obj branch from e48eada to 97922c1 Compare October 22, 2022 13:00

github-actions bot added the lang-jakt label Oct 22, 2022

dkm force-pushed the dkm/compile_binary_obj branch from 97922c1 to 14cc1d5 Compare November 4, 2022 18:19

dkm force-pushed the dkm/compile_binary_obj branch from 14cc1d5 to c5e225f Compare November 17, 2022 20:12

github-actions bot added the lang-c++ label Nov 17, 2022

mattgodbolt self-requested a review January 8, 2023 23:14

mattgodbolt reviewed Jan 8, 2023

View reviewed changes

dkm added 2 commits January 9, 2023 20:02

fixup! Support for compiling to binary object

ed802bf

fixup! Support for compiling to binary object

a80cf32

partouf reviewed Jan 9, 2023

View reviewed changes

lib/base-compiler.ts Outdated Show resolved Hide resolved

fixup! Support for compiling to binary object

158a78f

partouf reviewed Jan 9, 2023

View reviewed changes

lib/base-compiler.ts Outdated Show resolved Hide resolved

dkm added 2 commits January 9, 2023 21:32

fixup! Support for compiling to binary object

e362173

fixup! Support for compiling to binary object

b832bee

github-actions bot added the lang-rust label Jan 9, 2023

dkm commented Jan 9, 2023

View reviewed changes

RubenRBS self-requested a review January 10, 2023 21:11

mattgodbolt approved these changes Jan 11, 2023

View reviewed changes

fixup! Support for compiling to binary object

c056a56

partouf merged commit 6905568 into main Jan 11, 2023

partouf deleted the dkm/compile_binary_obj branch January 11, 2023 18:39

dkm mentioned this pull request Jan 11, 2023

rename: compile to binary -> link to binary #4577

Merged

Alcaro mentioned this pull request Jan 11, 2023

[REQUEST]: Checking compile to binary object should uncheck link to binary #4578

Open

dkm added a commit that referenced this pull request Jan 12, 2023

fix: pahole wrongly refuses to do anything

d202281

Condition was badly changed in #3232 (compiling to binary objects)

dkm mentioned this pull request Jan 12, 2023

fix: pahole wrongly refuses to do anything #4581

Merged

dkm added a commit that referenced this pull request Jan 12, 2023

fix: pahole wrongly refuses to do anything (#4581)

6d0d81d

Condition was badly changed in #3232 (compiling to binary objects)

mattgodbolt pushed a commit that referenced this pull request Jan 24, 2023

Support for compiling to binary object (#3232)

7403be8

mattgodbolt pushed a commit that referenced this pull request Jan 24, 2023

fix: pahole wrongly refuses to do anything (#4581)

1751ad3

Condition was badly changed in #3232 (compiling to binary objects)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for compiling to binary object #3232

Support for compiling to binary object #3232

dkm commented Jan 4, 2022 •

edited

dkm commented Jan 4, 2022

dkm commented Jan 5, 2022

dkm commented Jan 5, 2022

dkm commented Jan 23, 2022

dkm commented Jan 23, 2022

partouf commented Apr 9, 2022

dkm commented May 3, 2022 •

edited

RubenRBS commented Nov 9, 2022 •

edited

partouf commented Jan 8, 2023

mattgodbolt left a comment

dkm commented Jan 9, 2023

dkm Jan 9, 2023

partouf Jan 9, 2023 •

edited

dkm Jan 10, 2023

dkm commented Jan 10, 2023

mattgodbolt left a comment

mattgodbolt Jan 11, 2023

partouf commented Jan 11, 2023 •

edited

partouf commented Jan 11, 2023

partouf commented Jan 11, 2023

Support for compiling to binary object #3232

Support for compiling to binary object #3232

Conversation

dkm commented Jan 4, 2022 • edited

dkm commented Jan 4, 2022

dkm commented Jan 5, 2022

dkm commented Jan 5, 2022

dkm commented Jan 23, 2022

dkm commented Jan 23, 2022

partouf commented Apr 9, 2022

dkm commented May 3, 2022 • edited

RubenRBS commented Nov 9, 2022 • edited

partouf commented Jan 8, 2023

mattgodbolt left a comment

Choose a reason for hiding this comment

dkm commented Jan 9, 2023

dkm Jan 9, 2023

Choose a reason for hiding this comment

partouf Jan 9, 2023 • edited

Choose a reason for hiding this comment

dkm Jan 10, 2023

Choose a reason for hiding this comment

dkm commented Jan 10, 2023

mattgodbolt left a comment

Choose a reason for hiding this comment

mattgodbolt Jan 11, 2023

Choose a reason for hiding this comment

partouf commented Jan 11, 2023 • edited

partouf commented Jan 11, 2023

partouf commented Jan 11, 2023

dkm commented Jan 4, 2022 •

edited

dkm commented May 3, 2022 •

edited

RubenRBS commented Nov 9, 2022 •

edited

partouf Jan 9, 2023 •

edited

partouf commented Jan 11, 2023 •

edited