ScanCode.io: Craft d2d pipeline steps for ELF

[pombredanne]

The design is to extend the "map_deploy_to_develop" with an "Elf" option given the framework in:

• ScanCode.io: Create a package deployment to development (d2d) framework #1111

The process will be to:

• Collect ELF DWARF symbols with the new elf-inspector. See https://github.com/nexB/elf-inspector
• Create a pipeline to inspect ELF binaries https://github.com/nexB/scancode.io/blob/d6389b28841c4edf25075208eaf0708658650d06/scanpipe/pipelines/inspect_elf_binaries.py
• Create D2D steps for ELFs using DWARF paths

[pombredanne]

Some status update:

• We have an ELF pipe wrapping the elf-inspector https://github.com/nexB/scancode.io/blob/main/scanpipe/pipes/elf.py
• We have a pipeline to inspect ELF binaries https://github.com/nexB/scancode.io/blob/d6389b28841c4edf25075208eaf0708658650d06/scanpipe/pipelines/inspect_elf_binaries.py
• We have an early prototype of ELF d2d in main...812-d2d-elf-dwarf ( or nexB:d6389b2...nexB:6a711f0 ) that can be reused as the base for the actual d2d steps

[TG1999]

This is now done and we have a set of proper steps grouped and tagged for JavaScript "binary/webpacked", Java and now this new ELF backtosource analysis that can be selectively used in the context of:

• ScanCode.io: Create a package deployment to development (d2d) framework #1111

The actual Elf-specific pipeline steps look like this:

To test the feature:

• Install the latest ScanCode.io
• Create a new project
• Add these two Download URLs exactly as below using these zip examples:
  • source: https://github.com/nexB/scancode.io/raw/main/scanpipe/tests/data/d2d-elfs/from-data.zip#from
  • binary: https://github.com/nexB/scancode.io/raw/main/scanpipe/tests/data/d2d-elfs/to-data.zip#to
• Select and execute the "map_deploy_to_develop" pipeline, and then click on the "Elf" option

Here is how the project looks like after creation:

• When pipeline run is finished, refresh

• and click on the "relations" , and you can see the mapping between source and binaries:

• At the resource page, there are also file-level mappings details: