Bdio 2 #85
Bdio 2 #85
Conversation
|
Will be adding tests soon. |
Black Duck Security ReportBranch master has no Black Duck results, and could not be compared to #85. Analyze branch master to get a change comparison. Removed ComponentsHigh Risk: 1 |
src/main/java/com/synopsys/integration/detect/workflow/bdio/CodeLocationBdioCreator.java
Outdated
Show resolved
Hide resolved
| final Bdio2Document bdio2Document = bdio2Factory.createBdio2Document(bdioMetadata, bdio2Project, dependencyGraph); | ||
|
|
||
| final Bdio2Writer bdio2Writer = new Bdio2Writer(); | ||
| final File bdio2OutputFile = new File(bdioOutput, bdioCodeLocation.getBdioName() + ".bdio"); |
There was a problem hiding this comment.
This .bdio file would be a zip file, the Blackduck endpoint for Polaris is good to consume chunks of the data and not the whole zip file.
There was a problem hiding this comment.
Can Polaris consume the zipped .bdio format?
There was a problem hiding this comment.
We are currently planning to only output the .bdio archive for now. If this is a problem for Polaris we need to reconsider our approach.
| final File bdio2OutputFile = new File(bdioOutput, bdioCodeLocation.getBdioName() + ".bdio"); | ||
| try { | ||
| final OutputStream outputStream = new FileOutputStream(bdio2OutputFile); | ||
| bdio2Writer.writeBdioDocument(outputStream, bdio2Document); |
There was a problem hiding this comment.
Some questions:
- Is chunking supported?
- Is this file an archive of multiple files (including the header and one or more entries)?
- Have you tried uploading the old generated BDIO1 and the new generated BDIO2 documents (against the same repo target) and confirmed the results and relationships are the same?
- Is it possible to dry-run the BDIO2 contents to disk?
There was a problem hiding this comment.
- Not for 6.1.0. That is coming later.
- Yes it is an archive with the .bdio file extension.
- Yes and results were the same.
- For 6.1.0 Detect can only generate bdio in offline mode, and in that mode files from the run are preserved on the disk. When we support bdio 2 in online mode the user can also set detect.cleanup=false to have the files persist.
There was a problem hiding this comment.
Thanks. For SCA on Polaris, the intent will be for the Polaris CLI to drive Detect dependency scans with output of the BDIO2 header and chunked entry files. For standalone Black Duck, the system primarily has accepted start mode header, multiple chunk additions, and a completion message. The archive has primarily been for generic download/upload via the UI, but we don't have any clients using this mechanism right now.
There was a problem hiding this comment.
The BDIO library (https://github.com/blackducksoftware/bdio) currently has no implementation of a StreamSupplier other than a BdioFile (https://github.com/blackducksoftware/bdio/blob/1cd704a24ee2affbf03b635e3e41d5be8b2fd329/bdio2/src/main/java/com/blackducksoftware/bdio2/BdioWriter.java#L74). If another implementation of StreamSupplier was implemented that only outputs the header and chunked entries instead of an archive, we could use that.
There was a problem hiding this comment.
@JakeMathews @ketul-shukla, can you get together with Jake to better understand this mechanism? Currently, we are definitely chunking BDIO2 files internally. What is the library mechanism to support this type of structure?
Added support for bdio 2 in Detect. There are a lot of changes here which mostly pertain to upgrading the version of integration-bdio we are using. We will generate bdio 2 but not upload it in 6.1.0 of Detect.