Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide posiblity to define if sources and/or binaries data type is supported by PDS #1172

Closed
de-jcup opened this issue Mar 29, 2022 · 1 comment · Fixed by #1773
Closed

Comments

@de-jcup
Copy link
Member

de-jcup commented Mar 29, 2022

Situation

This is a kind of sub issue of #729, #1098 but mainly of #1166

At the moment, every code scan will insist on fetching SOURCE.zip from s3.
Because we introduce BINARY.tar for binary uploads, we must have a possibility to define which one shall be handled

Wanted

We introduce new configuration entries

Solution

  • Introduce new enum StorageType with SOURCE and BINARY inside

  • Introduce StorageSupport class which will provide logic to fetch streams automatically and also to
    handle missing mandatory storages (depending on type)

  • PDS shall use the the runtime product executor configuration to determine the supported data storage types

  • At PDS configuration we introduce pds.config.supported.datatype=$typedefinition.
    The type definition can be either binary , source or 'none'. Also a combination as comma separated entries - e.g. binary,source is possible . It can be either optional or mandatory - it does not matter. PDS solutions should have a default value here. If nothing is defined (neither in executor configuration or by a default value inside the PDS configuration) there will be a fallback to the current situation (binary AND source are downloaded + a WARNING will be logged about a missing configuration, but only one time at PDS instance start)

  • We will reuse SecHubDataConfigurationType enum here

  • When a data type is NOT supported, than we do NOT extract or download those files

  • When a data type is supported we try to extract and download the file

@de-jcup de-jcup added this to the Server 0.30.0 milestone Mar 29, 2022
@de-jcup de-jcup changed the title Provide posiblity to define if sourcecode.zip and/or binary.tar shall be fetched from storage Provide posiblity to define if sourcecode.zip and/or binaries.tar shall be fetched from storage Apr 11, 2022
@Jeeppler Jeeppler assigned Jeeppler and de-jcup and unassigned Jeeppler Apr 25, 2022
@sven-dmlr sven-dmlr modified the milestones: Server 0.31.0, Server 0.32.0 May 2, 2022
@sven-dmlr sven-dmlr modified the milestones: Server 0.33.0, Server 0.34.0 Jun 2, 2022
@de-jcup
Copy link
Member Author

de-jcup commented Nov 11, 2022

Some of the parts were already done inside other issues (these are crossed out in the comment) - but the configuration if a PDS does need sourcecode and/or binaries is still missing.
This is important for #1744

@de-jcup de-jcup modified the milestones: Server 0.36.0, PDS 0.34.0 Nov 14, 2022
@de-jcup de-jcup added the pds label Nov 14, 2022
@de-jcup de-jcup changed the title Provide posiblity to define if sourcecode.zip and/or binaries.tar shall be fetched from storage Provide posiblity to define if sources and/or binaries data type is supported by PDS Nov 14, 2022
de-jcup added a commit that referenced this issue Nov 14, 2022
- introduced 'pds.config.supported.datatypes' as job parameter
- introduced NONE in SecHubDataConfigurationType
- introduced SecHUbDataConfigurationTypeListParser
- introduced configuration autofix on PDS start
- added tests
- changed integration test setup
- fixed typos
- renamed some methods
de-jcup added a commit that referenced this issue Nov 22, 2022
- introduced 'pds.config.supported.datatypes' as job parameter
- introduced NONE in SecHubDataConfigurationType
- introduced SecHUbDataConfigurationTypeListParser
- introduced configuration autofix on PDS start
- added tests
- changed integration test setup
- fixed typos
- renamed some methods
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants