Introduce data section inside sechub configuration file

[company-jcup]

Situation

We currently provide only source code uploads inside codeScan configuration areas.
But we have planned to support also

• secretScan
• codeScan (with binaries)
• dependencyScan (with sources and binaries)
• licenseScan
• ... more ...

Which contains binary content as well - and this is currently not possible inside the configuration.
Also it is not possible to reuse an upload for all kind of scan types (e.g. when defining a secretScan and a codeScan in same configuration file )

Wanted

As a SecHub user I want to define a configuration file where I define my data (binary and/or source content), upload it once and use this for different kind of scans in ONE job.

Existing job configurations for users currently only performing code scans shall still work.

Solution

Codescan can be defined as before

We will provide following as before:

{
      "apiVersion": "1.0",
      "server"    : "https://sechub.example.org",

      "project"   : "gamechanger",

      "codeScan": {
         "fileSystem": {
              "folders": ["gamechanger-android/src/main/java",
                          "gamechanger-server/src/main/java"]
         },
         "excludes": ["**/mytestcode/**","**/documentation/**"],
         "additionalFilenameExtensions": [".cplusplus",".py9"]
      }
}

But the segment for code scan ("fileSystem" ) is only a shortcut for:

{
      "apiVersion": "1.0",
      "server"    : "https://sechub.example.org",

      "project"   : "gamechanger",
      
      "data" : {
           "sources": [
             {
                "fileSystem": { 
                    "folders": ["gamechanger-android/src/main/java",
                            "gamechanger-server/src/main/java"] 
                 },
              "excludes": ["**/mytestcode/**","**/documentation/**"],  
              "additionalFilenameExtensions": [".cplusplus",".py9"]  
            }
          ]
      },

      "codeScan": { 
         
      }
}

New possiblities for multiple scans for same upload or binary content

While introducing "data" as new element inside our json configuration, we will also

• provide mandatory name attribute inside data object elements
• we introduce also files inside fileSystem so we can define also single files
• we can reference named object elements via "use" attribute inside different scan type configurations

Here a full blown example (with massive configurations):

{
  "apiVersion" : "1.0",
  "server" : "https://sechub.example.org",
  "project" : "gamechanger",
  "data" : { // contains the different kind of data objects ( sources, binaries)
    "sources" : [ {
      "name" : "android-sources",
      "fileSystem" : {
        "folders" : [ "gamechanger-android/src/main/java", "gamechanger-server/src/main/java" ]
      },
      "excludes" : [ "**/mytestcode/**", "**/documentation/**" ],
      "additionalFilenameExtensions" : [ ".cplusplus", ".py9" ]
    }, {
      "name" : "openapi-reference",
      "fileSystem" : {
        "files" : [ "gamechanger-webapp/src/main/resources/openapi3.json" ]
      }
    }, {
      "name" : "secret-scanner-config",
      "fileSystem" : {
        "files" : [ "gamechanger-data/src/main/resources/secret-scanner-dictionary.json" ] // new: files
      }
    } ],
    "binaries" : [ { // new: binaries!
      "name" : "android-libraries",
      "fileSystem" : {
        "folders" : [ "gamechanger-android/lib/dlls", "gamechanger-server/lib/so" ]
      },
      "excludes" : [ "**/*.jpg", "myimages/**" ]
    } ]
  },
  // Define wanted scan and reference - if wanted the named data object
  "codeScan" : {
    "use" : [ "android-sources","android-libraries","openapi-reference"]
  },
  "dependencyScan" : { // fictional: currently this scan type does not exist, just an example
    "use" : [ "android-sources", "android-libraries" ]
  },
  "secretScan" : {// fictional: currently this scan type does not exist, just an example
    "use" : [ "secret-scanner-config" ],
    "featureToggle" : {
      "entropy" : "off"
    }
  },
  "licenseScan" : {// fictional: currently this scan type does not exist, just an example
    "use" : [ "android-sources","android-libraries" ]
  },
  "webScan" : {
    "url" : "https://example.com",
    "openApi" : {
      "use" : [ "openapi-reference"] 
    }
  }
}

[de-jcup]

• The names of referenced data configuration objects must be unique. There must be a validation inside the model.
• Names inside data section may not be empty or null
• Referenced names inside data section must exist
• A code scan configuration shall not have the possibility to define a name - so it has no name cannot be referenced outside
  code scan

[de-jcup]

Remark: This issue is a precondition for #1152