[ci][cli] Moving ensures script to config-based batch generation of samples

[jimschubert]

All samples scripts are now config based with a single bash entry point: bin/generate-samples.sh.

File cleanup using the FILES file will be done later.

Elm is currently skipped as it requires that we skip validation of the spec, so will need to be configured.

Running against the 179 configurations in bin/configs I see the following stats:

[SUCCESS] Batch generation completed successfully.
./bin/generate-samples.sh  54.20s user 6.64s system 243% cpu 24.954 total

This should address conversation in #6333.

PR checklist

• Read the contribution guidelines.
• If contributing template-only or documentation-only changes which will change sample output, build the project before.
• Run the shell script(s) under ./bin/ (or Windows batch scripts under.\bin\windows) to update Petstore samples related to your fix. This is important, as CI jobs will verify all generator outputs of your HEAD commit, and these must match the expectations made by your contribution. You only need to run ./bin/{LANG}-petstore.sh, ./bin/openapi3/{LANG}-petstore.sh if updating the code or mustache templates for a language ({LANG}) (e.g. php, ruby, python, etc).
• File the PR against the correct branch: master, 4.3.x, 5.0.x. Default: master.
• Copy the technical committee to review the pull request if your PR is targeting a particular programming language.

[auto-labeler]

👍 Thanks for opening this issue!
🏷 I have applied any labels matching special text in your issue.

The team will review the labels and make any necessary changes.

[spacether]

What is the other folder name meant to tell us or mean?
https://github.com/OpenAPITools/openapi-generator/tree/ensures/bin/configs/other

[spacether]

How about changing the name of the configs folder to something like samples_configs to convey that these are the config files for sample generation? configs is very generic and samples_configs helps people understand that this folder exists to configure our sample generation.

[jimschubert]

I'll write up more when this isn't a wip. But "other" are those that aren't in the ensure-up-to-date script in current master.

As for the suggested rename of the folder, I don't think that's necessary. When I'm done and I've updated docs, it'll be obvious enough what these configs are, especially with the naming of each yaml file.

[jimschubert]

Regarding the two TODO items... swift checks will have to be post-merge. Also for appveyor, we may consider using the following GitHub workflow so we have consistency in how we execute the script via bash:

name: Build
on:
  push:
    branches:  
      - master
  pull_request:
    branches:
      - master

jobs:
  build:
    name: Build on JDK ${{ matrix.java }} and ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        java: [8, 11]
        os: [ubuntu-latest, windows-latest]
        
    steps:
      - name: Check out code
        uses: actions/checkout@v2

      - name: Set up JDK ${{ matrix.java }}
        uses: actions/setup-java@v1 
        with:
          java-version: ${{ matrix.java }}
          
      - uses: actions/cache@v1
        with:
          path: ~/.m2/repository
          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
          restore-keys: |
            ${{ runner.os }}-maven-
        
      - name: Build with Maven
        shell: bash
        run: mvn -nsu -B --quiet -Djacoco.skip=true -Dorg.slf4j.simpleLogger.defaultLogLevel=error --no-transfer-progress clean install --file pom.xml ${{ matrix.flags }}
        
      - name: Test gradle
        shell: bash
        run: gradle -b modules/openapi-generator-gradle-plugin/samples/local-spec/build.gradle buildGoSdk --stacktrace

      - name: Upload Maven build artifact
        uses: actions/upload-artifact@v1
        if: matrix.java == '8'
        with:
          name: artifact
          path: modules/openapi-generator-cli/target/openapi-generator-cli.jar

  verify:
    name: Verifies integrity of the commit on ${{ matrix.os }}
    needs: build
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest]
    steps:
      - name: Check out code
        uses: actions/checkout@v2
      - name: Download build artifact
        uses: actions/download-artifact@v1
        with:
          name: artifact
      - name: Run Ensures Script
        run: |
          mkdir -p modules/openapi-generator-cli/target/
          mv artifact/openapi-generator-cli.jar modules/openapi-generator-cli/target/
          ./bin/utils/ensure-up-to-date

I've tested this in a private repository against this branch and it takes about 12 minutes to:

• Build Java 8 (Linux)
  • smoke test generation via gradle
  • (for Java 8 only) Cache the jar for use in the verification step
• Build Java 8 (Windows)
  • smoke test generation via gradle
• Build Java 11 (Linux)
  • smoke test generation via gradle
• Build Java 11 (Windows)
  • smoke test generation via gradle
• Run ensure-up-to-date (Linux)
• Run ensure-up-to-date (Windows)

The 4 build jobs run in parallel, once completed the verify job runs Linux+Windows in parallel.

As a separate verification, we could potentially move a couple sample tests here such as Go and Swift.

[spacether]

Looks good to me

[wing328]

Overall it looks good 👍

Did some tests locally (mac, windows) and here are 3 minor feedback:

• In https://github.com/OpenAPITools/openapi-generator/pull/6509/files#diff-195a635ad245ded9076330e31134bd80R8, I would suggest adding a line or 2 to ask Windows users install Git BASH as all the batch files are gone (Git bash works for me under Windows to run the generate script under ./bin/ to update samples) so we need put down some instructions to guide Windows users on how to update the sample.
• personally I prefer the generate script shows what files have been generated and also some generators will show useful information such as file post-processing, which will be helpful to new users/contributors.
• the script will wait for 5 seconds no matter what so I'll see if I can file a PR later to skip the 5-second wait time if there's at least one argument provided (so not updating all samples) as I update the samples a lot :)

[wing328]

I'll try to address the bitrise.io CI failure tomorrow and merge if no further questions/feedback on this PR.

[wing328]

I'll file follow-up PRs for what I mentioned above and also for some recent changes (e.g. rust client petstore)

[jimschubert]

@wing328 regarding

personally I prefer the generate script shows what files have been generated and also some generators will show useful information such as file post-processing, which will be helpful to new users/contributors.

I guess if the input to the script is a single yaml, we could revert to calling generate rather than batch. I think that information is too verbose when running more than one generator at a time.