Make repositories configurable for enterprise developers

[githendrik]

Allow enterprise developers to build it behind corporate firewalls

Issue faced

At my company we're prohibited to pull from most public registries (Gradle, Maven, Npm, ...) directly and are forced to go via artifactory mirrors. Unfortunately Datahub doesn't currently support this.

I've dug into the code and propse the following changes to allow customisation of the docker builds. I've changed all Dockerfiles and the required build.gradle files.

How I now build the project is by setting the following GRADLE_OPTS:

GRADLE_OPTS_LIST=(
 "-Dorg.gradle.internal.plugins.portal.url.override=https://jfrog.enterprise.corp/artifactory/gradle-plugins-release-remote"
 "-Dorg.gradle.project.nodeDistBaseUrl=https://jfrog.enterprise.corp/artifactory/nodejs-dist-remote/"
 "-Dorg.gradle.project.alpineApkRepositoryUrl=https://jfrog.enterprise.corp/artifactory/alpine-remote"
 "-Dorg.gradle.project.debianAptRepositoryUrl=https://jfrog.enterprise.corp/artifactory/debian-deb-remote"
 "-Dorg.gradle.project.githubMirrorUrl=https://jfrog.enterprise.corp/artifactory/github-generic-remote"
 "-Dorg.gradle.project.mavenCentralRepositoryUrl=https://jfrog.enterprise.corp/artifactory/maven-remote"
 "-Dorg.gradle.project.apacheMavenRepositoryUrl=https://jfrog.enterprise.corp/artifactory/apache-maven-remote"
 "-Dorg.gradle.project.confluentMavenRepositoryUrl=https://jfrog.enterprise.corp/artifactory/confluent-maven-remote"
 "-Dorg.gradle.project.linkedinOpenSourceRepositoryUrl=https://jfrog.enterprise.corp/artifactory/linkedin-open-source-maven-remote"
 "-Dorg.gradle.project.pipMirrorUrl=https://jfrog.enterprise.corp/artifactory/api/pypi/pypi-remote/simple"
 "-Dorg.gradle.project.apacheDownloadUrl=https://jfrog.enterprise.corp/artifactory/apache-generic-remote"
)

Once these are set, I'm able to build the following gradle targets successfully:

:datahub-frontend:docker -x yarnTest -x yarnLint
:metadata-service:war:docker
:datahub-upgrade:docker
:docker:datahub-ingestion:docker
:docker:datahub-ingestion-base:docker
:metadata-jobs:mae-consumer-job:docker
:metadata-jobs:mce-consumer-job:docker
:docker:elasticsearch-setup:docker
:docker:kafka-setup:docker
:docker:mysql-setup:docker
:docker:postgres-setup:docker

When exiting the corporate network, I'm still able to build the same targets after unsetting the GRADLE_OPTS, so I assume everything will simply keep working for everyone else.

Please let me know if you see something missing (or, come to think of it, if I should add these configuration flags to a documentation page somewhere).

Checklist

• The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
• Links to related issues (if applicable)
• Tests for the changes have been added/updated (if applicable)
• [] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
• For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

[david-leifker]

LGTM! Thank you!