Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streamline how VERSION is provided for all assembly and deployment rules #150

Closed
haikalpribadi opened this issue Jul 3, 2019 · 1 comment · Fixed by vaticle/typedb#5468

Comments

@haikalpribadi
Copy link
Member

At the moment, we have different ways to VERSION our distribution for assembly and deployment.

For example, for assemble_rpm and deploy_rpm, we strictly rely on the content of the //:VERSION file, so we overwrite the file before calling assembling or deploying.

echo $(cat VERSION)_$CIRCLE_SHA1 > VERSION && cat VERSION
bazel run //:deploy-rpm -- snapshot

For assemble_maven and deploy_maven, we repackage the JAR at deployment with the version we want:

bazel run //common:deploy-maven -- snapshot $CIRCLE_SHA1
bazel run //common:deploy-maven -- release $(cat VERSION)

This is very inconsistent, and in cases where we need to overwrite the //:VERSION file, it's actually problematic/error-prone: what if we a developer forgets to update the //:VERSION file and calls bazel run //:deploy-rpm -- snapshot, because it's not one canonical step?

Let's streamline the way version our distribution for all assembly and deployment rules as described below.

  1. Assembly rules do not take in version_file = "//:VERSION" any more, and can optionally be versioned using Bazel environment variables. For example:

    • bazel build //common:assemble-maven will produce a Maven JAR with version 0.0.0 (aka. "unversioned").
    • bazel build --define version=$GIT_SHA //common:assemble-maven will produce Maven JAR with a git commit as the version.
    • bazel build --define version=$(cat VERSION) //common:assemble-maven will produce a Maven JAR with the content of //:VERSION (e.g. 1.5.7) as the version.
  2. Deployment rules remain to be parameterised with release or snapshot, but do not take in any version arguments any more. They will verify the distribution they receive from assembly rules is versioned properly, unique to each platform's versioning syntax requirements. For example:

    • bazel run //common:deploy-maven -- snapshot will make sure that the Maven jar is not versioned 0.0.0 (i.e. not "unversioned")
    • bazel run //common:deploy-maven -- snapshot will make sure that they Maven jar is semantically version (X.Y.Z-*) (this just an example)
    • bazel run //:deploy-rpm -- release will make sure that the RPM distro is versioned X.Y.Z-{architecture} (this is just an example)

The above design should be applied to all assemble_* and deploy_* rules.

@haikalpribadi
Copy link
Member Author

Make sure to also apply the same design to assemble_zip and assemble_targz, so we can get rid of this annoying wait time @vmax :
versioning distribution

vmax added a commit that referenced this issue Jul 12, 2019
## What is the goal of this PR?

Incremental work on #150 for `deploy_brew` rule

## What are the changes implemented in this PR?

- Make `version_file` in `deploy_brew` optional
- Exposes version from Bazel command line as a file if `version_file` is not present.

### Sample usage
Invoking assembly would be the same; `--define version=<VERSION>` would be added as command-line argument:
`bazel run --define version=$(git rev-parse HEAD) //:deploy-brew`
vmax added a commit that referenced this issue Jul 12, 2019
## What is the goal of this PR?

Incremental work on #150 for `assemble_apt` rule

## What are the changes implemented in this PR?

- Make `version_file` in `assemble_apt` optional
- Add private rule (`_assemble_apt_version_file`) which exposes version from Bazel command line as a file. Use this file in absence of `version_file` in `assemble_apt`.

### Sample usage
Invoking assembly would be the same; `--define version=<VERSION>` would be added as command-line argument:
`bazel build --define version=$(git rev-parse HEAD) //console:assemble-linux-apt`
vmax added a commit that referenced this issue Jul 12, 2019
## What is the goal of this PR?

Incremental work on #150 for `java_deps` rule

## What are the changes implemented in this PR?

- Make `version_file` in `java_deps` optional
- Exposes version from Bazel command line as a file if `version_file` is not present.

### Sample usage
Invoking assembly would be the same; `--define version=<VERSION>` would be added as command-line argument:
`bazel build --define version=$(git rev-parse HEAD) //console:console-deps`
vmax added a commit that referenced this issue Jul 15, 2019
## What is the goal of this PR?

Incremental work on #150 for `deploy_github` rule

## What are the changes implemented in this PR?

- Make `version_file` in `deploy_github` optional
- Add missing symlink to `VERSION`
- Exposes version from Bazel command line as a file if `version_file` is not present.

### Sample usage
Invoking assembly would be the same; `--define version=<VERSION>` would be added as command-line argument:
`bazel run --define version=$(git rev-parse HEAD) //:deploy-github`
vmax added a commit that referenced this issue Jul 15, 2019
## What is the goal of this PR?

Incremental work on #150 for `assemble_maven` rule

## What are the changes implemented in this PR?

- Make `version_file` in `assemble_maven` optional
- Exposes version from Bazel command line as a file if `version_file` is not present.
- Replace `{pom_version}` [placeholder] in `pom.xml`
- Remove `version_file` from runfiles of assemble script — it's not used there

### Sample usage
Invoking assembly would be the same; `--define version=<VERSION>` would be added as command-line argument:
`bazel build --define version=$(git rev-parse HEAD) //console:assemble-maven`
vmax added a commit that referenced this issue Jul 15, 2019
## What is the goal of this PR?

Incremental work on #150 for `assemble_npm` rule

## What are the changes implemented in this PR?

- Make `version_file` in `assemble_npm` optional
- Exposes version from Bazel command line as a file if `version_file` is not present.
- Follow-up to #164: add docs to `version_file` of `assemble_maven`

### Sample usage
Invoking assembly would be the same; `--define version=<VERSION>` would be added as command-line argument:
`bazel build --define version=$(git rev-parse HEAD) //:assemble-npm`
vmax added a commit that referenced this issue Jul 15, 2019
## What is the goal of this PR?

Incremental work on #150 for `assemble_pip` rule

## What are the changes implemented in this PR?

- Make `version_file` in `assemble_pip` optional
- Exposes version from Bazel command line as a file if `version_file` is not present.

### Sample usage
Invoking assembly would be the same; `--define version=<VERSION>` would be added as command-line argument:
`bazel build --define version=$(git rev-parse HEAD) //:assemble-pip`
vmax added a commit that referenced this issue Jul 15, 2019
## What is the goal of this PR?

Incremental work on #150 for `assemble_rpm` rule

## What are the changes implemented in this PR?

- Make `version_file` in `assemble_rpm` optional
- Exposes version from Bazel command line as a file if `version_file` is not present.

### Sample usage
Invoking assembly would be the same; `--define version=<VERSION>` would be added as command-line argument:
`bazel build --define version=$(git rev-parse HEAD) //console:assemble-rpm`
haikalpribadi pushed a commit that referenced this issue Oct 9, 2019
## What is the goal of this PR?

Partial work on #150

Previously, we had to repack already-built `tar.gz` and `zip` distribution once again due to two reasons:
(1) - archive name doesn't have version in it (`grakn-core-all-linux.tar.gz` instead of `grakn-core-all-linux-1.5.9.tar.gz`)
(2) root folder _inside_ the archive didn't have version in it (`grakn-core-all-linux` instead of `grakn-core-all-linux-1.5.9`)

This pull request addresses (2).
Now, when `--define version=<>` is supplied during build time, it is used to construct root folder name inside the archive.

## What are the changes implemented in this PR?

* `pkg_tar` and `pkg_deb` are now loaded from `@rules_pkg` — version in `@bazel_tools` is going to be deprecated soon
* new rule `_assemble_targz_package_dir_file`: reads `package_file` from attributes and `version` from `--define version=<>` during build time; outputs a file which contains `package_file` and `version` separated by dash (if both are present). Output of this rule is later used as a root folder for `.tar.gz` package
* new rule `_assemble_zip_prefix_file`: reads `prefix` from attributes and `version` from `--define version=<>` during build time; outputs a file which contains `prefix` and `version` separated by dash (if both are present).  Output of this rule is later used as a root folder for `.zip` package. Additionally, `tgz2zip` rule is modified to be able to accept `prefix_file` [purpose is the same as `prefix`]
* `assemble_versioned` rule no longer changes files _inside_ the archives — only renaming of archives themselves is done
* `assemble_versioned` rule no longer performs compression on archive it generated (`ZIP_STORED` instead of `ZIP_DEFLATED`). While generated archive takes more space, build performance is 4x faster this way.

Please note: changes in invocation of `assemble_versioned` targets (as well as on targets which transitively depend on it, such as `deploy_github`) are needed. Specifically, they now have to be invoked as:

```
bazel run --define version=$(cat VERSION) //:deploy-github -- <old arguments>
```
haikalpribadi pushed a commit that referenced this issue Oct 10, 2019
## What is the goal of this PR?

Partial work on #150

## What are the changes implemented in this PR?

- `deploy_maven` no longer accepts version as command-line argument. Additionally, it no longer modifies `pom.xml` and `jar` which are to be deployed. Instead, it assumes that dependent `assemble_maven` target is already properly versioned by providing `--define version=<VERSION>`
- chore: regenerate docs
vmax added a commit that referenced this issue Oct 10, 2019
## What is the goal of this PR?

Partial work on #150

## What are the changes implemented in this PR?

* `deploy_npm` no longer modifies the archive provided by `assemble_npm` if `repo_type` is `snapshot`. Instead, it assumes archive generated by `assemble_npm`  already contains correct version.
* `assemble_npm` now prepends `0.0.0-` if version resembles a commit SHA
haikalpribadi pushed a commit that referenced this issue Oct 11, 2019
## What is the goal of this PR?

Partial work on #150 

## What are the changes implemented in this PR?

* Cleanup: remove old `setup.py` and `deploy.sh`
* if version resembles a commit, prepend `0.0.0` to it
vmax added a commit that referenced this issue Oct 15, 2019
## What is the goal of this PR?

Partial work on #150

## What are the changes implemented in this PR?

`assemble_versioned` no longer requires user to provide `version_file`. Instead it can be specified as `bazel build --define version=$(cat VERSION) <assemble_versioned_target>`
vmax added a commit that referenced this issue Oct 15, 2019
)

## What is the goal of this PR?

Partial work on #150

## What are the changes implemented in this PR?

* If version resembles commit, prepend `0.0.0_` to it
* Rewrite `deploy_rpm` in Python and remove old bash implementation
vmax added a commit to vaticle/typedb that referenced this issue Oct 16, 2019
## What is the goal of this PR?

Fix vaticle/bazel-distribution#150

## What are the changes implemented in this PR?

`version_file` argument is no longer used to provide version. As instead, `--define version=<>` argument is passed to Bazel everywhere a version is required
flyingsilverfin pushed a commit to vaticle/typedb-driver-python that referenced this issue Oct 16, 2019
## What is the goal of this PR?

Adapt `@graknlabs_client_python` to latest changes in bazel-distribution (in particular, vaticle/bazel-distribution#150)

## What are the changes implemented in this PR?

Instead of supplying `version_file` everywhere, use `--define version=<>` as a Bazel argument to supply version.
vmax added a commit to vaticle/typedb-console that referenced this issue Oct 16, 2019
## What is the goal of this PR?

Adapt `@graknlabs_console` to latest changes in bazel-distribution (in particular, vaticle/bazel-distribution#150)

## What are the changes implemented in this PR?

Instead of supplying `version_file` everywhere, use `--define version=<>` as a Bazel argument to supply version.
lolski pushed a commit to vaticle/typedb-driver-nodejs that referenced this issue Oct 17, 2019
## What is the goal of this PR?

Adapt `@graknlabs_client_nodejs` to latest changes in bazel-distribution (in particular, vaticle/bazel-distribution#150)

## What are the changes implemented in this PR?

Instead of supplying `version_file` everywhere, use `--define version=<>` as a Bazel argument to supply version.
lolski pushed a commit to vaticle/typedb-studio that referenced this issue Oct 17, 2019
## What is the goal of this PR?

Adapt `@graknlabs_workbase` to latest changes in bazel-distribution (in particular, vaticle/bazel-distribution#150)

## What are the changes implemented in this PR?

Instead of supplying `version_file` everywhere, use `--define version=<>` as a Bazel argument to supply version.
lolski pushed a commit to vaticle/typedb-protocol that referenced this issue Oct 17, 2019
## What is the goal of this PR?

Adapt `@graknlabs_protocol` to latest changes in bazel-distribution (in particular, vaticle/bazel-distribution#150)

## What are the changes implemented in this PR?

Instead of supplying `version_file` everywhere, use `--define version=<>` as a Bazel argument to supply version.
lolski pushed a commit to vaticle/typedb-common that referenced this issue Oct 17, 2019
## What is the goal of this PR?

Adapt `@graknlabs_common` to latest changes in bazel-distribution (in particular, vaticle/bazel-distribution#150)

## What are the changes implemented in this PR?

Instead of supplying `version_file` everywhere, use `--define version=<>` as a Bazel argument to supply version.
lolski pushed a commit to vaticle/typeql that referenced this issue Oct 17, 2019
## What is the goal of this PR?

Adapt `@graknlabs_graql` to latest changes in bazel-distribution (in particular, vaticle/bazel-distribution#150)

## What are the changes implemented in this PR?

Instead of supplying `version_file` everywhere, use `--define version=<>` as a Bazel argument to supply version.
jmsfltchr pushed a commit to vaticle/typedb-ml that referenced this issue Oct 17, 2019
## What is the goal of this PR?

Adapt `@graknlabs_kglib` to latest changes in bazel-distribution (in particular, vaticle/bazel-distribution#150)

## What are the changes implemented in this PR?

Instead of supplying `version_file` everywhere, use `--define version=<>` as a Bazel argument to supply version.
vmax added a commit to vaticle/typedb-driver that referenced this issue Oct 17, 2019
## What is the goal of this PR?

Adapt `@graknlabs_client_java` to latest changes in bazel-distribution (in particular, vaticle/bazel-distribution#150)

## What are the changes implemented in this PR?

Instead of supplying `version_file` everywhere, use `--define version=<>` as a Bazel argument to supply version.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants