Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/pmi 91 update version to 1.1.0 #37

Merged
merged 3 commits into from
Oct 4, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,6 @@ matrix:
include:
- jdk: "oraclejdk8"

before_script: ./jdbc-adapter/tools/version.sh verify

script: ./jdbc-adapter/integration-test-data/run_integration_tests.sh
8 changes: 4 additions & 4 deletions jdbc-adapter/doc/deploying_the_virtual_schema_adapter.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ cd virtual-schemas/jdbc-adapter/
mvn clean -DskipTests package
```

The resulting fat JAR is stored in `virtualschema-jdbc-adapter-dist/target/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar`.
The resulting fat JAR is stored in `virtualschema-jdbc-adapter-dist/target/virtualschema-jdbc-adapter-dist-1.1.0.jar`.

## Uploading the Adapter JAR Archive

Expand All @@ -42,8 +42,8 @@ Following steps are required to upload a file to a bucket:
1. Now upload the file into this bucket, e.g. using curl (adapt the hostname, BucketFS port, bucket name and bucket write password).

```bash
curl -X PUT -T virtualschema-jdbc-adapter-dist/target/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar \
http://w:write-password@your.exasol.host.com:2580/bucket1/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar
curl -X PUT -T virtualschema-jdbc-adapter-dist/target/virtualschema-jdbc-adapter-dist-1.1.0.jar \
http://w:write-password@your.exasol.host.com:2580/bucket1/virtualschema-jdbc-adapter-dist-1.1.0.jar
```

See chapter 3.6.4. "The synchronous cluster file system BucketFS" in the EXASolution User Manual for more details about BucketFS.
Expand Down Expand Up @@ -75,7 +75,7 @@ CREATE JAVA ADAPTER SCRIPT adapter.jdbc_adapter AS

// This will add the adapter jar to the classpath so that it can be used inside the adapter script
// Replace the names of the bucketfs and the bucket with the ones you used.
%jar /buckets/your-bucket-fs/your-bucket/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar;
%jar /buckets/your-bucket-fs/your-bucket/virtualschema-jdbc-adapter-dist-1.1.0.jar;

// You have to add all files of the data source jdbc driver here (e.g. Hive JDBC driver files)
%jar /buckets/your-bucket-fs/your-bucket/name-of-data-source-jdbc-driver.jar;
Expand Down
22 changes: 21 additions & 1 deletion jdbc-adapter/doc/developing_an_sql_dialect.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,7 +228,7 @@ CREATE OR REPLACE JAVA ADAPTER SCRIPT adapter.jdbc_adapter

// This will add the adapter jar to the classpath so that it can be used inside the adapter script
// Replace the names of the bucketfs and the bucket with the ones you used.
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-1.0.2-SNAPSHOT.jar;
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.1.0.jar;

// You have to add all files of the data source jdbc driver here (e.g. MySQL or Hive)

Expand All @@ -241,6 +241,26 @@ In eclipse (or any other Java IDE) you can then attach remotely to the Java Adap

The switch `suspend=y` tells the Java-process to wait until the debugger connects to the Java UDF.

## Version Management

All dialects have the same version as the master project. In the master `pom.xml` file a property called `product-version` is set. Use this in as the artifact version number in the JDBC adapter and all dialects.

Run the script

```bash
jdbc-adapter/tools/version.sh verify
```

To check that all documentation and templates reference the same version number. This script is also used as a build breaker in the continuous integration script.

To update documentation files run

```bash
jdbc-adapter/tools/version.sh unify
```

Note that the script must be run from the root directory of the virtual schema project.

## Troubleshooting

### Setting the Right IP Addresses for Database Connections
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/db2.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ CREATE or replace JAVA ADAPTER SCRIPT adapter.jdbc_adapter AS

// This will add the adapter jar to the classpath so that it can be used inside the adapter script
// Replace the names of the bucketfs and the bucket with the ones you used.
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar;
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.1.0.jar;

// DB2 Driver files
%jar /buckets/bucketfs1/bucket1/db2jcc4.jar;
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/exasol.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ After uploading the adapter jar, the adapter script can be created as follows:
CREATE SCHEMA adapter;
CREATE JAVA ADAPTER SCRIPT adapter.jdbc_adapter AS
%scriptclass com.exasol.adapter.jdbc.JdbcAdapter;
%jar /buckets/your-bucket-fs/your-bucket/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar;
%jar /buckets/your-bucket-fs/your-bucket/virtualschema-jdbc-adapter-dist-1.1.0.jar;
/
```

Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/hive.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ CREATE SCHEMA adapter;
CREATE JAVA ADAPTER SCRIPT jdbc_adapter AS
%scriptclass com.exasol.adapter.jdbc.JdbcAdapter;

%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar;
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.1.0.jar;

%jar /buckets/bucketfs1/bucket1/hive_metastore.jar;
%jar /buckets/bucketfs1/bucket1/hive_service.jar;
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/impala.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ CREATE SCHEMA adapter;
CREATE JAVA ADAPTER SCRIPT jdbc_adapter AS
%scriptclass com.exasol.adapter.jdbc.JdbcAdapter;

%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar;
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.1.0.jar;

%jar /buckets/bucketfs1/bucket1/hive_metastore.jar;
%jar /buckets/bucketfs1/bucket1/hive_service.jar;
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/oracle.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ CREATE JAVA ADAPTER SCRIPT adapter.jdbc_oracle AS

// You need to replace `your-bucket-fs` and `your-bucket` to match the actual location
// of the adapter jar.
%jar /buckets/your-bucket-fs/your-bucket/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar;
%jar /buckets/your-bucket-fs/your-bucket/virtualschema-jdbc-adapter-dist-1.1.0.jar;

// Add the oracle jdbc driver to the classpath
%jar /buckets/bucketfs1/bucket1/ojdbc7-12.1.0.2.jar
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/postgresql.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ CREATE OR REPLACE JAVA ADAPTER SCRIPT adapter.jdbc_adapter

// This will add the adapter jar to the classpath so that it can be used inside the adapter script
// Replace the names of the bucketfs and the bucket with the ones you used.
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar;
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.1.0.jar;

// You have to add all files of the data source jdbc driver here (e.g. MySQL or Hive)
%jar /buckets/bucketfs1/bucket1/postgresql-42.0.0.jar;
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/redshift.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ CREATE OR REPLACE JAVA ADAPTER SCRIPT adapter.jdbc_adapter

// This will add the adapter jar to the classpath so that it can be used inside the adapter script
// Replace the names of the bucketfs and the bucket with the ones you used.
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar;
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.1.0.jar;

// You have to add all files of the data source jdbc driver here (e.g. MySQL or Hive)

Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/sql_server.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ CREATE OR REPLACE JAVA ADAPTER SCRIPT adapter.sql_server_jdbc_adapter

// This will add the adapter jar to the classpath so that it can be used inside the adapter script
// Replace the names of the bucketfs and the bucket with the ones you used.
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar;
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.1.0.jar;

// You have to add all files of the data source jdbc driver here
%jar /buckets/bucketfs1/bucket1/jtds.jar;
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/sybase.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ CREATE OR REPLACE JAVA ADAPTER SCRIPT adapter.jdbc_adapter
AS

%scriptclass com.exasol.adapter.jdbc.JdbcAdapter;
%jar /buckets/bucketfs1/virtualschema/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar;
%jar /buckets/bucketfs1/virtualschema/virtualschema-jdbc-adapter-dist-1.1.0.jar;
%jar /buckets/bucketfs1/virtualschema/jtds-1.3.1.jar;
/
```
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/teradata.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ CREATE OR REPLACE JAVA ADAPTER SCRIPT adapter.jdbc_adapter

// This will add the adapter jar to the classpath so that it can be used inside the adapter script
// Replace the names of the bucketfs and the bucket with the ones you used.
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar;
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.1.0.jar;

// You have to add all files of the data source jdbc driver here (e.g. MySQL or Hive)
%jar /buckets/bucketfs1/bucket1/terajdbc4.jar;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ general:
debugAddress: '192.168.0.12:3000' # Address which will be defined as DEBUG_ADDRESS in the virtual schemas
bucketFsUrl: http://exasol-host:2580/bucket1
bucketFsPassword: bucket1
jdbcAdapterPath: /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar
jdbcAdapterPath: /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.1.0.jar

exasol:
runIntegrationTests: true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ general:
debugAddress: '192.168.0.12:3000' # Address which will be defined as DEBUG_ADDRESS in the virtual schemas
bucketFsUrl: http://exasol-host:2580/bucket1
bucketFsPassword: bucket1
jdbcAdapterPath: /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar
jdbcAdapterPath: /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.1.0.jar

exasol:
runIntegrationTests: true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ general:
debugAddress: ''
bucketFsUrl: http://127.0.0.1:6594/default
bucketFsPassword: write
jdbcAdapterPath: /buckets/bfsdefault/default/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar
jdbcAdapterPath: /buckets/bfsdefault/default/virtualschema-jdbc-adapter-dist-1.1.0.jar

exasol:
runIntegrationTests: true
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/local/integration-test-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ general:
debugAddress: '10.44.1.228:3000' # Address which will be defined as DEBUG_ADDRESS in the virtual schemas
bucketFsUrl: http://localhost:2580/jars
bucketFsPassword: public
jdbcAdapterPath: /buckets/bfsdefault/jars/virtualschema-jdbc-adapter-dist-1.0.2-SNAPSHOT.jar
jdbcAdapterPath: /buckets/bfsdefault/jars/virtualschema-jdbc-adapter-dist-1.1.0.jar

exasol:
runIntegrationTests: true
Expand Down
3 changes: 2 additions & 1 deletion jdbc-adapter/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>com.exasol</groupId>
<artifactId>virtualschema-jdbc-adapter-main</artifactId>
<version>1.0.2-SNAPSHOT</version>
<version>${product.version}</version>

<packaging>pom</packaging>

Expand All @@ -14,6 +14,7 @@
</modules>

<properties>
<product.version>1.1.0</product.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
Expand Down
15 changes: 0 additions & 15 deletions jdbc-adapter/tools/increment_version.sh

This file was deleted.

100 changes: 100 additions & 0 deletions jdbc-adapter/tools/version.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
#!/bin/bash
readonly vs_jar_prefix='virtualschema-jdbc-adapter-dist'
readonly jar_suffix='jar'
readonly vs_jar_pattern="$vs_jar_prefix-.*\.$jar_suffix"
readonly root_dir='virtual-schemas'
readonly master_pom='jdbc-adapter/pom.xml'
readonly file_find_regex='.*\.(md|yaml)'
readonly script=$(basename $0)

main() {
case "$1" in
help)
usage
;;
verify)
verify
;;
unify)
unify
;;
*)
log "Unknown command: \"$1\""
log
usage
exit 1
;;
esac
}

usage () {
log "Usage: $script help"
log " $script verify"
log " $script unify"
log
log "Run from the root directory \"$root_dir\""
log
log "This script can serve as a checkpoint using 'verify' as command. The exit value"
log "is zero when all detected version numbers match the ones on the master POM file."
log "It is non-zero if there is a mismatch."
log
log "Used with the command 'unify' this script rewrites all occurrences of divergent"
log "version numbers with the one found in the master POM file."
}

verify () {
prepare
verify_no_other_version_numbers "$version"
}

prepare() {
verify_current_directory "$root_dir"
readonly version=$(extract_product_version "$master_pom")
log "Found version $version in master file \"$master_pom\""
}

verify_current_directory() {
if [[ "$(basename $PWD)" != "$root_dir" ]]
then
log "Must be in root directory '$root_dir' to execute this script."
exit 1
fi
}

extract_product_version() {
grep -oP "product\.version>[^<]*<" "$1" | sed -e's/^.*>\s*//' -e's/\s*<//'
}

log () {
echo "$@"
}

verify_no_other_version_numbers() {
find -type f -regextype posix-extended -regex "$file_find_regex" \
-exec grep -Hnor $vs_jar_pattern {} \; | grep -v "$1"
if [[ $? -eq 0 ]]
then
log
log "Verification failed."
log "Found version mismatches that need to be fixed. Try the following command"
log
log " $script unify"
exit 1
else
log "Verification successful."
fi
}

unify() {
prepare
update_documentation
}

update_documentation() {
log "Checking all files matching \"$file_find_regex\""
find -type f -regextype posix-extended -regex "$file_find_regex" \
-exec echo "Processing \"{}\"" \; \
-exec sed -i s/"$vs_jar_pattern"/"$vs_jar_prefix-$version.$jar_suffix"/g {} \;
}

main "$@"
2 changes: 1 addition & 1 deletion jdbc-adapter/virtualschema-common/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.exasol</groupId>
<artifactId>virtualschema-jdbc-adapter-main</artifactId>
<version>1.0.2-SNAPSHOT</version>
<version>${product.version}</version>
</parent>

<artifactId>virtualschema-common</artifactId>
Expand Down
6 changes: 3 additions & 3 deletions jdbc-adapter/virtualschema-jdbc-adapter-dist/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.exasol</groupId>
<artifactId>virtualschema-jdbc-adapter-main</artifactId>
<version>1.0.2-SNAPSHOT</version>
<version>${product.version}</version>
</parent>
<artifactId>virtualschema-jdbc-adapter-dist</artifactId>

Expand All @@ -21,12 +21,12 @@
<dependency>
<groupId>com.exasol</groupId>
<artifactId>virtualschema-common</artifactId>
<version>1.0.2-SNAPSHOT</version>
<version>${product.version}</version>
</dependency>
<dependency>
<groupId>com.exasol</groupId>
<artifactId>virtualschema-jdbc-adapter</artifactId>
<version>1.0.2-SNAPSHOT</version>
<version>${product.version}</version>
</dependency>
</dependencies>

Expand Down
4 changes: 2 additions & 2 deletions jdbc-adapter/virtualschema-jdbc-adapter/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.exasol</groupId>
<artifactId>virtualschema-jdbc-adapter-main</artifactId>
<version>1.0.2-SNAPSHOT</version>
<version>${product.version}</version>
</parent>
<artifactId>virtualschema-jdbc-adapter</artifactId>

Expand Down Expand Up @@ -95,7 +95,7 @@
<dependency>
<groupId>com.exasol</groupId>
<artifactId>virtualschema-common</artifactId>
<version>1.0.2-SNAPSHOT</version>
<version>${product.version}</version>
</dependency>
<!-- We need this only for the integration test profile, however without adding it here you would have to enable the profile in the IDE to have the dependency -->
<dependency>
Expand Down