This is a grails plugin for the NSL project that maps the Domain objects of NSL. It is used to generate and update the schema for the NSL.
-
To build the plugin you need a database called
nslimport -
To install the plugin you need grails3 and Java 8
~/IdeaProjects/NSL/nsl-domain-plugin-g3$ grails --version
| Grails Version: 5.2.5
| JVM Version: 1.8.0_232-- Create role and database from sql prompt
CREATE ROLE nsldev with LOGIN PASSWORD 'nsldev';
CREATE DATABASE nslimport WITH OWNER nsldev;-
With that in place, you can now run
./gradlew buildfrom the project root -
Once that succeeds, you can install the plugin using the command
./gradlew publishToMavenLocal -
This installs the plugin in the maven directory as below
mo@local:~/nsl-domain-plugin-g3$ ls -l ~/.m2/repository/au/org/biodiversity/grails/plugins/nsl-domain-plugin-g3
total 32
drwxrwxr-x. 2 mo mo 4096 Jun 17 2019 1.17-SNAPSHOT
drwxrwxr-x. 2 mo mo 4096 Jul 25 2019 1.18-SNAPSHOT
drwxrwxr-x. 2 mo mo 4096 Aug 9 2019 1.19-SNAPSHOT
drwxrwxr-x. 2 mo mo 4096 Aug 14 2019 1.19.SNAPSHOT
drwxrwxr-x. 2 mo mo 4096 Aug 27 2019 3.1.19-SNAPSHOT
drwxrwxr-x. 2 mo mo 4096 Sep 2 2019 3.1.20-SNAPSHOT
drwxrwxr-x. 2 mo mo 4096 Jun 29 11:48 3.1.21-SNAPSHOT # Current version
-rw-rw-r--. 1 mo mo 553 Nov 23 09:48 maven-metadata-local.xmlIf you run into issues, checkout our known issues section
When making schema changes we have to consider two cases:
-
creating the schema for a new database
-
modifying the schema in an existing database
and this means your changes need to appear in an update script and the nsl-ddl.sql file.
If you are adding a view, trigger, index or default data:
1. first add your changes to an appropriate file in the /src/main/resources/sql/views directory
1. run grails generate-nsl-ddl to regenerate the nsl-ddl.sql file with your changes included
1. Add your change to the latest update-to-nn.sql file in the src/main/resources/sql directory
Remember the update-to-nn script is updating a database from the previous version, so you must take into account existing views and not loose data.
If you are changing a table or tables you must make those changes in the Grails domain classes which will
generate the appropriate sql when the grails generate-nsl-ddl command is run. This keeps the grails domain
classes,sql and DB schema in sync.
-
make your changes to the grails domain classes in
/grails-app/domain/au/org/biodiversity/nsl -
preferably test your changes using
gradle installlocally -
run
gradle cleanto remove the old ddl build directory -
run
grails generate-nsl-ddlto regenerate the /src/main/resources/nsl-ddl.sql file with your changes included -
diff nsl-ddl.sql to extract changes to put into the latest
update-to-nn.sqlfile in the/src/main/resources/sqldirectory
Make sure the version number you are working on is not the currently released version, if necessary create a new
version and update the default version number in sql/views/other-setup.sql as well as
grails-app/services/au/org/biodiversity/nsl/NslDomainService.groovy.
The simplest way to make the update sql is the re-generate the DDL from grails using the script:
grails generate-nsl-ddl
|
Warning
|
You should run grails clean before doing grails generate-nsl-ddl or you may end up with the hibernate generated
ddl in there twice.
|
and diff the result to see what needs to be changed.
When starting off with a fresh NSL shard it’s probably best to execute the nsl-ddl.sql file against your database (traditionally called nsl).
When writing update-to-nn.sql scripts we try to make the update script idempotent, so that if it is run twice (either
deliberately or accidentaly) it results in the same thing. Note data loss is possible it the scripts are run again after
a while.
This is part of the NSL (National Species List) Services Application developed at the Australian National Botanic Gardens in association with CHAH, the ALA and the IBIS team.
This code is Licensed under the Apache 2 License and we welcome contributions from the taxonomic community.
Please use Github issues for any bug reports.
For Australian NSL infrastructure we are splitting the ID ranges used across different known shards. This is not essential but it does mean we have a means to discriminate data sources on IDs if somehow we manage to loose track.
The current allocated ranges are:
-
Vascular (APNI) 1000 - 10,000,000 + 50,000,001 - 60,000,000
-
Moss 10,000,001 - 20,000,000
-
Algae 20,000,001 - 30,000,000
-
Lichen 30,000,001 - 40,000,000
-
DAWR 40,000,001 - 50,000,000 (Aust. Dept. of Agriculture and Water Resources)
-
Fungi 60,000,001 - 70,000,000
-
AFD 70,000,001 - 80,000,000
There is a confluence page for the NSL infrastructure as well at https://www.anbg.gov.au/ibis25/pages/viewpage.action?spaceKey=NSL&title=NSL+Project+2.0
You need to set the shard sequence rang in the nsl-ddl.sql script before running it. e.g.
create sequence nsl_global_seq minvalue 80000001 maxvalue 90000000;
This repository is automatically built by Travis CI (https://travis-ci.org/bio-org-au/nsl-domain-plugin-g3) and deployed to 'OJO' (http://oss.jfrog.org), which is an artifact (maven) repository for open source projects.
We publish the plugin to a oss.jfrog.org. To do this we need to set the BINTRAY_USER and BINTRAY_KEY environment
variables and the call ./gradlew publish to publish to 'OJO'.
To use the snapshot you need to include the snapshot repository by adding this to your gradle repositories block:
repositories {
maven {
url "https://oss.jfrog.org/artifactory/oss-snapshot-local"
}
}to consume the snapshot add the following to the dependencies:
compile "au.org.biodiversity.grails.plugins:nsl-domain-plugin-g3:3.1.19-SNAPSHOT"
To release the plugin we merge master into the release branch and remove the SNAPSHOT status from the plugin version. Then run the bintrayUpload gradle command to upload the released plugin to bintray https://bintray.com/beta/#/bio-org-au/plugins/nsl-domain-plugin-g3?tab=overview
|
Note
|
No releases have been pushed yet for G3. We will add a separate travis.yml in the release branch to build the release. |
-
Create a clean DB
-
use psql to run the nsl-ddl.sql command (in
src/main/resources/sql)
creatdb mynsl && psql -f nsl-ddl.sql mynsl
You might get the below error when you run grails install
$ grails install --stacktrace
| Error Error occurred running Grails CLI: No profile found for name [web-plugin]. (NOTE: Stack trace has been filtered. Use --verbose to see entire trace.)
java.lang.IllegalStateException: No profile found for name [web-plugin].
at org.grails.cli.GrailsCli.initializeProfile(GrailsCli.groovy:518)
at org.grails.cli.GrailsCli.initializeApplication(GrailsCli.groovy:306)
at org.grails.cli.GrailsCli.execute(GrailsCli.groovy:269)
at org.grails.cli.GrailsCli.main(GrailsCli.groovy:159)
| Error Error occurred running Grails CLI: No profile found for name [web-plugin].$ ./gradlew clean
user null
:clean
$ ./gradlew install
user null
:generatePomFileForMavenPublication
:compileAstJava NO-SOURCE
:compileAstGroovy NO-SOURCE
:processAstResources NO-SOURCE
:astClasses UP-TO-DATE
:compileJava NO-SOURCE
:configScript
:compileGroovy
:copyAstClasses NO-SOURCE
:assetPluginPackage
:copyCommands
:copyTemplates NO-SOURCE
:processResources
:classes
:compileWebappGroovyPages NO-SOURCE
:compileGroovyPages
:jar
:groovydoc
Trying to override old definition of task fileScanner
[ant:groovydoc] Resource org/codehaus/groovy/tools/groovy.ico not found so skipped
:javadoc
:javadocJar
:sourcesJar
:publishMavenPublicationToMavenLocal
:publishToMavenLocal
:install
BUILD SUCCESSFUL
Total time: 16.441 secs