How to transfer Score 1.7 data to Score 2.0

The following instructions only work for Score 1.7.x. Thus, if you are not using 1.7.x, please see Release Detail page to upgrade to 1.7.x. Also, you should know how to connect the database through MySQL client programs.

0. Pre-Data Validation

Over the course of upgrades OAGIS versions and Score versions, there might be some issues with data due to for example missing applications of database upgrade/fixes. This is a data validation step helps ensure that current data in Score 1.7.x are correct based on upgrading tests on various Score data sets the development team has access to. When you execute the following queries, the results should be empty if your data are valid.

1. Check if there is a referential integrity problem in ASBIEP, i.e., ASBIEP has no link to ASCCP.

SELECT `asbiep`.`guid`, `ascc`.`den`, `target_asccp`.`asccp_id`
FROM `asbiep`
JOIN `asbie` ON `asbiep`.`asbiep_id` = `asbie`.`to_asbiep_id`
JOIN `ascc` ON `asbie`.`based_ascc_id` = `ascc`.`ascc_id`
JOIN `asccp` AS `target_asccp` ON `ascc`.`to_asccp_id` = `target_asccp`.`current_asccp_id`
LEFT JOIN `asccp` ON `asbiep`.`based_asccp_id` = `asccp`.`asccp_id`
WHERE `asccp`.`asccp_id` IS NULL;

If you found rows in results, consider execute queries like below based on the results from the above query.

UPDATE `asbiep` SET `based_asccp_id` = {`target_asccp`.`asccp_id`} WHERE `guid` = '{`asbiep`.`guid`}';
-- Example
-- UPDATE `asbiep` SET `based_asccp_id` = 7468 WHERE `guid` = 'oagis-id-35553cae45db42b5a07234dff2d18fc0';

2. Check if there is a null reference to the primitives in BBIE_SC

SELECT `dt_sc`.`guid`, `dt_sc`.`property_term`, `dt_sc`.`representation_term`
FROM `bbie_sc`
JOIN `dt_sc` ON `bbie_sc`.`dt_sc_id` = `dt_sc`.`dt_sc_id`
WHERE `bbie_sc`.`dt_sc_pri_restri_id` IS NULL AND `bbie_sc`.`code_list_id` IS NULL AND `bbie_sc`.`agency_id_list_id` IS NULL;

If you found rows in results, the following query fixes them to use default primitive based on the CC data. However, you may want to consult the users/owners of the top-level BIE whether it should be updated to something else, if that BBIE_SC node is used. If it is not used, the update query below is good to go.

UPDATE
    `bbie_sc` 
     JOIN `dt_sc` ON `bbie_sc`.`dt_sc_id` = `dt_sc`.`dt_sc_id`
     JOIN `bdt_sc_pri_restri` ON `bdt_sc_pri_restri`.`bdt_sc_id` = `dt_sc`.`dt_sc_id` AND `bdt_sc_pri_restri`.`is_default` = 1
SET
    `bbie_sc`.`dt_sc_pri_restri_id`  = `bdt_sc_pri_restri`.`bdt_sc_pri_restri_id`
WHERE
    `bbie_sc`.`dt_sc_pri_restri_id` IS NULL AND `bbie_sc`.`code_list_id` IS NULL AND `bbie_sc`.`agency_id_list_id` IS NULL;

3. Check whether there is ABIEs refers to ACC whose component_type is a group. These are redundant records.

SELECT
    `acc`.`guid`, `acc`.`object_class_term`
FROM `abie`
JOIN `acc` ON `abie`.`based_acc_id` = `acc`.`acc_id`
WHERE
    `acc`.`oagis_component_type` = 3;

If you found rows in results, please execute mig_1.1.2_to_1.1.2.3_mysql.sql.

4. Check whether there is 'Carrier Route Base. Status. Status' ASBIEP. This shouldn't exist since the Status was removed in OAGIS 10.6 due to refactoring the based ACC of the Carrier Route Base ACC.

SELECT `asbie`.`guid`, `ascc`.`den`, `asbie`.`is_used`
FROM `asbie`
JOIN `ascc` ON `asbie`.`based_ascc_id` = `ascc`.`ascc_id`
WHERE `ascc`.`den` = 'Carrier Route Base. Status. Status';

If you found rows in results, please execute patch_oagis_10.5_to_10.6.sql.

1. Copy and create new docker-compose file

• Add transfer service into docker-compose.yml

version: "3.1"
services:
  frontend:
    image: oagi1docker/srt-web:1.7.0.2
    ports:
      - 80:80
    links:
      - backend
    environment:
      - GATEWAY_HOST=backend
      - GATEWAY_PORT=8080
    depends_on:
      - backend

  backend:
    image: oagi1docker/srt-http-gateway:1.7.0.2
    ports:
      - 8080:8080
    links:
      - db
      - redis
    environment:
      - DB_HOST=db
      - DB_PORT=3306
      - REDIS_HOST=redis
      - REDIS_PORT=6379
    depends_on:
      - db
      - redis

  transfer:
    image: oagi1docker/srt-transfer:1.0
    ports:
      - 5000:5000
    links:
      - db
    depends_on:
      - db

  db:
    image: oagi1docker/srt-repo:1.7.0
    ports:
      - 3306:3306
    volumes:
      - db-volume:/var/lib/mysql

  redis:
    image: redis:4
    ports:
      - 6379:6379
    volumes:
      - redis-volume:/data

volumes:
  db-volume:
  redis-volume:

The default port for the Score transfer service is 5000.

The network where Score 1.7 is being serviced must be accessible as port 5000. Or change it to an available port.

2. Restart docker containers

• Stop docker containers

shell> docker-compose down

• Restart Score 1.7 with the new settings.

shell> docker-compose up -d

3. Score 1.7 data export

You can now access the transfer service at http://{your host name}:{your port number} through a browser.

Based on localhost, the connection address is http://localhost:5000.

Connect to Score 1.7 database

Enter the information to access the Score 1.7 database.

Hostname should be service name of the docker image which is db.

After filling the information out, click Save & Next button to move to the next step.

Exporting Score 1.7 data

At this stage, you can export the data by clicking the Start button.

When exporting is complete, the browser automatically starts downloading the score.xml file.

Now, exporting step is over. If you want to launch Score 2.0 in the same machine, you should stop docker containers via the following command,

shell> docker-compose down

Export 1.7 BIE data for validation.

Express BIEs using the BIEExpress.jar using the binary and instruction provided here. The tool will download BIE XML schema files to your local system. Note: If you do not have large number of BIE, you can export all BIEs into a single schema file making it easier to compare in Step 5.

4. Run Score 2.0

Run Score 2.0 with the Transfer service with the following settings,

version: "3.1"
services:
  frontend:
    image: oagi1docker/srt-web:2.0.0
    ports:
      - 80:80
    links:
      - backend
    environment:
      - GATEWAY_HOST=backend
      - GATEWAY_PORT=8080
    depends_on:
      - backend

  backend:
    image: oagi1docker/srt-http-gateway:2.0.0
    ports:
      - 8080:8080
    links:
      - db
      - redis
    environment:
      - DB_HOST=db
      - DB_PORT=3306
      - REDIS_HOST=redis
      - REDIS_PORT=6379
    depends_on:
      - db
      - redis

  transfer:
    image: oagi1docker/srt-transfer:1.0
    ports:
      - 5000:5000
    links:
      - db
    depends_on:
      - db

  db:
    image: oagi1docker/srt-repo:2.0.0
    ports:
      - 3306:3306
    volumes:
      - db-volume:/var/lib/mysql

  redis:
    image: redis:4
    ports:
      - 6379:6379
    volumes:
      - redis-volume:/data

volumes:
  db-volume:
  redis-volume:

Same as Score 1.7, the transfer service running at port 5000. Connect to the Transfer service with the hostname of Score 2.0 and port number.

Based on localhost, the connection address is also same as http://localhost:5000.

Connect to Score 2.0 database

Hostname should be service name of the docker image which is db.

After filling the information out, click Save & Next button to move to the next step.

Upload the score.xml file

Upload the score.xml file that you received from the exporting procedure.

Setup a non-standard namespace

In Score 2.0, every core component including user extensions and code lists should have a namespace. Thus, a namespace for those should be created before the import. To do so, choose any end-user account to specify the owner of the namespace and put URI (mandatory, must follow the URI format). You also can put prefix and description for the namespace. This information can be updated afterward.

Import data to Score 2.0

You can import the data by clicking the Start button.

Once the import process is completed, validation procedure will proceed and the Diff.txt file will be automatically downloaded at the completion.

If there are differences in between 1.7 and 2.0, they will show up in the diff file. Contact the Score development team hakju.oh@nist.gov to help resolve issues. Differences due to ordering of SSO account are expected.

5. Validate BIE data, as an additional data transfer assurance.

Express BIEs from Score 2.0 using the BIEExpress.jar provided here. Use your available diff tool to compare BIE XML schema files from Step 3. If there are any differences, Contact the Score development team hakju.oh@nist.gov to help resolve issues.

6. Disable the oagis developer accounts

Because new releases of OAGIS standard are developed using Score. The standard is contributed by OAGIS members. Therefore, Score 2.0 data have those member accounts in the developer role. Although your Score instance maybe behind the firewall and they should not be able to log in, you can use the Admin menu on the Score tool to disable those accounts if you would like.