Skip to content

Commit

Permalink
chore: adapt shortcode of import scripts (DEV-1925) (#340)
Browse files Browse the repository at this point in the history
  • Loading branch information
jnussbaum committed Mar 20, 2023
1 parent b176b7c commit f440bda
Show file tree
Hide file tree
Showing 10 changed files with 22 additions and 22 deletions.
2 changes: 1 addition & 1 deletion .gitmodules
@@ -1,3 +1,3 @@
[submodule "knora/dsplib/import_scripts"]
path = src/dsp_tools/import_scripts
url = https://github.com/dasch-swiss/0123-import-scripts
url = https://github.com/dasch-swiss/00A1-import-scripts
16 changes: 8 additions & 8 deletions docs/developers/git-submodules.md
Expand Up @@ -2,20 +2,20 @@

# Git submodules

This repository embeds [https://github.com/dasch-swiss/0123-import-scripts](https://github.com/dasch-swiss/0123-import-scripts)
This repository embeds [https://github.com/dasch-swiss/00A1-import-scripts](https://github.com/dasch-swiss/00A1-import-scripts)
as a Git submodule in `src/dsp_tools/import_scripts`. That means that `src/dsp_tools/import_scripts` has no contents, but
only a reference to a certain commit in the main branch of `0123-import-scripts`. When you clone DSP-TOOLS from GitHub
only a reference to a certain commit in the main branch of `00A1-import-scripts`. When you clone DSP-TOOLS from GitHub
as usual, `src/dsp_tools/import_scripts` will be empty.



## Rationale to use a git submodule

The code of the `0123-import-scripts` repository is closely related to the documentation of the `excel2xml` module.
The code of the `00A1-import-scripts` repository is closely related to the documentation of the `excel2xml` module.
When something changes in `excel2xml`, the changes need not only be reflected in the docs, but also in
`0123-import-scripts`. This can easily be forgotten. The decision to embed it as a submodule is meant to bring
`0123-import-scripts` closer to the attention of the developers of DSP-TOOLS. For example, a repo-wide search for a
string or the usage of a method will also yield results from `0123-import-scripts`.
`00A1-import-scripts`. This can easily be forgotten. The decision to embed it as a submodule is meant to bring
`00A1-import-scripts` closer to the attention of the developers of DSP-TOOLS. For example, a repo-wide search for a
string or the usage of a method will also yield results from `00A1-import-scripts`.

The example project [rosetta](https://github.com/dasch-swiss/082E-rosetta-scripts) is a similar case. Changes in
DSP-TOOLS sometimes need to be reflected in rosetta. But since rosetta is not embedded as submodule, the developers
Expand Down Expand Up @@ -84,11 +84,11 @@ committing to it, the `main` branch needs to be checked out. The order how to pr

```bash
cd src/dsp_tools/import_scripts
git checkout main # check out main branch of 0123-import-scripts
git checkout main # check out main branch of 00A1-import-scripts
# (modify contents of submodule)
git add .
git commit -m "modify submodule"
git push origin main # push to origin of 0123-import-scripts
git push origin main # push to origin of 00A1-import-scripts
cd ../../..
git add src/dsp_tools/import_scripts
git commit -m "modify submodule"
Expand Down
8 changes: 4 additions & 4 deletions docs/excel2xml-module.md
Expand Up @@ -26,14 +26,14 @@ This can be done with the [CLI command `excel2xml`](./cli-commands.md#excel2xml)

## Module `excel2xml`: Convert a data source to XML

To demonstrate the usage of the `excel2xml` module, there is a GitHub repository named `0123-import-scripts`. It
To demonstrate the usage of the `excel2xml` module, there is a GitHub repository named `00A1-import-scripts`. It
contains:

- a sample JSON project file
- sample data that fits the data model of the JSON project file
- a sample Python script that demonstrates how to use the module `excel2xml`.

Navigate to [https://github.com/dasch-swiss/0123-import-scripts](https://github.com/dasch-swiss/0123-import-scripts) and
Navigate to [https://github.com/dasch-swiss/00A1-import-scripts](https://github.com/dasch-swiss/00A1-import-scripts) and
follow the steps described there. The README will teach you some basics that will be necessary to work with `excel2xml`.
Once you are familiar with the basics, return to this page to learn how the sample Python script works.

Expand Down Expand Up @@ -97,7 +97,7 @@ instead of the "name" which is needed for the `dsp-tools xmlupload`. You need a
If your data source uses the labels correctly, this is an easy task: The method `create_json_list_mapping()` creates a
dictionary that maps the labels to the names:

The list "category" in `0123-import-scripts/import_project.json` looks as follows:
The list "category" in `00A1-import-scripts/import_project.json` looks as follows:
```json
{
"name": "category",
Expand Down Expand Up @@ -142,7 +142,7 @@ If, however, your data source has spelling variants, you need the more sophistic
correct JSON project node name. This happens based on string similarity. Please carefully check the result if there are
no false matches!

The column "Category" in `0123-import-scripts/data_raw.csv` has spelling mistakes:
The column "Category" in `00A1-import-scripts/data_raw.csv` has spelling mistakes:
![column category](./assets/images/img-excel2xml-raw-data-category.png)

The dictionary that results if you call `create_json_excel_list_mapping()`:
Expand Down
6 changes: 3 additions & 3 deletions docs/file-formats/excel2json.md
Expand Up @@ -63,7 +63,7 @@ The `resources` section can be inserted into the ontology file and then be uploa

**An Excel file template can be found [here](../assets/data_model_templates/rosetta%20(rosetta)/resources.xlsx)
or also in the
[`data_model_files` folder of `0123-import-scripts`](https://github.com/dasch-swiss/0123-import-scripts/tree/main/data_model_files).
[`data_model_files` folder of `00A1-import-scripts`](https://github.com/dasch-swiss/00A1-import-scripts/tree/main/data_model_files).
It is recommended to work from the template.**

The expected worksheets of the Excel file are:
Expand Down Expand Up @@ -120,7 +120,7 @@ The `properties` section can be inserted into the ontology file and then be uplo

**An Excel file template can be found [here](../assets/data_model_templates/rosetta%20(rosetta)/properties.xlsx)
or also in the
[`data_model_files` folder of `0123-import-scripts`](https://github.com/dasch-swiss/0123-import-scripts/tree/main/data_model_files).
[`data_model_files` folder of `00A1-import-scripts`](https://github.com/dasch-swiss/00A1-import-scripts/tree/main/data_model_files).
It is recommended to work from the template.**

The Excel sheet must have the following structure:
Expand Down Expand Up @@ -200,7 +200,7 @@ Some notes:
- [en.xlsx](../assets/data_model_templates/lists/en.xlsx): File with the English labels
- [de.xlsx](../assets/data_model_templates/lists/de.xlsx): File with the German labels
- or alternatively from the
[`data_model_files` folder of `0123-import-scripts`](https://github.com/dasch-swiss/0123-import-scripts/tree/main/data_model_files)
[`data_model_files` folder of `00A1-import-scripts`](https://github.com/dasch-swiss/00A1-import-scripts/tree/main/data_model_files)

The output of the above command, with the template files, is:

Expand Down
2 changes: 1 addition & 1 deletion docs/file-formats/json-project/overview.md
Expand Up @@ -532,7 +532,7 @@ The `ontologies` array is [documented here](./ontologies.md)
DaSCH provides you with two example repositories that contain everything which is necessary to create a project and
upload data. Both of them also contain a JSON project definition file. You can find them here:

- [https://github.com/dasch-swiss/0123-import-scripts](https://github.com/dasch-swiss/0123-import-scripts)
- [https://github.com/dasch-swiss/00A1-import-scripts](https://github.com/dasch-swiss/00A1-import-scripts)
- [https://github.com/dasch-swiss/082E-rosetta-scripts](https://github.com/dasch-swiss/082E-rosetta-scripts)

In addition, there is another complete example of a JSON project file here:
Expand Down
2 changes: 1 addition & 1 deletion docs/file-formats/xml-data-file.md
Expand Up @@ -911,7 +911,7 @@ shortcut, so that the XML file can be validated more precisely.
DaSCH provides you with two example repositories that contain everything which is necessary to create a project and
upload data. Both of them also contain an XML data file. You can find them here:

- [https://github.com/dasch-swiss/0123-import-scripts](https://github.com/dasch-swiss/0123-import-scripts)
- [https://github.com/dasch-swiss/00A1-import-scripts](https://github.com/dasch-swiss/00A1-import-scripts)
- [https://github.com/dasch-swiss/082E-rosetta-scripts](https://github.com/dasch-swiss/082E-rosetta-scripts)

In addition, there is another complete example of an XML data file here:
Expand Down
Expand Up @@ -37,7 +37,7 @@ def test_import_scripts(self) -> None:
os.chdir(old_working_directory)

# check the output XML (but before, remove random components from resource IDs and resptr targets)
with open("testdata/excel2xml/0123-data-processed-expected.xml") as f:
with open("testdata/excel2xml/00A1-data-processed-expected.xml") as f:
xml_expected = _derandomize_xsd_id(f.read(), multiple_occurrences=True)
with open("src/dsp_tools/import_scripts/data-processed.xml") as f:
xml_returned = _derandomize_xsd_id(f.read(), multiple_occurrences=True)
Expand Down
2 changes: 1 addition & 1 deletion test/unittests/test_excel2xml.py
Expand Up @@ -148,7 +148,7 @@ def test_make_xsd_id_compatible(self) -> None:

# test that the special characters in the "Label" row of excel2xml-testdata-special-characters.xlsx are replaced
special_characters_df = pd.read_excel("testdata/excel2xml/excel2xml-testdata-special-characters.xlsx")
root = excel2xml.make_root("0123", "test")
root = excel2xml.make_root("00A1", "test")
root = excel2xml.append_permissions(root)
for i, row in special_characters_df.iterrows():
root.append(
Expand Down
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<knora xmlns="https://dasch.swiss/schema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://dasch.swiss/schema https://raw.githubusercontent.com/dasch-swiss/dsp-tools/main/src/dsp_tools/resources/schema/data.xsd" shortcode="0123" default-ontology="import">
<knora xmlns="https://dasch.swiss/schema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://dasch.swiss/schema https://raw.githubusercontent.com/dasch-swiss/dsp-tools/main/src/dsp_tools/resources/schema/data.xsd" shortcode="00A1" default-ontology="import">
<permissions id="res-default">
<allow group="UnknownUser">V</allow>
<allow group="KnownUser">V</allow>
Expand Down

0 comments on commit f440bda

Please sign in to comment.