Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: reformat code (DEV-2263) #399

Merged
merged 4 commits into from
Jun 9, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions .markdownlint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,16 +20,15 @@ MD012: false

# MD013/line-length - Line length
MD013:
line_length: 150
heading_line_length: 150
code_block_line_length: 150
line_length: 120
heading_line_length: 120
code_block_line_length: 120
# Include code blocks
code_blocks: true
# Include tables
tables: false
# Include headings
headings: true
# Include headings
headers: true
# Strict length checking
strict: false
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,8 @@ Find more information in the
## Publishing/distribution

Publishing is automated with GitHub Actions and should _not_ be done manually.
Please follow the [Pull Request Guidelines](https://docs.dasch.swiss/latest/developers/dsp/contribution/#pull-request-guidelines).
Please follow the
[Pull Request Guidelines](https://docs.dasch.swiss/latest/developers/dsp/contribution/#pull-request-guidelines).
If done correctly, when merging a pull request into `main`,
the `release-please` action will create or update a release PR.
This PR will follow semantic versioning and update the change log.
Expand Down
3 changes: 2 additions & 1 deletion docs/cli-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,8 @@ The following options are available:
- `-s` | `--server` (optional, default: `0.0.0.0:3333`): URL of the DSP server where DSP-TOOLS sends the data to
- `-u` | `--user` (optional, default: `root@example.com`): username (e-mail) used for authentication with the DSP-API
- `-p` | `--password` (optional, default: `test`): password used for authentication with the DSP-API
- `-S` | `--sipi` (optional, default: `http://0.0.0.0:1024`): URL of the SIPI server where DSP-TOOLS sends the multimedia files to
- `-S` | `--sipi` (optional, default: `http://0.0.0.0:1024`):
URL of the SIPI server where DSP-TOOLS sends the multimedia files to
- `-i` | `--imgdir` (optional, default: `.`): folder from where the paths in the `<bitstream>` tags are evaluated
- `-I` | `--incremental` (optional) : The links in the XML file point to IRIs (on the server)
instead of IDs (in the same XML file).
Expand Down
10 changes: 7 additions & 3 deletions docs/developers/git-submodules.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@

# Git submodules

This repository embeds [https://github.com/dasch-swiss/00A1-import-scripts](https://github.com/dasch-swiss/00A1-import-scripts)
This repository embeds
[https://github.com/dasch-swiss/00A1-import-scripts](https://github.com/dasch-swiss/00A1-import-scripts)
as a Git submodule in `src/dsp_tools/import_scripts`. That means that `src/dsp_tools/import_scripts` has no contents, but
only a reference to a certain commit in the main branch of `00A1-import-scripts`. When you clone DSP-TOOLS from GitHub
as usual, `src/dsp_tools/import_scripts` will be empty.
Expand Down Expand Up @@ -80,8 +81,11 @@ confused that the path to the submodule changed. If this doesn't help, it might

## Actively working with the contents of the submodule

After retrieving the contents of a submodule as described in the paragraph above, it is in "detached HEAD" state. Before
committing to it, the `main` branch needs to be checked out. The order how to proceed is the following:
After retrieving the contents of a submodule as described in the paragraph above,
it is in "detached HEAD" state.
Before committing to it,
the `main` branch needs to be checked out.
The order how to proceed is the following:

```bash
cd src/dsp_tools/import_scripts
Expand Down
3 changes: 2 additions & 1 deletion docs/developers/mkdocs.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,8 @@ without anyone noticing.
(see [here](https://facelessuser.github.io/pymdown-extensions/extras/slugs/)).
- markdown-link-validator uses [uslug](https://www.npmjs.com/package/uslug) to create the slugs
(see [here](https://github.com/webhintio/markdown-link-validator/blob/main/src/lib/mdfile.ts)).
- VS Code targets the CommonMark Markdown specification using the [markdown-it](https://github.com/markdown-it/markdown-it) library
- VS Code targets the CommonMark Markdown specification using the
[markdown-it](https://github.com/markdown-it/markdown-it) library
(see [here](https://code.visualstudio.com/docs/languages/markdown#_does-vs-code-support-github-flavored-markdown)).

Another useful reading is [here](https://github.com/yzhang-gh/vscode-markdown/issues/807).
42 changes: 27 additions & 15 deletions docs/excel2xml-module.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,19 @@ This can be done with the [CLI command `excel2xml`](./cli-commands.md#excel2xml)

## Module `excel2xml`: Convert a data source to XML

To demonstrate the usage of the `excel2xml` module, there is a GitHub repository named `00A1-import-scripts`. It
contains:
To demonstrate the usage of the `excel2xml` module,
there is a GitHub repository named `00A1-import-scripts`.
It contains:

- a sample JSON project file
- sample data that fits the data model of the JSON project file
- a sample Python script that demonstrates how to use the module `excel2xml`.

Navigate to [https://github.com/dasch-swiss/00A1-import-scripts](https://github.com/dasch-swiss/00A1-import-scripts) and
follow the steps described there. The README will teach you some basics that will be necessary to work with `excel2xml`.
Once you are familiar with the basics, return to this page to learn how the sample Python script works.
Navigate to [https://github.com/dasch-swiss/00A1-import-scripts](https://github.com/dasch-swiss/00A1-import-scripts)
and follow the steps described there.
The README will teach you some basics that will be necessary to work with `excel2xml`.
Once you are familiar with the basics,
return to this page to learn how the sample Python script works.

This is the simplified pattern how the Python script works:

Expand Down Expand Up @@ -93,10 +96,13 @@ here](./file-formats/xml-data-file.md#using-permissions-with-the-permissions-att

## 4. Create list mappings

Let's assume that your data source has a column containing list values named after the "label" of the JSON project list,
instead of the "name" which is needed for the `dsp-tools xmlupload`. You need a way to get the names from the labels.
If your data source uses the labels correctly, this is an easy task: The method `create_json_list_mapping()` creates a
dictionary that maps the labels to the names:
Let's assume that your data source has a column
containing list values named after the "label" of the JSON project list,
instead of the "name" which is needed for the `dsp-tools xmlupload`.
You need a way to get the names from the labels.
If your data source uses the labels correctly,
this is an easy task:
The method `create_json_list_mapping()` creates a dictionary that maps the labels to the names:

The list "category" in `00A1-import-scripts/import_project.json` looks as follows:

Expand Down Expand Up @@ -140,10 +146,13 @@ If you pass this list to `create_json_list_mapping()`, it creates the following
```


If, however, your data source has spelling variants, you need the more sophisticated approach of
`create_json_excel_list_mapping()`: This method creates a dictionary that maps the list values in your data source to their
correct JSON project node name. This happens based on string similarity. Please carefully check the result if there are
no false matches!
If, however, your data source has spelling variants,
you need the more sophisticated approach of `create_json_excel_list_mapping()`:
This method creates a dictionary
that maps the list values in your data source
to their correct JSON project node name.
This happens based on string similarity.
Please carefully check the result if there are no false matches!

The column "Category" in `00A1-import-scripts/data_raw.csv` has spelling mistakes:
![column category](./assets/images/img-excel2xml-raw-data-category.png)
Expand Down Expand Up @@ -254,8 +263,11 @@ For `make_boolean_prop(cell)`, the following formats are supported:
- true: True, "true", "True", "1", 1, "yes", "Yes"
- false: False, "false", "False", "0", 0, "no", "No"

N/A-like values will raise an Error. So if your cell is empty, this method will not count it as false, but will raise an
Error. If you want N/A-like values to be counted as false, you may use a construct like this:
N/A-like values will raise an Error.
So if your cell is empty, this method will not count it as false,
but will raise an Error.
If you want N/A-like values to be counted as false,
you may use a construct like this:

```python
if excel2xml.check_notna(cell):
Expand Down
4 changes: 2 additions & 2 deletions docs/file-formats/excel2json.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,8 +119,8 @@ Only the first worksheet of the Excel file is considered and only XLSX files are
The `properties` section can be inserted into the ontology file and then be uploaded onto a DSP server.

**An Excel file template can be found [here](../assets/data_model_templates/rosetta%20(rosetta)/properties.xlsx)
or also in the
[`data_model_files` folder of `00A1-import-scripts`](https://github.com/dasch-swiss/00A1-import-scripts/tree/main/data_model_files).
or also in the `data_model_files` folder of
[`00A1-import-scripts`](https://github.com/dasch-swiss/00A1-import-scripts/tree/main/data_model_files).
It is recommended to work from the template.**

The Excel sheet must have the following structure:
Expand Down
3 changes: 2 additions & 1 deletion docs/file-formats/json-project/caveats.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,8 @@ subclassed and used in a resource class.
[&lt;link&gt; tag](../xml-data-file.md#link)
- `hasGeometry`: Defines a geometry value (a JSON describing a polygon, circle or rectangle).
- must be used directly in the XML data file in the [&lt;region&gt; tag](../xml-data-file.md#region)
- `isRegionOf`: A special variant of `hasLinkTo`. It means that the given resource class is a region of interest in an image.
- `isRegionOf`: A special variant of `hasLinkTo`.
It means that the given resource class is a region of interest in an image.
- must be used directly in the XML data file in the [&lt;region&gt; tag](../xml-data-file.md#region)
- `isAnnotationOf`: A special variant of `hasLinkTo`. It means that the given resource class is an annotation to another
resource class.
Expand Down
30 changes: 18 additions & 12 deletions docs/file-formats/json-project/ontologies.md
Original file line number Diff line number Diff line change
Expand Up @@ -466,16 +466,18 @@ Example:

`"object": "ListValue"`

Represents a node of a (possibly hierarchical) list. See the [xmlupload documentation](../xml-data-file.md#list-prop)
Represents a node of a (possibly hierarchical) list.
See the [xmlupload documentation](../xml-data-file.md#list-prop)
for more information.

*gui_elements / gui_attributes*:

- `List`: A GUI element for *ListValue*. A dropdown to select a list node. This GUI element should be chosen for
hierarchical lists or flat lists that could be expanded to hierarchical lists in the future.
- `List`: A GUI element for *ListValue*. A dropdown to select a list node.
- *gui_attributes*:
- `hlist=<list-name>` (required): The name of a list defined in the ["lists" section](./overview.md#lists).
- `Radio` and `Pulldown` are allowed, too, but they don't have a different behaviour than `List`. It is recommended to use `List`.
- `Radio` and `Pulldown` are allowed, too,
but they don't have a different behaviour than `List`.
It is recommended to use `List`.


Example:
Expand Down Expand Up @@ -713,14 +715,18 @@ Example:

`"object": "<resourceclass>"`

A special case of linked resources are resources in a part-whole relation, i.e. resources that are composed of
other resources. A `isPartOf` property has to be added to the resource that is part of another resource. In case of
resources that are of type `StillImageRepresentation`, an additional property derived from `seqnum` with object `IntValue`
is required. When defined, the user is able to leaf through the parts of a compound object, p.ex. to leaf through pages
of a book.

The DSP base properties `isPartOf` and `seqnum` can be used to derive a custom property from them, or they can be used
directly as cardinalities in a resource. The example belows shows both possibilities.
A special case of linked resources are resources in a part-whole relation,
i.e. resources that are composed of other resources.
A `isPartOf` property has to be added to the resource that is part of another resource.
In case of resources that are of type `StillImageRepresentation`,
an additional property derived from `seqnum` with object `IntValue` is required.
When defined, the user is able to leaf through the parts of a compound object,
e.g. to leaf through pages of a book.

The DSP base properties `isPartOf` and `seqnum`
can be used to derive a custom property from them,
or they can be used directly as cardinalities in a resource.
The example belows shows both possibilities.

*gui_elements/gui_attributes*:

Expand Down
17 changes: 11 additions & 6 deletions docs/file-formats/json-project/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,15 @@

# JSON project definition format

This document describes the structure of a JSON project definition file that can be uploaded to a DSP server
This document describes the structure of a JSON project definition file
that can be uploaded to a DSP server
with the [`create`](../../cli-commands.md#create) command.

A project on a DSP server is like a container for data. It defines some basic metadata, the data model(s) and optionally
the user(s) who will be able to access the data. After the creation of a project, data can be uploaded that conforms
with the data model(s).
A project on a DSP server is like a container for data.
It defines some basic metadata, the data model(s)
and optionally the user(s) who will be able to access the data.
After the creation of a project,
data can be uploaded that conforms with the data model(s).

This documentation is divided into the following parts:

Expand Down Expand Up @@ -140,8 +143,10 @@ the DaSCH.

`"shortname": "<string>"`

The shortname has to be unique. It should be in the form of a [xsd:NCNAME](https://www.w3.org/TR/xmlschema11-2/#NCName).
This means a string without blanks or special characters but `-` and `_` are allowed (although not as first character).
The shortname has to be unique.
It should be in the form of a [xsd:NCNAME](https://www.w3.org/TR/xmlschema11-2/#NCName).
This means a string without blanks or special characters,
but with `-` and `_` are allowed (although not as first character).



Expand Down
Loading
Loading