Skip to content

Commit 591b5ad

Browse files
irinaschubertBalduinLandolt
irinaschubert
and
BalduinLandolt
authored
docs: improve documentation (DSP-1693) (#62)
* fix error in documentation * fix errors, clarify things, fix styling issues * remove doc from disallowed prefixes in the check-pr-title action * Update documentation * Update documentation * Update documentation * Update docs/dsp-tools-create.md Co-authored-by: Balduin Landolt <33053745+BalduinLandolt@users.noreply.github.com> * add suggestions of code review * split index into several subpages Co-authored-by: Balduin Landolt <33053745+BalduinLandolt@users.noreply.github.com>
1 parent 69772f4 commit 591b5ad

8 files changed

Lines changed: 1094 additions & 949 deletions

‎.github/workflows/ckeck-pr-title.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,8 @@ jobs:
1313
- uses: deepakputhraya/action-pr-title@master
1414
with:
1515
regex: '([a-z])+(\(([a-z\-_ ])+\))?!?: [a-z]([a-zA-Z-\.\d \(\)\[\]#_])+$' # Regex the title should match.
16-
allowed_prefixes: "fix,refactor,feat,docs,chore,style,test" # title should start with the given prefix
17-
disallowed_prefixes: "feature,hotfix,doc" # title should not start with the given prefix
16+
allowed_prefixes: 'fix,refactor,feat,docs,chore,style,test' # title should start with the given prefix
17+
disallowed_prefixes: 'feature,hotfix' # title should not start with the given prefix
1818
prefix_case_sensitive: true # title prefix are case insensitive
1919
min_length: 7 # Min length of the title
2020
max_length: 120 # Max length of the title

‎docs/dsp-tools-create.md‎

Lines changed: 396 additions & 356 deletions
Large diffs are not rendered by default.

‎docs/dsp-tools-excel.md‎

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,17 @@
1-
[![PyPI version](https://badge.fury.io/py/knora.svg)](https://badge.fury.io/py/knora)
1+
[![PyPI version](https://badge.fury.io/py/dsp-tools.svg)](https://badge.fury.io/py/dsp-tools)
22

3-
# DSP tools to use Excel-files for data modelling and data import
4-
Dsp-tools is able to directly read and process excel files and output the appropriate JSON and/or XML-files for
5-
importing data to the dsp-repository.
3+
# Excel files for data modelling and data import
4+
dsp-tools is able to process Excel files and output the appropriate JSON or XML file. The JSON/XML file can then
5+
be used to create the ontology on the DSP server or import data to the DSP repository. dsp-tools can also be used to
6+
create a list from an Excel file.
67

7-
## Flat and hierarchical lists
8-
Lists or "controlled vocabularies" are sets of fixed terms that are used to characterize something. Hierarchical lists
9-
correspond to classifications or taxonomies.
10-
11-
The format of the excel is described [here](./dsp-tools-create.md#lists-from-excel).
8+
## Create the data model JSON from an Excel file
9+
[not yet implemented]
1210

11+
## Create a DSP-conform XML file from an Excel file
12+
[not yet implemented]
1313

14+
## Create flat or hierarchical lists from an Excel file
15+
Lists or controlled vocabularies are sets of fixed terms that are used to characterize objects. Hierarchical lists
16+
correspond to classifications or taxonomies. With dsp-tools a list can be created from an Excel file. The expected
17+
format of the Excel file is described [here](./dsp-tools-create.md#lists-from-excel).

‎docs/dsp-tools-information-for-developers.md‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
[![PyPI version](https://badge.fury.io/py/dsp-tools.svg)](https://badge.fury.io/py/dsp-tools)
2+
3+
There is a `Makefile` for all the following tasks (and more). Type `make` to print the available targets.
4+
5+
## Install from source
6+
7+
To install from source run:
8+
```bash
9+
python3 setup.py install
10+
```
11+
12+
## Install the requirements
13+
14+
To install the requirements run:
15+
16+
```bash
17+
pip3 install -r requirements.txt
18+
```
19+
20+
To generate a requirements file (usually requirements.txt), that you commit with your project, run:
21+
22+
```bash
23+
pip3 freeze > requirements.txt
24+
```
25+
26+
## Testing
27+
Please note that testing requires launching the complete DSP API stack which is based on docker images. Therefore, we
28+
recommend installing the [docker desktop client](https://www.docker.com/products).
29+
30+
To run the complete test suite:
31+
32+
```bash
33+
make test
34+
```
35+
36+
## Publishing
37+
38+
Publishing is automated with GitHub Actions and should _not_ be done manually. Please follow the
39+
[Pull Request Guidelines](https://docs.dasch.swiss/developers/dsp/contribution/#pull-request-guidelines). If done
40+
correctly, when merging a pull request into `main`, the `release-please` action will create or update a pull request for
41+
a release. This pull request will follow semantic versioning and update the change log. Once all desired features are
42+
merged, the release can be executed by merging this release pull request into `main`. This will trigger actions that
43+
create a release on GitHub, on PyPI and the docs.
44+
45+
Please ensure you have only one pull request per feature.
46+
47+
## Publishing manually
48+
49+
Publishing is automated with GitHub Actions and should _not_ be done manually. If you still need to do it, follow the
50+
steps below.
51+
52+
Generate the distribution package. Make sure you have the latest versions of `setuptools` and `wheel` installed:
53+
54+
```bash
55+
python3 -m pip install --upgrade pip setuptools wheel
56+
python3 setup.py sdist bdist_wheel
57+
```
58+
59+
You can install the package locally from the dist:
60+
61+
```bash
62+
python3 -m pip ./dist/some_name.whl
63+
```
64+
65+
Upload package works also with `make`:
66+
67+
```bash
68+
make dist
69+
make upload
70+
```
71+
72+
For local development:
73+
74+
```bash
75+
python3 setup.py develop
76+
```
77+
78+
## Contributing to the documentation
79+
80+
The documentation is a collection of [markdown](https://en.wikipedia.org/wiki/Markdown) files in the `docs` folder.
81+
After updates of the files, build and check the result with the following commands:
82+
83+
```bash
84+
make build-docs
85+
make serve-docs
86+
```
87+
88+
To update the changes to the official documentation pages run:
89+
90+
```bash
91+
make publish-docs
92+
```

‎docs/dsp-tools-usage.md‎

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
[![PyPI version](https://badge.fury.io/py/dsp-tools.svg)](https://badge.fury.io/py/dsp-tools)
2+
3+
## Installation
4+
5+
To install the latest version run:
6+
7+
```
8+
pip3 install dsp-tools
9+
```
10+
11+
To update to the latest version run:
12+
13+
```
14+
pip3 install --upgrade dsp-tools
15+
```
16+
17+
## Create a data model on a DSP server
18+
19+
```bash
20+
dsp-tools create [options] data_model_definition.json
21+
```
22+
23+
The following options are available:
24+
25+
- `-s` | `--server` _server_: URL of the DSP server (default: localhost:3333)
26+
- `-u` | `--user` _username_: username used for authentication with the DSP API (default: root@example.com)
27+
- `-p` | `--password` _password_: password used for authentication with the DSP API (default: test)
28+
- `-V` | `--validate`: If set, only the validation of the JSON file is performed.
29+
- `-l` | `--lists`: If set, only the lists are created using a [simplified schema](./dsp-tools-create.md#lists). Please note
30+
that in this case the project must already exist.
31+
- `-v` | `--verbose`: If set, some information about the progress is printed to the console.
32+
33+
The command is used to read the definition of a data model (provided in a JSON file) and create it on the
34+
DSP server. The following example shows how to load the ontology defined in `data_model_definition.json` onto the DSP
35+
server `https://api.dsl.server.org` provided with the `-s` option. The username `root@example.com` and the password
36+
`test` are used.
37+
38+
```bash
39+
dsp-tools create -s https://api.dsl.server.org -u root@example.com -p test data_model_definition.json
40+
```
41+
42+
The description of the expected JSON format can be found [here](./dsp-tools-create.md).
43+
44+
## Get a data model from a DSP server
45+
46+
```bash
47+
dsp-tools get [options] output_file.json
48+
```
49+
50+
The following options are available:
51+
52+
- `-s` | `--server` _server_: URL of the DSP server (default: localhost:3333)
53+
- `-u` | `--user` _username_: username used for authentication with the DSP API (default: root@example.com)
54+
- `-p` | `--password` _password_: password used for authentication with the DSP API (default: test)
55+
- `-P` | `--project` _shortcode_ | _shortname_ | _iri_: shortcode, shortname or
56+
[IRI](https://en.wikipedia.org/wiki/Internationalized_Resource_Identifier) of the project
57+
- `-v` | `--verbose`: If set, some information about the progress is printed to the console.
58+
59+
The command is used to get the definition of a data model from a DSP server and write it into a JSON file. This JSON file
60+
could then be used to upload the data model to another DSP server. The following example shows how to get the data model
61+
from a DSP server `https://api.dsl.server.org` provided with the `-s` option. The username `root@example.com` and the
62+
password `test` are used. The data model is saved into the output file `output_file.json`.
63+
64+
```bash
65+
dsp-tools get -s https://api.dsl.server.org -u root@example.com -p test output_file.json
66+
```
67+
68+
## Upload data to a DSP server
69+
70+
```bash
71+
dsp-tools xmlupload [options] xml_data_file.xml
72+
```
73+
74+
The following options are available:
75+
76+
- `-s` | `--server` _server_: URL of the DSP server (default: localhost:3333)
77+
- `-u` | `--user` _username_: username used for authentication with the DSP API (default: root@example.com)
78+
- `-p` | `--password` _password_: password used for authentication with the DSP API (default: test)
79+
- `-i` | `--imgdir` _dirpath_: path to the directory where the bitstream objects are stored (default: .)
80+
- `-S` | `--sipi` _SIPIserver_: URL of the SIPI IIIF server (default: http://0.0.0.0:1024)
81+
82+
The command is used to upload data defined in an XML file onto a DSP server. The following example shows how to upload
83+
data from an XML file `xml_data_file.xml` onto the DSP server `https://api.dsl.server.org` provided with the `-s` option.
84+
The username `root@example.com` and the password `test` are used. The interface for the SIPI IIIF server is provided
85+
with the `-S` option (`https://iiif.dsl.server.org`).
86+
87+
```bash
88+
dsp-tools xmlupload -s https://api.dsl.server.org -u root@example.com -p test -S https://iiif.dsl.server.org xml_data_file.xml
89+
```
90+
91+
The description of the expected XML format can be found [here](./dsp-tools-xmlupload.md).
92+
93+
## Convert an Excel file into a JSON file that is compatible with dsp-tools
94+
95+
```bash
96+
dsp-tools excel [options] excel_file.xlsx output_file.json
97+
```
98+
99+
The following options are available:
100+
101+
- `-S` | `--sheet` _sheetname_: name of the Excel worksheet to use (default: Tabelle1)
102+
- `-s` | `--shortcode` _shortcode_: shortcode of the project (required)
103+
- `-l` | `--listname` _listname_: name to be used for the list and the list definition file (required)
104+
- `-L` | `--label` _label_: label to be used for the list (required)
105+
- `-x` | `--lang` _lang_: language used for the list labels and commentaries (default: en)
106+
- `-v` | `--verbose`: If set, some information about the progress is printed to the console.
107+
108+
The description of the expected Excel format can be found [here](./dsp-tools-create.md#lists-from-excel).

0 commit comments

Comments
 (0)