From 25e3c210d7ba5873adae071bad0c39a684d5220e Mon Sep 17 00:00:00 2001 From: arnoweiss Date: Thu, 14 Dec 2023 22:01:05 +0100 Subject: [PATCH 1/6] add guidance on quality assets --- .../page_software-development-view.md | 95 ++++++++++++++++++- 1 file changed, 92 insertions(+), 3 deletions(-) diff --git a/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md b/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md index 107bc0b0264..1c5e77e1c16 100644 --- a/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md +++ b/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md @@ -11,7 +11,89 @@ sidebar_position: 4 ## Quality Data Offers at EDC -For the process described in the KIT Quality, data exchange between the participating partner companies is necessarily to be done for large vehicle and product populations. The data exchange therefore tshould be done as a file download via EDC according to the following specifications. +When a Data Consumer calls the Catalog of a Data Provider, the Data Provider must signal in each Data Offer what exactly +a Consumer could negotiate for. Data Offers in the Catalog are sorted by dcat:Datasets which are registered in the EDC- +Management API as edc:Asset. Each Asset has private and public properties. The public properties are shown in the +catalog and give hints to the Data Consumer what API and data they may expect. There are some properties that are +mandatory for the entire Catena-X network and some that are mandatory only in specific Business Scenarios (like +Quality). + +The dataAddress object's structure is determined by the dataplane implementation as it configures the details of the +data transfer. They are not visible via the catalog. + +The following suggestion is a draft how Assets (and thus by proxy, dcat:Datasets) should be registered +in the Quality Use-Case. + +```json +{ + "@context": { + "cx-taxo": "https://w3id.org/catenax/taxonomy#", + "cx-common": "https://w3id.org/catenax/ontology/common#", + "dct": "https://purl.org/dc/terms/", + "dcat": "http://www.w3.org/ns/dcat#", + "edc": "https://w3id.org/edc/v0.0.1/ns/" + }, + "@id": "someId", + "@type": "edc:Asset", + "edc:properties": { + "dct:type": { + "@id": "cx-taxo:ProductDescription" + }, + "cx-common:version": "0.9", + "dct:language": { + "@id": "https://w3id.org/idsa/code/EN" + }, + "dcat:qualifiedRelation": { + "dct:isPartOf": { + "@id": "http://my.quality/task" + } + }, + "dct:description": "TBD", + "dct:format": "application/octet-stream;type=parquet-snappy", + "edc:type": "AmazonS3" + }, + "edc:dataAddress": { + "@type": "edc:DataAddress", + "edc:type": "AmazonS3", + "edc:region": "eu-west-1", + "edc:bucketName": "int-xcod-quality-aspect-models-eu-west-1", + "edc:keyName": "myCompany/myTag/QualityTask.parquet", + "edc:accessKeyId": "…", + "edc:secretAccessKey": "…" + } +} + +``` + +### S3 Data Address + +This section is not use-case specific but since the EDC's AmazonS3 dataplane is basically undocumented, here is an +explanation: + +| Property | Value | Description | +|-----------------------|------------------------|-----------------------------------------------------------------------------------------------------------------------------------| +| `edc:type` | `"AmazonS3"` | This shows which data source the Data Plane will query. It also determines what other content the `dataAddress` object must hold. | +| `edc:region` | `"eu-west-1"` | This property represents the AWS-region where the source bucket is located. | +| `edc:bucketName` | `""` | This is the name of the source bucket that the data to-be-transferred resides in. | +| `edc:keyName` | `""` | This is the path of the file that shall be offered to the dataspace. | +| `edc:accessKeyId` | `""` | TODO | +| `edc:secretAccessKey` | `""` | TODO | + +### Properties + +| Property | Value | Description | +|----------------------------------------------------|-------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `https://purl.org/dc/terms/type` | `{"@id": "cx-taxo:{{qualityAssetType}}"}` | CX-0018 mandates the usage of the dct:type property to signal what kind of Asset a consumer can expect behind a dcat:Dataset. In the Quality Use-Case, this one of
- `https://w3id.org/catenax/taxonomy#QualityTask`
- `https://w3id.org/catenax/taxonomy#ProductDescription`
- ??
- ??
All must be added to the cx-taxonomy. | +| `https://w3id.org/catenax/ontology/common#version` | `"0.9"` | CX-0018 recommends to use cx-common:version to signify the API's version. Since QM has a tight connection between the API and the datamodel, this value could describe the version of the CX-API-standard for the Quality use-case. | +| `https://purl.org/dc/terms/language` | `{"@id": "https://w3id.org/idsa/code/EN"}` | This property is QM-specific. As it points to an IRI, it must be embedded in a json-object with the `@id` key. The use of this is unclear. | +| `http://www.w3.org/ns/dcat#qualifiedRelation` | `{"dct:isPartOf": {"@id": "http://my.quality/task"}}` | This property is QM-specific. All QM-Asset types that are not cx-taxo:QualityTask must include this property to enable integrating the data behind this asset with the correct QualityTask. | +| `https://purl.org/dc/terms/description` | `` | This property is QM-specific. For human-readable content, rdfs:comment is the usual property but would introduce another namespace so the dct-native property is chosen here. | +| `https://purl.org/dc/terms/format` | `"application/octet-stream;type=parquet-snappy"` | This property is QM-specific. dct:format usually points to the correct IANA Media Type. As currently only parquet files are used, the type application/octet-stream with the added property type=parquet-snappy must be used. The syntax is expained [here](https://www.iana.org/assignments/media-types-parameters/media-types-parameters.xhtml). If in the future csv shall be supported, the value could also be `text/csv`. | +| `https://w3id.org/edc/v0.0.1/ns/type` | `AmazonS3` | This property signifies the EDC dataplane that the QM data will be transferred over. The expectation that this would be signaled via the dcat:DataSet-dcat:distribution property of the catalog currently isn't implemented in the EDC. Thus the data must be replicated here and is presented via the same property that the consumer-side `transferprocesses` API uses for this same signal. | + +For the process described in the KIT Quality, data exchange between the participating partner companies is necessarily +to be done for large vehicle and product populations. The data exchange therefore should be done as a file download via +EDC according to the following specifications. ### **Asset File type** @@ -23,13 +105,18 @@ File transfer is recommended to be done via EDC S3 plane, The transfer via EDC h ### **Asset consumption** -**File** flattening **rules**: The data provided in the asset is build from 4 to 6 structures. To assure a secure and smooth exchange flatting rules for the file (csv / xls / Parquet / json) must be applied. This includes checks for format and possible values for each column and will be part of next version regulations. If the rules are not applied correctly the mapping of content will not be possible without manual handling effort. +**File** flattening **rules**: The data provided in the asset is build from 4 to 6 structures. To assure a secure and +smooth exchange flatting rules for the file (csv / xls / Parquet / json) must be applied. This includes checks for +format and possible values for each column and will be part of next version regulations. If the rules are not applied +correctly the mapping of content will not be possible without manual handling effort. ## Sample Data Standard version from: 09.2023 -In the following, example data for the standardized data models are provided as download in zip format. The sample data is generated according the current standards. It contains a virtual fleet of 50.000 vehicles where two quality issues are implemented. +In the following, example data for the standardized data models are provided as download in zip format. The sample data +is generated according the current standards. It contains a virtual fleet of 50.000 vehicles where two quality issues +are implemented. - Production failure of product "zehn" at Tier 1 - Specification failure @@ -143,3 +230,5 @@ CX_release32_partsanalyses_200_testdata_100_json As **data provider** please add the **PARQUET file** from folder tesdata_CX32 as EDC asset id to **EDC S3 data plane**: CX_release32_partsanalyses_200_testdata_100_parquet + +## \ No newline at end of file From d84935b660282f86edd718d357ecf9aea2d4c2a5 Mon Sep 17 00:00:00 2001 From: arnoweiss Date: Fri, 15 Dec 2023 08:56:21 +0100 Subject: [PATCH 2/6] add explanation of s3 authentication --- .../page_software-development-view.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md b/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md index 1c5e77e1c16..3b7397fcab7 100644 --- a/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md +++ b/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md @@ -21,7 +21,7 @@ Quality). The dataAddress object's structure is determined by the dataplane implementation as it configures the details of the data transfer. They are not visible via the catalog. -The following suggestion is a draft how Assets (and thus by proxy, dcat:Datasets) should be registered +The following suggestion is a non-standardized draft how Assets (and thus by proxy, dcat:Datasets) should be registered in the Quality Use-Case. ```json @@ -70,14 +70,14 @@ in the Quality Use-Case. This section is not use-case specific but since the EDC's AmazonS3 dataplane is basically undocumented, here is an explanation: -| Property | Value | Description | -|-----------------------|------------------------|-----------------------------------------------------------------------------------------------------------------------------------| -| `edc:type` | `"AmazonS3"` | This shows which data source the Data Plane will query. It also determines what other content the `dataAddress` object must hold. | -| `edc:region` | `"eu-west-1"` | This property represents the AWS-region where the source bucket is located. | -| `edc:bucketName` | `""` | This is the name of the source bucket that the data to-be-transferred resides in. | -| `edc:keyName` | `""` | This is the path of the file that shall be offered to the dataspace. | -| `edc:accessKeyId` | `""` | TODO | -| `edc:secretAccessKey` | `""` | TODO | +| Property | Value | Description | +|-----------------------|------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `edc:type` | `"AmazonS3"` | This shows which data source the Data Plane will query. It also determines what other content the `dataAddress` object must hold. | +| `edc:region` | `"eu-west-1"` | This property represents the AWS-region where the source bucket is located. | +| `edc:bucketName` | `"provider-quality-bucket"` | This is the name of the source bucket that the data to-be-transferred resides in. | +| `edc:keyName` | `"path/through/provider/s3"` | This is the path of the file that shall be offered to the dataspace. | +| `edc:accessKeyId` | `""` | Amazon S3 uses this property similarly to how oauth2 client credentials use the `clientId`. Note that this can also be set during deployment-time for the whole S3-dataplane. If it's set here, it will override the default config. | +| `edc:secretAccessKey` | `""` | This secret is used similarly to a `clientSecret` in oauth2 client credentials | ### Properties From ac074d505630a5b1b82d789a393464bca0ede82b Mon Sep 17 00:00:00 2001 From: arnoweiss Date: Wed, 20 Dec 2023 09:39:08 +0100 Subject: [PATCH 3/6] add dct:conformsTo, dcat:keyword --- .../page_software-development-view.md | 26 ++++++++++++------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md b/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md index 3b7397fcab7..067d4d2c868 100644 --- a/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md +++ b/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md @@ -39,7 +39,7 @@ in the Quality Use-Case. "dct:type": { "@id": "cx-taxo:ProductDescription" }, - "cx-common:version": "0.9", + "cx-common:version": "1.0", "dct:language": { "@id": "https://w3id.org/idsa/code/EN" }, @@ -48,6 +48,10 @@ in the Quality Use-Case. "@id": "http://my.quality/task" } }, + "dcat:keyword": "Quality", + "dct:conformsTo": { + "@id": "urn:samm:io.catenax.vehicle.product_description:3.0.0#ProductDescription" + }, "dct:description": "TBD", "dct:format": "application/octet-stream;type=parquet-snappy", "edc:type": "AmazonS3" @@ -81,15 +85,17 @@ explanation: ### Properties -| Property | Value | Description | -|----------------------------------------------------|-------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `https://purl.org/dc/terms/type` | `{"@id": "cx-taxo:{{qualityAssetType}}"}` | CX-0018 mandates the usage of the dct:type property to signal what kind of Asset a consumer can expect behind a dcat:Dataset. In the Quality Use-Case, this one of
- `https://w3id.org/catenax/taxonomy#QualityTask`
- `https://w3id.org/catenax/taxonomy#ProductDescription`
- ??
- ??
All must be added to the cx-taxonomy. | -| `https://w3id.org/catenax/ontology/common#version` | `"0.9"` | CX-0018 recommends to use cx-common:version to signify the API's version. Since QM has a tight connection between the API and the datamodel, this value could describe the version of the CX-API-standard for the Quality use-case. | -| `https://purl.org/dc/terms/language` | `{"@id": "https://w3id.org/idsa/code/EN"}` | This property is QM-specific. As it points to an IRI, it must be embedded in a json-object with the `@id` key. The use of this is unclear. | -| `http://www.w3.org/ns/dcat#qualifiedRelation` | `{"dct:isPartOf": {"@id": "http://my.quality/task"}}` | This property is QM-specific. All QM-Asset types that are not cx-taxo:QualityTask must include this property to enable integrating the data behind this asset with the correct QualityTask. | -| `https://purl.org/dc/terms/description` | `` | This property is QM-specific. For human-readable content, rdfs:comment is the usual property but would introduce another namespace so the dct-native property is chosen here. | -| `https://purl.org/dc/terms/format` | `"application/octet-stream;type=parquet-snappy"` | This property is QM-specific. dct:format usually points to the correct IANA Media Type. As currently only parquet files are used, the type application/octet-stream with the added property type=parquet-snappy must be used. The syntax is expained [here](https://www.iana.org/assignments/media-types-parameters/media-types-parameters.xhtml). If in the future csv shall be supported, the value could also be `text/csv`. | -| `https://w3id.org/edc/v0.0.1/ns/type` | `AmazonS3` | This property signifies the EDC dataplane that the QM data will be transferred over. The expectation that this would be signaled via the dcat:DataSet-dcat:distribution property of the catalog currently isn't implemented in the EDC. Thus the data must be replicated here and is presented via the same property that the consumer-side `transferprocesses` API uses for this same signal. | +| Property | Value | Description | +|----------------------------------------------------|------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `https://purl.org/dc/terms/type` | `{"@id": "cx-taxo:{{qualityAssetType}}"}` | CX-0018 mandates the usage of the dct:type property to signal what kind of Asset a consumer can expect behind a dcat:Dataset. In the Quality Use-Case, this one of
- `https://w3id.org/catenax/taxonomy#QualityTask`
- `https://w3id.org/catenax/taxonomy#ProductDescription`
- ??
- ??
All must be added to the cx-taxonomy. | +| `https://purl.org/dc/terms/language` | `{"@id": "https://w3id.org/idsa/code/EN"}` | This property is QM-specific. As it points to an IRI, it must be embedded in a json-object with the `@id` key. The use of this is unclear. | +| `https://purl.org/dc/terms/format` | `"application/octet-stream;type=parquet-snappy"` | This property is QM-specific. dct:format usually points to the correct IANA Media Type. As currently only parquet files are used, the type application/octet-stream with the added property type=parquet-snappy must be used. The syntax is expained [here](https://www.iana.org/assignments/media-types-parameters/media-types-parameters.xhtml). If in the future csv shall be supported, the value could also be `text/csv`. | +| `https://purl.org/dc/terms/description` | `` | This property is QM-specific. For human-readable content, rdfs:comment is the usual property but would introduce another namespace so the dct-native property is chosen here. | +| `https://purl.org/dc/terms/conformsTo` | `{"@id":""}` | This property is QM-specific. It holds the exact aspect-model-URN that defines the schema of the presented dataset including its version. The version in here refers to the data model's version while the EDC-property `cx-common:version` defines the version of the underlying API serving the data. | +| `http://www.w3.org/ns/dcat#qualifiedRelation` | `{"dct:isPartOf": {"@id": ""}}` | This property is QM-specific. All Asset types defined in this Kit must include this property as it links the data behind an asset with the correct QualityTask. Note that the id of the QualityTask must be used, not the id of the EDC-Asset shielding said QualityTask. | +| `http://www.w3.org/ns/dcat#keyword` | `Quality` | This property is QM-specific. To enable consumer-side filtering for all Data Offers related to the exchange of quality-relevant information, this keyword must be by all Data Providers set to `"quality"`. | +| `https://w3id.org/edc/v0.0.1/ns/type` | `AmazonS3` | This property signifies the EDC dataplane that the QM data will be transferred over. The expectation that this would be signaled via the dcat:DataSet-dcat:distribution property of the catalog currently isn't implemented in the EDC. Thus the data must be replicated here and is presented via the same property that the consumer-side `transferprocesses` API uses for this same signal. | +| `https://w3id.org/catenax/ontology/common#version` | `"1.0"` | CX-0018 recommends to use cx-common:version to signify the API's version. Since QM has a tight connection between the API and the datamodel, this value could describe the version of the CX-API-standard for the Quality use-case. | For the process described in the KIT Quality, data exchange between the participating partner companies is necessarily to be done for large vehicle and product populations. The data exchange therefore should be done as a file download via From b402cdd6cf49ca97c7fea949deb3827daf05f879 Mon Sep 17 00:00:00 2001 From: arnoweiss Date: Wed, 17 Jan 2024 17:46:07 +0100 Subject: [PATCH 4/6] remove dcat:keyword and clarify dct:type property --- .../page_software-development-view.md | 20 +++++++++---------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md b/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md index 067d4d2c868..e67137e4b2c 100644 --- a/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md +++ b/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md @@ -48,7 +48,6 @@ in the Quality Use-Case. "@id": "http://my.quality/task" } }, - "dcat:keyword": "Quality", "dct:conformsTo": { "@id": "urn:samm:io.catenax.vehicle.product_description:3.0.0#ProductDescription" }, @@ -85,17 +84,16 @@ explanation: ### Properties -| Property | Value | Description | -|----------------------------------------------------|------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `https://purl.org/dc/terms/type` | `{"@id": "cx-taxo:{{qualityAssetType}}"}` | CX-0018 mandates the usage of the dct:type property to signal what kind of Asset a consumer can expect behind a dcat:Dataset. In the Quality Use-Case, this one of
- `https://w3id.org/catenax/taxonomy#QualityTask`
- `https://w3id.org/catenax/taxonomy#ProductDescription`
- ??
- ??
All must be added to the cx-taxonomy. | -| `https://purl.org/dc/terms/language` | `{"@id": "https://w3id.org/idsa/code/EN"}` | This property is QM-specific. As it points to an IRI, it must be embedded in a json-object with the `@id` key. The use of this is unclear. | +| Property | Value | Description | +|----------------------------------------------------|------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `https://purl.org/dc/terms/type` | `{"@id": "cx-taxo:QualityAsset"}` | CX-0018 mandates the usage of the dct:type property to signal what kind of Asset a consumer can expect behind a dcat:Dataset. In the Quality Use-Case, this is identified as `https://w3id.org/catenax/taxonomy#QualityAsset`. The expected payload this API serves is determined by the `dcat:conformsTo` property. | +| `https://purl.org/dc/terms/language` | `{"@id": "https://w3id.org/idsa/code/EN"}` | This property is QM-specific. As it points to an IRI, it must be embedded in a json-object with the `@id` key. The use of this is unclear. | | `https://purl.org/dc/terms/format` | `"application/octet-stream;type=parquet-snappy"` | This property is QM-specific. dct:format usually points to the correct IANA Media Type. As currently only parquet files are used, the type application/octet-stream with the added property type=parquet-snappy must be used. The syntax is expained [here](https://www.iana.org/assignments/media-types-parameters/media-types-parameters.xhtml). If in the future csv shall be supported, the value could also be `text/csv`. | -| `https://purl.org/dc/terms/description` | `` | This property is QM-specific. For human-readable content, rdfs:comment is the usual property but would introduce another namespace so the dct-native property is chosen here. | -| `https://purl.org/dc/terms/conformsTo` | `{"@id":""}` | This property is QM-specific. It holds the exact aspect-model-URN that defines the schema of the presented dataset including its version. The version in here refers to the data model's version while the EDC-property `cx-common:version` defines the version of the underlying API serving the data. | -| `http://www.w3.org/ns/dcat#qualifiedRelation` | `{"dct:isPartOf": {"@id": ""}}` | This property is QM-specific. All Asset types defined in this Kit must include this property as it links the data behind an asset with the correct QualityTask. Note that the id of the QualityTask must be used, not the id of the EDC-Asset shielding said QualityTask. | -| `http://www.w3.org/ns/dcat#keyword` | `Quality` | This property is QM-specific. To enable consumer-side filtering for all Data Offers related to the exchange of quality-relevant information, this keyword must be by all Data Providers set to `"quality"`. | -| `https://w3id.org/edc/v0.0.1/ns/type` | `AmazonS3` | This property signifies the EDC dataplane that the QM data will be transferred over. The expectation that this would be signaled via the dcat:DataSet-dcat:distribution property of the catalog currently isn't implemented in the EDC. Thus the data must be replicated here and is presented via the same property that the consumer-side `transferprocesses` API uses for this same signal. | -| `https://w3id.org/catenax/ontology/common#version` | `"1.0"` | CX-0018 recommends to use cx-common:version to signify the API's version. Since QM has a tight connection between the API and the datamodel, this value could describe the version of the CX-API-standard for the Quality use-case. | +| `https://purl.org/dc/terms/description` | `` | This property is QM-specific. For human-readable content, rdfs:comment is the usual property but would introduce another namespace so the dct-native property is chosen here. | +| `https://purl.org/dc/terms/conformsTo` | `{"@id":""}` | This property is QM-specific. It holds the exact aspect-model-URN that defines the schema of the presented dataset including its version. The version in here refers to the data model's version while the EDC-property `cx-common:version` defines the version of the underlying API serving the data. | +| `http://www.w3.org/ns/dcat#qualifiedRelation` | `{"dct:isPartOf": {"@id": ""}}` | This property is QM-specific. All Asset types defined in this Kit must include this property as it links the data behind an asset with the correct QualityTask. Note that the id of the QualityTask must be used, not the id of the EDC-Asset shielding said QualityTask. | +| `https://w3id.org/edc/v0.0.1/ns/type` | `AmazonS3` | This property signifies the EDC dataplane that the QM data will be transferred over. The expectation that this would be signaled via the dcat:DataSet-dcat:distribution property of the catalog currently isn't implemented in the EDC. Thus the data must be replicated here and is presented via the same property that the consumer-side `transferprocesses` API uses for this same signal. | +| `https://w3id.org/catenax/ontology/common#version` | `"1.0"` | CX-0018 recommends to use cx-common:version to signify the API's version. Since QM has a tight connection between the API and the datamodel, this value could describe the version of the CX-API-standard for the Quality use-case. Creation is currently in progress as CX-0123 v1.0.0. | For the process described in the KIT Quality, data exchange between the participating partner companies is necessarily to be done for large vehicle and product populations. The data exchange therefore should be done as a file download via From 67ed26b9027ac0dcbe4898bbdbfb9b8a661cef0e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Arno=20Wei=C3=9F?= <86715435+arnoweiss@users.noreply.github.com> Date: Mon, 22 Jan 2024 11:19:47 +0100 Subject: [PATCH 5/6] correct bug in quality-kit --- .../page_software-development-view.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md b/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md index e67137e4b2c..fae2cd666a1 100644 --- a/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md +++ b/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md @@ -37,7 +37,7 @@ in the Quality Use-Case. "@type": "edc:Asset", "edc:properties": { "dct:type": { - "@id": "cx-taxo:ProductDescription" + "@id": "cx-taxo:QualityAsset" }, "cx-common:version": "1.0", "dct:language": { @@ -235,4 +235,4 @@ As **data provider** please add the **PARQUET file** from folder tesdata_CX32 as EDC asset id to **EDC S3 data plane**: CX_release32_partsanalyses_200_testdata_100_parquet -## \ No newline at end of file +## From a6981e5324603260f915a5b685ac09c1801e335a Mon Sep 17 00:00:00 2001 From: arnoweiss Date: Mon, 22 Jan 2024 15:31:08 +0100 Subject: [PATCH 6/6] integrate review comments --- .../page_software-development-view.md | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md b/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md index fae2cd666a1..ad867611e6f 100644 --- a/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md +++ b/docs-kits/kits/Quality-Kit/Software Development View/page_software-development-view.md @@ -45,7 +45,7 @@ in the Quality Use-Case. }, "dcat:qualifiedRelation": { "dct:isPartOf": { - "@id": "http://my.quality/task" + "@id": "f7574ad6-95ee-46e2-8a45-6fa1782ba426" } }, "dct:conformsTo": { @@ -80,20 +80,20 @@ explanation: | `edc:bucketName` | `"provider-quality-bucket"` | This is the name of the source bucket that the data to-be-transferred resides in. | | `edc:keyName` | `"path/through/provider/s3"` | This is the path of the file that shall be offered to the dataspace. | | `edc:accessKeyId` | `""` | Amazon S3 uses this property similarly to how oauth2 client credentials use the `clientId`. Note that this can also be set during deployment-time for the whole S3-dataplane. If it's set here, it will override the default config. | -| `edc:secretAccessKey` | `""` | This secret is used similarly to a `clientSecret` in oauth2 client credentials | +| `edc:secretAccessKey` | `""` | This secret is used similarly to a `clientSecret` in oauth2 client credentials. | ### Properties -| Property | Value | Description | -|----------------------------------------------------|------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `https://purl.org/dc/terms/type` | `{"@id": "cx-taxo:QualityAsset"}` | CX-0018 mandates the usage of the dct:type property to signal what kind of Asset a consumer can expect behind a dcat:Dataset. In the Quality Use-Case, this is identified as `https://w3id.org/catenax/taxonomy#QualityAsset`. The expected payload this API serves is determined by the `dcat:conformsTo` property. | -| `https://purl.org/dc/terms/language` | `{"@id": "https://w3id.org/idsa/code/EN"}` | This property is QM-specific. As it points to an IRI, it must be embedded in a json-object with the `@id` key. The use of this is unclear. | -| `https://purl.org/dc/terms/format` | `"application/octet-stream;type=parquet-snappy"` | This property is QM-specific. dct:format usually points to the correct IANA Media Type. As currently only parquet files are used, the type application/octet-stream with the added property type=parquet-snappy must be used. The syntax is expained [here](https://www.iana.org/assignments/media-types-parameters/media-types-parameters.xhtml). If in the future csv shall be supported, the value could also be `text/csv`. | -| `https://purl.org/dc/terms/description` | `` | This property is QM-specific. For human-readable content, rdfs:comment is the usual property but would introduce another namespace so the dct-native property is chosen here. | -| `https://purl.org/dc/terms/conformsTo` | `{"@id":""}` | This property is QM-specific. It holds the exact aspect-model-URN that defines the schema of the presented dataset including its version. The version in here refers to the data model's version while the EDC-property `cx-common:version` defines the version of the underlying API serving the data. | -| `http://www.w3.org/ns/dcat#qualifiedRelation` | `{"dct:isPartOf": {"@id": ""}}` | This property is QM-specific. All Asset types defined in this Kit must include this property as it links the data behind an asset with the correct QualityTask. Note that the id of the QualityTask must be used, not the id of the EDC-Asset shielding said QualityTask. | -| `https://w3id.org/edc/v0.0.1/ns/type` | `AmazonS3` | This property signifies the EDC dataplane that the QM data will be transferred over. The expectation that this would be signaled via the dcat:DataSet-dcat:distribution property of the catalog currently isn't implemented in the EDC. Thus the data must be replicated here and is presented via the same property that the consumer-side `transferprocesses` API uses for this same signal. | -| `https://w3id.org/catenax/ontology/common#version` | `"1.0"` | CX-0018 recommends to use cx-common:version to signify the API's version. Since QM has a tight connection between the API and the datamodel, this value could describe the version of the CX-API-standard for the Quality use-case. Creation is currently in progress as CX-0123 v1.0.0. | +| Property | Value | Optional | Description | +|----------------------------------------------------|------------------------------------------------------------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `https://purl.org/dc/terms/type` | `{"@id": "cx-taxo:QualityAsset"}` | | CX-0018 mandates the usage of the dct:type property to signal what kind of Asset a consumer can expect behind a dcat:Dataset. In the Quality Use-Case, this is identified as `https://w3id.org/catenax/taxonomy#QualityAsset`. The expected payload this API serves is determined by the `dcat:conformsTo` property. | +| `https://purl.org/dc/terms/language` | `{"@id": "https://w3id.org/idsa/code/EN"}` | x | This property is QM-specific. As it points to an IRI, it must be embedded in a json-object with the `@id` key. The use of this is unclear. | +| `https://purl.org/dc/terms/format` | `"application/octet-stream;type=parquet-snappy"` | | This property is QM-specific. dct:format usually points to the correct IANA Media Type. As currently only parquet files are used, the type application/octet-stream with the added property type=parquet-snappy must be used. The syntax is expained [here](https://www.iana.org/assignments/media-types-parameters/media-types-parameters.xhtml). If in the future csv shall be supported, the value could also be `text/csv`. | +| `https://purl.org/dc/terms/description` | `` | x | This property is QM-specific. For human-readable content, rdfs:comment is the usual property but would introduce another namespace so the dct-native property is chosen here. | +| `https://purl.org/dc/terms/conformsTo` | `{"@id":""}` | | This property is QM-specific. It holds the exact aspect-model-URN that defines the schema of the presented dataset including its version. The version in here refers to the data model's version while the EDC-property `cx-common:version` defines the version of the underlying API serving the data. | +| `http://www.w3.org/ns/dcat#qualifiedRelation` | `{"dct:isPartOf": {"@id": ""}}` | | This property is QM-specific. All Asset types defined in this Kit must include this property as it links the data behind an asset with the correct QualityTask. Note that the id of the QualityTask must be used, not the id of the EDC-Asset shielding said QualityTask. | +| `https://w3id.org/edc/v0.0.1/ns/type` | `AmazonS3` | | This property signifies the EDC dataplane that the QM data will be transferred over. The expectation that this would be signaled via the dcat:DataSet-dcat:distribution property of the catalog currently isn't implemented in the EDC. Thus the data must be replicated here and is presented via the same property that the consumer-side `transferprocesses` API uses for this same signal. | +| `https://w3id.org/catenax/ontology/common#version` | `"1.0"` | | CX-0018 recommends to use cx-common:version to signal the API's version. Here, the API's version is equivalent to the version of the CX-standard for the Quality domain. Creation is currently in progress as CX-0123 v1.0.0. In this EDC-property, only major and minor increments should be added. | For the process described in the KIT Quality, data exchange between the participating partner companies is necessarily to be done for large vehicle and product populations. The data exchange therefore should be done as a file download via