The attributes of the current dataset in the lineage relationship module are not displayed correctly #1827

cobolbaby · 2020-08-27T08:33:23Z

cptran777 · 2020-08-27T18:00:57Z

Hey @cobolbaby , this looks like an issue with accommodating different types being presented by our OS and internal logic and possibly the logic you're currently using. Could you give a sample response of what your current dataset vs upstream vs downstream api looks like? (doesn't have to be any real information included just need a sample structure to debug)

cobolbaby · 2020-08-28T03:53:11Z

@cptran777 Can you give a curl example of how to call the related interface?

cptran777 · 2020-08-28T20:24:12Z

@cobolbaby Here's a curl call for upstreams using the fake quickstart data:

curl http://localhost:9001/api/v2/datasets/urn%3Ali%3Adataset%3A%28urn%3Ali%3AdataPlatform%3Ahdfs%2CSampleHdfsDataset%2CPROD%29/upstreams -H "Cookie:<COOKIE INFO HERE"

// Response
[
  {
    "actor": "urn:li:corpuser:jdoe",
    "dataset": {
      "createdTime": null,
      "decommissionTime": 1598684400000,
      "deprecated": true,
      "deprecationNote": "<p>deprecated</p>",
      "description": "",
      "fabric": "PROD",
      "modifiedTime": 1581407189000,
      "nativeName": "SampleKafkaDataset",
      "nativeType": null,
      "platform": "kafka",
      "properties": null,
      "removed": null,
      "tags": [],
      "uri": "urn:li:dataset:(urn:li:dataPlatform:kafka,SampleKafkaDataset,PROD)"
    },
    "modified": null,
    "type": "TRANSFORMED"
  }
]

cobolbaby · 2020-08-31T05:34:43Z

@cptran777

curl -X GET \
  'http://10.191.7.21:9001/api/v2/datasets/urn:li:dataset:%28urn:li:dataPlatform:bdc,dw.fact_cpu_sn,DEV%29/upstreams' \
  -H 'Content-Type: application/json' \
  -H 'Cookie: PLAY_SESSION=6f8d95f03e771c4ba9d24505afff8ef3dafb1ac6-auth_token=ZGF0YWh1YjoxNTk3MDMzNjA4MjUzOjBFNjUwNUVENzk2QjQwNTg5RTM3QkI0MkEyREQ1Nzg0NjkyN0VGRTRGNUFFRDQxMjkwOEU4RDIyNUY4RTk0OEQ%3D&uuid=36dee7c4-d330-46a0-a8da-c4ce5553e955&user=datahub'

[
    {
        "dataset": {
            "platform": "bdc",
            "nativeName": "fis.mds_sno",
            "fabric": "DEV",
            "uri": "urn:li:dataset:(urn:li:dataPlatform:bdc,fis.mds_sno,DEV)",
            "description": "",
            "nativeType": null,
            "properties": null,
            "tags": [
                "aaa",
                "bbb"
            ],
            "removed": null,
            "deprecated": null,
            "deprecationNote": null,
            "decommissionTime": null,
            "createdTime": null,
            "modifiedTime": 1596614389000
        },
        "type": "TRANSFORMED",
        "actor": "urn:li:corpuser:ITC180016",
        "modified": null
    },
    {
        "dataset": {
            "platform": "bdc",
            "nativeName": "fis.mds_unit_log",
            "fabric": "DEV",
            "uri": "urn:li:dataset:(urn:li:dataPlatform:bdc,fis.mds_unit_log,DEV)",
            "description": "",
            "nativeType": null,
            "properties": null,
            "tags": [
                "aaa",
                "bbb"
            ],
            "removed": null,
            "deprecated": null,
            "deprecationNote": null,
            "decommissionTime": null,
            "createdTime": null,
            "modifiedTime": 1596614389000
        },
        "type": "TRANSFORMED",
        "actor": "urn:li:corpuser:ITC180016",
        "modified": null
    },
    {
        "dataset": {
            "platform": "bdc",
            "nativeName": "fis.mds_sfc",
            "fabric": "DEV",
            "uri": "urn:li:dataset:(urn:li:dataPlatform:bdc,fis.mds_sfc,DEV)",
            "description": "",
            "nativeType": null,
            "properties": null,
            "tags": [
                "aaa",
                "bbb"
            ],
            "removed": null,
            "deprecated": null,
            "deprecationNote": null,
            "decommissionTime": null,
            "createdTime": null,
            "modifiedTime": 1596614389000
        },
        "type": "TRANSFORMED",
        "actor": "urn:li:corpuser:ITC180016",
        "modified": null
    },
    {
        "dataset": {
            "platform": "bdc",
            "nativeName": "fis.mds_dn",
            "fabric": "DEV",
            "uri": "urn:li:dataset:(urn:li:dataPlatform:bdc,fis.mds_dn,DEV)",
            "description": "",
            "nativeType": null,
            "properties": null,
            "tags": [
                "aaa",
                "bbb"
            ],
            "removed": null,
            "deprecated": null,
            "deprecationNote": null,
            "decommissionTime": null,
            "createdTime": null,
            "modifiedTime": 1596614389000
        },
        "type": "TRANSFORMED",
        "actor": "urn:li:corpuser:ITC180016",
        "modified": null
    },
    {
        "dataset": {
            "platform": "bdc",
            "nativeName": "fis.mds_parts",
            "fabric": "DEV",
            "uri": "urn:li:dataset:(urn:li:dataPlatform:bdc,fis.mds_parts,DEV)",
            "description": "",
            "nativeType": null,
            "properties": null,
            "tags": [
                "aaa",
                "bbb"
            ],
            "removed": null,
            "deprecated": null,
            "deprecationNote": null,
            "decommissionTime": null,
            "createdTime": null,
            "modifiedTime": 1596614389000
        },
        "type": "TRANSFORMED",
        "actor": "urn:li:corpuser:ITC180016",
        "modified": null
    },
    {
        "dataset": {
            "platform": "bdc",
            "nativeName": "fis.cto_sno",
            "fabric": "DEV",
            "uri": "urn:li:dataset:(urn:li:dataPlatform:bdc,fis.cto_sno,DEV)",
            "description": "",
            "nativeType": null,
            "properties": null,
            "tags": [
                "aaa",
                "bbb"
            ],
            "removed": null,
            "deprecated": null,
            "deprecationNote": null,
            "decommissionTime": null,
            "createdTime": null,
            "modifiedTime": 1596614389000
        },
        "type": "TRANSFORMED",
        "actor": "urn:li:corpuser:ITC180016",
        "modified": null
    },
    {
        "dataset": {
            "platform": "bdc",
            "nativeName": "fis.cto_unit_log",
            "fabric": "DEV",
            "uri": "urn:li:dataset:(urn:li:dataPlatform:bdc,fis.cto_unit_log,DEV)",
            "description": "",
            "nativeType": null,
            "properties": null,
            "tags": [
                "aaa",
                "bbb"
            ],
            "removed": null,
            "deprecated": null,
            "deprecationNote": null,
            "decommissionTime": null,
            "createdTime": null,
            "modifiedTime": 1596614389000
        },
        "type": "TRANSFORMED",
        "actor": "urn:li:corpuser:ITC180016",
        "modified": null
    },
    {
        "dataset": {
            "platform": "bdc",
            "nativeName": "fis.cto_sfc",
            "fabric": "DEV",
            "uri": "urn:li:dataset:(urn:li:dataPlatform:bdc,fis.cto_sfc,DEV)",
            "description": "",
            "nativeType": null,
            "properties": null,
            "tags": [
                "aaa",
                "bbb"
            ],
            "removed": null,
            "deprecated": null,
            "deprecationNote": null,
            "decommissionTime": null,
            "createdTime": null,
            "modifiedTime": 1596614389000
        },
        "type": "TRANSFORMED",
        "actor": "urn:li:corpuser:ITC180016",
        "modified": null
    },
    {
        "dataset": {
            "platform": "bdc",
            "nativeName": "fis.cto_dn",
            "fabric": "DEV",
            "uri": "urn:li:dataset:(urn:li:dataPlatform:bdc,fis.cto_dn,DEV)",
            "description": "",
            "nativeType": null,
            "properties": null,
            "tags": [
                "aaa",
                "bbb"
            ],
            "removed": null,
            "deprecated": null,
            "deprecationNote": null,
            "decommissionTime": null,
            "createdTime": null,
            "modifiedTime": 1596614389000
        },
        "type": "TRANSFORMED",
        "actor": "urn:li:corpuser:ITC180016",
        "modified": null
    },
    {
        "dataset": {
            "platform": "bdc",
            "nativeName": "fis.cto_parts",
            "fabric": "DEV",
            "uri": "urn:li:dataset:(urn:li:dataPlatform:bdc,fis.cto_parts,DEV)",
            "description": "",
            "nativeType": null,
            "properties": null,
            "tags": [
                "aaa",
                "bbb"
            ],
            "removed": null,
            "deprecated": null,
            "deprecationNote": null,
            "decommissionTime": null,
            "createdTime": null,
            "modifiedTime": 1596614389000
        },
        "type": "TRANSFORMED",
        "actor": "urn:li:corpuser:ITC180016",
        "modified": null
    }
]

curl -X GET \
  'http://10.191.7.21:9001/api/v2/datasets/urn:li:dataset:%28urn:li:dataPlatform:bdc,dw.fact_cpu_sn,DEV%29/downstreams' \
  -H 'Content-Type: application/json' \
  -H 'Cookie: PLAY_SESSION=6f8d95f03e771c4ba9d24505afff8ef3dafb1ac6-auth_token=ZGF0YWh1YjoxNTk3MDMzNjA4MjUzOjBFNjUwNUVENzk2QjQwNTg5RTM3QkI0MkEyREQ1Nzg0NjkyN0VGRTRGNUFFRDQxMjkwOEU4RDIyNUY4RTk0OEQ%3D&uuid=36dee7c4-d330-46a0-a8da-c4ce5553e955&user=datahub' 

[]

cptran777 · 2020-09-01T22:54:51Z

Thanks! I'll start investigating based on the information we have right now.

cptran777 · 2020-09-01T22:55:48Z

Also @cobolbaby thank you for being willing to try out this beta of our new FE and helping discover these bugs/issues. Really appreciate it.

cptran777 · 2020-09-15T02:14:41Z

Hey @cobolbaby, could you pull down the latest changes from #1874 and confirm if the issue is fixed for you?

frsann · 2020-09-15T03:21:15Z

I had similar problems (in addition the graph visualisation wouldn't render correctly) with the quickstart example + dummy data. I can confirm that this change fixes those issue ✅

cobolbaby · 2020-09-15T10:48:51Z

Confirmed.

cobolbaby added the bug Bug report label Aug 27, 2020

cobolbaby changed the title ~~The attributes of the current data set in the blood relationship module are not displayed correctly~~ The attributes of the current dataset in the lineage relationship module are not displayed correctly Aug 27, 2020

mars-lan assigned cptran777 Aug 27, 2020

This was referenced Sep 15, 2020

Social function issues introduced by the latest version of the frontend #1824

Closed

fix (frontend): Partially fixes lineage issues and dataset API handling #1874

Merged

cobolbaby closed this as completed Sep 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The attributes of the current dataset in the lineage relationship module are not displayed correctly #1827

The attributes of the current dataset in the lineage relationship module are not displayed correctly #1827

cobolbaby commented Aug 27, 2020

cptran777 commented Aug 27, 2020

cobolbaby commented Aug 28, 2020

cptran777 commented Aug 28, 2020 •

edited

cobolbaby commented Aug 31, 2020 •

edited

cptran777 commented Sep 1, 2020

cptran777 commented Sep 1, 2020

cptran777 commented Sep 15, 2020

frsann commented Sep 15, 2020

cobolbaby commented Sep 15, 2020

The attributes of the current dataset in the lineage relationship module are not displayed correctly #1827

The attributes of the current dataset in the lineage relationship module are not displayed correctly #1827

Comments

cobolbaby commented Aug 27, 2020

cptran777 commented Aug 27, 2020

cobolbaby commented Aug 28, 2020

cptran777 commented Aug 28, 2020 • edited

cobolbaby commented Aug 31, 2020 • edited

cptran777 commented Sep 1, 2020

cptran777 commented Sep 1, 2020

cptran777 commented Sep 15, 2020

frsann commented Sep 15, 2020

cobolbaby commented Sep 15, 2020

cptran777 commented Aug 28, 2020 •

edited

cobolbaby commented Aug 31, 2020 •

edited