Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A short description of the bug #9111

Closed
mattmm1 opened this issue Oct 26, 2023 · 1 comment · Fixed by #9157
Closed

A short description of the bug #9111

mattmm1 opened this issue Oct 26, 2023 · 1 comment · Fixed by #9157
Labels
bug Bug report

Comments

@mattmm1
Copy link

mattmm1 commented Oct 26, 2023

Describe the bug
HI in our powerBI ingest we're seeing this ocasionally in our logs:

AssertionError
{datahub.ingestion.api.source_helpers:188} Failed to lowercase urns for Mapper.EquableMetadataWorkUnit(id='powerbi-urn:li:dataset:(urn:li:dataPlatform:powerbi,PROD.ent_support_-_powerbi_dashboards_%28will_h%29.axis_live_sites,PROD)-upstreamLineage', metadata=MetadataChangeProposalWrapper(entityType='dataset', changeType='UPSERT', entityUrn='urn:li:dataset:(urn:li:dataPlatform:powerbi,prod.ent_support_-_powerbi_dashboards_%28will_h%29.axis_live_sites,PROD)', entityKeyAspect=None, auditHeader=None, aspectName='upstreamLineage', aspect=UpstreamLineageClass({'upstreams': [UpstreamClass({'auditStamp': AuditStampClass({'time': 0, 'actor': 'urn:li:corpuser:unknown', 'impersonator': None, 'message': None}), 'created': None, 'dataset': 'urn:li:dataset:(urn:li:dataPlatform:snowflake,PROD.operations_analytics.test.willh_axis_site_info,PROD)', 'type': 'TRANSFORMED', 'properties': None})], 'fineGrainedLineages': None}), systemMetadata=None), treat_errors_as_warnings=False, is_primary_source=True):
Traceback (most recent call last):
  File "/home/python/venv/dnap-datahub-ingest-8zpDmlFD-py3.10/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 183, in auto_lowercase_urns
    lowercase_dataset_urns(wu.metadata)
  File "/home/python/venv/dnap-datahub-ingest-8zpDmlFD-py3.10/lib/python3.10/site-packages/datahub/utilities/urns/urn_iter.py", line 156, in lowercase_dataset_urns
    transform_urns(model, modify_urn)
  File "/home/python/venv/dnap-datahub-ingest-8zpDmlFD-py3.10/lib/python3.10/site-packages/datahub/utilities/urns/urn_iter.py", line 106, in transform_urns
    _modify_at_path(model, path, new_urn)
  File "/home/python/venv/dnap-datahub-ingest-8zpDmlFD-py3.10/lib/python3.10/site-packages/datahub/utilities/urns/urn_iter.py", line 129, in _modify_at_path
    assert isinstance(model, DictWrapper)
We're running this with :
Python 3.10.13
DATAHUB_VERSION=0.11.0.5

I can see that something about the model isn't ACTUALLY a dathub model (aka DictWrapper) but because there's asserts :( there I can't tell what it SHOULD be handing to _modify_at_path.

Is this something to be concerned about (e.g., not lowercasing might affect lineage or something else). I'm guessing here that the only effect is it isn't lowercasing something it should? OR what's being passed in here isn't relevant for this and the ERROR message should be perhaps a warning?

Happy to add more or contribute - i just can't tell at the moment what SHOULD be being passed there.

@hsheth2
Copy link
Collaborator

hsheth2 commented Nov 1, 2023

@mattmm1 I have a PR up that should fix this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug report
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants