Skip to content
This repository was archived by the owner on May 17, 2024. It is now read-only.

add json format for dbt diff values #612

Merged
merged 13 commits into from
Jun 21, 2023
Merged

add json format for dbt diff values #612

merged 13 commits into from
Jun 21, 2023

Conversation

vvkh
Copy link
Contributor

@vvkh vvkh commented Jun 14, 2023

Add --json-output parameter support for dbt diffs. See type hints in data_diff/format.py and tests/test_format.py for documentation of the format.

Main things to discuss @nolar @dagadbm

  • For what is already being shown, are we happy with the current format?
  • Is there anything important you can think of that is missing in the current format?

@vvkh vvkh requested review from nolar and dlawin June 14, 2023 15:26
@dlawin
Copy link
Contributor

dlawin commented Jun 14, 2023

I'm getting an exception

  File "/Users/dan/repos/data-diff/data_diff/info_tree.py", line 21, in SegmentInfo
    def set_diff(self, diff: list[tuple[Any, ...]], schema: tuple[tuple[str, type]] | None = None):
TypeError: unsupported operand type(s) for |: 'types.GenericAlias' and 'NoneType'

@vvkh
Copy link
Contributor Author

vvkh commented Jun 15, 2023

@dlawin forgot that those type hints were not supported by the older versions of Python. I just pushed a fix.

@vvkh vvkh force-pushed the json-output branch 2 times, most recently from 600680d to da572c1 Compare June 15, 2023 12:32
@vvkh vvkh requested a review from dlawin June 15, 2023 12:32
@dlawin
Copy link
Contributor

dlawin commented Jun 16, 2023

I'm seeing this when I run against it the test project:

  File "/Users/dan/repos/data-diff/data_diff/dbt.py", line 130, in dbt_diff
    _local_diff(diff_vars, json_output)
  File "/Users/dan/repos/data-diff/data_diff/dbt.py", line 286, in _local_diff
    jsonify(
  File "/Users/dan/repos/data-diff/data_diff/format.py", line 24, in jsonify
    schema = [field for field, _ in diff_info.diff_schema]
TypeError: 'NoneType' object is not iterable

poetry run python -m data_diff --dbt --dbt-project-dir tests/dbt_artifacts --dbt-profiles-dir tests/dbt_artifacts --json

@vvkh
Copy link
Contributor Author

vvkh commented Jun 21, 2023

@dlawin I've added a fix for the exception you mentioned, thanks for the hint how to solve it 🙇

@dlawin dlawin merged commit 533d88b into master Jun 21, 2023
@dlawin
Copy link
Contributor

dlawin commented Jun 23, 2023

Resolves #607

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants