Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add jinja templating; remove MarkdownTransformer (put logic in WhaleLoader); small QOL improvements #96

Merged
merged 14 commits into from
Nov 21, 2020

Conversation

rsyi
Copy link
Owner

@rsyi rsyi commented Nov 21, 2020

This is a bit of a heavy PR, with the following changes:

  • Jinja templating support, which tacks on any templates found in the ~/.whale/templates/<connection-name>.sql file by default.
  • Fix: bigquery database was not correctly rendering in the markdown.
  • Refactored the logic to create markdown so it all lives in WhaleLoader. Eventually, this probably should be refactored again so it cleanly lives in methods on the models, but it's fine for now.

@codecov-io
Copy link

codecov-io commented Nov 21, 2020

Codecov Report

Merging #96 (5a0f26e) into master (42fc1a1) will decrease coverage by 0.03%.
The diff coverage is 78.43%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #96      +/-   ##
==========================================
- Coverage   71.86%   71.83%   -0.04%     
==========================================
  Files          31       31              
  Lines        1628     1658      +30     
==========================================
+ Hits         1170     1191      +21     
- Misses        458      467       +9     
Impacted Files Coverage Δ
...nes/whale/extractor/bigquery_metadata_extractor.py 94.66% <ø> (ø)
...es/whale/extractor/bigquery_watermark_extractor.py 30.15% <ø> (ø)
pipelines/whale/utils/extractor_wrappers.py 76.66% <ø> (ø)
pipelines/whale/loader/whale_loader.py 59.61% <66.00%> (+1.49%) ⬆️
pipelines/whale/utils/sql.py 87.50% <87.50%> (ø)
...pelines/whale/extractor/base_bigquery_extractor.py 88.11% <100.00%> (+0.24%) ⬆️
pipelines/whale/extractor/metric_runner.py 82.05% <100.00%> (-0.62%) ⬇️
pipelines/whale/utils/markdown_delimiters.py 100.00% <100.00%> (ø)
pipelines/whale/utils/paths.py 100.00% <100.00%> (ø)
pipelines/whale/utils/task_wrappers.py 61.29% <100.00%> (+1.29%) ⬆️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 42fc1a1...5a0f26e. Read the comment docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants