Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bug on select node add exclude selector subset ids logic #463

Merged
merged 17 commits into from
Aug 16, 2023

Conversation

jensenity
Copy link
Contributor

@jensenity jensenity commented Aug 14, 2023

Description

  1. Add Skip case if all configs are None
  2. Exclude subset id to not be overwrite as it iterates
  3. Add more test cases

Related Issue(s)

It closes: #464

Breaking Change?

Checklist

  • I have made corresponding changes to the documentation (if required)
  • I have added tests that prove my fix is effective or that my feature works

@jensenity jensenity requested a review from a team as a code owner August 14, 2023 13:42
@jensenity jensenity requested a review from a team August 14, 2023 13:42
@netlify
Copy link

netlify bot commented Aug 14, 2023

👷 Deploy Preview for amazing-pothos-a3bca0 processing.

Name Link
🔨 Latest commit a77e3b9
🔍 Latest deploy log https://app.netlify.com/sites/amazing-pothos-a3bca0/deploys/64dd2b3587a3ec00082c04c4

@jensenity jensenity changed the title add exclude test chore: fix bug on select node add exclude selector subset ids logic Aug 14, 2023
@codecov
Copy link

codecov bot commented Aug 14, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.14% 🎉

Comparison is base (b9fdc92) 91.52% compared to head (a77e3b9) 91.67%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #463      +/-   ##
==========================================
+ Coverage   91.52%   91.67%   +0.14%     
==========================================
  Files          50       50              
  Lines        1770     1777       +7     
==========================================
+ Hits         1620     1629       +9     
+ Misses        150      148       -2     
Files Changed Coverage Δ
cosmos/dbt/selector.py 98.93% <100.00%> (+2.38%) ⬆️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jensenity jensenity changed the title chore: fix bug on select node add exclude selector subset ids logic Fix bug on select node add exclude selector subset ids logic Aug 14, 2023
@jensenity jensenity temporarily deployed to external August 14, 2023 15:29 — with GitHub Actions Inactive
Copy link
Collaborator

@tatiana tatiana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much, @jensenity! This looks great.
I left some minor comments inline. I would love your thoughts on them. Also, if you could rebase, we can aim to merge this change today.

cosmos/dbt/selector.py Outdated Show resolved Hide resolved
cosmos/dbt/selector.py Show resolved Hide resolved
@jensenity jensenity temporarily deployed to internal August 15, 2023 13:01 — with GitHub Actions Inactive
@tatiana tatiana temporarily deployed to external August 15, 2023 20:15 — with GitHub Actions Inactive
@tatiana tatiana self-requested a review August 15, 2023 20:27
cosmos/dbt/selector.py Outdated Show resolved Hide resolved
cosmos/dbt/selector.py Outdated Show resolved Hide resolved
@tatiana tatiana temporarily deployed to external August 16, 2023 09:58 — with GitHub Actions Inactive
cosmos/dbt/selector.py Outdated Show resolved Hide resolved
jensenity and others added 2 commits August 16, 2023 19:09
@pre-commit-ci pre-commit-ci bot temporarily deployed to external August 16, 2023 10:10 Inactive
@pre-commit-ci pre-commit-ci bot temporarily deployed to external August 16, 2023 11:17 Inactive
@jensenity jensenity temporarily deployed to external August 16, 2023 11:55 — with GitHub Actions Inactive
cosmos/dbt/selector.py Outdated Show resolved Hide resolved
@jensenity jensenity temporarily deployed to internal August 16, 2023 14:12 — with GitHub Actions Inactive
@tatiana tatiana temporarily deployed to external August 16, 2023 20:02 — with GitHub Actions Inactive
Copy link
Collaborator

@tatiana tatiana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the contribution and for addressing all the feedback, @jensenity !

@tatiana tatiana merged commit c676429 into astronomer:main Aug 16, 2023
43 checks passed
tatiana added a commit that referenced this pull request Aug 16, 2023
Feature (pending documentation!)
* Support dbt global flags (via dbt_cmd_global_flags in `operator_args` by @tatiana in #469

Enhancements
* Hide sensitive field when using BigQuery keyfile_dict profile mapping by @jbandoro in #471

Bug fixes
* Fix bug on select node add exclude selector subset ids logic by @jensenity in #463
* Refactor dbt ls to run from a temporary directory, to avoid Read-only file system errors during DAG parsing, by @tatiana in #414

Others
* Docs: Fix RenderConfig load argument by @jbandoro in #466
* Enable CI integration tests from external forks by @tatiana in #458
* Improve CI tests runtime by @tatiana in #457
* Change CI to run coverage after tests pass by @tatiana in #461
* Fix forks code revision in code coverage by @tatiana in #472
* [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #467"
i
@tatiana tatiana mentioned this pull request Aug 16, 2023
@tatiana tatiana added this to the 1.1.0 milestone Aug 16, 2023
@tatiana tatiana mentioned this pull request Sep 6, 2023
tatiana added a commit that referenced this pull request Sep 6, 2023
**Features**

* Support dbt global flags (via dbt_cmd_global_flags in operator_args)
by @tatiana in #469
* Support parsing DAGs when there are no connections by @jlaneve in #489

**Enhancements**

* Hide sensitive field when using BigQuery keyfile_dict profile mapping
by @jbandoro in #471
* Consistent Airflow Dataset URIs, inlets and outlets with `Openlineage
package <https://pypi.org/project/openlineage-integration-common/>`_ by
@tatiana in #485. `Read more
<https://astronomer.github.io/astronomer-cosmos/configuration/lineage.html>`_.
* Refactor ``LoadMethod.DBT_LS`` to run from a temporary directory with
symbolic links by @tatiana in #488
* Run ``dbt deps`` when using ``LoadMethod.DBT_LS`` by @DanMawdsleyBA in
#481
* Update Cosmos log color to purple by @harels in #494
* Change operators to log ``dbt`` commands output as opposed to
recording to XCom by @tatiana in #513

**Bug fixes**

* Fix bug on select node add exclude selector subset ids logic by
@jensenity in #463
* Refactor dbt ls to run from a temporary directory, to avoid Read-only
file system errors during DAG parsing, by @tatiana in #414
* Fix profile_config arg in DbtKubernetesBaseOperator by @david-mag in
#505
* Fix SnowflakePrivateKeyPemProfileMapping private_key reference by
@nacpacheco in #501
* Fix incorrect temporary directory creation in VirtualenvOperator init
by @tatiana in #500
* Fix log propagation issue by @tatiana in #498
* Fix PostgresUserPasswordProfileMapping to retrieve port from
connection by @jlneve in #511

**Others**

* Docs: Fix RenderConfig load argument by @jbandoro in #466
* Enable CI integration tests from external forks by @tatiana in #458
* Improve CI tests runtime by @tatiana in #457
* Change CI to run coverage after tests pass by @tatiana in #461
* Fix forks code revision in code coverage by @tatiana in #472
* [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #467
* Drop support to Python 3.7 in the CI test matrix by @harels in #490
* Add Airflow 2.7 to the CI test matrix by @tatiana in #487
* Add MyPy type checks to CI since we exceeded pre-commit disk quota
usage by @tatiana in #510
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bug on Select Node Logic in v1.0.5
2 participants