Skip to content

Fix show_deleted query param for /materializations, rename it show_inactive#1386

Merged
samredai merged 8 commits intoDataJunction:mainfrom
samredai:materializationconfigs
May 22, 2025
Merged

Fix show_deleted query param for /materializations, rename it show_inactive#1386
samredai merged 8 commits intoDataJunction:mainfrom
samredai:materializationconfigs

Conversation

@samredai
Copy link
Copy Markdown
Contributor

Summary

When calling /nodes/{node_name}/materializations?show_deleted=true, it wasn't returning all of the materializations for a given node because it only looks at the current version (via node.current). This fixes that by pulling all materializations for all node revisions, only when that query param is enabled, otherwise the behavior remains the same.

I also renamed it to include_all since previous versions of the node could have materializations that aren't deleted, but still can only be accessed via this flag, so viewing deleted materializations is just a subset of why this would be used. I don't think this query param is used anywhere so changing this shouldn't break anything, but let me know if I missed somewhere it is being used.

Test Plan

Added test_include_all_materialization_configs.

  • PR has an associated issue: #
  • make check passes
  • make test shows 100% unit test coverage

Deployment Plan

@netlify
Copy link
Copy Markdown

netlify bot commented May 21, 2025

Deploy Preview for thriving-cassata-78ae72 canceled.

Name Link
🔨 Latest commit cc1e55d
🔍 Latest deploy log https://app.netlify.com/projects/thriving-cassata-78ae72/deploys/682e867e4c85850008a2ba52

)
else: # Get just the active materializations on the current node revision
for materialization in node.current.materializations: # type: ignore
if not materialization.deactivated_at: # pragma: no cover
Copy link
Copy Markdown
Member

@agorajek agorajek May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this code now would not give you the option of seeing just the current materialization regardless of their status. I wonder if it would be cleaner to have two params on this whole method:

  • show_deleted as before (although I think show_inactive sounds better)
  • include_all_revisions as a param to scan previos node states.

I would lean to longer params/names that are obvious, rather than shorter ones that look nice.

Copy link
Copy Markdown
Contributor Author

@samredai samredai May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see what you're saying. So show_inactive would basically ignore the deactivated_at value and include_all_revisions would iterate over all of the node revisions to pull the materialization. Let me update this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated 👍

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, i think this now supports all the query options. Thanks!

Copy link
Copy Markdown
Member

@agorajek agorajek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some unit tests to fix...

if (
not materialization.deactivated_at or show_inactive
): # pragma: no cover
materializations.append(
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this part be pulled into a separate function that operates on node_revision, whether it's current or an older revision?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, done!

for materialization in node_revision.materializations:
if (
not materialization.deactivated_at or show_inactive
): # pragma: no cover
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is coverage for this line right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, I'm not sure why I put this here. 🤔

@samredai samredai merged commit ef7546a into DataJunction:main May 22, 2025
16 checks passed
@samredai samredai changed the title Fix show_deleted query param for /materializations, rename it include_all Fix show_deleted query param for /materializations, rename it show_inactive May 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants