Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect / persistent new data version flag #19823

Open
BeigeOne opened this issue Feb 15, 2024 · 2 comments
Open

Incorrect / persistent new data version flag #19823

BeigeOne opened this issue Feb 15, 2024 · 2 comments
Assignees
Labels
type: bug Something isn't working

Comments

@BeigeOne
Copy link
Contributor

BeigeOne commented Feb 15, 2024

Dagster version

1.6.4

What's the issue?

I made some changes to my project which involved changing an asset (call it X) to an observable source asset and removing some other assets. The issue I have is that the asset downstream of X is now persistently reporting in the UI that "Changes since last materialization: X has a new data version"

What did you expect to happen?

I expected the process to be:
observe asset X -> materialize ds_asset -> expected the 'X has a new data version' flag on ds_asset to disappear
and for this to be true independent of former incarnations of X

How to reproduce?

# first set up using X as asset
@asset()
def X():
    pass

@asset(deps=["X"])
def ds_asset():
    pass

UI: deployment-reload
UI: materialize X
UI: materialize ds_asset
UI: materialize X
->ds_asset now flagged with 'Changes since last materialization: X has a new materialization'
UI: materialize ds_asset
->ds_asset flag disappears

# first set up using X as asset
#@asset()
#def X():
#    pass

# subsequently change X to observable_source_asset
@observable_source_asset() 
def X():
     return DataVersion('dummy-version-id')

@asset(deps=["X"])
def ds_asset():
    pass

UI: deployment-reload
UI: observe X
->ds_asset now flagged with 'Changes since last materialization: X has a new materialization'
UI: materialize ds_asset
->ds_asset flag remains

Deployment type

Local

Deployment details

windows-local

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

@BeigeOne BeigeOne added the type: bug Something isn't working label Feb 15, 2024
@BeigeOne
Copy link
Contributor Author

Is there a workaround? Some way I can easily clear the history of X?

@smackesey smackesey self-assigned this Feb 17, 2024
@BeigeOne
Copy link
Contributor Author

I have been using a workaround:

  • use UI asset catalog to 'wipe materializations'
  • remove X from code
  • redeploy code
  • materialise first downstream asset
  • reinstate code
  • observe X
  • materialise first downstream asset

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants