Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect documentation (code sample) for jobs with static partitions #19378

Open
ericct opened this issue Jan 24, 2024 · 1 comment
Open

Incorrect documentation (code sample) for jobs with static partitions #19378

ericct opened this issue Jan 24, 2024 · 1 comment

Comments

@ericct
Copy link
Contributor

ericct commented Jan 24, 2024

What's the issue or suggestion?

Code sample in the section https://docs.dagster.io/concepts/partitions-schedules-sensors/partitioning-ops#defining-jobs-with-static-partitions is not working. Configuration is not propagated to the op, leading to this error (I've just copy-pasted the code and run it)

dagster._core.errors.DagsterInvalidConfigError: Error in config for job continent_job
    Error 1: Missing required config entry "ops" at the root. Sample config for missing entry: {'ops': {'continent_op': {'config': {'continent_name': '...'}}}}

I'm using latest version as of today, 1.6.1.

Additional information

I will try to

  1. bissect to see if it's a "regression" (or documentation lacking update)
  2. understand what is wrong with the code and propose doc update

Note that I do not know Dagster code though.

(not relevant for the issue - but why it's important to me!) I'm experimenting with Dagster and I have to prove feasibility for static partitioned jobs and dynamic outs before adoption. The full pipeline is 1) a partitioned asset (from pyarrow dataset based on hive partitioning) and 2 job/sequence of ops for chunking an parallel processing (but with this partition context)

Thanks!

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

@ericct ericct added the area: docs Related to documentation in general label Jan 24, 2024
@ericct
Copy link
Contributor Author

ericct commented Jan 24, 2024

I've trie with multiple version, down to the latest 1.4, with the same result.

Note that it's not related to a missing configuration for execution. The code fails at the job definition (execution of @job decorator and subsequent validation)

@yuhan yuhan removed the area: docs Related to documentation in general label Apr 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants