Add docs as a real node config and support `node_color` for coloring the DAG #5397

b-per · 2022-06-21T07:18:53Z

related to #5343
related to #5333

This is a variant of #5343 where the docs config can now be defined in dbt_project.yml as well as individual yml files and the config() block.

The original docs config that accepts show:false has not been touched for now to avoid breaking changes for people using the original config.

This PR goes along the other PR dbt-labs/dbt-docs#281 on the dbt-docs repository.
Testing showed that the dbt-docsDAG crashes if invalid color names are provided hence why inputs for node_color are checked on the dbt-core side

I have not added tests so far as I would be keen to get feedback on the approach first but I will add tests if we decide to go forward with that code.

Copilot description

This pull request includes various changes to improve the functionality and documentation of the codebase. The most important changes include adding a new function for validating HTML colors, updating functions to handle the presence of a docs key, modifying the expected manifest to correctly set the node_color attribute, and adding new attributes and helper methods for handling documentation.

Main functionality changes:

core/dbt/contracts/graph/utils.py: Added a new function validate_color for validating HTML colors.
core/dbt/parser/base.py: Updated the update_parsed_node_config function to handle the presence of a docs key in the config_dict and import the Docs class. [1] [2]
tests/functional/artifacts/expected_manifest.py: Modified the expected_manifest.py file to correctly set the node_color attribute in the expected manifest. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17]
core/dbt/contracts/graph/model_config.py: Updated the same_contents function and added a new docs field to the NodeConfig class. [1] [2]

Documentation changes:

core/dbt/contracts/graph/unparsed.py: Added a new optional attribute node_color to the Docs dataclass.
.changes/unreleased/Features-20220803-104230.yaml: Added a new entry to make docs configurable in dbt_project.yml and add a node_color attribute.
tests/functional/configs/test_custom_node_colors_configs.py: Added a new test file to validate the behavior of custom node colors at different levels. (tests/functional/configs/test_custom_node_colors_configs.py)

github-actions · 2022-06-21T07:19:16Z

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

sungchun12 · 2022-06-21T12:38:38Z

core/dbt/contracts/graph/model_config.py

+    )
+
+    # we validate that node_color has a suitable value to prevent dbt-docs from crashing
+    def __post_init__(self):


@stu-k How do you feel about implementing color validation at the dbt-core layer vs. the dbt-docs layer?

dbt-labs/dbt-docs#279 (review)

If all CSS/HTML/JS is currently isolated in dbt-docs, I don't feel great about including CSS specific logic in dbt-core. This would introduce a case where if you needed to change the functionality of a thing, you'd have to modify a codebase where it is counterintuitive to do so.

cc: @jtcohen6

Hey Stu!

Makes sense to me. We can go ahead and include this defensive code in the dbt-docs codebase.

Part of our thinking was to make sure dbt tells the developer that they have the wrong colors when compiling their code. I know a lot of dbt users that don't run dbt docs generate until it hits a scheduled job. Therefore, the troubleshooting happens on the tailend vs. upfront. But sounds like we're okay with that tradeoff given maintainability of the codebase matters more.

Thanks both for sharing your perspectives! I see compelling points on both sides, and I could go either way.

Truth: There's really zero other CSS/HTML/JS in dbt-core. dbt-core does know a teensy bit about Markdown, just enough to make description fields populate correctly in dbt-docs, but it doesn't do any of the rendering itself.

Also, users should include docs generation + manual inspection of docs during development / CI, if they're messing with custom node colors, to makes sure that what they're doing doesn't break their docs site. (I bet we can also add some defensive logic to the docs site that ignores invalid color codes, rather than crashing.)

At the same time — by adding docs.node_color as a first-class config/property in dbt-core, similar to docs.show — we are setting an expectation that dbt-core knows about it and takes an interest in validating this config, more than it would for "any" fields like meta. That, along with making this configurable according to file paths (FQNs) and other rule sets, is one of the reasons I'd be supportive of incorporating this logic into dbt-core directly.

If it makes sense to add it to dbt-core the next question would be whether it should also be in dbt-docs. I modified dbt-labs/dbt-docs#281 a few days ago to add the validation on the dbt-docs side as well.

Having said my piece above, I'm happy to defer to @stu-k's preference, motivated by separation of concerns between dbt-core and dbt-docs

sungchun12 · 2022-06-21T12:49:19Z

core/dbt/contracts/graph/model_config.py

@@ -285,7 +286,7 @@ def same_contents(cls, unrendered: Dict[str, Any], other: Dict[str, Any]) -> boo
    # 'meta' moved here from node
    mergebehavior = {
        "append": ["pre-hook", "pre_hook", "post-hook", "post_hook", "tags"],
-        "update": ["quoting", "column_types", "meta"],
+        "update": ["quoting", "column_types", "meta", "docs"],
        "dict_key_append": ["grants"],


I pulled down this branch locally and it didn't work. Do you need to add to this dict_key_append similar to https://github.com/dbt-labs/dbt-core/pull/5343/files#diff-007a86f768b23e1516e6d253d71656b96d46672fc7432dbc81d1d9445c98ca05R289 ?

Hold on, I may have messed up my local dev env. One moment please.

Did you get an error or did it just not add values under model_id.config.docs?

I feel like "update" is what we want to do. If a model has docs defined because of the folder it is in, if we redefine it at the model level we just want the most specific value to be kept (rather than combining with the folder one).

I got it working at the folder level!

However, it overrides the schema.yml configs when it should be the other way around.

When I tested, the config at the individual YAML file was correctly overwriting the folder config. I think what is missing is that in the YAML file, it needs to be defined under config, like

models: - name: xxxx config: docs: node_color: "#e0115f"

Ahhh yes, this must be it. Thanks for writing this out explicitly. I'm aligned with your approach!

jtcohen6

Very cool! I weighed in on the thread about where validation should happen, and also gave me $0.02 about how we'd want to move forward with the docs config/property merge

jtcohen6 · 2022-06-28T10:28:14Z

core/dbt/contracts/graph/model_config.py

+    )
+
+    # we validate that node_color has a suitable value to prevent dbt-docs from crashing
+    def __post_init__(self):


Thanks both for sharing your perspectives! I see compelling points on both sides, and I could go either way.

Truth: There's really zero other CSS/HTML/JS in dbt-core. dbt-core does know a teensy bit about Markdown, just enough to make description fields populate correctly in dbt-docs, but it doesn't do any of the rendering itself.

Also, users should include docs generation + manual inspection of docs during development / CI, if they're messing with custom node colors, to makes sure that what they're doing doesn't break their docs site. (I bet we can also add some defensive logic to the docs site that ignores invalid color codes, rather than crashing.)

At the same time — by adding docs.node_color as a first-class config/property in dbt-core, similar to docs.show — we are setting an expectation that dbt-core knows about it and takes an interest in validating this config, more than it would for "any" fields like meta. That, along with making this configurable according to file paths (FQNs) and other rule sets, is one of the reasons I'd be supportive of incorporating this logic into dbt-core directly.

jtcohen6 · 2022-06-28T10:30:15Z

core/dbt/contracts/graph/model_config.py

+    docs: Dict[str, Any] = field(
+        default_factory=dict,
+        metadata=MergeBehavior.Update.meta(),
+    )


Indeed, what we really want here is to merge the values of node.docs and node.config.docs, just as we do today with node.meta and node.config.meta. If we can do that, the change will be both sensible and backwards compatible.

There are a few relevant spots for that (and I'm probably not catching all of them, so @dbt-labs/core-language should really advise):

dbt-core/core/dbt/parser/base.py

Lines 284 to 287 in d5608dc

# If we have meta in the config, copy to node level, for backwards

# compatibility with earlier node-only config.

if "meta" in config_dict and config_dict["meta"]:

parsed_node.meta = config_dict["meta"]

dbt-core/core/dbt/parser/schemas.py

Lines 807 to 821 in d5608dc

# We want to raise an error if 'meta' is in two places, and move 'meta'

# from toplevel to config if necessary

def normalize_meta_attribute(self, data, path):

if "meta" in data:

if "config" in data and "meta" in data["config"]:

raise ParsingException(

f"""

In {path}: found meta dictionary in 'config' dictionary and as top-level key.

Remove the top-level key and define it under 'config' dictionary only.

""".strip()

)

else:

if "config" not in data:

data["config"] = {}

data["config"]["meta"] = data.pop("meta")

We're moving toward a world where:

config can accept any values, set/inherit them via config hierarchy, and get them for use at runtime

any config values with important metadata implications should move up to top-level node properties (tags, meta, etc today; probably materialized, too)

Thanks @jtcohen6 , I will have a look.

Just adding an additional .02 - it seems that doing the work to support this config in dbt Core means we should also validate it's correct there, despite that introducing some new knowledge to the codebase around valid colors. Basically a +1 for @jtcohen6 's second point above.

From developer experience perspective, it seems very possible that someone could set this incorrectly, and then continue adding / running for a good long time without in the interim doing a dbt docs generate to refresh (and then catch the error). Totally agree that docs generation should be part of CI also FWIW.

Thanks for the pointers @jtcohen6

We actually also need to remove/modify this part handling the "legacy" docs (I spent way too much time figuring out why node.docs was not getting populated):

dbt-core/core/dbt/contracts/graph/parsed.py

Line 524 in d63ad4c

self.docs = patch.docs

Is the goal to make the docs dict attached to a dataclass with show/node_color or would we want it to be just a generic dict like meta?

I will update the PR soon with my progress.

Is the goal to make the docs dict attached to a dataclass with show/node_color or would we want it to be just a generic dict like meta?

I think the former. meta should remain a sorta special case in its acceptance of absolutely anything, since dbt makes zero promises as far as validating its contents.

Of course, when docs is serialized into manifest.json, it should just be a standard dictionary with up to 2 keys, show (always set) and node_color (optionally set).

I have a branch working locally when making docs a dict but making it a dataclass is more complex and raises a few errors.

Making node_color optional still requires setting a Default value, so we might need to have some specific code to handle the deletion of the attribute if node_color = None. The error I get without setting a default value is TypeError: non-default argument 'node_color' follows default argument

When adding the code to populate docs from config.docs with the following snippets, mahsumaro raises the error NameError: name 'dbt' is not defined which I can't pinpoint why. If I remove the new lines for "docs", docs is not moved from across from config (expected) but it also doesn't raise the error.

Code in core/dbt/parser/base.py:

# the original code for "meta" if "meta" in config_dict and config_dict["meta"]: parsed_node.meta = config_dict["meta"] # the new code for "docs" if "docs" in config_dict and config_dict["docs"]: parsed_node.docs = config_dict["docs"]

Debug error:

name 'dbt' is not defined 17:25:10.583468 [debug] [MainThread]: Traceback (most recent call last): File "/Users/bper/dev/dbt-core2/core/dbt/main.py", line 129, in main results, succeeded = handle_and_check(args) File "/Users/bper/dev/dbt-core2/core/dbt/main.py", line 191, in handle_and_check task, res = run_from_args(parsed) File "/Users/bper/dev/dbt-core2/core/dbt/main.py", line 238, in run_from_args results = task.run() File "/Users/bper/dev/dbt-core2/core/dbt/task/runnable.py", line 451, in run self._runtime_initialize() File "/Users/bper/dev/dbt-core2/core/dbt/task/runnable.py", line 159, in _runtime_initialize super()._runtime_initialize() File "/Users/bper/dev/dbt-core2/core/dbt/task/runnable.py", line 92, in _runtime_initialize self.load_manifest() File "/Users/bper/dev/dbt-core2/core/dbt/task/runnable.py", line 81, in load_manifest self.manifest = ManifestLoader.get_full_manifest(self.config) File "/Users/bper/dev/dbt-core2/core/dbt/parser/manifest.py", line 218, in get_full_manifest manifest = loader.load() File "/Users/bper/dev/dbt-core2/core/dbt/parser/manifest.py", line 403, in load self.write_manifest_for_partial_parse() File "/Users/bper/dev/dbt-core2/core/dbt/parser/manifest.py", line 549, in write_manifest_for_partial_parse manifest_msgpack = self.manifest.to_msgpack() File "/Users/bper/.venv/dbt-core2/lib/python3.9/site-packages/mashumaro/serializer/msgpack.py", line 40, in to_msgpack self.to_dict(**dict(DEFAULT_DICT_PARAMS, **dict_params)), File "<string>", line 9, in to_dict File "<string>", line 9, in <dictcomp> File "<string>", line 74, in __pack_union_Manifest_nodes__a52031cbc3504db8b4c0b1b1952cd791 NameError: name 'dbt' is not defined

sungchun12 · 2022-06-30T19:13:21Z

@b-per Love the error handling at compile time!

Can you dynamically update the log output to point to the node/folder config that needs to be fixed?

example: Invalid color name for docs.node_color: invalid. It is neither a valid HTML color name nor a valid HEX code. Update ['dim_cusomters.config.docs.node_color', 'models.core.config.docs.node_color'] to resolve this error!

jtcohen6 · 2022-07-18T16:00:27Z

After live conversation with @b-per @matt-winkler @sungchun12:

Benoit has this just about working, by treating docs: {} as a dict, just like meta: {}, instead of a dataclass. This works To Spec TM, for all intended user-facing functionality: configuring in dbt_project.yml, overriding for one model, copying the values out of the config() dict to the top-level node.docs.
I defer to the Language team in code review if we feel we want to preserve docs as an opinionated dataclass that accepts exactly two attributes (show + node_color). When attempting this with dataclasses, we ran into some serialization errors, which a Core engineer might be able to debug fairly quickly / easily. Benoit will try to push a branch for this too, just to see how close he got :)
We like the idea of validating docs.node_color BOTH in dbt-core (opinionated feature! compile-time errors!) AND in dbt-docs (avoid crashing the browser, just to be safe)
Benoit + Matt + Sung will work on writing tests for this:
- Different configuration options → assert that get_manifest has the correct contents
- Assert dbt raises a validation error at parse time if node_color is invalid
Sung will own updating Support node_color and docs.show under the config object dbt-docs#281 to coordinate with any changes we make here. (Specifically, only pulling from top-level node.docs, and never node.config.docs.)

Pending Benoit's latest code changes, I'm going to mark this one ready_for_review!

b-per · 2022-07-20T10:54:24Z

We still need to add tests but I have pushed the latest changes with node_color working as a dict and being pushed down from config.docs to docs.

b-per · 2022-07-26T17:43:17Z

FYI, I have pushed the branch test-docs-dataclass with my attempt at making the docs a dataclasss instead of a dictionary. That branch is currently failing with the error:

17:36:43  Running with dbt=1.3.0-a1
Traceback (most recent call last):
  File "/Users/bper/dev/dbt-core/core/dbt/main.py", line 129, in main
    results, succeeded = handle_and_check(args)
  File "/Users/bper/dev/dbt-core/core/dbt/main.py", line 192, in handle_and_check
    task, res = run_from_args(parsed)
  File "/Users/bper/dev/dbt-core/core/dbt/main.py", line 239, in run_from_args
    results = task.run()
  File "/Users/bper/dev/dbt-core/core/dbt/task/runnable.py", line 451, in run
    self._runtime_initialize()
  File "/Users/bper/dev/dbt-core/core/dbt/task/runnable.py", line 159, in _runtime_initialize
    super()._runtime_initialize()
  File "/Users/bper/dev/dbt-core/core/dbt/task/runnable.py", line 92, in _runtime_initialize
    self.load_manifest()
  File "/Users/bper/dev/dbt-core/core/dbt/task/runnable.py", line 81, in load_manifest
    self.manifest = ManifestLoader.get_full_manifest(self.config)
  File "/Users/bper/dev/dbt-core/core/dbt/parser/manifest.py", line 219, in get_full_manifest
    manifest = loader.load()
  File "/Users/bper/dev/dbt-core/core/dbt/parser/manifest.py", line 405, in load
    self.write_manifest_for_partial_parse()
  File "/Users/bper/dev/dbt-core/core/dbt/parser/manifest.py", line 551, in write_manifest_for_partial_parse
    manifest_msgpack = self.manifest.to_msgpack()
  File "/Users/bper/.venv/dbt-core/lib/python3.9/site-packages/mashumaro/serializer/msgpack.py", line 40, in to_msgpack
    self.to_dict(**dict(DEFAULT_DICT_PARAMS, **dict_params)),
  File "<string>", line 9, in to_dict
  File "<string>", line 9, in <dictcomp>
  File "<string>", line 74, in __pack_union_Manifest_nodes__ab08a427686c4289b27e5c8dd37d1694
NameError: name 'dbt' is not defined
17:36:44  Encountered an error:
name 'dbt' is not defined

emmyoop · 2022-07-27T14:36:27Z

@b-per I bumped our manifest version to v7 as it's needed for the config change. This will get some of the artifact tests passing. I didn't see this as in scope for this work specifically. You can't merge in the PR since many tests are still failing but you can merge my branch directly into yours.

Related PR: #5561

b-per · 2022-07-28T10:02:39Z

Thanks to you and Gerda for the pointer in this commit!
I had to tweak the logic a bit to populate the object correctly but it unblocked me.

I checked with a test project and I am getting the same output for docs in the manifest with dict and with dataclass. The only difference is that if there is no node_color configured, the dataclass version still creates the field but it is assigned to none.

In the last commits, I have:

merged your new manifest
moved Docs to a dataclass

I will work with the team in getting the tests passing and new ones created

matt-winkler · 2022-08-01T15:47:25Z

@b-per added tests for overriding show: at the various levels per our conversation this AM.

matt-winkler · 2022-08-01T18:50:43Z

Hi @emmyoop I wanted to clarify your comment on artifact tests from above. Are you saying that this branch is dependent on those being resolved before it's merge-able, or can we proceed independently with testing this feature?

emmyoop · 2022-08-01T19:01:11Z

@matt-winkler all tests will need to be passing before this is mergable. These changes were causing artifacts tests to fail because of the new node config. That should have been resolved by the code @b-per merged in from my branch.

I see there are some conflicts with the code that was merged in last week for the 1.3.0 beta. I'll see if I can resolve these later today.

matt-winkler · 2022-08-01T19:04:52Z

Good deal thanks @emmyoop, so in that case it seems like next steps are:

Resolve merge conflicts - emmyoop handling
Additional functional tests on this new build which @b-per is taking down beyond what @sungchun12 and I have already done.
Other testing we identify as-needed

Anything I'm missing?

b-per · 2022-08-02T11:39:27Z

I just pushed some changes to fix the unit tests. In a similar way that meta exists in the root of the model config and under the config key, docs also shows under the config key now.

emmyoop

This looks good!

Thank you for all your hard work and coordination on this one @b-per, @sungchun12 and @matt-winkler! ❤️

sungchun12 · 2022-08-04T17:49:58Z

I'm so proud of everyone involved in this PR. This was a meaningful, hard problem to solve for. I honestly think we'll be the first mainstream data tool to add custom colors to data lineage(anecdotally). When people talk about the bronze, silver, gold layers, the conversation will be literal now :)

@b-per @matt-winkler, this was so fun and yet another role model for open source, async work together!

@emmyoop @stu-k @jtcohen6, thank you for the excellent input on mechanical design. You role modeled what great open source stewardship looks and feels like.

Everyone, be ready to be celebrated publicly because I'm going to demo this live in front of a face to face audience on 08/18/2022 at this event: https://www.getdbt.com/resources/drinks-data-dbt-ca/

…the DAG (#5397) * add Optional node_color config in Docs dataclass * Remove node_color from the original docs config * Add docs config and input validation * Handle when docs is both under docs and config.docs * Add node_color to Docs * Make docs a Dict to avoid parsing errors * Make docs a dataclass instead of a Dict * Fix error when using docs as dataclass * Simplify generator for the default value * skeleton for test fixtures * bump manifest to v7 * + config hierarchy tests * add show override tests * update manifest * Remove node_color from the original docs config * Add node_color to Docs * Make docs a Dict to avoid parsing errors * Make docs a dataclass instead of a Dict * Simplify generator for the default value * + config hierarchy tests * add show override tests * Fix unit tests * Add tests in case of incorrect input for node_color * Rename tests and Fix typos * Fix functional tests * Fix issues with remote branch * Add changie entry * modify tests to meet standards (#5608) Co-authored-by: Matt Winkler <matt.winkler@fishtownanalytics.com> Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com>

…the DAG (dbt-labs#5397) * add Optional node_color config in Docs dataclass * Remove node_color from the original docs config * Add docs config and input validation * Handle when docs is both under docs and config.docs * Add node_color to Docs * Make docs a Dict to avoid parsing errors * Make docs a dataclass instead of a Dict * Fix error when using docs as dataclass * Simplify generator for the default value * skeleton for test fixtures * bump manifest to v7 * + config hierarchy tests * add show override tests * update manifest * Remove node_color from the original docs config * Add node_color to Docs * Make docs a Dict to avoid parsing errors * Make docs a dataclass instead of a Dict * Simplify generator for the default value * + config hierarchy tests * add show override tests * Fix unit tests * Add tests in case of incorrect input for node_color * Rename tests and Fix typos * Fix functional tests * Fix issues with remote branch * Add changie entry * modify tests to meet standards (dbt-labs#5608) Co-authored-by: Matt Winkler <matt.winkler@fishtownanalytics.com> Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com>

b-per requested a review from a team as a code owner June 21, 2022 07:18

b-per requested a review from emmyoop June 21, 2022 07:18

cla-bot bot added the cla:yes label Jun 21, 2022

This was referenced Jun 21, 2022

[Draft] add Optional node_color config in Docs dataclass #5343

Closed

Support node_color and docs.show under the config object dbt-labs/dbt-docs#281

Merged

sungchun12 reviewed Jun 21, 2022

View reviewed changes

jtcohen6 reviewed Jun 28, 2022

View reviewed changes

jtcohen6 added Team:Language ready_for_review Externally contributed PR has functional approval, ready for code review from Core engineering labels Jul 18, 2022

b-per requested a review from a team as a code owner July 22, 2022 20:51

b-per requested a review from iknox-fa July 22, 2022 20:51

b-per requested a review from a team August 1, 2022 21:19

b-per requested review from a team and leahwicz as code owners August 1, 2022 21:19

jtcohen6 mentioned this pull request Aug 2, 2022

[CT-967] [Feature] Hide docs using dbt_project.yml #5591

Closed

3 tasks

matt-winkler and others added 2 commits August 2, 2022 10:56

add Optional node_color config in Docs dataclass

9903356

Remove node_color from the original docs config

1762f56

matt-winkler and others added 13 commits August 2, 2022 10:56

+ config hierarchy tests

619c6b0

add show override tests

724b20d

update manifest

02fd429

Remove node_color from the original docs config

9657fd8

Add node_color to Docs

75d47a4

Make docs a Dict to avoid parsing errors

79c3242

Make docs a dataclass instead of a Dict

a9d16b1

Simplify generator for the default value

84fb626

+ config hierarchy tests

fd46565

add show override tests

9c3ef8b

Fix unit tests

b15fc03

Add tests in case of incorrect input for node_color

3f3d14a

Rename tests and Fix typos

b3ce967

emmyoop force-pushed the feature/custom-node-colors-dbt_project branch from eb3dd02 to b3ce967 Compare August 2, 2022 16:14

b-per added 3 commits August 2, 2022 21:26

Fix functional tests

f19285c

Fix issues with remote branch

5b702cf

Add changie entry

dad425e

b-per changed the title ~~[Draft] Add docs as a real node config and support node_color for coloring the DAG~~ Add docs as a real node config and support node_color for coloring the DAG Aug 3, 2022

modify tests to meet standards (#5608)

1aab0ef

emmyoop approved these changes Aug 3, 2022

View reviewed changes

emmyoop merged commit 32415e3 into main Aug 3, 2022

emmyoop deleted the feature/custom-node-colors-dbt_project branch August 3, 2022 19:31

grahamcarman mentioned this pull request Aug 3, 2022

[CT-727] Hide Package Names from Documentation #5333

Closed

1 task

emmyoop mentioned this pull request Aug 4, 2022

[CT-991] [Feature] Bring in node_color support from dbt-docs #5613

Closed

jtcohen6 mentioned this pull request Aug 4, 2022

Custom node coloring in docs site dbt-labs/docs.getdbt.com#1835

Closed

1 task

jtcohen6 mentioned this pull request Aug 17, 2022

[CT-249] Ability to hide all models in a directory from project config #4739

Closed

dbeatty10 mentioned this pull request Nov 16, 2023

Update docs for node_color dbt-labs/docs.getdbt.com#4492

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add docs as a real node config and support `node_color` for coloring the DAG #5397

Add docs as a real node config and support `node_color` for coloring the DAG #5397

b-per commented Jun 21, 2022 •

edited by runleonarun

Loading

github-actions bot commented Jun 21, 2022

sungchun12 Jun 21, 2022 •

edited

Loading

stu-k Jun 21, 2022 •

edited

Loading

sungchun12 Jun 22, 2022

jtcohen6 Jun 28, 2022

b-per Jun 29, 2022

jtcohen6 Jun 29, 2022

sungchun12 Jun 21, 2022

sungchun12 Jun 21, 2022

b-per Jun 21, 2022

sungchun12 Jun 21, 2022

b-per Jun 23, 2022

sungchun12 Jun 23, 2022

jtcohen6 left a comment

jtcohen6 Jun 28, 2022

jtcohen6 Jun 28, 2022

b-per Jun 29, 2022

matt-winkler Jun 29, 2022

b-per Jun 30, 2022

jtcohen6 Jun 30, 2022

b-per Jul 1, 2022

sungchun12 commented Jun 30, 2022

jtcohen6 commented Jul 18, 2022

b-per commented Jul 20, 2022

b-per commented Jul 26, 2022

emmyoop commented Jul 27, 2022

b-per commented Jul 28, 2022

matt-winkler commented Aug 1, 2022

matt-winkler commented Aug 1, 2022

emmyoop commented Aug 1, 2022

matt-winkler commented Aug 1, 2022 •

edited

Loading

b-per commented Aug 2, 2022

emmyoop left a comment

sungchun12 commented Aug 4, 2022

	# If we have meta in the config, copy to node level, for backwards
	# compatibility with earlier node-only config.
	if "meta" in config_dict and config_dict["meta"]:
	parsed_node.meta = config_dict["meta"]

	# We want to raise an error if 'meta' is in two places, and move 'meta'
	# from toplevel to config if necessary
	def normalize_meta_attribute(self, data, path):
	if "meta" in data:
	if "config" in data and "meta" in data["config"]:
	raise ParsingException(
	f"""
	In {path}: found meta dictionary in 'config' dictionary and as top-level key.
	Remove the top-level key and define it under 'config' dictionary only.
	""".strip()
	)
	else:
	if "config" not in data:
	data["config"] = {}
	data["config"]["meta"] = data.pop("meta")

Add docs as a real node config and support node_color for coloring the DAG #5397

Add docs as a real node config and support node_color for coloring the DAG #5397

Conversation

b-per commented Jun 21, 2022 • edited by runleonarun Loading

Copilot description

github-actions bot commented Jun 21, 2022

sungchun12 Jun 21, 2022 • edited Loading

Choose a reason for hiding this comment

stu-k Jun 21, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jtcohen6 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sungchun12 commented Jun 30, 2022

jtcohen6 commented Jul 18, 2022

b-per commented Jul 20, 2022

b-per commented Jul 26, 2022

emmyoop commented Jul 27, 2022

b-per commented Jul 28, 2022

matt-winkler commented Aug 1, 2022

matt-winkler commented Aug 1, 2022

emmyoop commented Aug 1, 2022

matt-winkler commented Aug 1, 2022 • edited Loading

b-per commented Aug 2, 2022

emmyoop left a comment

Choose a reason for hiding this comment

sungchun12 commented Aug 4, 2022

Add docs as a real node config and support `node_color` for coloring the DAG #5397

Add docs as a real node config and support `node_color` for coloring the DAG #5397

b-per commented Jun 21, 2022 •

edited by runleonarun

Loading

sungchun12 Jun 21, 2022 •

edited

Loading

stu-k Jun 21, 2022 •

edited

Loading

matt-winkler commented Aug 1, 2022 •

edited

Loading