3482 Add ConfigComponent for config parsing

[Nic-Ma]

Task step 2 of #3482 .

Description

This PR implemented the ConfigComponent feature for config parsing, it's for task 2 of #3482 .
The whole proposal is in draft PR: #3593.

Status

Ready

Types of changes

• Non-breaking change (fix or new feature that would not break existing functionality).
• Breaking change (fix or new feature that would cause existing functionality to change).
• New tests added to cover the changes.
• Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
• Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
• In-line docstrings updated.
• Documentation updated, tested make html command in the docs/ folder.

[Nic-Ma]

/black

[Nic-Ma]

/build

[Nic-Ma]

I tried to cover all the possible config cases in the unit tests.

Thanks.

[wyli]

(Still under discussion)

[Nic-Ma]

/build

[Nic-Ma]

/build

[wyli]

I'm still looking into the existing implementations outside of monai...

for this pr,

• at the component config level looks like it is decoupled from the format-specific elements of json and yaml. could we have a yaml demo? looks like multiple research repositories are using yaml for workflow configs.
• the concept of unique id is unclear to me, why and where to ensure the uniqueness? also class_table[name] = modname doesn't allow the same class name under different packages, this could be an issue when importing from different packages
• the class scanner could be mostly replaced by pydoc.locate? see https://github.com/facebookresearch/hydra/blob/5f605a863dd4aed8f432884fcd1e1b388c8fb559/hydra/_internal/utils.py#L554-L558
  https://github.com/python/cpython/blob/main/Lib/pydoc.py#L1707
• this repo is achieving a similar goal https://github.com/catalyst-team/hydra-slayer
• the hydra implementation for class instantiation also looks powerful https://github.com/facebookresearch/hydra/blob/5f605a863dd4aed8f432884fcd1e1b388c8fb559/hydra/_internal/instantiate/_instantiate2.py#L121 (test cases: https://github.com/facebookresearch/hydra/blob/5f605a863dd4aed8f432884fcd1e1b388c8fb559/tests/instantiate/test_instantiate.py#L64), do we really want to implement from scratch here...

cc @ericspod @rijobro

[Nic-Ma]

Thanks for @wyli 's review and suggestions.

Hi @ericspod @rijobro ,

Could you please help also take a look and share some opinions?
I will come back to work on this ticket tomorrow.

Thanks in advance.

[ericspod]

Hydra does do some of the things we want to include here, do we want to add it as a dependency? I'm not sure it could be an optional dependency since the model archiving functionality would need it to be there and it's a pretty important that it's present in MONAI.

[wyli]

this one also looks interesting https://github.com/jmespath/jmespath.py

[ericspod]

this one also looks interesting https://github.com/jmespath/jmespath.py

Something structured like this looks like a good idea, I do like the "foo.baz" syntax. It's rather more complex that we need for this application perhaps, again though the concern is adding another dependency.

I have a bit of code in a tool for generating random data that instantiates objects from a YAML schema. This function to creates objects from the dictionary derived from the schema recursively so that new objects can be used as constructor arguments when instantiating others. The find_type_def is pretty well the same as get_class and much of the other code here is doing a very similar thing.

I'm good with this going forward if we want to keep the momentum going on the model archive.

[Nic-Ma]

Hi @ericspod @wyli ,

Thanks very much for your review and comments, suggestions.
I was busy with NVIDIA Clara issues when came back from the Chinese New Year holiday this week, now I am checking other repos' implementations of config parsing and trying to resolve your comments.

Thanks.

[Nic-Ma]

/black

[Nic-Ma]

/build

[Nic-Ma]

/build

[Nic-Ma]

/build

[Nic-Ma]

/build

now simplified to the essential components

[Nic-Ma]

/black

[Nic-Ma]

/build

[Nic-Ma]

/build

[Nic-Ma]

/build

[pxLi]

/build

[pxLi]

/build

[Nic-Ma]

/black

[Nic-Ma]

/build

[Nic-Ma]

/build

[Nic-Ma]

I have merged this PR into the whole draft PR: #3593 and verified the overall features.
Will merge this PR and create following PRs for other parts.

Thanks.