Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added suggestion for mistyped/wrong parameters #300

Closed
wants to merge 21 commits into from

Conversation

Hoxbro
Copy link
Member

@Hoxbro Hoxbro commented Aug 17, 2022

It can be hard to know if something is transform or transforms. I have tried to take account of this by using difflib to come up with suggestions if parameters are mistyped/wrong.

I have chosen to raise an error when an input parameter does not match the class parameters. My reasoning for this is that the error usually will arrive later, making it harder to debug.

I have used the following example to see if I can provoke an error by adding or removing an s.

from lumen.pipeline import Pipeline

data_url = 'https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-07-28/penguins.csv'

pipeline = Pipeline.from_spec({
    'source': {
        'type': 'file',
        'table': {
            'penguins': data_url
        }
    },
    'filters': [
        {'type': 'widget', 'field': 'species'},
        {'type': 'widget', 'field': 'island'},
        {'type': 'widget', 'field': 'sex'},
        {'type': 'widget', 'field': 'year'}
    ],
    'transforms': [
        {'type': 'aggregate', 'method': 'mean', 'by': ['species', 'sex', 'year']}
    ]
})

The example will raise an error (try to find out what raises it).

Error message

Before:

image

After:

image

@codecov-commenter
Copy link

codecov-commenter commented Aug 17, 2022

Codecov Report

Merging #300 (9dd2aca) into master (e93e63d) will increase coverage by 0.48%.
The diff coverage is 76.58%.

@@            Coverage Diff             @@
##           master     #300      +/-   ##
==========================================
+ Coverage   66.55%   67.04%   +0.48%     
==========================================
  Files          59       60       +1     
  Lines        6196     6308     +112     
==========================================
+ Hits         4124     4229     +105     
- Misses       2072     2079       +7     
Impacted Files Coverage Δ
lumen/views/base.py 61.25% <33.33%> (-0.19%) ⬇️
lumen/sources/base.py 62.00% <42.10%> (-0.66%) ⬇️
lumen/dashboard.py 76.71% <50.00%> (ø)
lumen/filters/base.py 68.35% <50.00%> (+0.55%) ⬆️
lumen/sources/intake.py 88.13% <50.00%> (ø)
lumen/sources/intake_sql.py 89.18% <50.00%> (ø)
lumen/state.py 73.37% <50.00%> (ø)
lumen/variables/base.py 77.97% <50.00%> (ø)
lumen/target.py 78.90% <53.84%> (-0.09%) ⬇️
lumen/pipeline.py 73.49% <66.66%> (+4.16%) ⬆️
... and 7 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

lumen/base.py Outdated
matches = "', '".join(get_close_matches(component_type, cls_types))
if matches:
msg += f" Did you mean: '{matches}'?"
raise ValueError(msg)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this code be in Param instead? It doesn't seem specific to Lumen.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The validate_parameters could/should be implemented in Param as an option instead of "only" emitting a warning. holoviz/param#174

For the code in this function, in particular, I don't think it can be ported to param.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't think that Param could raise a helpful exception when given an attribute name that is not a parameter, guessing the right parameter name the same as done here? I must be missing something, then.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! A helpful exception for Param would be a very nice addition.

But that is not what this function does, which tries to match the input string component_type to a lumen component and return it.

@@ -16,6 +18,8 @@ class Component(param.Parameterized):

def __init__(self, **params):
self._refs = params.pop('refs', {})
expected = list(self.param.params())
validate_parameters(params, expected, type(self).name)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reasonable at this level, note though that a lot of components simply collect any parameters that haven't been explicitly declared into self.kwargs so this will never be triggered in those cases.

lumen/base.py Outdated Show resolved Hide resolved
lumen/target.py Outdated Show resolved Hide resolved
@Hoxbro
Copy link
Member Author

Hoxbro commented Aug 24, 2022

I have found another s-problem with this yaml file

config:
  title: Lumen

sources:
  penguins:
    type: file
    tables:
      penguins: https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-07-28/penguins.csv

targets:
  - title: Table
    source: penguins
    views:
      table:
        type: table
        table: penguins
        transform:  # missing an s here
          - type: iloc
            end: 100

Which does not raise an error, but the following warning:

WARNING:param.Tabulator00842: Setting non-parameter attribute transform=[{'type': 'iloc', 'end': 100}] using a mechanism intended only for parameters

lumen/target.py Outdated Show resolved Hide resolved
@Hoxbro
Copy link
Member Author

Hoxbro commented Aug 25, 2022

I think it could make sense to have a custom error like SpecificationError.

Maybe when this error is raised, it would only show the message and not the traceback.

lumen/target.py Outdated Show resolved Hide resolved
Co-authored-by: Philipp Rudiger <prudiger@anaconda.com>
@philippjfr
Copy link
Member

I think it could make sense to have a custom error like SpecificationError.

Fully agree!

@philippjfr
Copy link
Member

Superseded by #312 (kept all your commits)

@philippjfr philippjfr closed this Sep 1, 2022
@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 11, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants