Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

For forbid_extra_keys raise custom ForbiddenExtraKeyError #225

Merged
merged 4 commits into from
Mar 25, 2022

Conversation

raabf
Copy link
Contributor

@raabf raabf commented Feb 7, 2022

When forbid_extra_keys is actived, I would like to be able to try-except that specific Problem (here ForbiddenExtraKeyError) instead of any Exception.

The unit tests also gain a little more percition through that change.

@codecov-commenter
Copy link

codecov-commenter commented Feb 7, 2022

Codecov Report

Merging #225 (5e136f5) into main (5675952) will increase coverage by 0.38%.
The diff coverage is 100.00%.

❗ Current head 5e136f5 differs from pull request most recent head b20eab5. Consider uploading reports for the commit b20eab5 to get more accurate results

@@            Coverage Diff             @@
##             main     #225      +/-   ##
==========================================
+ Coverage   97.94%   98.33%   +0.38%     
==========================================
  Files          16       16              
  Lines         779      719      -60     
==========================================
- Hits          763      707      -56     
+ Misses         16       12       -4     
Impacted Files Coverage Δ
src/cattr/errors.py 100.00% <100.00%> (ø)
src/cattr/dispatch.py 100.00% <0.00%> (ø)
src/cattr/preconf/json.py 100.00% <0.00%> (ø)
src/cattr/preconf/ujson.py 100.00% <0.00%> (ø)
src/cattr/disambiguators.py 100.00% <0.00%> (ø)
src/cattr/preconf/orjson.py 100.00% <0.00%> (ø)
src/cattr/_compat.py 94.77% <0.00%> (+0.54%) ⬆️
src/cattr/converters.py 99.04% <0.00%> (+0.64%) ⬆️

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@Tinche
Copy link
Member

Tinche commented Feb 8, 2022

I think this is a good idea but it should wait until after the extended validation branch is merged, which is blocked by other things temporarily.

@raabf
Copy link
Contributor Author

raabf commented Feb 8, 2022

Sure, order the MR as it fits best. 😄

@Tinche Tinche mentioned this pull request Mar 24, 2022
@Tinche
Copy link
Member

Tinche commented Mar 24, 2022

Hello, I'm preparing a new release and let's get this merged!

So please rebase, and I would ask for a change: add a field to the exception (extra_keys) that carries the list of extra keys. This will make it easier to introspect these exceptions programmatically.

@raabf raabf force-pushed the forbid_extra_key_custom_error branch from 5e136f5 to b20eab5 Compare March 25, 2022 17:12
@raabf
Copy link
Contributor Author

raabf commented Mar 25, 2022

add a field to the exception (extra_keys) that carries the list of extra keys. This will make it easier to introspect these exceptions programmatically.

Yea that is nice – I have added this and a reference to class to which the extra keys should be used (i.e. to which class constructor the extra keys were passed). I also extended unit test to check that attributes.

However, there is a small problem, since ForbiddenExtraKeysError can conflict with ClassValidationError. I assumed that an ForbiddenExtraKeysError means that the Class Validation failed. That means if ClassValidationError is used, I add ForbiddenExtraKeysError to the errors list which is then passed to the ClassValidationError. This is more or less anyway implicitly done, because if ForbiddenExtraKeysError in a nested class is raised it will be catched anyway by its upper class because you have added except Exception as e: to every attribute. See this commit where I have implemented this behaviour fd5a9b8 . I hope this is what you have intended.

@raabf
Copy link
Contributor Author

raabf commented Mar 25, 2022

So that it is easy to understand this is an example generated structure function (unknows_fields are added to errors list):

def structure_A(o, _, *, __cl=__cl, __c_cve=__c_cve, __c_structure_c=__c_structure_c, __c_type_c=__c_type_c, __c_structure_a=__c_structure_a):
  res = {}
  errors = []
  try:
    res['c'] = __c_structure_c(o['c'], __c_type_c)
  except Exception as e:
    e.__note__ = 'Structuring class test_forbid_extra_keys_nested_override.<locals>.A @ attribute c'
    errors.append(e)
  if 'a' in o:
    try:
      res['a'] = __c_structure_a(o['a'])
    except Exception as e:
      e.__note__ = 'Structuring class test_forbid_extra_keys_nested_override.<locals>.A @ attribute a'
      errors.append(e)
  unknown_fields = set(o.keys()) - __c_a
  if unknown_fields:
    errors.append(__c_feke('', __cl, unknown_fields)) 
  if errors: raise __c_cve('While structuring A', errors, __cl)
  try:
    return __cl(
      **res,
    )
  except Exception as exc: raise __c_cve('While structuring A', [exc], __cl)

@Tinche
Copy link
Member

Tinche commented Mar 25, 2022

Yep, that looks great. Thanks a lot!

@Tinche Tinche merged commit 031eeef into python-attrs:main Mar 25, 2022
mergify bot pushed a commit to aws/jsii that referenced this pull request Apr 4, 2022
…2 in /packages/@jsii/python-runtime (#3470)

Updates the requirements on [cattrs](https://github.com/python-attrs/cattrs) to permit the latest version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/python-attrs/cattrs/blob/main/HISTORY.rst">cattrs's changelog</a>.</em></p>
<blockquote>
<h2>22.1.0 (2022-04-03)</h2>
<ul>
<li>cattrs now uses the CalVer versioning convention.</li>
<li>cattrs now has a detailed validation mode, which is enabled by default. Learn more <code>here &lt;https://cattrs.readthedocs.io/en/latest/validation.html&gt;</code>_.
The old behavior can be restored by creating the converter with <code>detailed_validation=False</code>.</li>
<li><code>attrs</code> and dataclass structuring is now ~25% faster.</li>
<li>Fix an issue structuring bare <code>typing.List</code> s on Pythons lower than 3.9.
(<code>[#209](python-attrs/cattrs#209) &lt;https://github.com/python-attrs/cattrs/issues/209&gt;</code>_)</li>
<li>Fix structuring of non-parametrized containers like <code>list/dict/...</code> on Pythons lower than 3.9.
(<code>[#218](python-attrs/cattrs#218) &lt;https://github.com/python-attrs/cattrs/issues/218&gt;</code>_)</li>
<li>Fix structuring bare <code>typing.Tuple</code> on Pythons lower than 3.9.
(<code>[#218](python-attrs/cattrs#218) &lt;https://github.com/python-attrs/cattrs/issues/218&gt;</code>_)</li>
<li>Fix a wrong <code>AttributeError</code> of an missing <code>__parameters__</code> attribute. This could happen
when inheriting certain generic classes – for example <code>typing.*</code> classes are affected.
(<code>[#217](python-attrs/cattrs#217) &lt;https://github.com/python-attrs/cattrs/issues/217&gt;</code>_)</li>
<li>Fix structuring of <code>enum.Enum</code> instances in <code>typing.Literal</code> types.
(<code>[#231](python-attrs/cattrs#231) &lt;https://github.com/python-attrs/cattrs/pull/231&gt;</code>_)</li>
<li>Fix unstructuring all tuples - unannotated, variable-length, homogenous and heterogenous - to <code>list</code>.
(<code>[#226](python-attrs/cattrs#226) &lt;https://github.com/python-attrs/cattrs/issues/226&gt;</code>_)</li>
<li>For <code>forbid_extra_keys</code> raise custom <code>ForbiddenExtraKeyError</code> instead of generic <code>Exception</code>.
(<code>[#225](python-attrs/cattrs#225) &lt;https://github.com/python-attrs/cattrs/pull/225&gt;</code>_)</li>
<li>All preconf converters now support <code>loads</code> and <code>dumps</code> directly. See an example <code>here &lt;https://cattrs.readthedocs.io/en/latest/preconf.html&gt;</code>_.</li>
<li>Fix mappings with byte keys for the orjson, bson and tomlkit converters.
(<code>[#241](python-attrs/cattrs#241) &lt;https://github.com/python-attrs/cattrs/issues/241&gt;</code>_)</li>
</ul>
<h2>1.10.0 (2022-01-04)</h2>
<ul>
<li>Add PEP 563 (string annotations) support for dataclasses.
(<code>[#195](python-attrs/cattrs#195) &lt;https://github.com/python-attrs/cattrs/issues/195&gt;</code>_)</li>
<li>Fix handling of dictionaries with string Enum keys for bson, orjson, and tomlkit.</li>
<li>Rename the <code>cattr.gen.make_dict_unstructure_fn.omit_if_default</code> parameter to <code>_cattrs_omit_if_default</code>, for consistency. The <code>omit_if_default</code> parameters to <code>GenConverter</code> and <code>override</code> are unchanged.</li>
<li>Following the changes in <code>attrs</code> 21.3.0, add a <code>cattrs</code> package mirroring the existing <code>cattr</code> package. Both package names may be used as desired, and the <code>cattr</code> package isn't going away.</li>
</ul>
<h2>1.9.0 (2021-12-06)</h2>
<ul>
<li>Python 3.10 support, including support for the new union syntax (<code>A | B</code> vs <code>Union[A, B]</code>).</li>
<li>The <code>GenConverter</code> can now properly structure generic classes with generic collection fields.
(<code>[#149](python-attrs/cattrs#149) &lt;https://github.com/python-attrs/cattrs/issues/149&gt;</code>_)</li>
<li><code>omit=True</code> now also affects generated structuring functions.
(<code>[#166](python-attrs/cattrs#166) &lt;https://github.com/python-attrs/cattrs/issues/166&gt;</code>_)</li>
<li><code>cattr.gen.{make_dict_structure_fn, make_dict_unstructure_fn}</code> now resolve type annotations automatically when PEP 563 is used.
(<code>[#169](python-attrs/cattrs#169) &lt;https://github.com/python-attrs/cattrs/issues/169&gt;</code>_)</li>
<li>Protocols are now unstructured as their runtime types.
(<code>[#177](python-attrs/cattrs#177) &lt;https://github.com/python-attrs/cattrs/pull/177&gt;</code>_)</li>
<li>Fix an issue generating structuring functions with renaming and <code>_cattrs_forbid_extra_keys=True</code>.
(<code>[#190](python-attrs/cattrs#190) &lt;https://github.com/python-attrs/cattrs/issues/190&gt;</code>_)</li>
</ul>
<h2>1.8.0 (2021-08-13)</h2>
<ul>
<li>Fix <code>GenConverter</code> mapping structuring for unannotated dicts on Python 3.8.</li>
</ul>

</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a href="https://github.com/python-attrs/cattrs/commits">compare view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
mergify bot pushed a commit to aws/jsii that referenced this pull request Oct 3, 2022
…3 in /packages/@jsii/python-runtime (#3785)

Updates the requirements on [cattrs](https://github.com/python-attrs/cattrs) to permit the latest version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/python-attrs/cattrs/blob/main/HISTORY.rst">cattrs's changelog</a>.</em></p>
<blockquote>
<h2>22.2.0 (2022-10-03)</h2>
<ul>
<li><em>Potentially breaking</em>: <code>cattrs.Converter</code> has been renamed to <code>cattrs.BaseConverter</code>, and <code>cattrs.GenConverter</code> to <code>cattrs.Converter</code>.
The <code>GenConverter</code> name is still available for backwards compatibility, but is deprecated.
If you were depending on functionality specific to the old <code>Converter</code>, change your import to <code>from cattrs import BaseConverter</code>.</li>
<li><code>NewTypes &lt;https://docs.python.org/3/library/typing.html#newtype&gt;</code>_ are now supported by the <code>cattrs.Converter</code>.
(<code>[#255](python-attrs/cattrs#255) &lt;https://github.com/python-attrs/cattrs/pull/255&gt;</code><em>, <code>[#94](python-attrs/cattrs#94) &lt;https://github.com/python-attrs/cattrs/issues/94&gt;</code></em>, <code>[#297](python-attrs/cattrs#297) &lt;https://github.com/python-attrs/cattrs/issues/297&gt;</code>_)</li>
<li><code>cattrs.Converter</code> and <code>cattrs.BaseConverter</code> can now copy themselves using the <code>copy</code> method.
(<code>[#284](python-attrs/cattrs#284) &lt;https://github.com/python-attrs/cattrs/pull/284&gt;</code>_)</li>
<li>Python 3.11 support.</li>
<li>cattrs now supports un/structuring <code>kw_only</code> fields on attrs classes into/from dictionaries.
(<code>[#247](python-attrs/cattrs#247) &lt;https://github.com/python-attrs/cattrs/pull/247&gt;</code>_)</li>
<li>PyPy support (and tests, using a minimal Hypothesis profile) restored.
(<code>[#253](python-attrs/cattrs#253) &lt;https://github.com/python-attrs/cattrs/issues/253&gt;</code>_)</li>
<li>Fix propagating the <code>detailed_validation</code> flag to mapping and counter structuring generators.</li>
<li>Fix <code>typing.Set</code> applying too broadly when used with the <code>GenConverter.unstruct_collection_overrides</code> parameter on Python versions below 3.9. Switch to <code>typing.AbstractSet</code> on those versions to restore the old behavior.
(<code>[#264](python-attrs/cattrs#264) &lt;https://github.com/python-attrs/cattrs/issues/264&gt;</code>_)</li>
<li>Uncap the required Python version, to avoid problems detailed in <a href="https://iscinumpy.dev/post/bound-version-constraints/#pinning-the-python-version-is-special">https://iscinumpy.dev/post/bound-version-constraints/#pinning-the-python-version-is-special</a>
(<code>[#275](python-attrs/cattrs#275) &lt;https://github.com/python-attrs/cattrs/issues/275&gt;</code>_)</li>
<li>Fix <code>Converter.register_structure_hook_factory</code> and <code>cattrs.gen.make_dict_unstructure_fn</code> type annotations.
(<code>[#281](python-attrs/cattrs#281) &lt;https://github.com/python-attrs/cattrs/issues/281&gt;</code>_)</li>
<li>Expose all error classes in the <code>cattr.errors</code> namespace. Note that it is deprecated, just use <code>cattrs.errors</code>.
(<code>[#252](python-attrs/cattrs#252) &lt;https://github.com/python-attrs/cattrs/issues/252&gt;</code>_)</li>
<li>Fix generating structuring functions for types with quotes in the name.
(<code>[#291](python-attrs/cattrs#291) &lt;https://github.com/python-attrs/cattrs/issues/291&gt;</code>_ <code>[#277](python-attrs/cattrs#277) &lt;https://github.com/python-attrs/cattrs/issues/277&gt;</code>_)</li>
<li>Fix usage of notes for the final version of <code>PEP 678 &lt;https://peps.python.org/pep-0678/&gt;</code><em>, supported since <code>exceptiongroup&gt;=1.0.0rc4</code>.
(<code>[#303](python-attrs/cattrs#303) &lt;303 &lt;https://github.com/python-attrs/cattrs/pull/303&gt;</code></em>)</li>
</ul>
<h2>22.1.0 (2022-04-03)</h2>
<ul>
<li>cattrs now uses the CalVer versioning convention.</li>
<li>cattrs now has a detailed validation mode, which is enabled by default. Learn more <code>here &lt;https://cattrs.readthedocs.io/en/latest/validation.html&gt;</code>_.
The old behavior can be restored by creating the converter with <code>detailed_validation=False</code>.</li>
<li><code>attrs</code> and dataclass structuring is now ~25% faster.</li>
<li>Fix an issue structuring bare <code>typing.List</code> s on Pythons lower than 3.9.
(<code>[#209](python-attrs/cattrs#209) &lt;https://github.com/python-attrs/cattrs/issues/209&gt;</code>_)</li>
<li>Fix structuring of non-parametrized containers like <code>list/dict/...</code> on Pythons lower than 3.9.
(<code>[#218](python-attrs/cattrs#218) &lt;https://github.com/python-attrs/cattrs/issues/218&gt;</code>_)</li>
<li>Fix structuring bare <code>typing.Tuple</code> on Pythons lower than 3.9.
(<code>[#218](python-attrs/cattrs#218) &lt;https://github.com/python-attrs/cattrs/issues/218&gt;</code>_)</li>
<li>Fix a wrong <code>AttributeError</code> of an missing <code>__parameters__</code> attribute. This could happen
when inheriting certain generic classes – for example <code>typing.*</code> classes are affected.
(<code>[#217](python-attrs/cattrs#217) &lt;https://github.com/python-attrs/cattrs/issues/217&gt;</code>_)</li>
<li>Fix structuring of <code>enum.Enum</code> instances in <code>typing.Literal</code> types.
(<code>[#231](python-attrs/cattrs#231) &lt;https://github.com/python-attrs/cattrs/pull/231&gt;</code>_)</li>
<li>Fix unstructuring all tuples - unannotated, variable-length, homogenous and heterogenous - to <code>list</code>.
(<code>[#226](python-attrs/cattrs#226) &lt;https://github.com/python-attrs/cattrs/issues/226&gt;</code>_)</li>
<li>For <code>forbid_extra_keys</code> raise custom <code>ForbiddenExtraKeyError</code> instead of generic <code>Exception</code>.
(<code>[#225](python-attrs/cattrs#225) &lt;https://github.com/python-attrs/cattrs/pull/225&gt;</code>_)</li>
<li>All preconf converters now support <code>loads</code> and <code>dumps</code> directly. See an example <code>here &lt;https://cattrs.readthedocs.io/en/latest/preconf.html&gt;</code>_.</li>
</ul>

</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/python-attrs/cattrs/commit/405f0291b958ae9eb45ee38febeb91fb65dd644f"><code>405f029</code></a> v22.2.0</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/89de04f57aa774d6abfb0ae62517dc8a8064e3c2"><code>89de04f</code></a> Fix some mor</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/0abbf271461babca203862f3581c20743f6118e0"><code>0abbf27</code></a> Fix tests</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/3b750439aec826a8fd20976ca113f30a27e75408"><code>3b75043</code></a> <strong>notes</strong> is list[str]</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/906b95cfef4903e2d6247abf0fdd72f7da21617a"><code>906b95c</code></a> Reorder HISTORY</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/ed7f86a0ccd9adeab895b9afac2dea69fa02bcff"><code>ed7f86a</code></a> Improve NewTypes (<a href="https://github-redirect.dependabot.com/python-attrs/cattrs/issues/310">#310</a>)</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/e7926599ad44e07d8325ae4072626e1a24705542"><code>e792659</code></a> Fix missing imports of preconf converters (<a href="https://github-redirect.dependabot.com/python-attrs/cattrs/issues/309">#309</a>)</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/e425d6378aa8dfc74bbdd9e152365e1b962fd8cf"><code>e425d63</code></a> Reorder HISTORY</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/cc56b2b873852a4e91b16706a39da00755c87759"><code>cc56b2b</code></a> Remove spurious type comment</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/cbd6f29d2c0ebc40805b3ca0d81accfc330eb10c"><code>cbd6f29</code></a> Reformat</li>
<li>Additional commits viewable in <a href="https://github.com/python-attrs/cattrs/compare/v1.8.0...v22.2.0">compare view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants