fix(annotations): handle required fields properly

[villebro]

SUMMARY

Currently it's possible to create annotations and annotation layers via the API that cause the frontend to crash. For instance, by defining an annotation layer with a null name, the edit window crashes with the following error:

Also, editing a previously created annotation where the json_metadata field had been unset (null), trying to save the annotation resulted in the following error:

This PR updates both the annotation layer and annotation marshmallow schemas to require the minimum amount of defined or non-null values. Note that when leaving out a property on a PUT request, the original object leaves the omitted values untouched, hence a very permissive schema for PUT operations (no required fields). In addition, the frontend logic is updated to no longer crash if an object that was created prior to this PR is edited.

Test cases are added to all relevant permutations of allowed and incorrect POST/PUT payloads. Affected API tests are also updated to correspond to the new minimum requirements.

ADDITIONAL INFORMATION

• Has associated issue:
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[codecov]

Codecov Report

Merging #17234 (7e34a51) into master (ef01cbb) will decrease coverage by 0.07%.
The diff coverage is 92.85%.

❗ Current head 7e34a51 differs from pull request most recent head 8feedc1. Consider uploading reports for the commit 8feedc1 to get more accurate results

@@            Coverage Diff             @@
##           master   #17234      +/-   ##
==========================================
- Coverage   76.82%   76.75%   -0.08%     
==========================================
  Files        1038     1037       -1     
  Lines       55597    55609      +12     
  Branches     7585     7588       +3     
==========================================
- Hits        42715    42680      -35     
- Misses      12632    12679      +47     
  Partials      250      250              

Flag	Coverage Δ
mysql	81.95% <100.00%> (+0.01%)	⬆️
postgres	81.96% <100.00%> (+0.01%)	⬆️
presto	?
python	82.04% <100.00%> (-0.16%)	⬇️
sqlite	81.63% <100.00%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
superset-frontend/src/components/Select/Select.tsx	92.35% <ø> (ø)
...d/components/DashboardBuilder/DashboardBuilder.tsx	90.17% <ø> (ø)
...tend/src/explore/components/ExploreChartHeader.jsx	6.84% <0.00%> (-0.20%)	⬇️
...ontend/src/filters/components/Select/buildQuery.ts	100.00% <ø> (ø)
superset-frontend/src/setup/setupColors.ts	100.00% <ø> (ø)
superset/annotation_layers/schemas.py	100.00% <ø> (ø)
...et-frontend/src/dashboard/components/Dashboard.jsx	81.88% <100.00%> (+3.79%)	⬆️
...ponents/filterscope/renderFilterScopeTreeNodes.jsx	87.50% <100.00%> (+0.83%)	⬆️
...tend/src/views/CRUD/annotation/AnnotationModal.tsx	60.44% <100.00%> (ø)
...ews/CRUD/annotationlayers/AnnotationLayerModal.tsx	69.47% <100.00%> (ø)
... and 7 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ef01cbb...8feedc1. Read the comment docs.

[geido]

LGTM!

[villebro]

required=True only makes sure the property is defined, so here we need to also add allow_none=False to make sure null values aren't accepted.

[zhaoyongjie]

LGTM, Thanks for the comprehensive unit test.

[dpgaspar]

Thank you for fixing this!

[dpgaspar]

Should we place these on tests/unit_tests/annotation_layers instead? they seem more like unit tests then integration tests. It would be nice also to place a test on the API integration tests that would fail on current master but succeed on this PR

[villebro]

Thanks for the comment @dpgaspar - you're absolutely right, these should have been placed on the unit tests (completely forgot about the new split). We'll do that + add an IT in a follow up shortly 👍