Skip to content

Conversation

@signekb
Copy link
Member

@signekb signekb commented Jun 12, 2025

Description

It's supposed to be a string, but there's an error in the JSON schema. According to a PR in the data package repo, this should be fixed in datapackage V2.1 🤞

I encountered this issue bc the website build fails bc there was an error in resources.qmd when writing the properties.

This PR needs a quick review.

Checklist

  • Ran just run-all

signekb added 2 commits June 12, 2025 10:01
…erties()`

It's supposed to be a string, but there's an error in the JSON schema. According to a PR in the datapackage repo, this should be fixed in datapackage V2.1
@signekb signekb requested a review from a team as a code owner June 12, 2025 08:08
@signekb signekb moved this from Todo to In Review in Iteration planning Jun 12, 2025
Copy link
Contributor

@martonvago martonvago left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah we're seeing this now because the camel/snake case fix was merged in!

But do we want to change the type of fields_match in TableSchemaProperties to array as well? Because now it's type hinted as a string (well, FieldsMatchType). There are also some other references to fields_match in the tests.

If we don't want to make a bigger change to the "wrong" type in TableSchemaProperties, we could

  • not set fields_match in Sprout until the issue is fixed
  • fix it ourselves in the schema, but not by editing the JSON directly, but by resetting that one property in check_properties / check_resource_properties when the schema is loaded

@signekb
Copy link
Member Author

signekb commented Jun 12, 2025

Ahhh, I was wondering why it didn't show up earlier. That makes sense.

I suspect that it might take a while before the PR that fixes this isse is merged, unfortunately. At least, it's been open since Oct 2024. But you're absolutely right, we should align however we decide to fix this across Sprout.

Did we decide on anything more general about how we handle these situations after we found that the list field type was missing in the schema (#1064)? There, we removed the field type instead of editing the schema. Would an equivalent in this situation be to just not set it in Sprout for now then (potentially with a TODO comment about this)? Don't think we use it a lot anyways.

What do you think, @martonvago?

@martonvago
Copy link
Contributor

@signekb Yeah, I think we don't use it in Sprout for anything, so it's okay to not set it at all for now. (We should set it eventually only because Sprout doesn't use the default value for it, which is not obvious if datapackage.json is ever used/shared outside of Sprout.)

@signekb
Copy link
Member Author

signekb commented Jun 13, 2025

@martonvago Hmmm, maybe we should set it in Sprout as the default then? With the default being the array bc that's what it is according to the schema?

@martonvago
Copy link
Contributor

Hmm, I don't feel strongly about either deleting it or changing it to an array.
I guess your argument about being consistent with how we dealt with the list type missing is convincing though. So I'm happy to change it to an array!

@lwjohnst86
Copy link
Member

I can't tell if there is a resolution to the discussion so @signekb I'll let you merge in when you are ready/the discussion has been resolved :)

@signekb
Copy link
Member Author

signekb commented Jun 19, 2025

@lwjohnst86 We briefly discussed it at the meeting this morning, and I think we agreed on changing it to an array to fit the current schema even though it's incorrect and should be fixed in datapackage V2.1. What do you think?

@lwjohnst86
Copy link
Member

I think we'll just have to fix it until they formally correct the issue. They have a call for frictionless on June 27 where I think they'll talk about the next version status

@signekb
Copy link
Member Author

signekb commented Jun 19, 2025

Ah, ok, should we attend that meeting?

@lwjohnst86
Copy link
Member

@signekb I might, haven't decided though.

@lwjohnst86
Copy link
Member

I'm going to merge this in, since the docs workflow keeps failing.

@lwjohnst86 lwjohnst86 merged commit cebf1f1 into main Jun 20, 2025
5 checks passed
@lwjohnst86 lwjohnst86 deleted the fix/change-fieldsmatch-to-be-an-array-in-extract_props branch June 20, 2025 13:55
@github-project-automation github-project-automation bot moved this from In Review to Done in Iteration planning Jun 20, 2025
signekb added a commit that referenced this pull request Jun 23, 2025
# Description

This PR changes the type of `fields_match` to list after
#1392.

Closes #1408 

This PR needs a quick review.

## Checklist

- [x] Added or updated tests
- [x] Updated documentation
- [x] Ran `just run-all`

Co-authored-by: Signe Kirk Brødbæk <40836345+signekb@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

4 participants