Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Default to eliding missing values in JSON representation #363

Closed
cnuernber opened this issue Jul 28, 2023 · 3 comments · Fixed by #364
Closed

Default to eliding missing values in JSON representation #363

cnuernber opened this issue Jul 28, 2023 · 3 comments · Fixed by #364

Comments

@cnuernber
Copy link
Collaborator

Currently they can end up as null or NAN or a few different things so it would be ideal to just elide the key. I currently believe that this behavior should be optional but defaulted on.

Concurrently ds/rows should have an option to elide missing keys - this should not be the default as there is a per-record perf cost inherent in this pathway but the option should be there and be clearly documented.

@genmeblog
Copy link

I want to share, that eliding missing values in mapseq-reader by default was really breaking change. It wasn't visible at the beginning. Since then several users of tablecloth report strange bugs which are caused only by this change in different places like pivoting, aggregation, grouping etc...

It's just a lesson that such kind of minor update can cause major breakage.

@cnuernber
Copy link
Collaborator Author

Thanks for the clear communication - I definitely did not intend such breakage. Glad things are now resolved.

@genmeblog
Copy link

I definitely should work more on tests to find such things earlier. Anyway, it looks like the transition to TMD7 was quite smooth which was really great result after such big refactoring.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants