Skip to content

Conversation

@eecavanna
Copy link
Collaborator

On this branch, I made the following two changes:

  1. When generating the properties of an Entity, instead of omitting all fields whose values are falsey, only omit the ones whose values are None, [], or {} — in other words, keep those whose values are "" or 0
  2. When dumping the JSON data, omit fields whose values are None

CC: @pkalita-lbl and @shreddd

@eecavanna eecavanna self-assigned this Sep 19, 2025
@eecavanna eecavanna linked an issue Sep 19, 2025 that may be closed by this pull request
@eecavanna eecavanna requested a review from Copilot September 19, 2025 03:08
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refines the NMDC ingest script's data handling behavior by improving field filtering logic and JSON serialization. The changes ensure that empty strings and zero values are preserved while removing truly empty containers and null values.

  • Modified entity property filtering to preserve empty strings and zero values while still omitting None, empty lists, and empty dictionaries
  • Updated JSON serialization to exclude None values from output
  • Added cache.json to gitignore for commonly used caching between script runs

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
contrib/nmdc/ingest.py Updated field filtering logic and JSON serialization to better handle empty vs. falsy values
contrib/nmdc/.gitignore Added cache.json file to ignore list for temporary data storage

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@eecavanna eecavanna merged commit 87fab60 into main Sep 19, 2025
@eecavanna eecavanna deleted the 28-omit-none-valued-fields-from-json-objects-dumped-by-ingester-nmdc branch September 19, 2025 03:14
@pkalita-lbl
Copy link
Contributor

Thanks I meant to follow up on your comment in the other PR. But this is exactly what I was going to suggest doing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Omit None-valued fields from JSON objects dumped by ingester (NMDC)

3 participants