Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support pathlib.Path for input_file_list #160

Closed
hombit opened this issue Nov 3, 2023 · 0 comments · Fixed by #235
Closed

Support pathlib.Path for input_file_list #160

hombit opened this issue Nov 3, 2023 · 0 comments · Fixed by #235
Assignees
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@hombit
Copy link
Contributor

hombit commented Nov 3, 2023

Currently, if I pass a list of pathlib.Path objects into ImportArguments(input_file_list=), the processing runs well until "Finishing" step. At this step metadata creation fails when trying to dump json, saying that PosixPath is not serialiasable. Since stdlib's pathlib.Path is a modern and convenient way of path manipulations, I believe we should support it.

File ~/.virtualenvs/hipscat-import-sfd/lib/python3.11/site-packages/hipscat/io/write_metadata.py:27, in write_json_file(metadata_dictionary, file_pointer, storage_options)
     15 def write_json_file(
     16     metadata_dictionary: dict,
     17     file_pointer: file_io.FilePointer,
     18     storage_options: Union[Dict[Any, Any], None] = None,
     19 ):
     20     """Convert metadata_dictionary to a json string and print to file.
     21 
     22     Args:
   (...)
     25         storage_options: dictionary that contains abstract filesystem credentials
     26     """
---> 27     dumped_metadata = json.dumps(metadata_dictionary, indent=4)
     28     file_io.write_string_to_file(file_pointer, dumped_metadata + "\n", storage_options=storage_options)

TypeError: Object of type PosixPath is not JSON serializable
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants