Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support ECSV format #225

Closed
3 tasks done
hombit opened this issue Feb 16, 2024 · 0 comments · Fixed by #288
Closed
3 tasks done

Support ECSV format #225

hombit opened this issue Feb 16, 2024 · 0 comments · Fixed by #288
Assignees
Labels
enhancement New feature or request

Comments

@hombit
Copy link
Contributor

hombit commented Feb 16, 2024

** Feature request**

It would be great if hipscat-import could infer data types from ECSV catalogs, like Gaia DR3. Currently, user is required to provide an empty parquet file with a schema for this catalog.

astropy could help with this task. While its ECSV parser looks very slow, we still could use it just for data type inference:

from astropy.io import ascii

empty_astropy_table = ascii.read(gaia_file, format='ecsv', data_end=1)
empty_astropy_table.write('schema.parquet')

Before submitting
Please check the following:

  • I have described the purpose of the suggested change, specifying what I need the enhancement to accomplish, i.e. what problem it solves.
  • I have included any relevant links, screenshots, environment information, and data relevant to implementing the requested feature, as well as pseudocode for how I want to access the new functionality.
  • If I have ideas for how the new feature could be implemented, I have provided explanations and/or pseudocode and/or task lists for the steps.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants