iterrows in catalog ingestion casts all dtypes to float

[bfhealy]

When ingesting a parquet file using ingest_catalog.py, this line casts all columns' dtypes to float. Thus the downstream checks for different dtypes will not work as intended.

Perhaps using itertuples instead would help by preserving the dtypes (at the expense of some re-working of the loop)? Note this could introduce a new issue where columns beginning with an underscore are renamed (e.g. _id, which gets mapped to _1). This might be avoided by setting name=None within itertuples.

[bfhealy]

For now I'm able to get around this issue by casting integers to the pandas Int64 type using column.astype("Int64"), then saving in parquet format.

[Theodlz]

Hi @bfhealy. Do you maybe have a sample that I can use to try this out?

[Theodlz]

a PR addressing this issue has been merged