Feat/pandas reader parquet

[flaviassantos]

This pull request adds parquet read and write functionality for issue #406.

Changes

The changes to the files are as described below:

pandas_extensions.py: Added the classes to read and write parquet files
test_pandas_extensions.py: Added a single test case that exercises the writing and reading functionality respectively
notebook.ipynb: Added the example of parquet materialization
my_script.py: Added the example of parquet materialization

How I tested this

By running successfully:

• unit test for the PandasParquetWriter and PandasParquetReader classes.
• the Jupyter notebook.
• the my_script.py
• cicleci job locally

Notes

Checklist

• PR has an informative and human-readable title (this will be pulled into the release notes)
• Changes are limited to a single goal (no scope creep)
• Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
• Any change in functionality is tested
• New functions are documented (with a description, list of inputs, and expected output)
• Placeholder code is flagged / future TODOs are captured in comments
• Project documentation has been updated if adding/changing functionality.

[flaviassantos]

@skrawcz , last point in the checklist. Can you point me to the "Project documentation" mentioned?

[skrawcz]

@skrawcz , last point in the checklist. Can you point me to the "Project documentation" mentioned?

Yep so that should be automatically updated for you. It should show up here (based on the build of this branch) https://hamilton--429.org.readthedocs.build/en/429/reference/io/available-data-adapters/.

[skrawcz]

looks good, just the minor comment on the test!

[skrawcz]

try:
result = self.api.parquet.read_table(
path_or_handle, columns=columns, **kwargs
).to_pandas(**to_pandas_kwargs)
E TypeError: read_table() got an unexpected keyword argument 'dtype_backend'

../venvs/hamilton-venv/lib/python3.7/site-packages/pandas/io/parquet.py:240: TypeError

You will need to gate this parameter for 3.7. So you'll see others have:

        if sys.version_info >= (3, 8) and self.dtype_backend is not None:
            kwargs["dtype_backend"] = self.dtype_backend