correct to_parquet example in docs #4641
jakirkham pushed a commit that referenced this issue
Apr 30, 2021
* Add test illustrating import issue * move layer materialization for shuffle * add missing layers.py file and revise org a bit * move layers.py * remove xfail * use import rather than pickle for functions * fix importlib error and use dict * roll back test changes from 7374 (let failures be resolved in that PR) * remove debug print * move Shuffle layers to layers.py completely * moving moving BroadcastJoinLayer to layers.py * tweak CallableLazyImport * move BlockwiseCreateArray into layers.py * import test coverage * incorperate testing idea from 7374 * comment tweaks * introduce new test_layers.py module * update comment in test * Update dask/layers.py Co-authored-by: James Bourbeau <email@example.com> * Only use CallableLazyImport when the graph is materialized within __dask_distributed_unpack__ * remove obsolete annotation handling * Update dask/layers.py Co-authored-by: James Bourbeau <firstname.lastname@example.org> * _construct_graph fix * migrate csv code * migrate orc changes * basic parquet migration - still need to handle serialization * add require_pickle option to DataFrameIOLayer * update testing * change 'column culling' language to 'column projection' * use ConcatAxesWrapper to avoid inline task * add test coverage * use serialize instead of pickle * use simpler SerializedFunction wrapper * update to rely on distributed#4575 * serialization experiments * align with current status of distributed-4641 * update/fix comment * align with latest distributed#4641 state * re-align with James' suggestion * Temporarily point to Distributed PR #4641 * point CI back to distributed main branch * avoid using distributed 2021.03.0 for mindeps test * start aligning with #7455 * begin requiring io_deps elements to inherit from base class * more simplification * fix indices init * Add .git suffix * Add explicit pip * serialization cleanup * address dep-name issue * strip out unnecessary import logic * use output_blocks in pack * more code review suggestions * use clearer tmp variable name * avoid io_deps copy * dictionary comp for BlockwiseDepDict pack * start working on csv problem * handle csv nested-task problem * further cleanup * apply fix * add produces_tasks * use place-holder required_indices variable * CreateArrayDeps fix * remove extra name def * Change test ordering * fix dumps_function mistake and account for required_indices being an empty collection * roll back (breaking) large_graph_objects change * minor blockwise changes to address code-review * more cleanup * use for column projection in csv * updating some comments * improve documentation * add project_columns to the functions to make things a bit more explicit for now * move timeseries to Blockwise * remove commented code * add daily-stock Co-authored-by: James Bourbeau <email@example.com> Co-authored-by: James Bourbeau <firstname.lastname@example.org>
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments.