Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

correct to_parquet example in docs #4641

Merged
merged 1 commit into from Mar 28, 2019

Conversation

@aaronfowles
Copy link
Contributor

@aaronfowles aaronfowles commented Mar 28, 2019

  • Tests added / passed
  • Passes flake8 dask

I've updated the example in the to_parquet to be correct. Also, seems to be getting skipped in doctesting - should I remove skip comments?

@martindurant
Copy link
Member

@martindurant martindurant commented Mar 28, 2019

Good catch, thank you.
It should remain skipped because the line before does not refer to any specific data to load, only gives the form.

@aaronfowles
Copy link
Contributor Author

@aaronfowles aaronfowles commented Mar 28, 2019

You're welcome!

@martindurant martindurant merged commit 44948ec into dask:master Mar 28, 2019
2 checks passed
jorge-pessoa pushed a commit to jorge-pessoa/dask that referenced this issue May 14, 2019
jrbourbeau added a commit to rjzamora/dask that referenced this issue Apr 14, 2021
jakirkham pushed a commit that referenced this issue Apr 30, 2021
* Add test illustrating import issue

* move layer materialization for shuffle

* add missing layers.py file and revise org a bit

* move layers.py

* remove xfail

* use import rather than pickle for functions

* fix importlib error and use dict

* roll back test changes from 7374 (let failures be resolved in that PR)

* remove debug print

* move Shuffle layers to layers.py completely

* moving moving BroadcastJoinLayer to layers.py

* tweak CallableLazyImport

* move BlockwiseCreateArray into layers.py

* import test coverage

* incorperate testing idea from 7374

* comment tweaks

* introduce new test_layers.py module

* update comment in test

* Update dask/layers.py

Co-authored-by: James Bourbeau <jrbourbeau@users.noreply.github.com>

* Only use CallableLazyImport when the graph is materialized within __dask_distributed_unpack__

* remove obsolete annotation handling

* Update dask/layers.py

Co-authored-by: James Bourbeau <jrbourbeau@users.noreply.github.com>

* _construct_graph fix

* migrate csv code

* migrate orc changes

* basic parquet migration - still need to handle serialization

* add require_pickle option to DataFrameIOLayer

* update testing

* change 'column culling' language to 'column projection'

* use ConcatAxesWrapper to avoid inline task

* add test coverage

* use serialize instead of pickle

* use simpler SerializedFunction wrapper

* update to rely on distributed#4575

* serialization experiments

* align with current status of distributed-4641

* update/fix comment

* align with latest distributed#4641 state

* re-align with James' suggestion

* Temporarily point to Distributed PR #4641

* point CI back to distributed main branch

* avoid using distributed 2021.03.0 for mindeps test

* start aligning with #7455

* begin requiring io_deps elements to inherit from base class

* more simplification

* fix indices init

* Add .git suffix

* Add explicit pip

* serialization cleanup

* address dep-name issue

* strip out unnecessary import logic

* use output_blocks in pack

* more code review suggestions

* use clearer tmp variable name

* avoid io_deps copy

* dictionary comp for BlockwiseDepDict pack

* start working on csv problem

* handle csv nested-task problem

* further cleanup

* apply fix

* add produces_tasks

* use place-holder required_indices variable

* CreateArrayDeps fix

* remove extra name def

* Change test ordering

* fix dumps_function mistake and account for required_indices being an empty collection

* roll back (breaking) large_graph_objects change

* minor blockwise changes to address code-review

* more cleanup

* use  for column projection in csv

* updating some comments

* improve documentation

* add project_columns to the functions to make things a bit more explicit for now

* move timeseries to Blockwise

* remove commented code

* add daily-stock

Co-authored-by: James Bourbeau <jrbourbeau@gmail.com>
Co-authored-by: James Bourbeau <jrbourbeau@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants