COMPAT: Support fastparquet 0.7.1

[lithomas1]

• closes CI: Fastparquet upgrade broke CI #42588
• tests added / passed
• Ensure all linting tests pass, see here for how to run them
• whatsnew entry

Also adds support for use_nullable_dtypes keyword. Technically, its not our enhancement its fastparquet's, so I think can backport. IMO, users shouldn't have to wait 6 months for 1.4 to get this new feature.

[jreback]

is this path tested?

[lithomas1]

https://github.com/pandas-dev/pandas/pull/42919/files#diff-c787f43da305dac82a3a8b918606b0dd8bd53fda24b26cabf71c28fff31d3699R1078

[jreback]

thanks @lithomas1

[jreback]

@meeseeksdev backport 1.3.x

[lumberbot-app]

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

$ git checkout 1.3.x
$ git pull

2. Cherry pick the first parent branch of the this PR on top of the older branch:

$ git cherry-pick -m1 e0422193ddb8830f91d6c3ac7c2c11971cc390bb

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

$ git commit -am 'Backport PR #42919: COMPAT: Support fastparquet 0.7.1'

4. Push to a named branch :

git push YOURFORK 1.3.x:auto-backport-of-pr-42919-on-1.3.x

5. Create a PR against branch 1.3.x, I would have named this PR:

"Backport PR #42919 on branch 1.3.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to suggest an improvement.

[lumberbot-app]

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

$ git checkout 1.3.x
$ git pull

2. Cherry pick the first parent branch of the this PR on top of the older branch:

$ git cherry-pick -m1 e0422193ddb8830f91d6c3ac7c2c11971cc390bb

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

$ git commit -am 'Backport PR #42919: COMPAT: Support fastparquet 0.7.1'

4. Push to a named branch :

git push YOURFORK 1.3.x:auto-backport-of-pr-42919-on-1.3.x

5. Create a PR against branch 1.3.x, I would have named this PR:

"Backport PR #42919 on branch 1.3.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to suggest an improvement.

[jorisvandenbossche]

As I mentioned (lately) on the issue, I am not in favor of enabling the use_nullable_dtypes keyword for the fastparquet engine if we don't guarantee a consistent behaviour (or at least I wouldn't backport that part, so we have a bit more time to flesh out the behaviour).

[simonjayhawkins]

hmm, we have the change note in 1.3.2 whatsnew.

can we revert this from master, reopen the PR to continue the discussion. and have two separate PRs if needed to avoid a partial backport.

[lithomas1]

Happy to talk it out. I think we can backport something that just turns of all the nullable dtypes for now until we get this sorted out. Will send PR soon.