ENH Add check for inferred compression before get_filepath_or_buffer

[stephen-hoover]

When reading CSVs, if compression='infer', check the input before calling get_filepath_or_buffer in the _read function. This way we can catch compresion extensions on S3 files. Partially resolves issue #11070 .

Checking for the file extension in the _read function should make the checks inside the parsers redundant. When I tried to remove them, however, I discovered that there's tests which assume the parsers can take an "infer" compression, so I left their checks.

I also discovered that the URL-reading code has a test which reads a URL ending in "gz" but which appears not to be gzip encoded, so this PR attempts to preserve its verdict in that case.

[jreback]

pls change tests which are incorrect as well
iow this should cause them to fail

[stephen-hoover]

I made this PR so that it didn't break any tests. Are the parsers ever accessed outside of the _read function? Do they need to be able to infer the compression type on their own? If not, I can remove that code from the parsers and change the tests.

[jreback]

the infer param can be moved higher up in the stack (eg in the get_filepath_or_buffer) - makes the readers simpler in that respect

[stephen-hoover]

Found it. I actually didn't need to change any tests. Now the only check for file extensions happens in the _read function, instead of separately inside each of the two parsers. By the time the parsers get called, any compression inference has already taken place.

[jreback]

gr8

[stephen-hoover]

Added a test using the new files in s3://pandas-test/.

[stephen-hoover]

Should I do anything else for this PR?

[jreback]

can you add a whatsnew note for this

[jreback]

pls rebase. ping when green.

[stephen-hoover]

@jreback , green! I found had to tweak get_filepath_or_buffer with an extra check for 'infer' compression.

[jreback]

thanks!