Test all available file formats and providers in source file

[sherifnada]

Tell us about the problem you're trying to solve

Source-file supports the following file formats:

  "csv",
  "json",
  "html",
  "excel",
  "feather",
  "parquet",
  "orc",
  "pickle"

and the following storage providers:

HTTPS
GCS
S3
SFTP
SSH
WebHDFS
Local filesystem

The connector is built on top of the smart-open package which is why it can support such a wide variety of files and providers. We'd like to use automated testing to verify all those options are working.

Acceptance Criteria

1. For each file format, place one instance of that file format on the local file system and write a test case to pull this file and verify the output is correct.
2. For each storage provider, place a simple CSV file on this storage provider and write a test case to pull from it.
3. Add credentials as needed to the .test-command.yml github workflow and the ci_credentials.sh script.

Feel free to break up the solution into as many PRs as needed

[sherifnada]

Blocked on providing credentials for any providers

[vitaliizazmic]

The estimate time to complete this task is 9h

Acceptance Criteria

1. Source files - 4h 30m
2. Storage provider - 4h
3. Add credentials - 30m