Connector for Google Cloud Storage

[cragwolfe]

Create a data connector that pulls documents from Google Cloud Storage, stores them locally (at least temporarily for processing), and runs them through unstructured.partition.auto.

See Adding Data Connectors for details on how to get started. Make sure to include a link to this issue when submitting a PR.

Definition of Done

• The checklist has been completed.
• The connector is able to process a single document.
• The connector is able to process all documents in a Google Cloud Storage folder, recursively.
• For now, it is OK to process only doc types that unstructured.partition.auto is capable of processing. Google Cloud Storage documents should be converted to PDF or Word Doc for processing (unless there is a better way).
• Bonus points: the ability to filter by document type.

[alvarobartt]

Thanks @cragwolfe I'll tackle this!

[cragwolfe]

Sounds good! Thanks @alvarobartt !

[alvarobartt]

Hi @benjats07, can you provide me a public GCS Bucket to test this integration too?

[alvarobartt]

Hi @cragwolfe so for this integration I think anyone could do it taking both AzureBlobStorageConnector and S3Connector as a reference, so feel free to label it as good first issue 🤗 In case no one tackles this during the upcoming week I'll do so as soon as we have a GCS Bucket to test it!