Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make filebeat pub/sub input work with Google pub/sub emulator #13511

Closed
jbogp opened this issue Sep 5, 2019 · 2 comments
Closed

Make filebeat pub/sub input work with Google pub/sub emulator #13511

jbogp opened this issue Sep 5, 2019 · 2 comments
Labels

Comments

@jbogp
Copy link

jbogp commented Sep 5, 2019

Hello

Describe the enhancement:
Google makes a pubsub emulator available for local development, https://cloud.google.com/pubsub/docs/emulator
I've tried to make the pub/sub filebeat input work with it by exporting the environment variables PUBSUB_EMULATOR_HOST and PUBSUB_PROJECT_ID inside the container running filebeat (following the emulator documentation linked above), unfortunately filebeat still crashes with a "missing API key" error which seems to indicate the pub/sub library used doesn't take into account the possibility of an emulator

Describe a specific use case for the enhancement or feature:
This would be NTH to be able to develop the pipeline entirely locally without relying on a real pub/sub instance. Also the emulator is an official Google product so it's maintained and here to stay.

@andrewkroh
Copy link
Member

andrewkroh commented Sep 5, 2019

The emulator is used by the googlepubsub input's integration tests so it works with it. In order to satisfy Filebeat's config validation try setting the credential_file option to a dummy value like the tests do and see if that works.

"credentials_file": "NONE FOR EMULATOR TESTING",

@jbogp
Copy link
Author

jbogp commented Sep 24, 2019

Thank you that worked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants