-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Documentation on how to set this up #1
Comments
Hi @amiel. I'll go ahead and do that. In the meantime, I'll assign this issue to myself. Thanks for reaching out. |
Thanks @jeffchao!!! I look forward to trying it out :) |
@amiel The README is up. I added some context as well. Feel free to file another issue if you have any other questions. |
@jeffchao Thanks so much for adding this documentation. I spent a day trying to get it set up. There were a few changes I had to make and I was planning to put a PR together with those changes when I got it working. However, I haven't been able to get the heroku process successfully running. I was able to connect the kafka cluster with kafkacat, but the heroku process eventually crashes with the following error:
I'm pretty much out of time to experiment with this right now, but if you have any advice, I would appreciate it for the next time I get a chance to experiment with Kafka on Heroku. I will also clean up my changes and submit a PR. Even if you don't merge it, it may help other people trying to see what changes I had to make. I'd love to get CDC working from Heroku Postgres to Heroku Kafka as I think this would improve the stability of data processing at work! |
@amiel thanks for the feedback. I look forward to your PR. Regarding the error, I suspect you are using the basic Kafka plans. Is this right? If so, then in these plans, you'll have to pre-create all internal Kafka topics using the kafka topics create command on the Heroku CLI. This is because Heroku basic Kafka plans are multitenant and provide isolation by leveraging Kafka ACLs. This is the most likely cause. Another option is to try this on a standard kafka plan, which runs expensive, but fortunately is pro-rated to your usage. That way, you can spin one up, test it out, and spin it down. |
@jeffchao Yes, I was on a basic Kafka plan. I added the |
Closed by #2. |
Hi @jeffchao,
I am curious about your example. Would you be willing to write up a README explaining how to set this up on Heroku?
Thanks!!
-Amiel
The text was updated successfully, but these errors were encountered: