Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP - V2 alpha #1

Merged
merged 91 commits into from May 20, 2020
Merged

WIP - V2 alpha #1

merged 91 commits into from May 20, 2020

Conversation

Yshayy
Copy link
Contributor

@Yshayy Yshayy commented Apr 22, 2020

Refactor everything
Change configuration format (breaking)
Added SQS provider
Add support for listeners/producers
Add support for multiple sources
Add support for multiple workers/pipes
Add minimal e2e test
Add usage documentation and project scope
Add support for multiple config files, dirs, command-line args and env variables

Need to add:

  • Observability (logs, metrics, tracing(?))
  • Implement Error handling
  • Stabilize Docker and tests
  • Extract some generic utils
  • Wrap generic consumer/producer with better defaults
  • Marshal configuration with normal golang objects
  • Output in pipe
  • Unit tests
  • Set CI

Investigate:

  • CloudEvents integration
  • Partition support (for reading with streams like Kafka)
  • Binary data support
  • GRPC support
  • Benchmarks
  • Tests against cloud environments

@Yshayy Yshayy merged commit 5e5cb60 into master May 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant