How to optimize performance when running in Docker Desktop (on Windows) #37842
Unanswered
LarsHanegraaf
asked this question in
Connector Questions
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi all,
I'm currently using Airbyte in Docker Desktop on Windows (I used the
abctl
command to install it). As I want to create a local development environment, I would like to run Airbyte alongside Dagster and a Postgres database.Currently I'm incidentally able to run a single sync from an API (custom connector) to a database (max. 30 MB of data). If it succeeds it often takes 3 to 4 minutes to run. If it doesn't succeed, it hangs forever and it looks like it is short on resources.
I think I have a decent sized machine (11th generation i7 and 16GB of RAM), but the Windows Resource Manager shows that the memory is always on 100% (probably because
abctl
requires Kubernetes, which doesn't play that nice on Windows due to WSL2).Happy to hear some ideas on how to make Airbyte performant on my local machine. I like Airbyte becasue of the ecosystem and community, but this performance kills the developer productivity unfortunately.
Thanks!
Lars
Beta Was this translation helpful? Give feedback.
All reactions