Connector generator: Investigate build issues

[erohmensing]

After moving the connector generator over to poetry, there is some cleanup that needs to be done. This was reported by a community user, I haven't reproduced yet.

These are issues with Step 7: building/running the connector in docker

Option A: Building the docker image with airbyte-ci

airbyte-ci build build fails with

Failed to build connector image for platform linux/amd64: lstat main.py: no such file or directory

This is likely because we have nested the generated connector into a src directory, but don't have handling for that in airbyte-ci build. Moving the generated files out of the src directory into the root fixes the issue as a workaround.

Option B: Building the docker image with a Dockerfile

Based on changes, the dockerfile here likely needs to be updated.

[thesven]

I have been running into the same thing with an HTTP source connector. Followed the directions as outlined in:

https://docs.airbyte.com/connector-development/tutorials/cdk-tutorial-python-http/getting-started

spec
check
discover
read

all appear to work correctly. Moving the generated files out of the src directory seems to solve some of the problems on the actual build side, but when that connector source is added to AirByte OSS through the web ui, it fails to discover the schema.

[natikgadzhi]

@alafanechere, you're probably the closest one to that change in the first place — want to clean this up?

[natikgadzhi]

Another user stumbled at this today — I'll try to get this sorted

[bodschut]

Stumbled onto the same issue today. We're trying to get an internal custom connector deployed and we can't get it to build atm using the proposed Dockerfile contents from the docs, neither with the airbyte-ci build command.

[natikgadzhi]

connector.code_directory is used so widely and assumes that it's in the root of the connector that I'm not sure if it's easier to change the generators to not use src. Sure, not Poetry standard, but we don't want that many new python source connectors anyway. I'll go that route.