Augustin/protocolv2 (#39863)

## What
<!--
* Describe what the change is solving. Link all GitHub issues related to this change.
-->

Separate out the `datamodel-codegen` workflow into a dagger workflow. This enables us to, upstack, properly generate the same v1 models as previously. Unfortunately datamodel-codegen's "pydantic v1" output on its v2 versions doesn't output what one would expect - see [issue](koxudaxi/datamodel-code-generator#1950) (thanks AJ!). 

## How
<!--
* Describe how code changes achieve the solution.
-->
* Convert the script from bash to python (in dagger) and run it via a shell script (to install dagger)

## User Impact
<!--
* What is the end result perceived by the user?
* If there are negative side effects, please list them. 
-->
None. Development experience is also the same

## Can this PR be safely reverted and rolled back?
<!--
* If unsure, leave it blank.
-->
- [x] YES 💚
- [ ] NO ❌

airbyte-cdk/python/airbyte_cdk/sources/declarative/models/__init__.py

@@ -1,2 +1,2 @@
-# generated by generate-component-manifest-files
+# generated by bin/generate_component_manifest_files.py
 from .declarative_component_schema import *

airbyte-cdk/python/bin/generate-component-manifest-dagger.sh

@@ -0,0 +1,6 @@
+#!/usr/bin/env bash
+
+set -e
+
+pip install dagger-io==0.9.6
+python bin/generate_component_manifest_files.py

airbyte-cdk/python/bin/generate_component_manifest_files.py

@@ -0,0 +1,75 @@
+# Copyright (c) 2024 Airbyte, Inc., all rights reserved.
+
+import sys
+from glob import glob
+from pathlib import Path
+
+import anyio
+import dagger
+
+PYTHON_IMAGE = "python:3.10"
+LOCAL_YAML_DIR_PATH = "airbyte_cdk/sources/declarative"
+LOCAL_OUTPUT_DIR_PATH = "airbyte_cdk/sources/declarative/models"
+
+
+PIP_DEPENDENCIES = [
+    "datamodel_code_generator==0.11.19",
+]
+
+
+def get_all_yaml_files_without_ext() -> list[str]:
+    return [Path(f).stem for f in glob(f"{LOCAL_YAML_DIR_PATH}/*.yaml")]
+
+
+def generate_init_module_content() -> str:
+    header = "# generated by bin/generate_component_manifest_files.py\n"
+    for module_name in get_all_yaml_files_without_ext():
+        header += f"from .{module_name} import *\n"
+    return header
+
+
+async def post_process_codegen(codegen_container: dagger.Container):
+    codegen_container = codegen_container.with_exec(["mkdir", "/generated_post_processed"])
+    for generated_file in await codegen_container.directory("/generated").entries():
+        if generated_file.endswith(".py"):
+            original_content = await codegen_container.file(f"/generated/{generated_file}").contents()
+            # the space before _parameters is intentional to avoid replacing things like `request_parameters:` with `requestparameters:`
+            post_processed_content = original_content.replace(" _parameters:", " parameters:")
+            codegen_container = codegen_container.with_new_file(
+                f"/generated_post_processed/{generated_file}", contents=post_processed_content
+            )
+    return codegen_container
+
+
+async def main():
+    init_module_content = generate_init_module_content()
+
+    async with dagger.Connection(dagger.Config(log_output=sys.stderr)) as dagger_client:
+
+        codegen_container = (
+            dagger_client.container()
+            .from_(PYTHON_IMAGE)
+            .with_exec(["mkdir", "/generated"])
+            .with_exec(["pip", "install", " ".join(PIP_DEPENDENCIES)])
+            .with_mounted_directory("/yaml", dagger_client.host().directory(LOCAL_YAML_DIR_PATH, include=["*.yaml"]))
+            .with_new_file("/generated/__init__.py", contents=init_module_content)
+        )
+        for yaml_file in get_all_yaml_files_without_ext():
+            codegen_container = codegen_container.with_exec(
+                [
+                    "datamodel-codegen",
+                    "--input",
+                    f"/yaml/{yaml_file}.yaml",
+                    "--output",
+                    f"/generated/{yaml_file}.py",
+                    "--disable-timestamp",
+                    "--enum-field-as-literal",
+                    "one",
+                    "--set-default-enum-member",
+                ]
+            )
+
+        await ((await post_process_codegen(codegen_container)).directory("/generated_post_processed").export(LOCAL_OUTPUT_DIR_PATH))
+
+
+anyio.run(main)