From c071fc54c5d2c8d571fb170c69f459bdb5fc1381 Mon Sep 17 00:00:00 2001 From: Benoit Moriceau Date: Fri, 6 Jan 2023 11:04:03 -0800 Subject: [PATCH] Convert the server to micronaut (#19194) * Extract Operation API * Extract scheduler API * Format * extract source api * Extract source definition api * Add path * Extract State API * extract webbackend api * extract webbackend api * extract workspace api * Extract source definition specification api * Remove configuration API * tmp * Checkstyle * tmp * tmp * Inject but don't resolve Bean * tmp * Tmp * fix build * TMP * Tmp * Clean up * better thread pool * Change port to 8080 * Fix port * Rm unused * Cors filter * Format * rename * Tmp * Config based * Rm health controller ref * tmp * Pool size * Mock healthcheck * Revert "Mock healthcheck" This reverts commit 466677636bda96bed7bbfd354a59abc38ff1e315. * Revert "Revert "Mock healthcheck"" This reverts commit 267094ae149f3c077bdba6f71646d99655ce81a0. * Restore health check * Tmp * format * Rm deprecated * Fix PMD * Tmp * Fix proxy test * Remove useless annotation * set auto commit as false * Clean up and PR comments * Bmoric/convert attempt micronaut (#19847) * tmp * Fix build * tmp * Tmp * tmp * tmp * Tmp * tmp * tmp * Clean up * Comments and banner * Non related files * rm tmp * Fix build * Format * Hit the micronaut server directly * micronaut OperationApiController (#20270) * micronaut OperationApiController * pass micronaut client to OperationApi * Bmoric/convert connection micronaut (#20211) * tmp * Fix build * tmp * Tmp * tmp * tmp * Tmp * tmp * tmp * Clean up * tmp * Convert Connection Api Controller * PR Comments * convert openapiapicontroller to micronaut (#20258) * convert openapiapicontroller to micronaut * merge health/openapi locations into one entry * Fix build * Format * Remove media type * Format Co-authored-by: Cole Snodgrass * Bmoric/convert destination controller micronaut (#20269) * tmp * Fix build * tmp * Tmp * tmp * tmp * Tmp * tmp * tmp * Clean up * tmp * Convert Connection Api Controller * Tmp * PR Comments * convert openapiapicontroller to micronaut (#20258) * convert openapiapicontroller to micronaut * merge health/openapi locations into one entry * Fix bean * Add JsonSchemaValidator as a Bean * Fix build * Format * Format * Test fix * Pr comments * Remove media type * Format * Remove media type * Format * format * Add missing airbyte api client Co-authored-by: Cole Snodgrass * Bmoric/convert destination definition controller micronaut (#20277) * tmp * Fix build * tmp * Tmp * tmp * tmp * Tmp * tmp * tmp * Clean up * tmp * Convert Connection Api Controller * Tmp * PR Comments * convert openapiapicontroller to micronaut (#20258) * convert openapiapicontroller to micronaut * merge health/openapi locations into one entry * Fix bean * Add JsonSchemaValidator as a Bean * Fix build * Format * Format * Test fix * Pr comments * Remove media type * Format * Remove media type * Format * Remove media type * Format * api client * missing annotation * format Co-authored-by: Cole Snodgrass * convert StateApiController to Micronaut (#20329) * convert to micronaut * nginx updates * format * Move dest oauth to micronaut (#20318) * tmp * Fix build * tmp * Tmp * tmp * tmp * Tmp * tmp * tmp * Clean up * tmp * Convert Connection Api Controller * PR Comments * convert openapiapicontroller to micronaut (#20258) * convert openapiapicontroller to micronaut * merge health/openapi locations into one entry * Fix build * Format * Remove media type * Format * Move dest oauth to micronaut * Pr comments * format Co-authored-by: Cole Snodgrass * Bmoric/convert source micronaut (#20334) * tmp * Fix build * tmp * Tmp * tmp * tmp * Tmp * tmp * tmp * Clean up * tmp * Convert Connection Api Controller * PR Comments * convert openapiapicontroller to micronaut (#20258) * convert openapiapicontroller to micronaut * merge health/openapi locations into one entry * Fix build * Format * Remove media type * Format * Tmp * tmp * Build * missing bean * format Co-authored-by: Cole Snodgrass * Migrate to micronaut (#20339) * Migrate source to micronaut * convert SchedulerApiController to Micronaut (#20337) * wip; SchedulerApiController * remove @Named * remove @Singleton * add back todo message * Bmoric/convert source definition micronaut (#20338) * tmp * Fix build * tmp * Tmp * tmp * tmp * Tmp * tmp * tmp * Clean up * tmp * Convert Connection Api Controller * PR Comments * convert openapiapicontroller to micronaut (#20258) * convert openapiapicontroller to micronaut * merge health/openapi locations into one entry * Fix build * Format * Remove media type * Format * Tmp * tmp * Build * missing bean * Tmp * Add Beans * fix Bean * Add passthrough * Clean up * Missing path * FIx typo * Fix conflicts * for mat Co-authored-by: Cole Snodgrass * update SourceOauthApiController to Micronaut (#20386) * convert SourceOauthApiController to Micronaut * remove SourceOauthApi reference * convert WorkspaceApiController to micronaut (#20214) * wip; broken * convert WorkspaceApiController to micronaut * remove test controller * format * format * add @Body to SourceOauthApiController * consolidate nginx settings * remove unnecessary factories * Bmoric/convert jobs micronaut (#20382) * Convert jobs to micronaut * Nit * Format * Bmoric/convert source definition specification micronaut (#20379) * Migrate source definition specifications to micronaut * Format * Format * convert database assert call to Micronaut (#20406) * remove dupe config section; add DatabaseEventListener * move eventlistner to correct package; update implementation * convert NotificationsApiController to Micronaut (#20396) * convert NotificationsApiController to Micronaut * format * Migrate logs to micronaut (#20400) * Bmoric/convert webbackend micronaut (#20403) * Convert jobs to micronaut * Nit * Format * Migrate the webbackend to micronaut * Add missing Bean * Cleanup (#20459) * Cleanup * More cleanup * Disable in order to test cloud * Restore missing files * Fix test * Format and fix pmd * Add transactional * Fix version * Tentative * Cleanup the cleanup * Rm reference to the micronaut server * format * pmd * more pmd * fix build * Delete logs API * Revert "Delete logs API" This reverts commit fcb271dcdf51d9904fa5d482e5c3e6984d5bc182. * Rm flaky test * Format * Try to fix test * Format * Remove optional * Rm import * Test sleep * Simplify injection * update import * Remove sleep * More injection * Remove more requirement * imports * Remove more requirement * Fix yaml * Remove unused conf * Add role * Test acceptance test * Update env * Revert "Update to Micronaut 3.8.0 (#20716)" This reverts commit a28f93747ffb7c8ad9e30a4a90ceada9d0d7dc0e. * Update helm chart * Fix helm chart * Convert Application Listener * Format * Add explicit deployment mode * Change check port * Update version and bump version to the right value * Cleanup * Update FE end to end test * Allow head request * Fix controller * Format * Fix http client Bean * Format Co-authored-by: Cole Snodgrass --- .env | 5 +- .../airbyte/api/client/AirbyteApiClient.java | 6 +- .../commons/temporal/TemporalClient.java | 3 - .../workers/config/ApiClientBeanFactory.java | 10 +- .../workers/helper/ConnectionHelper.java | 3 - .../features/EnvVariableFeatureFlags.java | 6 +- .../config/helpers/LogClientSingleton.java | 4 + .../config/persistence/ConfigRepository.java | 8 +- .../config/persistence/ConfigWriter.java | 3 +- .../persistence/StandardSyncPersistence.java | 3 +- .../StandardSyncPersistenceTest.java | 3 +- .../persistence/job/WorkspaceHelper.java | 4 +- airbyte-proxy/test.sh | 3 +- airbyte-server/build.gradle | 23 ++- .../java/io/airbyte/server/Application.java | 15 ++ .../airbyte/server/DatabaseEventListener.java | 50 ++++++ .../java/io/airbyte/server/ServerApp.java | 81 +++------ .../java/io/airbyte/server/ServerFactory.java | 163 +----------------- .../server/apis/AttemptApiController.java | 13 +- .../server/apis/ConnectionApiController.java | 35 ++-- .../server/apis/DestinationApiController.java | 33 ++-- .../DestinationDefinitionApiController.java | 24 ++- ...nDefinitionSpecificationApiController.java | 12 +- .../apis/DestinationOauthApiController.java | 16 +- .../server/apis/HealthApiController.java | 13 +- .../server/apis/JobsApiController.java | 20 ++- .../server/apis/LogsApiController.java | 14 +- .../server/apis/NotFoundController.java | 30 ++++ .../apis/NotificationsApiController.java | 15 +- .../server/apis/OpenapiApiController.java | 12 +- .../server/apis/OperationApiController.java | 30 ++-- .../server/apis/SchedulerApiController.java | 12 +- .../server/apis/SourceApiController.java | 23 ++- .../apis/SourceDefinitionApiController.java | 24 ++- ...eDefinitionSpecificationApiController.java | 12 +- .../server/apis/SourceOauthApiController.java | 21 ++- .../server/apis/StateApiController.java | 13 +- .../server/apis/WebBackendApiController.java | 23 ++- .../server/apis/WorkspaceApiController.java | 37 ++-- .../server/apis/binders/AttemptApiBinder.java | 21 --- .../apis/binders/ConnectionApiBinder.java | 21 --- .../apis/binders/DestinationApiBinder.java | 21 --- .../DestinationDefinitionApiBinder.java | 21 --- ...ationDefinitionSpecificationApiBinder.java | 21 --- .../binders/DestinationOauthApiBinder.java | 21 --- .../server/apis/binders/HealthApiBinder.java | 21 --- .../server/apis/binders/JobsApiBinder.java | 21 --- .../server/apis/binders/LogsApiBinder.java | 21 --- .../apis/binders/NotificationApiBinder.java | 21 --- .../server/apis/binders/OpenapiApiBinder.java | 21 --- .../apis/binders/OperationApiBinder.java | 21 --- .../apis/binders/SchedulerApiBinder.java | 21 --- .../server/apis/binders/SourceApiBinder.java | 21 --- .../binders/SourceDefinitionApiBinder.java | 21 --- ...ourceDefinitionSpecificationApiBinder.java | 21 --- .../apis/binders/SourceOauthApiBinder.java | 21 --- .../server/apis/binders/StateApiBinder.java | 21 --- .../apis/binders/WebBackendApiBinder.java | 21 --- .../apis/binders/WorkspaceApiBinder.java | 21 --- .../apis/factories/AttemptApiFactory.java | 35 ---- .../apis/factories/ConnectionApiFactory.java | 44 ----- .../apis/factories/DestinationApiFactory.java | 40 ----- .../DestinationDefinitionApiFactory.java | 29 ---- ...tionDefinitionSpecificationApiFactory.java | 29 ---- .../factories/DestinationOauthApiFactory.java | 29 ---- .../apis/factories/HealthApiFactory.java | 29 ---- .../server/apis/factories/JobsApiFactory.java | 32 ---- .../server/apis/factories/LogsApiFactory.java | 29 ---- .../factories/NotificationsApiFactory.java | 29 ---- .../apis/factories/OpenapiApiFactory.java | 29 ---- .../apis/factories/OperationApiFactory.java | 29 ---- .../apis/factories/SchedulerApiFactory.java | 29 ---- .../apis/factories/SourceApiFactory.java | 32 ---- .../factories/SourceDefinitionApiFactory.java | 29 ---- ...urceDefinitionSpecificationApiFactory.java | 29 ---- .../apis/factories/SourceOauthApiFactory.java | 29 ---- .../apis/factories/StateApiFactory.java | 29 ---- .../apis/factories/WebBackendApiFactory.java | 38 ---- .../apis/factories/WorkspaceApiFactory.java | 29 ---- .../server/config/ApplicationBeanFactory.java | 116 +++++++++++++ .../config/CloudStorageBeanFactory.java | 92 ++++++++++ .../server/config/DatabaseBeanFactory.java | 125 ++++++++++++++ .../server/config/HelperBeanFactory.java | 21 +++ .../config/JobErrorReportingBeanFactory.java | 61 +++++++ .../config/SecretPersistenceBeanFactory.java | 114 ++++++++++++ .../server/config/TemporalBeanFactory.java | 67 +++++++ .../converters/ConfigurationUpdate.java | 2 + .../server/converters/JobConverter.java | 6 +- .../converters/WorkflowStateConverter.java | 2 + .../errors/IdNotFoundExceptionMapper.java | 35 ++++ .../errors/InvalidInputExceptionMapper.java | 26 +-- .../errors/InvalidJsonExceptionMapper.java | 27 +-- .../InvalidJsonInputExceptionMapper.java | 29 ++-- .../server/errors/KnownExceptionMapper.java | 8 +- .../errors/NotFoundExceptionMapper.java | 30 ++-- .../errors/UncaughtExceptionMapper.java | 31 ++-- .../io/airbyte/server/filter/CorsFilter.java | 35 ++++ .../server/handlers/AttemptHandler.java | 8 +- .../server/handlers/ConnectionsHandler.java | 3 + .../DestinationDefinitionsHandler.java | 21 +-- .../server/handlers/DestinationHandler.java | 4 + .../server/handlers/HealthCheckHandler.java | 5 +- .../server/handlers/JobHistoryHandler.java | 2 + .../airbyte/server/handlers/LogsHandler.java | 32 +++- .../airbyte/server/handlers/OAuthHandler.java | 2 + .../server/handlers/OpenApiConfigHandler.java | 2 + .../server/handlers/OperationsHandler.java | 12 +- .../server/handlers/SchedulerHandler.java | 28 +-- .../handlers/SourceDefinitionsHandler.java | 20 +-- .../server/handlers/SourceHandler.java | 4 + .../airbyte/server/handlers/StateHandler.java | 2 + .../WebBackendCheckUpdatesHandler.java | 9 +- .../WebBackendConnectionsHandler.java | 36 ++-- .../WebBackendGeographiesHandler.java | 6 +- .../server/handlers/WorkspacesHandler.java | 4 + .../src/main/resources/application.yml | 130 ++++++++++++++ .../banner.txt => micronaut-banner.txt} | 7 +- .../DestinationDefinitionsHandlerTest.java | 4 +- .../server/handlers/SchedulerHandlerTest.java | 9 +- .../SourceDefinitionsHandlerTest.java | 7 +- .../acceptance/AdvancedAcceptanceTests.java | 28 --- .../test/acceptance/BasicAcceptanceTests.java | 2 +- .../workers/config/TemporalBeanFactory.java | 13 +- .../airbyte-server/templates/deployment.yaml | 30 ++++ deps.toml | 3 + docker-compose.yaml | 1 + kube/resources/server.yaml | 30 ++++ 127 files changed, 1585 insertions(+), 1578 deletions(-) create mode 100644 airbyte-server/src/main/java/io/airbyte/server/Application.java create mode 100644 airbyte-server/src/main/java/io/airbyte/server/DatabaseEventListener.java create mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/NotFoundController.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/AttemptApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/ConnectionApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/DestinationApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/DestinationDefinitionApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/DestinationDefinitionSpecificationApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/DestinationOauthApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/HealthApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/JobsApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/LogsApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/NotificationApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/OpenapiApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/OperationApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/SchedulerApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/SourceApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/SourceDefinitionApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/SourceDefinitionSpecificationApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/SourceOauthApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/StateApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/WebBackendApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/binders/WorkspaceApiBinder.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/AttemptApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/ConnectionApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/DestinationApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/DestinationDefinitionApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/DestinationDefinitionSpecificationApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/DestinationOauthApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/HealthApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/JobsApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/LogsApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/NotificationsApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/OpenapiApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/OperationApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/SchedulerApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/SourceApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/SourceDefinitionApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/SourceDefinitionSpecificationApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/SourceOauthApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/StateApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/WebBackendApiFactory.java delete mode 100644 airbyte-server/src/main/java/io/airbyte/server/apis/factories/WorkspaceApiFactory.java create mode 100644 airbyte-server/src/main/java/io/airbyte/server/config/ApplicationBeanFactory.java create mode 100644 airbyte-server/src/main/java/io/airbyte/server/config/CloudStorageBeanFactory.java create mode 100644 airbyte-server/src/main/java/io/airbyte/server/config/DatabaseBeanFactory.java create mode 100644 airbyte-server/src/main/java/io/airbyte/server/config/HelperBeanFactory.java create mode 100644 airbyte-server/src/main/java/io/airbyte/server/config/JobErrorReportingBeanFactory.java create mode 100644 airbyte-server/src/main/java/io/airbyte/server/config/SecretPersistenceBeanFactory.java create mode 100644 airbyte-server/src/main/java/io/airbyte/server/config/TemporalBeanFactory.java create mode 100644 airbyte-server/src/main/java/io/airbyte/server/errors/IdNotFoundExceptionMapper.java create mode 100644 airbyte-server/src/main/java/io/airbyte/server/filter/CorsFilter.java create mode 100644 airbyte-server/src/main/resources/application.yml rename airbyte-server/src/main/resources/{banner/banner.txt => micronaut-banner.txt} (54%) diff --git a/.env b/.env index 32c5ba3c4952..79742071a91d 100644 --- a/.env +++ b/.env @@ -47,13 +47,13 @@ DATABASE_PORT=5432 DATABASE_DB=airbyte # translate manually DATABASE_URL=jdbc:postgresql://${DATABASE_HOST}:${DATABASE_PORT}/${DATABASE_DB} (do not include the username or password here) DATABASE_URL=jdbc:postgresql://db:5432/airbyte -JOBS_DATABASE_MINIMUM_FLYWAY_MIGRATION_VERSION=0.29.15.001 +JOBS_DATABASE_MINIMUM_FLYWAY_MIGRATION_VERSION=0.40.26.001 # Airbyte Internal Config Database, defaults to Job Database if empty. Explicitly left empty to mute docker compose warnings. CONFIG_DATABASE_USER= CONFIG_DATABASE_PASSWORD= CONFIG_DATABASE_URL= -CONFIGS_DATABASE_MINIMUM_FLYWAY_MIGRATION_VERSION=0.35.15.001 +CONFIGS_DATABASE_MINIMUM_FLYWAY_MIGRATION_VERSION=0.40.23.002 ### AIRBYTE SERVICES ### TEMPORAL_HOST=airbyte-temporal:7233 @@ -115,4 +115,3 @@ OTEL_COLLECTOR_ENDPOINT="http://host.docker.internal:4317" USE_STREAM_CAPABLE_STATE=true AUTO_DETECT_SCHEMA=false - diff --git a/airbyte-api/src/main/java/io/airbyte/api/client/AirbyteApiClient.java b/airbyte-api/src/main/java/io/airbyte/api/client/AirbyteApiClient.java index ce1fd820d9e5..08f2c74c03b9 100644 --- a/airbyte-api/src/main/java/io/airbyte/api/client/AirbyteApiClient.java +++ b/airbyte-api/src/main/java/io/airbyte/api/client/AirbyteApiClient.java @@ -28,12 +28,12 @@ * This class is meant to consolidate all our API endpoints into a fluent-ish client. Currently, all * open API generators create a separate class per API "root-route". For example, if our API has two * routes "/v1/First/get" and "/v1/Second/get", OpenAPI generates (essentially) the following files: - * + *

* ApiClient.java, FirstApi.java, SecondApi.java - * + *

* To call the API type-safely, we'd do new FirstApi(new ApiClient()).get() or new SecondApi(new * ApiClient()).get(), which can get cumbersome if we're interacting with many pieces of the API. - * + *

* This is currently manually maintained. We could look into autogenerating it if needed. */ public class AirbyteApiClient { diff --git a/airbyte-commons-temporal/src/main/java/io/airbyte/commons/temporal/TemporalClient.java b/airbyte-commons-temporal/src/main/java/io/airbyte/commons/temporal/TemporalClient.java index 3c8ec42e4985..2d01a0a5c803 100644 --- a/airbyte-commons-temporal/src/main/java/io/airbyte/commons/temporal/TemporalClient.java +++ b/airbyte-commons-temporal/src/main/java/io/airbyte/commons/temporal/TemporalClient.java @@ -8,7 +8,6 @@ import com.google.common.annotations.VisibleForTesting; import com.google.protobuf.ByteString; -import io.airbyte.commons.temporal.config.WorkerMode; import io.airbyte.commons.temporal.exception.DeletedWorkflowException; import io.airbyte.commons.temporal.exception.UnreachableWorkflowException; import io.airbyte.commons.temporal.scheduling.CheckConnectionWorkflow; @@ -30,7 +29,6 @@ import io.airbyte.persistence.job.models.IntegrationLauncherConfig; import io.airbyte.persistence.job.models.JobRunConfig; import io.airbyte.protocol.models.StreamDescriptor; -import io.micronaut.context.annotation.Requires; import io.temporal.api.common.v1.WorkflowType; import io.temporal.api.enums.v1.WorkflowExecutionStatus; import io.temporal.api.workflowservice.v1.ListClosedWorkflowExecutionsRequest; @@ -62,7 +60,6 @@ @Slf4j @Singleton -@Requires(env = WorkerMode.CONTROL_PLANE) public class TemporalClient { /** diff --git a/airbyte-commons-worker/src/main/java/io/airbyte/workers/config/ApiClientBeanFactory.java b/airbyte-commons-worker/src/main/java/io/airbyte/workers/config/ApiClientBeanFactory.java index cd0b7b4c1934..d7e5d80407c0 100644 --- a/airbyte-commons-worker/src/main/java/io/airbyte/workers/config/ApiClientBeanFactory.java +++ b/airbyte-commons-worker/src/main/java/io/airbyte/workers/config/ApiClientBeanFactory.java @@ -40,11 +40,13 @@ public class ApiClientBeanFactory { private static final int JWT_TTL_MINUTES = 5; @Singleton - public ApiClient apiClient(@Value("${airbyte.internal.api.auth-header.name}") final String airbyteApiAuthHeaderName, + @Named("apiClient") + public ApiClient apiClient( + @Value("${airbyte.internal.api.auth-header.name}") final String airbyteApiAuthHeaderName, @Value("${airbyte.internal.api.host}") final String airbyteApiHost, @Named("internalApiAuthToken") final BeanProvider internalApiAuthToken, @Named("internalApiScheme") final String internalApiScheme) { - return new io.airbyte.api.client.invoker.generated.ApiClient() + return new ApiClient() .setScheme(internalApiScheme) .setHost(parseHostName(airbyteApiHost)) .setPort(parsePort(airbyteApiHost)) @@ -66,7 +68,7 @@ public AirbyteApiClient airbyteApiClient(final ApiClient apiClient) { } @Singleton - public SourceApi sourceApi(final ApiClient apiClient) { + public SourceApi sourceApi(@Named("apiClient") final ApiClient apiClient) { return new SourceApi(apiClient); } @@ -87,7 +89,7 @@ public WorkspaceApi workspaceApi(final ApiClient apiClient) { @Singleton public HttpClient httpClient() { - return HttpClient.newHttpClient(); + return HttpClient.newBuilder().version(HttpClient.Version.HTTP_1_1).build(); } @Singleton diff --git a/airbyte-commons-worker/src/main/java/io/airbyte/workers/helper/ConnectionHelper.java b/airbyte-commons-worker/src/main/java/io/airbyte/workers/helper/ConnectionHelper.java index 628b2b2f070f..832b1c30392f 100644 --- a/airbyte-commons-worker/src/main/java/io/airbyte/workers/helper/ConnectionHelper.java +++ b/airbyte-commons-worker/src/main/java/io/airbyte/workers/helper/ConnectionHelper.java @@ -7,7 +7,6 @@ import com.google.common.base.Preconditions; import io.airbyte.commons.enums.Enums; import io.airbyte.commons.json.Jsons; -import io.airbyte.commons.temporal.config.WorkerMode; import io.airbyte.config.BasicSchedule; import io.airbyte.config.JobSyncConfig.NamespaceDefinitionType; import io.airbyte.config.Schedule; @@ -18,7 +17,6 @@ import io.airbyte.config.persistence.ConfigRepository; import io.airbyte.persistence.job.WorkspaceHelper; import io.airbyte.validation.json.JsonValidationException; -import io.micronaut.context.annotation.Requires; import jakarta.inject.Singleton; import java.io.IOException; import java.util.List; @@ -28,7 +26,6 @@ // todo (cgardens) - we are not getting any value out of instantiating this class. we should just // use it as statics. not doing it now, because already in the middle of another refactor. @Singleton -@Requires(env = WorkerMode.CONTROL_PLANE) public class ConnectionHelper { private final ConfigRepository configRepository; diff --git a/airbyte-commons/src/main/java/io/airbyte/commons/features/EnvVariableFeatureFlags.java b/airbyte-commons/src/main/java/io/airbyte/commons/features/EnvVariableFeatureFlags.java index 2038fd893459..4302875bee55 100644 --- a/airbyte-commons/src/main/java/io/airbyte/commons/features/EnvVariableFeatureFlags.java +++ b/airbyte-commons/src/main/java/io/airbyte/commons/features/EnvVariableFeatureFlags.java @@ -5,11 +5,13 @@ package io.airbyte.commons.features; import java.util.function.Function; -import lombok.extern.slf4j.Slf4j; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; -@Slf4j public class EnvVariableFeatureFlags implements FeatureFlags { + private static final Logger log = LoggerFactory.getLogger(EnvVariableFeatureFlags.class); + public static final String USE_STREAM_CAPABLE_STATE = "USE_STREAM_CAPABLE_STATE"; public static final String AUTO_DETECT_SCHEMA = "AUTO_DETECT_SCHEMA"; // Set this value to true to see all messages from the source to destination, set to one second diff --git a/airbyte-config/config-models/src/main/java/io/airbyte/config/helpers/LogClientSingleton.java b/airbyte-config/config-models/src/main/java/io/airbyte/config/helpers/LogClientSingleton.java index 0ef0dd211e2d..eb9ad5e7e5ad 100644 --- a/airbyte-config/config-models/src/main/java/io/airbyte/config/helpers/LogClientSingleton.java +++ b/airbyte-config/config-models/src/main/java/io/airbyte/config/helpers/LogClientSingleton.java @@ -82,6 +82,7 @@ public File getServerLogFile(final Path workspaceRoot, final WorkerEnvironment w } final var cloudLogPath = sanitisePath(APP_LOGGING_CLOUD_PREFIX, getServerLogsRoot(workspaceRoot)); try { + createCloudClientIfNull(logConfigs); return logClient.downloadCloudLog(logConfigs, cloudLogPath); } catch (final IOException e) { throw new RuntimeException("Error retrieving log file: " + cloudLogPath + " from S3", e); @@ -95,6 +96,7 @@ public File getSchedulerLogFile(final Path workspaceRoot, final WorkerEnvironmen final var cloudLogPath = APP_LOGGING_CLOUD_PREFIX + getSchedulerLogsRoot(workspaceRoot); try { + createCloudClientIfNull(logConfigs); return logClient.downloadCloudLog(logConfigs, cloudLogPath); } catch (final IOException e) { throw new RuntimeException("Error retrieving log file: " + cloudLogPath + " from S3", e); @@ -111,6 +113,7 @@ public List getJobLogFile(final WorkerEnvironment workerEnvironment, fin } final var cloudLogPath = sanitisePath(JOB_LOGGING_CLOUD_PREFIX, logPath); + createCloudClientIfNull(logConfigs); return logClient.tailCloudLog(logConfigs, cloudLogPath, LOG_TAIL_SIZE); } @@ -127,6 +130,7 @@ public void deleteLogs(final WorkerEnvironment workerEnvironment, final LogConfi throw new NotImplementedException("Local log deletes not supported."); } final var cloudLogPath = sanitisePath(JOB_LOGGING_CLOUD_PREFIX, Path.of(logPath)); + createCloudClientIfNull(logConfigs); logClient.deleteLogs(logConfigs, cloudLogPath); } diff --git a/airbyte-config/config-persistence/src/main/java/io/airbyte/config/persistence/ConfigRepository.java b/airbyte-config/config-persistence/src/main/java/io/airbyte/config/persistence/ConfigRepository.java index 78c111a6bbf8..554ccf031167 100644 --- a/airbyte-config/config-persistence/src/main/java/io/airbyte/config/persistence/ConfigRepository.java +++ b/airbyte-config/config-persistence/src/main/java/io/airbyte/config/persistence/ConfigRepository.java @@ -126,7 +126,7 @@ public ConfigRepository(final Database database) { */ public boolean healthCheck() { try { - database.query(ctx -> ctx.select(WORKSPACE.ID).from(WORKSPACE).limit(1).fetch()); + database.query(ctx -> ctx.select(WORKSPACE.ID).from(WORKSPACE).limit(1).fetch()).stream().count(); } catch (final Exception e) { LOGGER.error("Health check error: ", e); return false; @@ -294,7 +294,8 @@ private Stream sourceDefQuery(final Optional sou .where(ACTOR_DEFINITION.ACTOR_TYPE.eq(ActorType.source)) .and(sourceDefId.map(ACTOR_DEFINITION.ID::eq).orElse(noCondition())) .and(includeTombstone ? noCondition() : ACTOR_DEFINITION.TOMBSTONE.notEqual(true)) - .fetchStream()) + .fetch()) + .stream() .map(DbConverter::buildStandardSourceDefinition) // Ensure version is set. Needed for connectors not upgraded since we added versioning. .map(def -> def.withProtocolVersion(AirbyteProtocolVersion.getWithDefault(def.getProtocolVersion()).serialize())); @@ -356,7 +357,8 @@ private Stream destDefQuery(final Optional .where(ACTOR_DEFINITION.ACTOR_TYPE.eq(ActorType.destination)) .and(destDefId.map(ACTOR_DEFINITION.ID::eq).orElse(noCondition())) .and(includeTombstone ? noCondition() : ACTOR_DEFINITION.TOMBSTONE.notEqual(true)) - .fetchStream()) + .fetch()) + .stream() .map(DbConverter::buildStandardDestinationDefinition) // Ensure version is set. Needed for connectors not upgraded since we added versioning. .map(def -> def.withProtocolVersion(AirbyteProtocolVersion.getWithDefault(def.getProtocolVersion()).serialize())); diff --git a/airbyte-config/config-persistence/src/main/java/io/airbyte/config/persistence/ConfigWriter.java b/airbyte-config/config-persistence/src/main/java/io/airbyte/config/persistence/ConfigWriter.java index 00640eadfd9b..8d8ea240a2f1 100644 --- a/airbyte-config/config-persistence/src/main/java/io/airbyte/config/persistence/ConfigWriter.java +++ b/airbyte-config/config-persistence/src/main/java/io/airbyte/config/persistence/ConfigWriter.java @@ -68,7 +68,8 @@ private static Stream> getActorDefiniti return ctx.select(ACTOR_DEFINITION.ID, ACTOR_DEFINITION.DOCKER_REPOSITORY, ACTOR_DEFINITION.ACTOR_TYPE, ACTOR_DEFINITION.PROTOCOL_VERSION) .from(ACTOR_DEFINITION) .join(ACTOR).on(ACTOR.ACTOR_DEFINITION_ID.equal(ACTOR_DEFINITION.ID)) - .fetchStream(); + .fetch() + .stream(); } static void writeStandardSourceDefinition(final List configs, final DSLContext ctx) { diff --git a/airbyte-config/config-persistence/src/main/java/io/airbyte/config/persistence/StandardSyncPersistence.java b/airbyte-config/config-persistence/src/main/java/io/airbyte/config/persistence/StandardSyncPersistence.java index 10b1d7a53a90..01f91cfe8e8a 100644 --- a/airbyte-config/config-persistence/src/main/java/io/airbyte/config/persistence/StandardSyncPersistence.java +++ b/airbyte-config/config-persistence/src/main/java/io/airbyte/config/persistence/StandardSyncPersistence.java @@ -289,7 +289,8 @@ private Stream findDisabledSyncs(final DSLC .where( CONNECTION.UNSUPPORTED_PROTOCOL_VERSION.eq(true).and( (actorType == ActorType.DESTINATION ? destDef : sourceDef).ID.eq(actorDefId))) - .fetchStream() + .fetch() + .stream() .map(r -> new StandardSyncIdsWithProtocolVersions( r.get(CONNECTION.ID), r.get(sourceDef.ID), diff --git a/airbyte-config/config-persistence/src/test/java/io/airbyte/config/persistence/StandardSyncPersistenceTest.java b/airbyte-config/config-persistence/src/test/java/io/airbyte/config/persistence/StandardSyncPersistenceTest.java index 332429c88f59..cbfb4dfd8b0e 100644 --- a/airbyte-config/config-persistence/src/test/java/io/airbyte/config/persistence/StandardSyncPersistenceTest.java +++ b/airbyte-config/config-persistence/src/test/java/io/airbyte/config/persistence/StandardSyncPersistenceTest.java @@ -237,7 +237,8 @@ private Set getProtocolVersionFlagForSyncs(fina .select(CONNECTION.ID, CONNECTION.UNSUPPORTED_PROTOCOL_VERSION) .from(CONNECTION) .where(CONNECTION.ID.in(standardSync.stream().map(StandardSync::getConnectionId).toList())) - .fetchStream()) + .fetch()) + .stream() .map(r -> new StandardSyncProtocolVersionFlag(r.get(CONNECTION.ID), r.get(CONNECTION.UNSUPPORTED_PROTOCOL_VERSION))) .collect(Collectors.toSet()); } diff --git a/airbyte-persistence/job-persistence/src/main/java/io/airbyte/persistence/job/WorkspaceHelper.java b/airbyte-persistence/job-persistence/src/main/java/io/airbyte/persistence/job/WorkspaceHelper.java index 66b24c82b25b..ceb76e1f0730 100644 --- a/airbyte-persistence/job-persistence/src/main/java/io/airbyte/persistence/job/WorkspaceHelper.java +++ b/airbyte-persistence/job-persistence/src/main/java/io/airbyte/persistence/job/WorkspaceHelper.java @@ -18,6 +18,7 @@ import io.airbyte.config.persistence.ConfigRepository; import io.airbyte.persistence.job.models.Job; import io.airbyte.validation.json.JsonValidationException; +import jakarta.inject.Singleton; import java.io.IOException; import java.util.Objects; import java.util.UUID; @@ -30,6 +31,7 @@ // scheduler:persistence in order to get workspace ids for configs (e.g. source). Our options are to // split this helper by database or put it in a new module. @SuppressWarnings("PMD.AvoidCatchingThrowable") +@Singleton public class WorkspaceHelper { private static final Logger LOGGER = LoggerFactory.getLogger(WorkspaceHelper.class); @@ -106,7 +108,7 @@ public UUID load(@NonNull final Long jobId) throws ConfigNotFoundException, IOEx * There are generally two kinds of helper methods present here. The first kind propagate exceptions * for the method backing the cache. The second ignores them. The former is meant to be used with * proper api calls, while the latter is meant to be use with asserts and precondtions checks. - * + *

* In API calls, distinguishing between various exceptions helps return the correct status code. */ diff --git a/airbyte-proxy/test.sh b/airbyte-proxy/test.sh index 3b97c60df974..c3e66d327361 100755 --- a/airbyte-proxy/test.sh +++ b/airbyte-proxy/test.sh @@ -19,7 +19,8 @@ function start_container () { } function start_container_with_proxy () { - CMD="docker run -d -p $PORT:8000 --env PROXY_PASS_WEB=$1 --env PROXY_PASS_API=$1 --name $NAME airbyte/proxy:$VERSION" + CMD="docker run -d -p $PORT:8000 --env PROXY_PASS_WEB=$1 --env PROXY_PASS_API=$1 --name $NAME + airbyte/proxy:$VERSION" echo $CMD eval $CMD wait_for_docker; diff --git a/airbyte-server/build.gradle b/airbyte-server/build.gradle index da2c5657f654..aa00568bb652 100644 --- a/airbyte-server/build.gradle +++ b/airbyte-server/build.gradle @@ -4,7 +4,6 @@ plugins { configurations.all { exclude group: 'io.micronaut.jaxrs' - exclude group: 'io.micronaut.sql' } dependencies { @@ -25,7 +24,6 @@ dependencies { implementation project(':airbyte-protocol:protocol-models') implementation project(':airbyte-persistence:job-persistence') - implementation libs.flyway.core implementation 'com.github.slugify:slugify:2.4' implementation 'commons-cli:commons-cli:1.4' implementation libs.temporal.sdk @@ -38,12 +36,28 @@ dependencies { implementation 'org.glassfish.jersey.media:jersey-media-json-jackson' implementation 'org.glassfish.jersey.ext:jersey-bean-validation' implementation 'org.quartz-scheduler:quartz:2.3.2' + implementation 'io.sentry:sentry:6.3.1' + implementation 'io.swagger:swagger-annotations:1.6.2' + annotationProcessor platform(libs.micronaut.bom) + annotationProcessor libs.bundles.micronaut.annotation.processor + annotationProcessor libs.micronaut.jaxrs.processor + + implementation platform(libs.micronaut.bom) + implementation libs.bundles.micronaut + implementation libs.micronaut.jaxrs.server + + // Ensure that the versions defined in deps.toml are used + // instead of versions from transitive dependencies + implementation(libs.flyway.core) { + force = true + } testImplementation project(':airbyte-test-utils') testImplementation libs.postgresql testImplementation libs.platform.testcontainers.postgresql testImplementation 'com.squareup.okhttp3:mockwebserver:4.9.1' + testImplementation 'org.mockito:mockito-inline:4.7.0' } // we want to be able to access the generated db files from config/init when we build the server docker image. @@ -57,9 +71,10 @@ task copySeed(type: Copy, dependsOn: [project(':airbyte-config:init').processRes test.dependsOn(project.tasks.copySeed) assemble.dependsOn(project.tasks.copySeed) -mainClassName = 'io.airbyte.server.ServerApp' +mainClassName = 'io.airbyte.server.Application' application { + applicationName = project.name mainClass = mainClassName applicationDefaultJvmArgs = ['-XX:+ExitOnOutOfMemoryError', '-XX:MaxRAMPercentage=75.0'] } @@ -87,6 +102,8 @@ run { environment "AIRBYTE_VERSION", env.VERSION environment "AIRBYTE_ROLE", System.getenv('AIRBYTE_ROLE') environment "TEMPORAL_HOST", "localhost:7233" + + environment 'MICRONAUT_ENVIRONMENTS', 'control-plane' } // produce reproducible archives diff --git a/airbyte-server/src/main/java/io/airbyte/server/Application.java b/airbyte-server/src/main/java/io/airbyte/server/Application.java new file mode 100644 index 000000000000..974d907d097d --- /dev/null +++ b/airbyte-server/src/main/java/io/airbyte/server/Application.java @@ -0,0 +1,15 @@ +/* + * Copyright (c) 2022 Airbyte, Inc., all rights reserved. + */ + +package io.airbyte.server; + +import io.micronaut.runtime.Micronaut; + +public class Application { + + public static void main(final String[] args) { + Micronaut.run(Application.class, args); + } + +} diff --git a/airbyte-server/src/main/java/io/airbyte/server/DatabaseEventListener.java b/airbyte-server/src/main/java/io/airbyte/server/DatabaseEventListener.java new file mode 100644 index 000000000000..0e24c3de184b --- /dev/null +++ b/airbyte-server/src/main/java/io/airbyte/server/DatabaseEventListener.java @@ -0,0 +1,50 @@ +/* + * Copyright (c) 2022 Airbyte, Inc., all rights reserved. + */ + +package io.airbyte.server; + +import io.airbyte.db.check.DatabaseCheckException; +import io.airbyte.db.check.DatabaseMigrationCheck; +import io.micronaut.context.event.ApplicationEventListener; +import io.micronaut.discovery.event.ServiceReadyEvent; +import jakarta.inject.Named; +import jakarta.inject.Singleton; +import java.lang.invoke.MethodHandles; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +@Singleton +public class DatabaseEventListener implements ApplicationEventListener { + + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + + private final DatabaseMigrationCheck configsMigrationCheck; + + private final DatabaseMigrationCheck jobsMigrationCheck; + + public DatabaseEventListener( + @Named("configsDatabaseMigrationCheck") final DatabaseMigrationCheck configsMigrationCheck, + @Named("jobsDatabaseMigrationCheck") final DatabaseMigrationCheck jobsMigrationCheck) { + this.configsMigrationCheck = configsMigrationCheck; + this.jobsMigrationCheck = jobsMigrationCheck; + } + + @Override + public void onApplicationEvent(final ServiceReadyEvent event) { + log.info("Checking configs database flyway migration version..."); + try { + configsMigrationCheck.check(); + } catch (final DatabaseCheckException e) { + throw new RuntimeException(e); + } + + log.info("Checking jobs database flyway migration version..."); + try { + jobsMigrationCheck.check(); + } catch (final DatabaseCheckException e) { + throw new RuntimeException(e); + } + } + +} diff --git a/airbyte-server/src/main/java/io/airbyte/server/ServerApp.java b/airbyte-server/src/main/java/io/airbyte/server/ServerApp.java index 7078edf375d3..e1cb903d05d1 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/ServerApp.java +++ b/airbyte-server/src/main/java/io/airbyte/server/ServerApp.java @@ -8,16 +8,16 @@ import io.airbyte.analytics.TrackingClient; import io.airbyte.analytics.TrackingClientSingleton; import io.airbyte.commons.features.EnvVariableFeatureFlags; -import io.airbyte.commons.lang.CloseableShutdownHook; +import io.airbyte.commons.features.FeatureFlags; import io.airbyte.commons.resources.MoreResources; import io.airbyte.commons.temporal.ConnectionManagerUtils; import io.airbyte.commons.temporal.StreamResetRecordsHelper; import io.airbyte.commons.temporal.TemporalClient; import io.airbyte.commons.temporal.TemporalUtils; import io.airbyte.commons.temporal.TemporalWorkflowUtils; +import io.airbyte.commons.version.AirbyteProtocolVersionRange; import io.airbyte.commons.version.AirbyteVersion; import io.airbyte.config.Configs; -import io.airbyte.config.EnvConfigs; import io.airbyte.config.helpers.LogClientSingleton; import io.airbyte.config.persistence.ConfigRepository; import io.airbyte.config.persistence.SecretsRepositoryReader; @@ -28,12 +28,7 @@ import io.airbyte.config.persistence.split_secrets.SecretsHydrator; import io.airbyte.db.Database; import io.airbyte.db.check.DatabaseCheckException; -import io.airbyte.db.factory.DSLContextFactory; -import io.airbyte.db.factory.DataSourceFactory; import io.airbyte.db.factory.DatabaseCheckFactory; -import io.airbyte.db.factory.FlywayFactory; -import io.airbyte.db.instance.configs.ConfigsDatabaseMigrator; -import io.airbyte.db.instance.jobs.JobsDatabaseMigrator; import io.airbyte.persistence.job.DefaultJobPersistence; import io.airbyte.persistence.job.JobPersistence; import io.airbyte.persistence.job.WebUrlHelper; @@ -57,7 +52,6 @@ import io.airbyte.server.handlers.JobHistoryHandler; import io.airbyte.server.handlers.LogsHandler; import io.airbyte.server.handlers.OAuthHandler; -import io.airbyte.server.handlers.OpenApiConfigHandler; import io.airbyte.server.handlers.OperationsHandler; import io.airbyte.server.handlers.SchedulerHandler; import io.airbyte.server.handlers.SourceDefinitionsHandler; @@ -78,7 +72,7 @@ import java.util.Map; import java.util.Optional; import java.util.Set; -import javax.sql.DataSource; +import java.util.UUID; import org.eclipse.jetty.server.Server; import org.eclipse.jetty.servlet.ServletContextHandler; import org.eclipse.jetty.servlet.ServletHolder; @@ -87,7 +81,6 @@ import org.glassfish.jersey.server.ResourceConfig; import org.glassfish.jersey.servlet.ServletContainer; import org.jooq.DSLContext; -import org.jooq.SQLDialect; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.slf4j.MDC; @@ -97,7 +90,6 @@ public class ServerApp implements ServerRunnable { private static final Logger LOGGER = LoggerFactory.getLogger(ServerApp.class); private static final int PORT = 8001; - private static final String DRIVER_CLASS_NAME = "org.postgresql.Driver"; private final AirbyteVersion airbyteVersion; private final Set> customComponentClasses; @@ -158,11 +150,11 @@ public void start() throws Exception { })); } - private static void assertDatabasesReady(final Configs configs, - final DSLContext configsDslContext, - final Flyway configsFlyway, - final DSLContext jobsDslContext, - final Flyway jobsFlyway) + public static void assertDatabasesReady(final Configs configs, + final DSLContext configsDslContext, + final Flyway configsFlyway, + final DSLContext jobsDslContext, + final Flyway jobsFlyway) throws DatabaseCheckException { LOGGER.info("Checking configs database flyway migration version.."); DatabaseCheckFactory @@ -197,8 +189,8 @@ public static ServerRunnable getServer(final ServerFactory apiFactory, final Optional ephemeralSecretPersistence = SecretPersistence.getEphemeral(configsDslContext, configs); final ConfigRepository configRepository = new ConfigRepository(configsDatabase); final SecretsRepositoryReader secretsRepositoryReader = new SecretsRepositoryReader(configRepository, secretsHydrator); - final SecretsRepositoryWriter secretsRepositoryWriter = - new SecretsRepositoryWriter(configRepository, secretPersistence, ephemeralSecretPersistence); + final SecretsRepositoryWriter secretsRepositoryWriter = new SecretsRepositoryWriter(configRepository, secretPersistence, + ephemeralSecretPersistence); LOGGER.info("Creating jobs persistence..."); final Database jobsDatabase = new Database(jobsDslContext); @@ -214,7 +206,7 @@ public static ServerRunnable getServer(final ServerFactory apiFactory, final TrackingClient trackingClient = TrackingClientSingleton.get(); final JobTracker jobTracker = new JobTracker(configRepository, jobPersistence, trackingClient); - final EnvVariableFeatureFlags envVariableFeatureFlags = new EnvVariableFeatureFlags(); + final FeatureFlags envVariableFeatureFlags = new EnvVariableFeatureFlags(); final WebUrlHelper webUrlHelper = new WebUrlHelper(configs.getWebappUrl()); final JobErrorReportingClient jobErrorReportingClient = JobErrorReportingClientFactory.getClient(configs.getJobErrorReportingStrategy(), configs); @@ -291,8 +283,17 @@ public static ServerRunnable getServer(final ServerFactory apiFactory, connectionsHandler, envVariableFeatureFlags); - final DestinationDefinitionsHandler destinationDefinitionsHandler = new DestinationDefinitionsHandler(configRepository, syncSchedulerClient, - destinationHandler); + final AirbyteProtocolVersionRange airbyteProtocolVersionRange = new AirbyteProtocolVersionRange(configs.getAirbyteProtocolVersionMin(), + configs.getAirbyteProtocolVersionMax()); + + final AirbyteGithubStore airbyteGithubStore = AirbyteGithubStore.production(); + + final DestinationDefinitionsHandler destinationDefinitionsHandler = new DestinationDefinitionsHandler(configRepository, + () -> UUID.randomUUID(), + syncSchedulerClient, + airbyteGithubStore, + destinationHandler, + airbyteProtocolVersionRange); final HealthCheckHandler healthCheckHandler = new HealthCheckHandler(configRepository); @@ -306,7 +307,9 @@ public static ServerRunnable getServer(final ServerFactory apiFactory, connectionsHandler, oAuthConfigSupplier); - final SourceDefinitionsHandler sourceDefinitionsHandler = new SourceDefinitionsHandler(configRepository, syncSchedulerClient, sourceHandler); + final SourceDefinitionsHandler sourceDefinitionsHandler = + new SourceDefinitionsHandler(configRepository, () -> UUID.randomUUID(), syncSchedulerClient, airbyteGithubStore, sourceHandler, + airbyteProtocolVersionRange); final JobHistoryHandler jobHistoryHandler = new JobHistoryHandler( jobPersistence, @@ -329,8 +332,6 @@ public static ServerRunnable getServer(final ServerFactory apiFactory, destinationHandler, sourceHandler); - final OpenApiConfigHandler openApiConfigHandler = new OpenApiConfigHandler(); - final StatePersistence statePersistence = new StatePersistence(configsDatabase); final StateHandler stateHandler = new StateHandler(statePersistence); @@ -378,46 +379,14 @@ public static ServerRunnable getServer(final ServerFactory apiFactory, jobHistoryHandler, logsHandler, oAuthHandler, - openApiConfigHandler, operationsHandler, schedulerHandler, sourceHandler, sourceDefinitionsHandler, - stateHandler, workspacesHandler, webBackendConnectionsHandler, webBackendGeographiesHandler, webBackendCheckUpdatesHandler); } - public static void main(final String[] args) { - try { - final Configs configs = new EnvConfigs(); - - // Manual configuration that will be replaced by Dependency Injection in the future - final DataSource configsDataSource = - DataSourceFactory.create(configs.getConfigDatabaseUser(), configs.getConfigDatabasePassword(), DRIVER_CLASS_NAME, - configs.getConfigDatabaseUrl()); - final DataSource jobsDataSource = - DataSourceFactory.create(configs.getDatabaseUser(), configs.getDatabasePassword(), DRIVER_CLASS_NAME, configs.getDatabaseUrl()); - - try (final DSLContext configsDslContext = DSLContextFactory.create(configsDataSource, SQLDialect.POSTGRES); - final DSLContext jobsDslContext = DSLContextFactory.create(jobsDataSource, SQLDialect.POSTGRES)) { - - // Ensure that the database resources are closed on application shutdown - CloseableShutdownHook.registerRuntimeShutdownHook(configsDataSource, jobsDataSource, configsDslContext, jobsDslContext); - - final Flyway configsFlyway = FlywayFactory.create(configsDataSource, ServerApp.class.getSimpleName(), - ConfigsDatabaseMigrator.DB_IDENTIFIER, ConfigsDatabaseMigrator.MIGRATION_FILE_LOCATION); - final Flyway jobsFlyway = FlywayFactory.create(jobsDataSource, ServerApp.class.getSimpleName(), JobsDatabaseMigrator.DB_IDENTIFIER, - JobsDatabaseMigrator.MIGRATION_FILE_LOCATION); - - getServer(new ServerFactory.Api(), configs, configsDslContext, configsFlyway, jobsDslContext, jobsFlyway).start(); - } - } catch (final Throwable e) { - LOGGER.error("Server failed", e); - System.exit(1); // so the app doesn't hang on background threads - } - } - } diff --git a/airbyte-server/src/main/java/io/airbyte/server/ServerFactory.java b/airbyte-server/src/main/java/io/airbyte/server/ServerFactory.java index 63b01db3815d..91c483a4124c 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/ServerFactory.java +++ b/airbyte-server/src/main/java/io/airbyte/server/ServerFactory.java @@ -13,66 +13,6 @@ import io.airbyte.config.persistence.SecretsRepositoryWriter; import io.airbyte.db.Database; import io.airbyte.persistence.job.JobPersistence; -import io.airbyte.server.apis.AttemptApiController; -import io.airbyte.server.apis.ConnectionApiController; -import io.airbyte.server.apis.DestinationApiController; -import io.airbyte.server.apis.DestinationDefinitionApiController; -import io.airbyte.server.apis.DestinationDefinitionSpecificationApiController; -import io.airbyte.server.apis.DestinationOauthApiController; -import io.airbyte.server.apis.HealthApiController; -import io.airbyte.server.apis.JobsApiController; -import io.airbyte.server.apis.LogsApiController; -import io.airbyte.server.apis.NotificationsApiController; -import io.airbyte.server.apis.OpenapiApiController; -import io.airbyte.server.apis.OperationApiController; -import io.airbyte.server.apis.SchedulerApiController; -import io.airbyte.server.apis.SourceApiController; -import io.airbyte.server.apis.SourceDefinitionApiController; -import io.airbyte.server.apis.SourceDefinitionSpecificationApiController; -import io.airbyte.server.apis.SourceOauthApiController; -import io.airbyte.server.apis.StateApiController; -import io.airbyte.server.apis.WebBackendApiController; -import io.airbyte.server.apis.WorkspaceApiController; -import io.airbyte.server.apis.binders.AttemptApiBinder; -import io.airbyte.server.apis.binders.ConnectionApiBinder; -import io.airbyte.server.apis.binders.DestinationApiBinder; -import io.airbyte.server.apis.binders.DestinationDefinitionApiBinder; -import io.airbyte.server.apis.binders.DestinationDefinitionSpecificationApiBinder; -import io.airbyte.server.apis.binders.DestinationOauthApiBinder; -import io.airbyte.server.apis.binders.HealthApiBinder; -import io.airbyte.server.apis.binders.JobsApiBinder; -import io.airbyte.server.apis.binders.LogsApiBinder; -import io.airbyte.server.apis.binders.NotificationApiBinder; -import io.airbyte.server.apis.binders.OpenapiApiBinder; -import io.airbyte.server.apis.binders.OperationApiBinder; -import io.airbyte.server.apis.binders.SchedulerApiBinder; -import io.airbyte.server.apis.binders.SourceApiBinder; -import io.airbyte.server.apis.binders.SourceDefinitionApiBinder; -import io.airbyte.server.apis.binders.SourceDefinitionSpecificationApiBinder; -import io.airbyte.server.apis.binders.SourceOauthApiBinder; -import io.airbyte.server.apis.binders.StateApiBinder; -import io.airbyte.server.apis.binders.WebBackendApiBinder; -import io.airbyte.server.apis.binders.WorkspaceApiBinder; -import io.airbyte.server.apis.factories.AttemptApiFactory; -import io.airbyte.server.apis.factories.ConnectionApiFactory; -import io.airbyte.server.apis.factories.DestinationApiFactory; -import io.airbyte.server.apis.factories.DestinationDefinitionApiFactory; -import io.airbyte.server.apis.factories.DestinationDefinitionSpecificationApiFactory; -import io.airbyte.server.apis.factories.DestinationOauthApiFactory; -import io.airbyte.server.apis.factories.HealthApiFactory; -import io.airbyte.server.apis.factories.JobsApiFactory; -import io.airbyte.server.apis.factories.LogsApiFactory; -import io.airbyte.server.apis.factories.NotificationsApiFactory; -import io.airbyte.server.apis.factories.OpenapiApiFactory; -import io.airbyte.server.apis.factories.OperationApiFactory; -import io.airbyte.server.apis.factories.SchedulerApiFactory; -import io.airbyte.server.apis.factories.SourceApiFactory; -import io.airbyte.server.apis.factories.SourceDefinitionApiFactory; -import io.airbyte.server.apis.factories.SourceDefinitionSpecificationApiFactory; -import io.airbyte.server.apis.factories.SourceOauthApiFactory; -import io.airbyte.server.apis.factories.StateApiFactory; -import io.airbyte.server.apis.factories.WebBackendApiFactory; -import io.airbyte.server.apis.factories.WorkspaceApiFactory; import io.airbyte.server.handlers.AttemptHandler; import io.airbyte.server.handlers.ConnectionsHandler; import io.airbyte.server.handlers.DestinationDefinitionsHandler; @@ -81,12 +21,10 @@ import io.airbyte.server.handlers.JobHistoryHandler; import io.airbyte.server.handlers.LogsHandler; import io.airbyte.server.handlers.OAuthHandler; -import io.airbyte.server.handlers.OpenApiConfigHandler; import io.airbyte.server.handlers.OperationsHandler; import io.airbyte.server.handlers.SchedulerHandler; import io.airbyte.server.handlers.SourceDefinitionsHandler; import io.airbyte.server.handlers.SourceHandler; -import io.airbyte.server.handlers.StateHandler; import io.airbyte.server.handlers.WebBackendCheckUpdatesHandler; import io.airbyte.server.handlers.WebBackendConnectionsHandler; import io.airbyte.server.handlers.WebBackendGeographiesHandler; @@ -95,10 +33,8 @@ import io.airbyte.server.scheduler.SynchronousSchedulerClient; import java.net.http.HttpClient; import java.nio.file.Path; -import java.util.Map; -import java.util.Set; +import java.util.HashSet; import org.flywaydb.core.Flyway; -import org.slf4j.MDC; public interface ServerFactory { @@ -126,12 +62,10 @@ ServerRunnable create(final SynchronousSchedulerClient synchronousSchedulerClien final JobHistoryHandler jobHistoryHandler, final LogsHandler logsHandler, final OAuthHandler oAuthHandler, - final OpenApiConfigHandler openApiConfigHandler, final OperationsHandler operationsHandler, final SchedulerHandler schedulerHandler, final SourceHandler sourceHandler, final SourceDefinitionsHandler sourceDefinitionsHandler, - final StateHandler stateHandler, final WorkspacesHandler workspacesHandler, final WebBackendConnectionsHandler webBackendConnectionsHandler, final WebBackendGeographiesHandler webBackendGeographiesHandler, @@ -164,110 +98,17 @@ public ServerRunnable create(final SynchronousSchedulerClient synchronousSchedul final JobHistoryHandler jobHistoryHandler, final LogsHandler logsHandler, final OAuthHandler oAuthHandler, - final OpenApiConfigHandler openApiConfigHandler, final OperationsHandler operationsHandler, final SchedulerHandler schedulerHandler, final SourceHandler sourceHandler, final SourceDefinitionsHandler sourceDefinitionsHandler, - final StateHandler stateHandler, final WorkspacesHandler workspacesHandler, final WebBackendConnectionsHandler webBackendConnectionsHandler, final WebBackendGeographiesHandler webBackendGeographiesHandler, final WebBackendCheckUpdatesHandler webBackendCheckUpdatesHandler) { - final Map mdc = MDC.getCopyOfContextMap(); - - AttemptApiFactory.setValues(attemptHandler, mdc); - - ConnectionApiFactory.setValues( - connectionsHandler, - operationsHandler, - schedulerHandler, - mdc); - - DestinationApiFactory.setValues(destinationApiHandler, schedulerHandler, mdc); - - DestinationDefinitionApiFactory.setValues(destinationDefinitionsHandler); - - DestinationDefinitionSpecificationApiFactory.setValues(schedulerHandler); - - HealthApiFactory.setValues(healthCheckHandler); - - DestinationOauthApiFactory.setValues(oAuthHandler); - - SourceOauthApiFactory.setValues(oAuthHandler); - - JobsApiFactory.setValues(jobHistoryHandler, schedulerHandler); - - LogsApiFactory.setValues(logsHandler); - - NotificationsApiFactory.setValues(workspacesHandler); - - OperationApiFactory.setValues(operationsHandler); - - OpenapiApiFactory.setValues(openApiConfigHandler); - - SchedulerApiFactory.setValues(schedulerHandler); - - SourceApiFactory.setValues(schedulerHandler, sourceHandler); - - SourceDefinitionApiFactory.setValues(sourceDefinitionsHandler); - - SourceDefinitionSpecificationApiFactory.setValues(schedulerHandler); - - StateApiFactory.setValues(stateHandler); - - WebBackendApiFactory.setValues(webBackendConnectionsHandler, webBackendGeographiesHandler, webBackendCheckUpdatesHandler); - - WorkspaceApiFactory.setValues(workspacesHandler); - - // server configurations - final Set> componentClasses = Set.of( - AttemptApiController.class, - ConnectionApiController.class, - DestinationApiController.class, - DestinationDefinitionApiController.class, - DestinationDefinitionSpecificationApiController.class, - DestinationOauthApiController.class, - HealthApiController.class, - JobsApiController.class, - LogsApiController.class, - NotificationsApiController.class, - OpenapiApiController.class, - OperationApiController.class, - SchedulerApiController.class, - SourceApiController.class, - SourceDefinitionApiController.class, - SourceDefinitionSpecificationApiController.class, - SourceOauthApiController.class, - StateApiController.class, - WebBackendApiController.class, - WorkspaceApiController.class); - - final Set components = Set.of( - new CorsFilter(), - new AttemptApiBinder(), - new ConnectionApiBinder(), - new DestinationApiBinder(), - new DestinationDefinitionApiBinder(), - new DestinationDefinitionSpecificationApiBinder(), - new DestinationOauthApiBinder(), - new HealthApiBinder(), - new JobsApiBinder(), - new LogsApiBinder(), - new NotificationApiBinder(), - new OpenapiApiBinder(), - new OperationApiBinder(), - new SchedulerApiBinder(), - new SourceApiBinder(), - new SourceDefinitionApiBinder(), - new SourceDefinitionSpecificationApiBinder(), - new SourceOauthApiBinder(), - new StateApiBinder(), - new WebBackendApiBinder(), - new WorkspaceApiBinder()); // construct server - return new ServerApp(airbyteVersion, componentClasses, components); + return new ServerApp(airbyteVersion, new HashSet<>(), new HashSet<>()); } } diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/AttemptApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/AttemptApiController.java index 66dd9cded057..c22fb95fd33e 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/AttemptApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/AttemptApiController.java @@ -9,9 +9,12 @@ import io.airbyte.api.model.generated.SaveStatsRequestBody; import io.airbyte.api.model.generated.SetWorkflowInAttemptRequestBody; import io.airbyte.server.handlers.AttemptHandler; -import javax.ws.rs.Path; +import io.micronaut.http.MediaType; +import io.micronaut.http.annotation.Body; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; -@Path("/v1/attempt/") +@Controller("/api/v1/attempt/") public class AttemptApiController implements AttemptApi { private final AttemptHandler attemptHandler; @@ -21,12 +24,16 @@ public AttemptApiController(final AttemptHandler attemptHandler) { } @Override + @Post(uri = "/save_stats", + processes = MediaType.APPLICATION_JSON) public InternalOperationResult saveStats(final SaveStatsRequestBody requestBody) { return ApiHelper.execute(() -> attemptHandler.saveStats(requestBody)); } @Override - public InternalOperationResult setWorkflowInAttempt(final SetWorkflowInAttemptRequestBody requestBody) { + @Post(uri = "/set_workflow_in_attempt", + processes = MediaType.APPLICATION_JSON) + public InternalOperationResult setWorkflowInAttempt(@Body final SetWorkflowInAttemptRequestBody requestBody) { return ApiHelper.execute(() -> attemptHandler.setWorkflowInAttempt(requestBody)); } diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/ConnectionApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/ConnectionApiController.java index fe29b42e1f33..0992b878f470 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/ConnectionApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/ConnectionApiController.java @@ -16,9 +16,13 @@ import io.airbyte.server.handlers.ConnectionsHandler; import io.airbyte.server.handlers.OperationsHandler; import io.airbyte.server.handlers.SchedulerHandler; -import javax.ws.rs.Path; +import io.micronaut.context.annotation.Context; +import io.micronaut.http.annotation.Body; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; -@Path("/v1/connections") +@Controller("/api/v1/connections") +@Context() public class ConnectionApiController implements ConnectionApi { private final ConnectionsHandler connectionsHandler; @@ -34,37 +38,44 @@ public ConnectionApiController(final ConnectionsHandler connectionsHandler, } @Override - public ConnectionRead createConnection(final ConnectionCreate connectionCreate) { + @Post(uri = "/create") + public ConnectionRead createConnection(@Body final ConnectionCreate connectionCreate) { return ApiHelper.execute(() -> connectionsHandler.createConnection(connectionCreate)); } @Override - public ConnectionRead updateConnection(final ConnectionUpdate connectionUpdate) { + @Post(uri = "/update") + public ConnectionRead updateConnection(@Body final ConnectionUpdate connectionUpdate) { return ApiHelper.execute(() -> connectionsHandler.updateConnection(connectionUpdate)); } @Override - public ConnectionReadList listConnectionsForWorkspace(final WorkspaceIdRequestBody workspaceIdRequestBody) { + @Post(uri = "/list") + public ConnectionReadList listConnectionsForWorkspace(@Body final WorkspaceIdRequestBody workspaceIdRequestBody) { return ApiHelper.execute(() -> connectionsHandler.listConnectionsForWorkspace(workspaceIdRequestBody)); } @Override - public ConnectionReadList listAllConnectionsForWorkspace(final WorkspaceIdRequestBody workspaceIdRequestBody) { + @Post(uri = "/list_all") + public ConnectionReadList listAllConnectionsForWorkspace(@Body final WorkspaceIdRequestBody workspaceIdRequestBody) { return ApiHelper.execute(() -> connectionsHandler.listAllConnectionsForWorkspace(workspaceIdRequestBody)); } @Override - public ConnectionReadList searchConnections(final ConnectionSearch connectionSearch) { + @Post(uri = "/search") + public ConnectionReadList searchConnections(@Body final ConnectionSearch connectionSearch) { return ApiHelper.execute(() -> connectionsHandler.searchConnections(connectionSearch)); } @Override - public ConnectionRead getConnection(final ConnectionIdRequestBody connectionIdRequestBody) { + @Post(uri = "/get") + public ConnectionRead getConnection(@Body final ConnectionIdRequestBody connectionIdRequestBody) { return ApiHelper.execute(() -> connectionsHandler.getConnection(connectionIdRequestBody.getConnectionId())); } @Override - public void deleteConnection(final ConnectionIdRequestBody connectionIdRequestBody) { + @Post(uri = "/delete") + public void deleteConnection(@Body final ConnectionIdRequestBody connectionIdRequestBody) { ApiHelper.execute(() -> { operationsHandler.deleteOperationsForConnection(connectionIdRequestBody); connectionsHandler.deleteConnection(connectionIdRequestBody.getConnectionId()); @@ -73,12 +84,14 @@ public void deleteConnection(final ConnectionIdRequestBody connectionIdRequestBo } @Override - public JobInfoRead syncConnection(final ConnectionIdRequestBody connectionIdRequestBody) { + @Post(uri = "/sync") + public JobInfoRead syncConnection(@Body final ConnectionIdRequestBody connectionIdRequestBody) { return ApiHelper.execute(() -> schedulerHandler.syncConnection(connectionIdRequestBody)); } @Override - public JobInfoRead resetConnection(final ConnectionIdRequestBody connectionIdRequestBody) { + @Post(uri = "/reset") + public JobInfoRead resetConnection(@Body final ConnectionIdRequestBody connectionIdRequestBody) { return ApiHelper.execute(() -> schedulerHandler.resetConnection(connectionIdRequestBody)); } diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/DestinationApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/DestinationApiController.java index 8cbdbb5cd6d0..208e39e1fbfd 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/DestinationApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/DestinationApiController.java @@ -16,61 +16,72 @@ import io.airbyte.api.model.generated.WorkspaceIdRequestBody; import io.airbyte.server.handlers.DestinationHandler; import io.airbyte.server.handlers.SchedulerHandler; -import javax.ws.rs.Path; +import io.micronaut.http.annotation.Body; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; import lombok.AllArgsConstructor; -@Path("/v1/destinations") +@Controller("/api/v1/destinations") @AllArgsConstructor public class DestinationApiController implements DestinationApi { private final DestinationHandler destinationHandler; private final SchedulerHandler schedulerHandler; + @Post(uri = "/check_connection") @Override - public CheckConnectionRead checkConnectionToDestination(final DestinationIdRequestBody destinationIdRequestBody) { + public CheckConnectionRead checkConnectionToDestination(@Body final DestinationIdRequestBody destinationIdRequestBody) { return ApiHelper.execute(() -> schedulerHandler.checkDestinationConnectionFromDestinationId(destinationIdRequestBody)); } + @Post(uri = "/check_connection_for_update") @Override - public CheckConnectionRead checkConnectionToDestinationForUpdate(final DestinationUpdate destinationUpdate) { + public CheckConnectionRead checkConnectionToDestinationForUpdate(@Body final DestinationUpdate destinationUpdate) { return ApiHelper.execute(() -> schedulerHandler.checkDestinationConnectionFromDestinationIdForUpdate(destinationUpdate)); } + @Post(uri = "/clone") @Override - public DestinationRead cloneDestination(final DestinationCloneRequestBody destinationCloneRequestBody) { + public DestinationRead cloneDestination(@Body final DestinationCloneRequestBody destinationCloneRequestBody) { return ApiHelper.execute(() -> destinationHandler.cloneDestination(destinationCloneRequestBody)); } + @Post(uri = "/create") @Override - public DestinationRead createDestination(final DestinationCreate destinationCreate) { + public DestinationRead createDestination(@Body final DestinationCreate destinationCreate) { return ApiHelper.execute(() -> destinationHandler.createDestination(destinationCreate)); } + @Post(uri = "/delete") @Override - public void deleteDestination(final DestinationIdRequestBody destinationIdRequestBody) { + public void deleteDestination(@Body final DestinationIdRequestBody destinationIdRequestBody) { ApiHelper.execute(() -> { destinationHandler.deleteDestination(destinationIdRequestBody); return null; }); } + @Post(uri = "/get") @Override - public DestinationRead getDestination(final DestinationIdRequestBody destinationIdRequestBody) { + public DestinationRead getDestination(@Body final DestinationIdRequestBody destinationIdRequestBody) { return ApiHelper.execute(() -> destinationHandler.getDestination(destinationIdRequestBody)); } + @Post(uri = "/list") @Override - public DestinationReadList listDestinationsForWorkspace(final WorkspaceIdRequestBody workspaceIdRequestBody) { + public DestinationReadList listDestinationsForWorkspace(@Body final WorkspaceIdRequestBody workspaceIdRequestBody) { return ApiHelper.execute(() -> destinationHandler.listDestinationsForWorkspace(workspaceIdRequestBody)); } + @Post(uri = "/search") @Override - public DestinationReadList searchDestinations(final DestinationSearch destinationSearch) { + public DestinationReadList searchDestinations(@Body final DestinationSearch destinationSearch) { return ApiHelper.execute(() -> destinationHandler.searchDestinations(destinationSearch)); } + @Post(uri = "/update") @Override - public DestinationRead updateDestination(final DestinationUpdate destinationUpdate) { + public DestinationRead updateDestination(@Body final DestinationUpdate destinationUpdate) { return ApiHelper.execute(() -> destinationHandler.updateDestination(destinationUpdate)); } diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/DestinationDefinitionApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/DestinationDefinitionApiController.java index d2325703298d..9ae9e4ff53f1 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/DestinationDefinitionApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/DestinationDefinitionApiController.java @@ -15,20 +15,27 @@ import io.airbyte.api.model.generated.PrivateDestinationDefinitionReadList; import io.airbyte.api.model.generated.WorkspaceIdRequestBody; import io.airbyte.server.handlers.DestinationDefinitionsHandler; -import javax.ws.rs.Path; -import lombok.AllArgsConstructor; +import io.micronaut.context.annotation.Context; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; -@Path("/v1/destination_definitions") -@AllArgsConstructor +@Controller("/api/v1/destination_definitions") +@Context public class DestinationDefinitionApiController implements DestinationDefinitionApi { private final DestinationDefinitionsHandler destinationDefinitionsHandler; + public DestinationDefinitionApiController(final DestinationDefinitionsHandler destinationDefinitionsHandler) { + this.destinationDefinitionsHandler = destinationDefinitionsHandler; + } + + @Post(uri = "/create_custom") @Override public DestinationDefinitionRead createCustomDestinationDefinition(final CustomDestinationDefinitionCreate customDestinationDefinitionCreate) { return ApiHelper.execute(() -> destinationDefinitionsHandler.createCustomDestinationDefinition(customDestinationDefinitionCreate)); } + @Post(uri = "/delete") @Override public void deleteDestinationDefinition(final DestinationDefinitionIdRequestBody destinationDefinitionIdRequestBody) { ApiHelper.execute(() -> { @@ -37,42 +44,50 @@ public void deleteDestinationDefinition(final DestinationDefinitionIdRequestBody }); } + @Post(uri = "/get") @Override public DestinationDefinitionRead getDestinationDefinition(final DestinationDefinitionIdRequestBody destinationDefinitionIdRequestBody) { return ApiHelper.execute(() -> destinationDefinitionsHandler.getDestinationDefinition(destinationDefinitionIdRequestBody)); } + @Post(uri = "/get_for_workspace") @Override public DestinationDefinitionRead getDestinationDefinitionForWorkspace(final DestinationDefinitionIdWithWorkspaceId destinationDefinitionIdWithWorkspaceId) { return ApiHelper.execute(() -> destinationDefinitionsHandler.getDestinationDefinitionForWorkspace(destinationDefinitionIdWithWorkspaceId)); } + @Post(uri = "/grant_definition") @Override public PrivateDestinationDefinitionRead grantDestinationDefinitionToWorkspace(final DestinationDefinitionIdWithWorkspaceId destinationDefinitionIdWithWorkspaceId) { return ApiHelper .execute(() -> destinationDefinitionsHandler.grantDestinationDefinitionToWorkspace(destinationDefinitionIdWithWorkspaceId)); } + @Post(uri = "/list") @Override public DestinationDefinitionReadList listDestinationDefinitions() { return ApiHelper.execute(destinationDefinitionsHandler::listDestinationDefinitions); } + @Post(uri = "/list_for_workspace") @Override public DestinationDefinitionReadList listDestinationDefinitionsForWorkspace(final WorkspaceIdRequestBody workspaceIdRequestBody) { return ApiHelper.execute(() -> destinationDefinitionsHandler.listDestinationDefinitionsForWorkspace(workspaceIdRequestBody)); } + @Post(uri = "/list_latest") @Override public DestinationDefinitionReadList listLatestDestinationDefinitions() { return ApiHelper.execute(destinationDefinitionsHandler::listLatestDestinationDefinitions); } + @Post(uri = "/list_private") @Override public PrivateDestinationDefinitionReadList listPrivateDestinationDefinitions(final WorkspaceIdRequestBody workspaceIdRequestBody) { return ApiHelper.execute(() -> destinationDefinitionsHandler.listPrivateDestinationDefinitions(workspaceIdRequestBody)); } + @Post(uri = "/revoke_definition") @Override public void revokeDestinationDefinitionFromWorkspace(final DestinationDefinitionIdWithWorkspaceId destinationDefinitionIdWithWorkspaceId) { ApiHelper.execute(() -> { @@ -81,6 +96,7 @@ public void revokeDestinationDefinitionFromWorkspace(final DestinationDefinition }); } + @Post(uri = "/update") @Override public DestinationDefinitionRead updateDestinationDefinition(final DestinationDefinitionUpdate destinationDefinitionUpdate) { return ApiHelper.execute(() -> destinationDefinitionsHandler.updateDestinationDefinition(destinationDefinitionUpdate)); diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/DestinationDefinitionSpecificationApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/DestinationDefinitionSpecificationApiController.java index 74853019c8dc..6d1ce0fcb3f3 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/DestinationDefinitionSpecificationApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/DestinationDefinitionSpecificationApiController.java @@ -8,15 +8,19 @@ import io.airbyte.api.model.generated.DestinationDefinitionIdWithWorkspaceId; import io.airbyte.api.model.generated.DestinationDefinitionSpecificationRead; import io.airbyte.server.handlers.SchedulerHandler; -import javax.ws.rs.Path; -import lombok.AllArgsConstructor; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; -@Path("/v1/destination_definition_specifications/get") -@AllArgsConstructor +@Controller("/api/v1/destination_definition_specifications") public class DestinationDefinitionSpecificationApiController implements DestinationDefinitionSpecificationApi { private final SchedulerHandler schedulerHandler; + public DestinationDefinitionSpecificationApiController(final SchedulerHandler schedulerHandler) { + this.schedulerHandler = schedulerHandler; + } + + @Post("/get") @Override public DestinationDefinitionSpecificationRead getDestinationDefinitionSpecification(final DestinationDefinitionIdWithWorkspaceId destinationDefinitionIdWithWorkspaceId) { return ApiHelper.execute(() -> schedulerHandler.getDestinationSpecification(destinationDefinitionIdWithWorkspaceId)); diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/DestinationOauthApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/DestinationOauthApiController.java index fcbaa9a2675f..966d6561f810 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/DestinationOauthApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/DestinationOauthApiController.java @@ -10,26 +10,34 @@ import io.airbyte.api.model.generated.OAuthConsentRead; import io.airbyte.api.model.generated.SetInstancewideDestinationOauthParamsRequestBody; import io.airbyte.server.handlers.OAuthHandler; +import io.micronaut.context.annotation.Context; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; import java.util.Map; -import javax.ws.rs.Path; -import lombok.AllArgsConstructor; -@Path("/v1/destination_oauths") -@AllArgsConstructor +@Controller("/api/v1/destination_oauths") +@Context public class DestinationOauthApiController implements DestinationOauthApi { private final OAuthHandler oAuthHandler; + public DestinationOauthApiController(final OAuthHandler oAuthHandler) { + this.oAuthHandler = oAuthHandler; + } + + @Post("/complete_oauth") @Override public Map completeDestinationOAuth(final CompleteDestinationOAuthRequest completeDestinationOAuthRequest) { return ApiHelper.execute(() -> oAuthHandler.completeDestinationOAuth(completeDestinationOAuthRequest)); } + @Post("/get_consent_url") @Override public OAuthConsentRead getDestinationOAuthConsent(final DestinationOauthConsentRequest destinationOauthConsentRequest) { return ApiHelper.execute(() -> oAuthHandler.getDestinationOAuthConsent(destinationOauthConsentRequest)); } + @Post("/oauth_params/create") @Override public void setInstancewideDestinationOauthParams(final SetInstancewideDestinationOauthParamsRequestBody setInstancewideDestinationOauthParamsRequestBody) { ApiHelper.execute(() -> { diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/HealthApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/HealthApiController.java index 75c10ca4852b..6acb5cd39cd2 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/HealthApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/HealthApiController.java @@ -7,16 +7,21 @@ import io.airbyte.api.generated.HealthApi; import io.airbyte.api.model.generated.HealthCheckRead; import io.airbyte.server.handlers.HealthCheckHandler; -import javax.ws.rs.Path; -import lombok.AllArgsConstructor; +import io.micronaut.http.MediaType; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Get; -@Path("/v1/health") -@AllArgsConstructor +@Controller("/api/v1/health") public class HealthApiController implements HealthApi { private final HealthCheckHandler healthCheckHandler; + public HealthApiController(final HealthCheckHandler healthCheckHandler) { + this.healthCheckHandler = healthCheckHandler; + } + @Override + @Get(produces = MediaType.APPLICATION_JSON) public HealthCheckRead getHealthCheck() { return healthCheckHandler.health(); } diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/JobsApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/JobsApiController.java index 197a861a63e5..05ff3a2ae8ca 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/JobsApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/JobsApiController.java @@ -14,41 +14,53 @@ import io.airbyte.api.model.generated.JobReadList; import io.airbyte.server.handlers.JobHistoryHandler; import io.airbyte.server.handlers.SchedulerHandler; -import javax.ws.rs.Path; -import lombok.AllArgsConstructor; +import io.micronaut.context.annotation.Context; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; -@Path("/v1/jobs") -@AllArgsConstructor +@Controller("/api/v1/jobs") +@Context public class JobsApiController implements JobsApi { private final JobHistoryHandler jobHistoryHandler; private final SchedulerHandler schedulerHandler; + public JobsApiController(final JobHistoryHandler jobHistoryHandler, final SchedulerHandler schedulerHandler) { + this.jobHistoryHandler = jobHistoryHandler; + this.schedulerHandler = schedulerHandler; + } + + @Post("/cancel") @Override public JobInfoRead cancelJob(final JobIdRequestBody jobIdRequestBody) { return ApiHelper.execute(() -> schedulerHandler.cancelJob(jobIdRequestBody)); } + @Post("/get_normalization_status") @Override public AttemptNormalizationStatusReadList getAttemptNormalizationStatusesForJob(final JobIdRequestBody jobIdRequestBody) { return ApiHelper.execute(() -> jobHistoryHandler.getAttemptNormalizationStatuses(jobIdRequestBody)); } + @Post("/get_debug_info") @Override public JobDebugInfoRead getJobDebugInfo(final JobIdRequestBody jobIdRequestBody) { return ApiHelper.execute(() -> jobHistoryHandler.getJobDebugInfo(jobIdRequestBody)); } + @Post("/get") @Override public JobInfoRead getJobInfo(final JobIdRequestBody jobIdRequestBody) { return ApiHelper.execute(() -> jobHistoryHandler.getJobInfo(jobIdRequestBody)); } + @Post("/get_light") @Override public JobInfoLightRead getJobInfoLight(final JobIdRequestBody jobIdRequestBody) { return ApiHelper.execute(() -> jobHistoryHandler.getJobInfoLight(jobIdRequestBody)); } + @Post("/list") @Override public JobReadList listJobsFor(final JobListRequestBody jobListRequestBody) { return ApiHelper.execute(() -> jobHistoryHandler.listJobsFor(jobListRequestBody)); diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/LogsApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/LogsApiController.java index 95e7758f4f68..f3c43fdfdf32 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/LogsApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/LogsApiController.java @@ -7,16 +7,22 @@ import io.airbyte.api.generated.LogsApi; import io.airbyte.api.model.generated.LogsRequestBody; import io.airbyte.server.handlers.LogsHandler; +import io.micronaut.context.annotation.Context; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; import java.io.File; -import javax.ws.rs.Path; -import lombok.AllArgsConstructor; -@Path("/v1/logs/get") -@AllArgsConstructor +@Controller("/api/v1/logs") +@Context public class LogsApiController implements LogsApi { private final LogsHandler logsHandler; + public LogsApiController(final LogsHandler logsHandler) { + this.logsHandler = logsHandler; + } + + @Post("/get") @Override public File getLogs(final LogsRequestBody logsRequestBody) { return ApiHelper.execute(() -> logsHandler.getLogs(logsRequestBody)); diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/NotFoundController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/NotFoundController.java new file mode 100644 index 000000000000..b898261dac47 --- /dev/null +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/NotFoundController.java @@ -0,0 +1,30 @@ +/* + * Copyright (c) 2022 Airbyte, Inc., all rights reserved. + */ + +package io.airbyte.server.apis; + +import io.micronaut.http.HttpRequest; +import io.micronaut.http.HttpResponse; +import io.micronaut.http.HttpStatus; +import io.micronaut.http.MediaType; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Error; + +/** + * Custom controller that handles global 404 responses for unknown/unmapped paths. + */ +@Controller("/api/notfound") +public class NotFoundController { + + @Error(status = HttpStatus.NOT_FOUND, + global = true) + public HttpResponse notFound(final HttpRequest request) { + // Would like to send the id along but we don't have access to the http request anymore to fetch it + // from. TODO: Come back to this with issue #4189 + return HttpResponse.status(HttpStatus.NOT_FOUND) + .body("Object not found.") + .contentType(MediaType.APPLICATION_JSON); + } + +} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/NotificationsApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/NotificationsApiController.java index 981ad80f7e8d..682ab6eb0d08 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/NotificationsApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/NotificationsApiController.java @@ -8,17 +8,22 @@ import io.airbyte.api.model.generated.Notification; import io.airbyte.api.model.generated.NotificationRead; import io.airbyte.server.handlers.WorkspacesHandler; -import javax.ws.rs.Path; -import lombok.AllArgsConstructor; +import io.micronaut.http.annotation.Body; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; -@Path("/v1/notifications/try") -@AllArgsConstructor +@Controller("/api/v1/notifications/try") public class NotificationsApiController implements NotificationsApi { private final WorkspacesHandler workspacesHandler; + public NotificationsApiController(final WorkspacesHandler workspacesHandler) { + this.workspacesHandler = workspacesHandler; + } + + @Post @Override - public NotificationRead tryNotificationConfig(final Notification notification) { + public NotificationRead tryNotificationConfig(@Body final Notification notification) { return ApiHelper.execute(() -> workspacesHandler.tryNotification(notification)); } diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/OpenapiApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/OpenapiApiController.java index ae749403b420..8d3ebadb4f62 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/OpenapiApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/OpenapiApiController.java @@ -6,16 +6,20 @@ import io.airbyte.api.generated.OpenapiApi; import io.airbyte.server.handlers.OpenApiConfigHandler; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Get; import java.io.File; -import javax.ws.rs.Path; -import lombok.AllArgsConstructor; -@Path("/v1/openapi") -@AllArgsConstructor +@Controller("/api/v1/openapi") public class OpenapiApiController implements OpenapiApi { private final OpenApiConfigHandler openApiConfigHandler; + public OpenapiApiController(final OpenApiConfigHandler openApiConfigHandler) { + this.openApiConfigHandler = openApiConfigHandler; + } + + @Get(produces = "text/plain") @Override public File getOpenApiSpec() { return ApiHelper.execute(openApiConfigHandler::getFile); diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/OperationApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/OperationApiController.java index 3b4eeb9ee431..10eabb1571fd 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/OperationApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/OperationApiController.java @@ -14,45 +14,55 @@ import io.airbyte.api.model.generated.OperationUpdate; import io.airbyte.api.model.generated.OperatorConfiguration; import io.airbyte.server.handlers.OperationsHandler; -import javax.ws.rs.Path; -import lombok.AllArgsConstructor; +import io.micronaut.http.annotation.Body; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; -@Path("/v1/operations") -@AllArgsConstructor +@Controller("/api/v1/operations") public class OperationApiController implements OperationApi { private final OperationsHandler operationsHandler; + public OperationApiController(final OperationsHandler operationsHandler) { + this.operationsHandler = operationsHandler; + } + + @Post("/check") @Override - public CheckOperationRead checkOperation(final OperatorConfiguration operatorConfiguration) { + public CheckOperationRead checkOperation(@Body final OperatorConfiguration operatorConfiguration) { return ApiHelper.execute(() -> operationsHandler.checkOperation(operatorConfiguration)); } + @Post("/create") @Override - public OperationRead createOperation(final OperationCreate operationCreate) { + public OperationRead createOperation(@Body final OperationCreate operationCreate) { return ApiHelper.execute(() -> operationsHandler.createOperation(operationCreate)); } + @Post("/delete") @Override - public void deleteOperation(final OperationIdRequestBody operationIdRequestBody) { + public void deleteOperation(@Body final OperationIdRequestBody operationIdRequestBody) { ApiHelper.execute(() -> { operationsHandler.deleteOperation(operationIdRequestBody); return null; }); } + @Post("/get") @Override - public OperationRead getOperation(final OperationIdRequestBody operationIdRequestBody) { + public OperationRead getOperation(@Body final OperationIdRequestBody operationIdRequestBody) { return ApiHelper.execute(() -> operationsHandler.getOperation(operationIdRequestBody)); } + @Post("/list") @Override - public OperationReadList listOperationsForConnection(final ConnectionIdRequestBody connectionIdRequestBody) { + public OperationReadList listOperationsForConnection(@Body final ConnectionIdRequestBody connectionIdRequestBody) { return ApiHelper.execute(() -> operationsHandler.listOperationsForConnection(connectionIdRequestBody)); } + @Post("/update") @Override - public OperationRead updateOperation(final OperationUpdate operationUpdate) { + public OperationRead updateOperation(@Body final OperationUpdate operationUpdate) { return ApiHelper.execute(() -> operationsHandler.updateOperation(operationUpdate)); } diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/SchedulerApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/SchedulerApiController.java index addc445d3c0f..009bec3d816b 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/SchedulerApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/SchedulerApiController.java @@ -10,23 +10,31 @@ import io.airbyte.api.model.generated.SourceCoreConfig; import io.airbyte.api.model.generated.SourceDiscoverSchemaRead; import io.airbyte.server.handlers.SchedulerHandler; -import lombok.AllArgsConstructor; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; -@AllArgsConstructor +@Controller("/api/v1/scheduler") public class SchedulerApiController implements SchedulerApi { private final SchedulerHandler schedulerHandler; + public SchedulerApiController(final SchedulerHandler schedulerHandler) { + this.schedulerHandler = schedulerHandler; + } + + @Post("/destinations/check_connection") @Override public CheckConnectionRead executeDestinationCheckConnection(final DestinationCoreConfig destinationCoreConfig) { return ApiHelper.execute(() -> schedulerHandler.checkDestinationConnectionFromDestinationCreate(destinationCoreConfig)); } + @Post("/sources/check_connection") @Override public CheckConnectionRead executeSourceCheckConnection(final SourceCoreConfig sourceCoreConfig) { return ApiHelper.execute(() -> schedulerHandler.checkSourceConnectionFromSourceCreate(sourceCoreConfig)); } + @Post("/sources/discover_schema") @Override public SourceDiscoverSchemaRead executeSourceDiscoverSchema(final SourceCoreConfig sourceCoreConfig) { return ApiHelper.execute(() -> schedulerHandler.discoverSchemaForSourceFromSourceCreate(sourceCoreConfig)); diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/SourceApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/SourceApiController.java index d0bd6152325a..4befc0806e5d 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/SourceApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/SourceApiController.java @@ -19,36 +19,45 @@ import io.airbyte.api.model.generated.WorkspaceIdRequestBody; import io.airbyte.server.handlers.SchedulerHandler; import io.airbyte.server.handlers.SourceHandler; -import javax.ws.rs.Path; -import lombok.AllArgsConstructor; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; -@Path("/v1/sources") -@AllArgsConstructor +@Controller("/api/v1/sources") public class SourceApiController implements SourceApi { private final SchedulerHandler schedulerHandler; private final SourceHandler sourceHandler; + public SourceApiController(final SchedulerHandler schedulerHandler, final SourceHandler sourceHandler) { + this.schedulerHandler = schedulerHandler; + this.sourceHandler = sourceHandler; + } + + @Post("/check_connection") @Override public CheckConnectionRead checkConnectionToSource(final SourceIdRequestBody sourceIdRequestBody) { return ApiHelper.execute(() -> schedulerHandler.checkSourceConnectionFromSourceId(sourceIdRequestBody)); } + @Post("/check_connection_for_update") @Override public CheckConnectionRead checkConnectionToSourceForUpdate(final SourceUpdate sourceUpdate) { return ApiHelper.execute(() -> schedulerHandler.checkSourceConnectionFromSourceIdForUpdate(sourceUpdate)); } + @Post("/clone") @Override public SourceRead cloneSource(final SourceCloneRequestBody sourceCloneRequestBody) { return ApiHelper.execute(() -> sourceHandler.cloneSource(sourceCloneRequestBody)); } + @Post("/create") @Override public SourceRead createSource(final SourceCreate sourceCreate) { return ApiHelper.execute(() -> sourceHandler.createSource(sourceCreate)); } + @Post("/delete") @Override public void deleteSource(final SourceIdRequestBody sourceIdRequestBody) { ApiHelper.execute(() -> { @@ -57,31 +66,37 @@ public void deleteSource(final SourceIdRequestBody sourceIdRequestBody) { }); } + @Post("/discover_schema") @Override public SourceDiscoverSchemaRead discoverSchemaForSource(final SourceDiscoverSchemaRequestBody sourceDiscoverSchemaRequestBody) { return ApiHelper.execute(() -> schedulerHandler.discoverSchemaForSourceFromSourceId(sourceDiscoverSchemaRequestBody)); } + @Post("/get") @Override public SourceRead getSource(final SourceIdRequestBody sourceIdRequestBody) { return ApiHelper.execute(() -> sourceHandler.getSource(sourceIdRequestBody)); } + @Post("/most_recent_source_actor_catalog") @Override public ActorCatalogWithUpdatedAt getMostRecentSourceActorCatalog(final SourceIdRequestBody sourceIdRequestBody) { return ApiHelper.execute(() -> sourceHandler.getMostRecentSourceActorCatalogWithUpdatedAt(sourceIdRequestBody)); } + @Post("/list") @Override public SourceReadList listSourcesForWorkspace(final WorkspaceIdRequestBody workspaceIdRequestBody) { return ApiHelper.execute(() -> sourceHandler.listSourcesForWorkspace(workspaceIdRequestBody)); } + @Post("/search") @Override public SourceReadList searchSources(final SourceSearch sourceSearch) { return ApiHelper.execute(() -> sourceHandler.searchSources(sourceSearch)); } + @Post("/update") @Override public SourceRead updateSource(final SourceUpdate sourceUpdate) { return ApiHelper.execute(() -> sourceHandler.updateSource(sourceUpdate)); diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/SourceDefinitionApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/SourceDefinitionApiController.java index 8820b6287e4f..ccfd35f41696 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/SourceDefinitionApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/SourceDefinitionApiController.java @@ -15,20 +15,27 @@ import io.airbyte.api.model.generated.SourceDefinitionUpdate; import io.airbyte.api.model.generated.WorkspaceIdRequestBody; import io.airbyte.server.handlers.SourceDefinitionsHandler; -import javax.ws.rs.Path; -import lombok.AllArgsConstructor; +import io.micronaut.context.annotation.Context; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; -@Path("/v1/source_definitions") -@AllArgsConstructor +@Controller("/api/v1/source_definitions") +@Context public class SourceDefinitionApiController implements SourceDefinitionApi { private final SourceDefinitionsHandler sourceDefinitionsHandler; + public SourceDefinitionApiController(final SourceDefinitionsHandler sourceDefinitionsHandler) { + this.sourceDefinitionsHandler = sourceDefinitionsHandler; + } + + @Post("/create_custom") @Override public SourceDefinitionRead createCustomSourceDefinition(final CustomSourceDefinitionCreate customSourceDefinitionCreate) { return ApiHelper.execute(() -> sourceDefinitionsHandler.createCustomSourceDefinition(customSourceDefinitionCreate)); } + @Post("/delete") @Override public void deleteSourceDefinition(final SourceDefinitionIdRequestBody sourceDefinitionIdRequestBody) { ApiHelper.execute(() -> { @@ -37,41 +44,49 @@ public void deleteSourceDefinition(final SourceDefinitionIdRequestBody sourceDef }); } + @Post("/get") @Override public SourceDefinitionRead getSourceDefinition(final SourceDefinitionIdRequestBody sourceDefinitionIdRequestBody) { return ApiHelper.execute(() -> sourceDefinitionsHandler.getSourceDefinition(sourceDefinitionIdRequestBody)); } + @Post("/get_for_workspace") @Override public SourceDefinitionRead getSourceDefinitionForWorkspace(final SourceDefinitionIdWithWorkspaceId sourceDefinitionIdWithWorkspaceId) { return ApiHelper.execute(() -> sourceDefinitionsHandler.getSourceDefinitionForWorkspace(sourceDefinitionIdWithWorkspaceId)); } + @Post("/grant_definition") @Override public PrivateSourceDefinitionRead grantSourceDefinitionToWorkspace(final SourceDefinitionIdWithWorkspaceId sourceDefinitionIdWithWorkspaceId) { return ApiHelper.execute(() -> sourceDefinitionsHandler.grantSourceDefinitionToWorkspace(sourceDefinitionIdWithWorkspaceId)); } + @Post("/list_latest") @Override public SourceDefinitionReadList listLatestSourceDefinitions() { return ApiHelper.execute(sourceDefinitionsHandler::listLatestSourceDefinitions); } + @Post("/list_private") @Override public PrivateSourceDefinitionReadList listPrivateSourceDefinitions(final WorkspaceIdRequestBody workspaceIdRequestBody) { return ApiHelper.execute(() -> sourceDefinitionsHandler.listPrivateSourceDefinitions(workspaceIdRequestBody)); } + @Post("/list") @Override public SourceDefinitionReadList listSourceDefinitions() { return ApiHelper.execute(sourceDefinitionsHandler::listSourceDefinitions); } + @Post("/list_for_workspace") @Override public SourceDefinitionReadList listSourceDefinitionsForWorkspace(final WorkspaceIdRequestBody workspaceIdRequestBody) { return ApiHelper.execute(() -> sourceDefinitionsHandler.listSourceDefinitionsForWorkspace(workspaceIdRequestBody)); } + @Post("/revoke_definition") @Override public void revokeSourceDefinitionFromWorkspace(final SourceDefinitionIdWithWorkspaceId sourceDefinitionIdWithWorkspaceId) { ApiHelper.execute(() -> { @@ -80,6 +95,7 @@ public void revokeSourceDefinitionFromWorkspace(final SourceDefinitionIdWithWork }); } + @Post("/update") @Override public SourceDefinitionRead updateSourceDefinition(final SourceDefinitionUpdate sourceDefinitionUpdate) { return ApiHelper.execute(() -> sourceDefinitionsHandler.updateSourceDefinition(sourceDefinitionUpdate)); diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/SourceDefinitionSpecificationApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/SourceDefinitionSpecificationApiController.java index 4068f1174048..2cae2b33f1f1 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/SourceDefinitionSpecificationApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/SourceDefinitionSpecificationApiController.java @@ -8,15 +8,19 @@ import io.airbyte.api.model.generated.SourceDefinitionIdWithWorkspaceId; import io.airbyte.api.model.generated.SourceDefinitionSpecificationRead; import io.airbyte.server.handlers.SchedulerHandler; -import javax.ws.rs.Path; -import lombok.AllArgsConstructor; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; -@Path("/v1/source_definition_specifications/get") -@AllArgsConstructor +@Controller("/api/v1/source_definition_specifications") public class SourceDefinitionSpecificationApiController implements SourceDefinitionSpecificationApi { private final SchedulerHandler schedulerHandler; + public SourceDefinitionSpecificationApiController(final SchedulerHandler schedulerHandler) { + this.schedulerHandler = schedulerHandler; + } + + @Post("/get") @Override public SourceDefinitionSpecificationRead getSourceDefinitionSpecification(final SourceDefinitionIdWithWorkspaceId sourceDefinitionIdWithWorkspaceId) { return ApiHelper.execute(() -> schedulerHandler.getSourceDefinitionSpecification(sourceDefinitionIdWithWorkspaceId)); diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/SourceOauthApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/SourceOauthApiController.java index 84e0782a3020..9d57cf3c8798 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/SourceOauthApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/SourceOauthApiController.java @@ -10,28 +10,35 @@ import io.airbyte.api.model.generated.SetInstancewideSourceOauthParamsRequestBody; import io.airbyte.api.model.generated.SourceOauthConsentRequest; import io.airbyte.server.handlers.OAuthHandler; +import io.micronaut.http.annotation.Body; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; import java.util.Map; -import javax.ws.rs.Path; -import lombok.AllArgsConstructor; -@Path("/v1/source_oauths") -@AllArgsConstructor +@Controller("/api/v1/source_oauths") public class SourceOauthApiController implements SourceOauthApi { private final OAuthHandler oAuthHandler; + public SourceOauthApiController(final OAuthHandler oAuthHandler) { + this.oAuthHandler = oAuthHandler; + } + + @Post("/complete_oauth") @Override - public Map completeSourceOAuth(final CompleteSourceOauthRequest completeSourceOauthRequest) { + public Map completeSourceOAuth(@Body final CompleteSourceOauthRequest completeSourceOauthRequest) { return ApiHelper.execute(() -> oAuthHandler.completeSourceOAuth(completeSourceOauthRequest)); } + @Post("/get_consent_url") @Override - public OAuthConsentRead getSourceOAuthConsent(final SourceOauthConsentRequest sourceOauthConsentRequest) { + public OAuthConsentRead getSourceOAuthConsent(@Body final SourceOauthConsentRequest sourceOauthConsentRequest) { return ApiHelper.execute(() -> oAuthHandler.getSourceOAuthConsent(sourceOauthConsentRequest)); } + @Post("/oauth_params/create") @Override - public void setInstancewideSourceOauthParams(final SetInstancewideSourceOauthParamsRequestBody setInstancewideSourceOauthParamsRequestBody) { + public void setInstancewideSourceOauthParams(@Body final SetInstancewideSourceOauthParamsRequestBody setInstancewideSourceOauthParamsRequestBody) { ApiHelper.execute(() -> { oAuthHandler.setSourceInstancewideOauthParams(setInstancewideSourceOauthParamsRequestBody); return null; diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/StateApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/StateApiController.java index e148c1010fe3..0692d4f475ba 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/StateApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/StateApiController.java @@ -9,20 +9,25 @@ import io.airbyte.api.model.generated.ConnectionState; import io.airbyte.api.model.generated.ConnectionStateCreateOrUpdate; import io.airbyte.server.handlers.StateHandler; -import javax.ws.rs.Path; -import lombok.AllArgsConstructor; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; -@Path("/v1/state") -@AllArgsConstructor +@Controller("/api/v1/state") public class StateApiController implements StateApi { private final StateHandler stateHandler; + public StateApiController(final StateHandler stateHandler) { + this.stateHandler = stateHandler; + } + + @Post("/create_or_update") @Override public ConnectionState createOrUpdateState(final ConnectionStateCreateOrUpdate connectionStateCreateOrUpdate) { return ApiHelper.execute(() -> stateHandler.createOrUpdateState(connectionStateCreateOrUpdate)); } + @Post("/get") @Override public ConnectionState getState(final ConnectionIdRequestBody connectionIdRequestBody) { return ApiHelper.execute(() -> stateHandler.getState(connectionIdRequestBody)); diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/WebBackendApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/WebBackendApiController.java index 046efc95fa5e..a31a5cfac73b 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/WebBackendApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/WebBackendApiController.java @@ -20,52 +20,67 @@ import io.airbyte.server.handlers.WebBackendCheckUpdatesHandler; import io.airbyte.server.handlers.WebBackendConnectionsHandler; import io.airbyte.server.handlers.WebBackendGeographiesHandler; -import javax.ws.rs.Path; -import lombok.AllArgsConstructor; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; -@Path("/v1/web_backend") -@AllArgsConstructor +@Controller("/api/v1/web_backend") public class WebBackendApiController implements WebBackendApi { private final WebBackendConnectionsHandler webBackendConnectionsHandler; private final WebBackendGeographiesHandler webBackendGeographiesHandler; private final WebBackendCheckUpdatesHandler webBackendCheckUpdatesHandler; + public WebBackendApiController(final WebBackendConnectionsHandler webBackendConnectionsHandler, + final WebBackendGeographiesHandler webBackendGeographiesHandler, + final WebBackendCheckUpdatesHandler webBackendCheckUpdatesHandler) { + this.webBackendConnectionsHandler = webBackendConnectionsHandler; + this.webBackendGeographiesHandler = webBackendGeographiesHandler; + this.webBackendCheckUpdatesHandler = webBackendCheckUpdatesHandler; + } + + @Post("/state/get_type") @Override public ConnectionStateType getStateType(final ConnectionIdRequestBody connectionIdRequestBody) { return ApiHelper.execute(() -> webBackendConnectionsHandler.getStateType(connectionIdRequestBody)); } + @Post("/check_updates") @Override public WebBackendCheckUpdatesRead webBackendCheckUpdates() { return ApiHelper.execute(webBackendCheckUpdatesHandler::checkUpdates); } + @Post("/connections/create") @Override public WebBackendConnectionRead webBackendCreateConnection(final WebBackendConnectionCreate webBackendConnectionCreate) { return ApiHelper.execute(() -> webBackendConnectionsHandler.webBackendCreateConnection(webBackendConnectionCreate)); } + @Post("/connections/get") @Override public WebBackendConnectionRead webBackendGetConnection(final WebBackendConnectionRequestBody webBackendConnectionRequestBody) { return ApiHelper.execute(() -> webBackendConnectionsHandler.webBackendGetConnection(webBackendConnectionRequestBody)); } + @Post("/workspace/state") @Override public WebBackendWorkspaceStateResult webBackendGetWorkspaceState(final WebBackendWorkspaceState webBackendWorkspaceState) { return ApiHelper.execute(() -> webBackendConnectionsHandler.getWorkspaceState(webBackendWorkspaceState)); } + @Post("/connections/list") @Override public WebBackendConnectionReadList webBackendListConnectionsForWorkspace(final WebBackendConnectionListRequestBody webBackendConnectionListRequestBody) { return ApiHelper.execute(() -> webBackendConnectionsHandler.webBackendListConnectionsForWorkspace(webBackendConnectionListRequestBody)); } + @Post("/geographies/list") @Override public WebBackendGeographiesListResult webBackendListGeographies() { return ApiHelper.execute(webBackendGeographiesHandler::listGeographiesOSS); } + @Post("/connections/update") @Override public WebBackendConnectionRead webBackendUpdateConnection(final WebBackendConnectionUpdate webBackendConnectionUpdate) { return ApiHelper.execute(() -> webBackendConnectionsHandler.webBackendUpdateConnection(webBackendConnectionUpdate)); diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/WorkspaceApiController.java b/airbyte-server/src/main/java/io/airbyte/server/apis/WorkspaceApiController.java index bfba9954cfaf..fa4242850d84 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/WorkspaceApiController.java +++ b/airbyte-server/src/main/java/io/airbyte/server/apis/WorkspaceApiController.java @@ -15,63 +15,76 @@ import io.airbyte.api.model.generated.WorkspaceUpdate; import io.airbyte.api.model.generated.WorkspaceUpdateName; import io.airbyte.server.handlers.WorkspacesHandler; -import javax.ws.rs.Path; -import lombok.AllArgsConstructor; +import io.micronaut.http.annotation.Body; +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Post; -@Path("/v1/workspaces") -@AllArgsConstructor +@Controller("/api/v1/workspaces") public class WorkspaceApiController implements WorkspaceApi { private final WorkspacesHandler workspacesHandler; + public WorkspaceApiController(final WorkspacesHandler workspacesHandler) { + this.workspacesHandler = workspacesHandler; + } + + @Post("/create") @Override - public WorkspaceRead createWorkspace(final WorkspaceCreate workspaceCreate) { + public WorkspaceRead createWorkspace(@Body final WorkspaceCreate workspaceCreate) { return ApiHelper.execute(() -> workspacesHandler.createWorkspace(workspaceCreate)); } + @Post("/delete") @Override - public void deleteWorkspace(final WorkspaceIdRequestBody workspaceIdRequestBody) { + public void deleteWorkspace(@Body final WorkspaceIdRequestBody workspaceIdRequestBody) { ApiHelper.execute(() -> { workspacesHandler.deleteWorkspace(workspaceIdRequestBody); return null; }); } + @Post("/get") @Override - public WorkspaceRead getWorkspace(final WorkspaceIdRequestBody workspaceIdRequestBody) { + public WorkspaceRead getWorkspace(@Body final WorkspaceIdRequestBody workspaceIdRequestBody) { return ApiHelper.execute(() -> workspacesHandler.getWorkspace(workspaceIdRequestBody)); } + @Post("/get_by_slug") @Override - public WorkspaceRead getWorkspaceBySlug(final SlugRequestBody slugRequestBody) { + public WorkspaceRead getWorkspaceBySlug(@Body final SlugRequestBody slugRequestBody) { return ApiHelper.execute(() -> workspacesHandler.getWorkspaceBySlug(slugRequestBody)); } + @Post("/list") @Override public WorkspaceReadList listWorkspaces() { return ApiHelper.execute(workspacesHandler::listWorkspaces); } + @Post("/update") @Override - public WorkspaceRead updateWorkspace(final WorkspaceUpdate workspaceUpdate) { + public WorkspaceRead updateWorkspace(@Body final WorkspaceUpdate workspaceUpdate) { return ApiHelper.execute(() -> workspacesHandler.updateWorkspace(workspaceUpdate)); } + @Post("/tag_feedback_status_as_done") @Override - public void updateWorkspaceFeedback(final WorkspaceGiveFeedback workspaceGiveFeedback) { + public void updateWorkspaceFeedback(@Body final WorkspaceGiveFeedback workspaceGiveFeedback) { ApiHelper.execute(() -> { workspacesHandler.setFeedbackDone(workspaceGiveFeedback); return null; }); } + @Post("/update_name") @Override - public WorkspaceRead updateWorkspaceName(final WorkspaceUpdateName workspaceUpdateName) { + public WorkspaceRead updateWorkspaceName(@Body final WorkspaceUpdateName workspaceUpdateName) { return ApiHelper.execute(() -> workspacesHandler.updateWorkspaceName(workspaceUpdateName)); } + @Post("/get_by_connection_id") @Override - public WorkspaceRead getWorkspaceByConnectionId(final ConnectionIdRequestBody connectionIdRequestBody) { + public WorkspaceRead getWorkspaceByConnectionId(@Body final ConnectionIdRequestBody connectionIdRequestBody) { return ApiHelper.execute(() -> workspacesHandler.getWorkspaceByConnectionId(connectionIdRequestBody)); } diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/AttemptApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/AttemptApiBinder.java deleted file mode 100644 index 2eb09dddaf02..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/AttemptApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.AttemptApiController; -import io.airbyte.server.apis.factories.AttemptApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class AttemptApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(AttemptApiFactory.class) - .to(AttemptApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/ConnectionApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/ConnectionApiBinder.java deleted file mode 100644 index d95fe3385514..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/ConnectionApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.ConnectionApiController; -import io.airbyte.server.apis.factories.ConnectionApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class ConnectionApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(ConnectionApiFactory.class) - .to(ConnectionApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/DestinationApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/DestinationApiBinder.java deleted file mode 100644 index a1ae3cd38e8a..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/DestinationApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.DestinationApiController; -import io.airbyte.server.apis.factories.DestinationApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class DestinationApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(DestinationApiFactory.class) - .to(DestinationApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/DestinationDefinitionApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/DestinationDefinitionApiBinder.java deleted file mode 100644 index 1d9279119796..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/DestinationDefinitionApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.DestinationDefinitionApiController; -import io.airbyte.server.apis.factories.DestinationDefinitionApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class DestinationDefinitionApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(DestinationDefinitionApiFactory.class) - .to(DestinationDefinitionApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/DestinationDefinitionSpecificationApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/DestinationDefinitionSpecificationApiBinder.java deleted file mode 100644 index 3d166bd139af..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/DestinationDefinitionSpecificationApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.DestinationDefinitionSpecificationApiController; -import io.airbyte.server.apis.factories.DestinationDefinitionSpecificationApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class DestinationDefinitionSpecificationApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(DestinationDefinitionSpecificationApiFactory.class) - .to(DestinationDefinitionSpecificationApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/DestinationOauthApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/DestinationOauthApiBinder.java deleted file mode 100644 index ff2da2c5f0e8..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/DestinationOauthApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.DestinationOauthApiController; -import io.airbyte.server.apis.factories.DestinationOauthApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class DestinationOauthApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(DestinationOauthApiFactory.class) - .to(DestinationOauthApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/HealthApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/HealthApiBinder.java deleted file mode 100644 index bfe6161529f8..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/HealthApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.HealthApiController; -import io.airbyte.server.apis.factories.HealthApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class HealthApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(HealthApiFactory.class) - .to(HealthApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/JobsApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/JobsApiBinder.java deleted file mode 100644 index 8f96f02357c0..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/JobsApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.JobsApiController; -import io.airbyte.server.apis.factories.JobsApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class JobsApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(JobsApiFactory.class) - .to(JobsApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/LogsApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/LogsApiBinder.java deleted file mode 100644 index 037597e9e5f1..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/LogsApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.LogsApiController; -import io.airbyte.server.apis.factories.LogsApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class LogsApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(LogsApiFactory.class) - .to(LogsApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/NotificationApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/NotificationApiBinder.java deleted file mode 100644 index 009c3e2d48b5..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/NotificationApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.NotificationsApiController; -import io.airbyte.server.apis.factories.NotificationsApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class NotificationApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(NotificationsApiFactory.class) - .to(NotificationsApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/OpenapiApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/OpenapiApiBinder.java deleted file mode 100644 index 820da88a9206..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/OpenapiApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.OpenapiApiController; -import io.airbyte.server.apis.factories.OpenapiApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class OpenapiApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(OpenapiApiFactory.class) - .to(OpenapiApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/OperationApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/OperationApiBinder.java deleted file mode 100644 index 52d55383aeca..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/OperationApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.OperationApiController; -import io.airbyte.server.apis.factories.OperationApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class OperationApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(OperationApiFactory.class) - .to(OperationApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/SchedulerApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/SchedulerApiBinder.java deleted file mode 100644 index 64a904193b90..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/SchedulerApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.SchedulerApiController; -import io.airbyte.server.apis.factories.SchedulerApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class SchedulerApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(SchedulerApiFactory.class) - .to(SchedulerApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/SourceApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/SourceApiBinder.java deleted file mode 100644 index 9707ee1d1477..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/SourceApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.SourceApiController; -import io.airbyte.server.apis.factories.SourceApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class SourceApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(SourceApiFactory.class) - .to(SourceApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/SourceDefinitionApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/SourceDefinitionApiBinder.java deleted file mode 100644 index dfc976191c14..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/SourceDefinitionApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.SourceDefinitionApiController; -import io.airbyte.server.apis.factories.SourceDefinitionApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class SourceDefinitionApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(SourceDefinitionApiFactory.class) - .to(SourceDefinitionApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/SourceDefinitionSpecificationApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/SourceDefinitionSpecificationApiBinder.java deleted file mode 100644 index 5a5ae758f019..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/SourceDefinitionSpecificationApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.SourceDefinitionSpecificationApiController; -import io.airbyte.server.apis.factories.SourceDefinitionSpecificationApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class SourceDefinitionSpecificationApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(SourceDefinitionSpecificationApiFactory.class) - .to(SourceDefinitionSpecificationApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/SourceOauthApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/SourceOauthApiBinder.java deleted file mode 100644 index 744099e4cf80..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/SourceOauthApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.SourceOauthApiController; -import io.airbyte.server.apis.factories.SourceOauthApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class SourceOauthApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(SourceOauthApiFactory.class) - .to(SourceOauthApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/StateApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/StateApiBinder.java deleted file mode 100644 index 65ab669528c0..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/StateApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.StateApiController; -import io.airbyte.server.apis.factories.StateApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class StateApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(StateApiFactory.class) - .to(StateApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/WebBackendApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/WebBackendApiBinder.java deleted file mode 100644 index 8ad66204be33..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/WebBackendApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.WebBackendApiController; -import io.airbyte.server.apis.factories.WebBackendApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class WebBackendApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(WebBackendApiFactory.class) - .to(WebBackendApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/WorkspaceApiBinder.java b/airbyte-server/src/main/java/io/airbyte/server/apis/binders/WorkspaceApiBinder.java deleted file mode 100644 index f114235bac50..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/binders/WorkspaceApiBinder.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.binders; - -import io.airbyte.server.apis.WorkspaceApiController; -import io.airbyte.server.apis.factories.WorkspaceApiFactory; -import org.glassfish.hk2.utilities.binding.AbstractBinder; -import org.glassfish.jersey.process.internal.RequestScoped; - -public class WorkspaceApiBinder extends AbstractBinder { - - @Override - protected void configure() { - bindFactory(WorkspaceApiFactory.class) - .to(WorkspaceApiController.class) - .in(RequestScoped.class); - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/AttemptApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/AttemptApiFactory.java deleted file mode 100644 index 27fb62696b41..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/AttemptApiFactory.java +++ /dev/null @@ -1,35 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.AttemptApiController; -import io.airbyte.server.handlers.AttemptHandler; -import java.util.Map; -import org.glassfish.hk2.api.Factory; -import org.slf4j.MDC; - -public class AttemptApiFactory implements Factory { - - private static AttemptHandler attemptHandler; - private static Map mdc; - - public static void setValues(final AttemptHandler attemptHandler, final Map mdc) { - AttemptApiFactory.attemptHandler = attemptHandler; - AttemptApiFactory.mdc = mdc; - } - - @Override - public AttemptApiController provide() { - MDC.setContextMap(AttemptApiFactory.mdc); - - return new AttemptApiController(attemptHandler); - } - - @Override - public void dispose(final AttemptApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/ConnectionApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/ConnectionApiFactory.java deleted file mode 100644 index 7378d342b65f..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/ConnectionApiFactory.java +++ /dev/null @@ -1,44 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.ConnectionApiController; -import io.airbyte.server.handlers.ConnectionsHandler; -import io.airbyte.server.handlers.OperationsHandler; -import io.airbyte.server.handlers.SchedulerHandler; -import java.util.Map; -import org.glassfish.hk2.api.Factory; -import org.slf4j.MDC; - -public class ConnectionApiFactory implements Factory { - - private static ConnectionsHandler connectionsHandler; - private static OperationsHandler operationsHandler; - private static SchedulerHandler schedulerHandler; - private static Map mdc; - - public static void setValues(final ConnectionsHandler connectionsHandler, - final OperationsHandler operationsHandler, - final SchedulerHandler schedulerHandler, - final Map mdc) { - ConnectionApiFactory.connectionsHandler = connectionsHandler; - ConnectionApiFactory.operationsHandler = operationsHandler; - ConnectionApiFactory.schedulerHandler = schedulerHandler; - ConnectionApiFactory.mdc = mdc; - } - - @Override - public ConnectionApiController provide() { - MDC.setContextMap(ConnectionApiFactory.mdc); - - return new ConnectionApiController(connectionsHandler, operationsHandler, schedulerHandler); - } - - @Override - public void dispose(final ConnectionApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/DestinationApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/DestinationApiFactory.java deleted file mode 100644 index 2dcd6aa62714..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/DestinationApiFactory.java +++ /dev/null @@ -1,40 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.DestinationApiController; -import io.airbyte.server.handlers.DestinationHandler; -import io.airbyte.server.handlers.SchedulerHandler; -import java.util.Map; -import org.glassfish.hk2.api.Factory; -import org.slf4j.MDC; - -public class DestinationApiFactory implements Factory { - - private static DestinationHandler destinationHandler; - private static SchedulerHandler schedulerHandler; - private static Map mdc; - - public static void setValues(final DestinationHandler destinationHandler, - final SchedulerHandler schedulerHandler, - final Map mdc) { - DestinationApiFactory.destinationHandler = destinationHandler; - DestinationApiFactory.schedulerHandler = schedulerHandler; - DestinationApiFactory.mdc = mdc; - } - - @Override - public DestinationApiController provide() { - MDC.setContextMap(DestinationApiFactory.mdc); - - return new DestinationApiController(destinationHandler, schedulerHandler); - } - - @Override - public void dispose(final DestinationApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/DestinationDefinitionApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/DestinationDefinitionApiFactory.java deleted file mode 100644 index f76f631145ce..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/DestinationDefinitionApiFactory.java +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.DestinationDefinitionApiController; -import io.airbyte.server.handlers.DestinationDefinitionsHandler; -import org.glassfish.hk2.api.Factory; - -public class DestinationDefinitionApiFactory implements Factory { - - private static DestinationDefinitionsHandler destinationDefinitionsHandler; - - public static void setValues(final DestinationDefinitionsHandler destinationDefinitionsHandler) { - DestinationDefinitionApiFactory.destinationDefinitionsHandler = destinationDefinitionsHandler; - } - - @Override - public DestinationDefinitionApiController provide() { - return new DestinationDefinitionApiController(destinationDefinitionsHandler); - } - - @Override - public void dispose(final DestinationDefinitionApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/DestinationDefinitionSpecificationApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/DestinationDefinitionSpecificationApiFactory.java deleted file mode 100644 index d7dcbfac229d..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/DestinationDefinitionSpecificationApiFactory.java +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.DestinationDefinitionSpecificationApiController; -import io.airbyte.server.handlers.SchedulerHandler; -import org.glassfish.hk2.api.Factory; - -public class DestinationDefinitionSpecificationApiFactory implements Factory { - - private static SchedulerHandler schedulerHandler; - - public static void setValues(final SchedulerHandler schedulerHandler) { - DestinationDefinitionSpecificationApiFactory.schedulerHandler = schedulerHandler; - } - - @Override - public DestinationDefinitionSpecificationApiController provide() { - return new DestinationDefinitionSpecificationApiController(DestinationDefinitionSpecificationApiFactory.schedulerHandler); - } - - @Override - public void dispose(final DestinationDefinitionSpecificationApiController instance) { - - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/DestinationOauthApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/DestinationOauthApiFactory.java deleted file mode 100644 index 0548b8a6b16e..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/DestinationOauthApiFactory.java +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.DestinationOauthApiController; -import io.airbyte.server.handlers.OAuthHandler; -import org.glassfish.hk2.api.Factory; - -public class DestinationOauthApiFactory implements Factory { - - private static OAuthHandler oAuthHandler; - - public static void setValues(final OAuthHandler oAuthHandler) { - DestinationOauthApiFactory.oAuthHandler = oAuthHandler; - } - - @Override - public DestinationOauthApiController provide() { - return new DestinationOauthApiController(oAuthHandler); - } - - @Override - public void dispose(final DestinationOauthApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/HealthApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/HealthApiFactory.java deleted file mode 100644 index b13c17d5ebb2..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/HealthApiFactory.java +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.HealthApiController; -import io.airbyte.server.handlers.HealthCheckHandler; -import org.glassfish.hk2.api.Factory; - -public class HealthApiFactory implements Factory { - - private static HealthCheckHandler healthCheckHandler; - - public static void setValues(final HealthCheckHandler healthCheckHandler) { - HealthApiFactory.healthCheckHandler = healthCheckHandler; - } - - @Override - public HealthApiController provide() { - return new HealthApiController(healthCheckHandler); - } - - @Override - public void dispose(final HealthApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/JobsApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/JobsApiFactory.java deleted file mode 100644 index 618a15dc1bf0..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/JobsApiFactory.java +++ /dev/null @@ -1,32 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.JobsApiController; -import io.airbyte.server.handlers.JobHistoryHandler; -import io.airbyte.server.handlers.SchedulerHandler; -import org.glassfish.hk2.api.Factory; - -public class JobsApiFactory implements Factory { - - private static JobHistoryHandler jobHistoryHandler; - private static SchedulerHandler schedulerHandler; - - public static void setValues(final JobHistoryHandler jobHistoryHandler, final SchedulerHandler schedulerHandler) { - JobsApiFactory.jobHistoryHandler = jobHistoryHandler; - JobsApiFactory.schedulerHandler = schedulerHandler; - } - - @Override - public JobsApiController provide() { - return new JobsApiController(jobHistoryHandler, schedulerHandler); - } - - @Override - public void dispose(final JobsApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/LogsApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/LogsApiFactory.java deleted file mode 100644 index 450419d6a515..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/LogsApiFactory.java +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.LogsApiController; -import io.airbyte.server.handlers.LogsHandler; -import org.glassfish.hk2.api.Factory; - -public class LogsApiFactory implements Factory { - - private static LogsHandler logsHandler; - - public static void setValues(final LogsHandler logsHandler) { - LogsApiFactory.logsHandler = logsHandler; - } - - @Override - public LogsApiController provide() { - return new LogsApiController(logsHandler); - } - - @Override - public void dispose(final LogsApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/NotificationsApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/NotificationsApiFactory.java deleted file mode 100644 index 29a3cf7e9096..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/NotificationsApiFactory.java +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.NotificationsApiController; -import io.airbyte.server.handlers.WorkspacesHandler; -import org.glassfish.hk2.api.Factory; - -public class NotificationsApiFactory implements Factory { - - private static WorkspacesHandler workspacesHandler; - - public static void setValues(final WorkspacesHandler workspacesHandler) { - NotificationsApiFactory.workspacesHandler = workspacesHandler; - } - - @Override - public NotificationsApiController provide() { - return new NotificationsApiController(NotificationsApiFactory.workspacesHandler); - } - - @Override - public void dispose(final NotificationsApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/OpenapiApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/OpenapiApiFactory.java deleted file mode 100644 index 239fc610188b..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/OpenapiApiFactory.java +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.OpenapiApiController; -import io.airbyte.server.handlers.OpenApiConfigHandler; -import org.glassfish.hk2.api.Factory; - -public class OpenapiApiFactory implements Factory { - - private static OpenApiConfigHandler openApiConfigHandler; - - public static void setValues(final OpenApiConfigHandler openApiConfigHandler) { - OpenapiApiFactory.openApiConfigHandler = openApiConfigHandler; - } - - @Override - public OpenapiApiController provide() { - return new OpenapiApiController(openApiConfigHandler); - } - - @Override - public void dispose(final OpenapiApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/OperationApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/OperationApiFactory.java deleted file mode 100644 index d74ad9138170..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/OperationApiFactory.java +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.OperationApiController; -import io.airbyte.server.handlers.OperationsHandler; -import org.glassfish.hk2.api.Factory; - -public class OperationApiFactory implements Factory { - - private static OperationsHandler operationsHandler; - - public static void setValues(final OperationsHandler operationsHandler) { - OperationApiFactory.operationsHandler = operationsHandler; - } - - @Override - public OperationApiController provide() { - return new OperationApiController(OperationApiFactory.operationsHandler); - } - - @Override - public void dispose(final OperationApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/SchedulerApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/SchedulerApiFactory.java deleted file mode 100644 index 7ba4cf9b9ff2..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/SchedulerApiFactory.java +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.SchedulerApiController; -import io.airbyte.server.handlers.SchedulerHandler; -import org.glassfish.hk2.api.Factory; - -public class SchedulerApiFactory implements Factory { - - private static SchedulerHandler schedulerHandler; - - public static void setValues(final SchedulerHandler schedulerHandler) { - SchedulerApiFactory.schedulerHandler = schedulerHandler; - } - - @Override - public SchedulerApiController provide() { - return new SchedulerApiController(SchedulerApiFactory.schedulerHandler); - } - - @Override - public void dispose(final SchedulerApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/SourceApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/SourceApiFactory.java deleted file mode 100644 index aceda0580dde..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/SourceApiFactory.java +++ /dev/null @@ -1,32 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.SourceApiController; -import io.airbyte.server.handlers.SchedulerHandler; -import io.airbyte.server.handlers.SourceHandler; -import org.glassfish.hk2.api.Factory; - -public class SourceApiFactory implements Factory { - - private static SchedulerHandler schedulerHandler; - private static SourceHandler sourceHandler; - - public static void setValues(final SchedulerHandler schedulerHandler, final SourceHandler sourceHandler) { - SourceApiFactory.schedulerHandler = schedulerHandler; - SourceApiFactory.sourceHandler = sourceHandler; - } - - @Override - public SourceApiController provide() { - return new SourceApiController(schedulerHandler, sourceHandler); - } - - @Override - public void dispose(final SourceApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/SourceDefinitionApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/SourceDefinitionApiFactory.java deleted file mode 100644 index 15a08c6c9e85..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/SourceDefinitionApiFactory.java +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.SourceDefinitionApiController; -import io.airbyte.server.handlers.SourceDefinitionsHandler; -import org.glassfish.hk2.api.Factory; - -public class SourceDefinitionApiFactory implements Factory { - - private static SourceDefinitionsHandler sourceDefinitionsHandler; - - public static void setValues(final SourceDefinitionsHandler sourceDefinitionsHandler) { - SourceDefinitionApiFactory.sourceDefinitionsHandler = sourceDefinitionsHandler; - } - - @Override - public SourceDefinitionApiController provide() { - return new SourceDefinitionApiController(SourceDefinitionApiFactory.sourceDefinitionsHandler); - } - - @Override - public void dispose(final SourceDefinitionApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/SourceDefinitionSpecificationApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/SourceDefinitionSpecificationApiFactory.java deleted file mode 100644 index 8ea1f7d456ec..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/SourceDefinitionSpecificationApiFactory.java +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.SourceDefinitionSpecificationApiController; -import io.airbyte.server.handlers.SchedulerHandler; -import org.glassfish.hk2.api.Factory; - -public class SourceDefinitionSpecificationApiFactory implements Factory { - - private static SchedulerHandler schedulerHandler; - - public static void setValues(final SchedulerHandler schedulerHandler) { - SourceDefinitionSpecificationApiFactory.schedulerHandler = schedulerHandler; - } - - @Override - public SourceDefinitionSpecificationApiController provide() { - return new SourceDefinitionSpecificationApiController(SourceDefinitionSpecificationApiFactory.schedulerHandler); - } - - @Override - public void dispose(final SourceDefinitionSpecificationApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/SourceOauthApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/SourceOauthApiFactory.java deleted file mode 100644 index a5aef45948ec..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/SourceOauthApiFactory.java +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.SourceOauthApiController; -import io.airbyte.server.handlers.OAuthHandler; -import org.glassfish.hk2.api.Factory; - -public class SourceOauthApiFactory implements Factory { - - private static OAuthHandler oAuthHandler; - - public static void setValues(final OAuthHandler oAuthHandler) { - SourceOauthApiFactory.oAuthHandler = oAuthHandler; - } - - @Override - public SourceOauthApiController provide() { - return new SourceOauthApiController(SourceOauthApiFactory.oAuthHandler); - } - - @Override - public void dispose(final SourceOauthApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/StateApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/StateApiFactory.java deleted file mode 100644 index 0498681d7629..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/StateApiFactory.java +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.StateApiController; -import io.airbyte.server.handlers.StateHandler; -import org.glassfish.hk2.api.Factory; - -public class StateApiFactory implements Factory { - - private static StateHandler stateHandler; - - public static void setValues(final StateHandler stateHandler) { - StateApiFactory.stateHandler = stateHandler; - } - - @Override - public StateApiController provide() { - return new StateApiController(StateApiFactory.stateHandler); - } - - @Override - public void dispose(final StateApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/WebBackendApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/WebBackendApiFactory.java deleted file mode 100644 index c3788637229c..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/WebBackendApiFactory.java +++ /dev/null @@ -1,38 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.WebBackendApiController; -import io.airbyte.server.handlers.WebBackendCheckUpdatesHandler; -import io.airbyte.server.handlers.WebBackendConnectionsHandler; -import io.airbyte.server.handlers.WebBackendGeographiesHandler; -import org.glassfish.hk2.api.Factory; - -public class WebBackendApiFactory implements Factory { - - private static WebBackendConnectionsHandler webBackendConnectionsHandler; - private static WebBackendGeographiesHandler webBackendGeographiesHandler; - private static WebBackendCheckUpdatesHandler webBackendCheckUpdatesHandler; - - public static void setValues(final WebBackendConnectionsHandler webBackendConnectionsHandler, - final WebBackendGeographiesHandler webBackendGeographiesHandler, - final WebBackendCheckUpdatesHandler webBackendCheckUpdatesHandler) { - WebBackendApiFactory.webBackendConnectionsHandler = webBackendConnectionsHandler; - WebBackendApiFactory.webBackendGeographiesHandler = webBackendGeographiesHandler; - WebBackendApiFactory.webBackendCheckUpdatesHandler = webBackendCheckUpdatesHandler; - } - - @Override - public WebBackendApiController provide() { - return new WebBackendApiController(WebBackendApiFactory.webBackendConnectionsHandler, WebBackendApiFactory.webBackendGeographiesHandler, - WebBackendApiFactory.webBackendCheckUpdatesHandler); - } - - @Override - public void dispose(final WebBackendApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/WorkspaceApiFactory.java b/airbyte-server/src/main/java/io/airbyte/server/apis/factories/WorkspaceApiFactory.java deleted file mode 100644 index 644c7fdadc9a..000000000000 --- a/airbyte-server/src/main/java/io/airbyte/server/apis/factories/WorkspaceApiFactory.java +++ /dev/null @@ -1,29 +0,0 @@ -/* - * Copyright (c) 2022 Airbyte, Inc., all rights reserved. - */ - -package io.airbyte.server.apis.factories; - -import io.airbyte.server.apis.WorkspaceApiController; -import io.airbyte.server.handlers.WorkspacesHandler; -import org.glassfish.hk2.api.Factory; - -public class WorkspaceApiFactory implements Factory { - - private static WorkspacesHandler workspacesHandler; - - public static void setValues(final WorkspacesHandler workspacesHandler) { - WorkspaceApiFactory.workspacesHandler = workspacesHandler; - } - - @Override - public WorkspaceApiController provide() { - return new WorkspaceApiController(WorkspaceApiFactory.workspacesHandler); - } - - @Override - public void dispose(final WorkspaceApiController instance) { - /* no op */ - } - -} diff --git a/airbyte-server/src/main/java/io/airbyte/server/config/ApplicationBeanFactory.java b/airbyte-server/src/main/java/io/airbyte/server/config/ApplicationBeanFactory.java new file mode 100644 index 000000000000..13111f657156 --- /dev/null +++ b/airbyte-server/src/main/java/io/airbyte/server/config/ApplicationBeanFactory.java @@ -0,0 +1,116 @@ +/* + * Copyright (c) 2022 Airbyte, Inc., all rights reserved. + */ + +package io.airbyte.server.config; + +import io.airbyte.analytics.TrackingClient; +import io.airbyte.commons.features.EnvVariableFeatureFlags; +import io.airbyte.commons.features.FeatureFlags; +import io.airbyte.commons.temporal.TemporalClient; +import io.airbyte.commons.version.AirbyteProtocolVersionRange; +import io.airbyte.commons.version.AirbyteVersion; +import io.airbyte.commons.version.Version; +import io.airbyte.config.Configs.DeploymentMode; +import io.airbyte.config.Configs.TrackingStrategy; +import io.airbyte.config.persistence.ConfigRepository; +import io.airbyte.config.persistence.split_secrets.JsonSecretsProcessor; +import io.airbyte.persistence.job.JobPersistence; +import io.airbyte.persistence.job.WebUrlHelper; +import io.airbyte.persistence.job.tracker.JobTracker; +import io.airbyte.server.scheduler.EventRunner; +import io.airbyte.server.scheduler.TemporalEventRunner; +import io.airbyte.server.services.AirbyteGithubStore; +import io.airbyte.validation.json.JsonSchemaValidator; +import io.micronaut.context.annotation.Factory; +import io.micronaut.context.annotation.Value; +import io.micronaut.core.util.StringUtils; +import jakarta.inject.Named; +import jakarta.inject.Singleton; +import java.nio.file.Path; +import java.util.Locale; +import java.util.UUID; +import java.util.function.Function; +import java.util.function.Supplier; + +@Factory +public class ApplicationBeanFactory { + + @Singleton + public Supplier randomUUIDSupplier() { + return () -> UUID.randomUUID(); + } + + @Singleton + public EventRunner eventRunner(final TemporalClient temporalClient) { + return new TemporalEventRunner(temporalClient); + } + + @Singleton + public TrackingStrategy trackingStrategy(@Value("${airbyte.tracking-strategy}") final String trackingStrategy) { + return convertToEnum(trackingStrategy, TrackingStrategy::valueOf, TrackingStrategy.LOGGING); + } + + @Singleton + public AirbyteVersion airbyteVersion(@Value("${airbyte.version}") final String airbyteVersion) { + return new AirbyteVersion(airbyteVersion); + } + + @Singleton + public DeploymentMode deploymentMode(@Value("${airbyte.deployment-mode}") final String deploymentMode) { + return convertToEnum(deploymentMode, DeploymentMode::valueOf, DeploymentMode.OSS); + } + + @Singleton + public JobTracker jobTracker( + final ConfigRepository configRepository, + final JobPersistence jobPersistence, + final TrackingClient trackingClient) { + return new JobTracker(configRepository, jobPersistence, trackingClient); + } + + @Singleton + public WebUrlHelper webUrlHelper(@Value("${airbyte.web-app.url}") final String webAppUrl) { + return new WebUrlHelper(webAppUrl); + } + + @Singleton + public FeatureFlags featureFlags() { + return new EnvVariableFeatureFlags(); + } + + @Singleton + @Named("workspaceRoot") + public Path workspaceRoot(@Value("${airbyte.workspace.root}") final String workspaceRoot) { + return Path.of(workspaceRoot); + } + + @Singleton + public JsonSecretsProcessor jsonSecretsProcessor(final FeatureFlags featureFlags) { + return JsonSecretsProcessor.builder() + .copySecrets(false) + .build(); + } + + @Singleton + public JsonSchemaValidator jsonSchemaValidator() { + return new JsonSchemaValidator(); + } + + @Singleton + public AirbyteGithubStore airbyteGithubStore() { + return AirbyteGithubStore.production(); + } + + @Singleton + public AirbyteProtocolVersionRange airbyteProtocolVersionRange( + @Value("${airbyte.protocol.min-version}") final String minVersion, + @Value("${airbyte.protocol.max-version}") final String maxVersion) { + return new AirbyteProtocolVersionRange(new Version(minVersion), new Version(maxVersion)); + } + + private T convertToEnum(final String value, final Function creatorFunction, final T defaultValue) { + return StringUtils.isNotEmpty(value) ? creatorFunction.apply(value.toUpperCase(Locale.ROOT)) : defaultValue; + } + +} diff --git a/airbyte-server/src/main/java/io/airbyte/server/config/CloudStorageBeanFactory.java b/airbyte-server/src/main/java/io/airbyte/server/config/CloudStorageBeanFactory.java new file mode 100644 index 000000000000..06ba03c9512c --- /dev/null +++ b/airbyte-server/src/main/java/io/airbyte/server/config/CloudStorageBeanFactory.java @@ -0,0 +1,92 @@ +/* + * Copyright (c) 2022 Airbyte, Inc., all rights reserved. + */ + +package io.airbyte.server.config; + +import io.airbyte.config.storage.CloudStorageConfigs; +import io.airbyte.config.storage.CloudStorageConfigs.GcsConfig; +import io.airbyte.config.storage.CloudStorageConfigs.MinioConfig; +import io.airbyte.config.storage.CloudStorageConfigs.S3Config; +import io.micronaut.context.annotation.Factory; +import io.micronaut.context.annotation.Requires; +import io.micronaut.context.annotation.Value; +import jakarta.inject.Named; +import jakarta.inject.Singleton; + +/** + * Micronaut bean factory for cloud storage-related singletons. + */ +@Factory +@SuppressWarnings("PMD.AvoidDuplicateLiterals") +public class CloudStorageBeanFactory { + + @Singleton + @Requires(property = "airbyte.cloud.storage.logs.type", + pattern = "(?i)^gcs$") + @Named("logStorageConfigs") + public CloudStorageConfigs gcsLogStorageConfigs( + @Value("${airbyte.cloud.storage.logs.gcs.bucket}") final String gcsLogBucket, + @Value("${airbyte.cloud.storage.logs.gcs.application-credentials}") final String googleApplicationCredentials) { + return CloudStorageConfigs.gcs(new GcsConfig(gcsLogBucket, googleApplicationCredentials)); + } + + @Singleton + @Requires(property = "airbyte.cloud.storage.logs.type", + pattern = "(?i)^minio$") + @Named("logStorageConfigs") + public CloudStorageConfigs minioLogStorageConfigs( + @Value("${airbyte.cloud.storage.logs.minio.access-key}") final String awsAccessKeyId, + @Value("${airbyte.cloud.storage.logs.minio.secret-access-key}") final String awsSecretAccessKey, + @Value("${airbyte.cloud.storage.logs.minio.bucket}") final String s3LogBucket, + @Value("${airbyte.cloud.storage.logs.minio.endpoint}") final String s3MinioEndpoint) { + return CloudStorageConfigs.minio(new MinioConfig(s3LogBucket, awsAccessKeyId, awsSecretAccessKey, s3MinioEndpoint)); + } + + @Singleton + @Requires(property = "airbyte.cloud.storage.logs.type", + pattern = "(?i)^s3$") + @Named("logStorageConfigs") + public CloudStorageConfigs s3LogStorageConfigs( + @Value("${airbyte.cloud.storage.logs.s3.access-key}") final String awsAccessKeyId, + @Value("${airbyte.cloud.storage.logs.s3.secret-access-key}") final String awsSecretAccessKey, + @Value("${airbyte.cloud.storage.logs.s3.bucket}") final String s3LogBucket, + @Value("${airbyte.cloud.storage.logs.s3.region}") final String s3LogBucketRegion) { + return CloudStorageConfigs.s3(new S3Config(s3LogBucket, awsAccessKeyId, awsSecretAccessKey, s3LogBucketRegion)); + } + + @Singleton + @Requires(property = "airbyte.cloud.storage.state.type", + pattern = "(?i)^gcs$") + @Named("stateStorageConfigs") + public CloudStorageConfigs gcsStateStorageConfiguration( + @Value("${airbyte.cloud.storage.state.gcs.bucket}") final String gcsBucketName, + @Value("${airbyte.cloud.storage.state.gcs.application-credentials}") final String gcsApplicationCredentials) { + return CloudStorageConfigs.gcs(new GcsConfig(gcsBucketName, gcsApplicationCredentials)); + } + + @Singleton + @Requires(property = "airbyte.cloud.storage.state.type", + pattern = "(?i)^minio$") + @Named("stateStorageConfigs") + public CloudStorageConfigs minioStateStorageConfiguration( + @Value("${airbyte.cloud.storage.state.minio.bucket}") final String bucketName, + @Value("${airbyte.cloud.storage.state.minio.access-key}") final String awsAccessKey, + @Value("${airbyte.cloud.storage.state.minio.secret-access-key}") final String secretAccessKey, + @Value("${airbyte.cloud.storage.state.minio.endpoint}") final String endpoint) { + return CloudStorageConfigs.minio(new MinioConfig(bucketName, awsAccessKey, secretAccessKey, endpoint)); + } + + @Singleton + @Requires(property = "airbyte.cloud.storage.state.type", + pattern = "(?i)^s3$") + @Named("stateStorageConfigs") + public CloudStorageConfigs s3StateStorageConfiguration( + @Value("${airbyte.cloud.storage.state.s3.bucket}") final String bucketName, + @Value("${airbyte.cloud.storage.state.s3.access-key}") final String awsAccessKey, + @Value("${airbyte.cloud.storage.state.s3.secret-access-key}") final String secretAcessKey, + @Value("${airbyte.cloud.storage.state.s3.region}") final String s3Region) { + return CloudStorageConfigs.s3(new S3Config(bucketName, awsAccessKey, secretAcessKey, s3Region)); + } + +} diff --git a/airbyte-server/src/main/java/io/airbyte/server/config/DatabaseBeanFactory.java b/airbyte-server/src/main/java/io/airbyte/server/config/DatabaseBeanFactory.java new file mode 100644 index 000000000000..999a40aa9db1 --- /dev/null +++ b/airbyte-server/src/main/java/io/airbyte/server/config/DatabaseBeanFactory.java @@ -0,0 +1,125 @@ +/* + * Copyright (c) 2022 Airbyte, Inc., all rights reserved. + */ + +package io.airbyte.server.config; + +import io.airbyte.config.persistence.ConfigRepository; +import io.airbyte.config.persistence.StatePersistence; +import io.airbyte.config.persistence.StreamResetPersistence; +import io.airbyte.db.Database; +import io.airbyte.db.check.DatabaseMigrationCheck; +import io.airbyte.db.check.impl.JobsDatabaseAvailabilityCheck; +import io.airbyte.db.factory.DatabaseCheckFactory; +import io.airbyte.db.instance.DatabaseConstants; +import io.airbyte.persistence.job.DefaultJobPersistence; +import io.airbyte.persistence.job.JobPersistence; +import io.micronaut.context.annotation.Factory; +import io.micronaut.context.annotation.Value; +import io.micronaut.flyway.FlywayConfigurationProperties; +import jakarta.inject.Named; +import jakarta.inject.Singleton; +import java.io.IOException; +import javax.sql.DataSource; +import lombok.extern.slf4j.Slf4j; +import org.flywaydb.core.Flyway; +import org.jooq.DSLContext; + +/** + * Micronaut bean factory for database-related singletons. + */ +@Factory +@Slf4j +@SuppressWarnings("PMD.AvoidDuplicateLiterals") +public class DatabaseBeanFactory { + + private static final String BASELINE_DESCRIPTION = "Baseline from file-based migration v1"; + private static final Boolean BASELINE_ON_MIGRATION = true; + private static final String INSTALLED_BY = "ServerApp"; + + @Singleton + @Named("configDatabase") + public Database configDatabase(@Named("config") final DSLContext dslContext) throws IOException { + return new Database(dslContext); + } + + @Singleton + @Named("configFlyway") + public Flyway configFlyway(@Named("config") final FlywayConfigurationProperties configFlywayConfigurationProperties, + @Named("config") final DataSource configDataSource, + @Value("${airbyte.flyway.configs.minimum-migration-version}") final String baselineVersion) { + return configFlywayConfigurationProperties.getFluentConfiguration() + .dataSource(configDataSource) + .baselineVersion(baselineVersion) + .baselineDescription(BASELINE_DESCRIPTION) + .baselineOnMigrate(BASELINE_ON_MIGRATION) + .installedBy(INSTALLED_BY) + .table(String.format("airbyte_%s_migrations", "configs")) + .load(); + } + + @Singleton + @Named("jobsFlyway") + public Flyway jobsFlyway(@Named("jobs") final FlywayConfigurationProperties jobsFlywayConfigurationProperties, + @Named("config") final DataSource jobsDataSource, + @Value("${airbyte.flyway.jobs.minimum-migration-version}") final String baselineVersion) { + return jobsFlywayConfigurationProperties.getFluentConfiguration() + .dataSource(jobsDataSource) + .baselineVersion(baselineVersion) + .baselineDescription(BASELINE_DESCRIPTION) + .baselineOnMigrate(BASELINE_ON_MIGRATION) + .installedBy(INSTALLED_BY) + .table(String.format("airbyte_%s_migrations", "jobs")) + .load(); + } + + @Singleton + public ConfigRepository configRepository(@Named("configDatabase") final Database configDatabase) { + return new ConfigRepository(configDatabase); + } + + @Singleton + public JobPersistence jobPersistence(@Named("configDatabase") final Database jobDatabase) { + return new DefaultJobPersistence(jobDatabase); + } + + @Singleton + public StatePersistence statePersistence(@Named("configDatabase") final Database configDatabase) { + return new StatePersistence(configDatabase); + } + + @Singleton + @Named("configsDatabaseMigrationCheck") + public DatabaseMigrationCheck configsDatabaseMigrationCheck(@Named("config") final DSLContext dslContext, + @Named("configFlyway") final Flyway configsFlyway, + @Value("${airbyte.flyway.configs.minimum-migration-version}") final String configsDatabaseMinimumFlywayMigrationVersion, + @Value("${airbyte.flyway.configs.initialization-timeout-ms}") final Long configsDatabaseInitializationTimeoutMs) { + log.info("Configs database configuration: {} {}", configsDatabaseMinimumFlywayMigrationVersion, configsDatabaseInitializationTimeoutMs); + return DatabaseCheckFactory + .createConfigsDatabaseMigrationCheck(dslContext, configsFlyway, configsDatabaseMinimumFlywayMigrationVersion, + configsDatabaseInitializationTimeoutMs); + } + + @Singleton + @Named("jobsDatabaseMigrationCheck") + public DatabaseMigrationCheck jobsDatabaseMigrationCheck(@Named("config") final DSLContext dslContext, + @Named("jobsFlyway") final Flyway jobsFlyway, + @Value("${airbyte.flyway.jobs.minimum-migration-version}") final String jobsDatabaseMinimumFlywayMigrationVersion, + @Value("${airbyte.flyway.jobs.initialization-timeout-ms}") final Long jobsDatabaseInitializationTimeoutMs) { + return DatabaseCheckFactory + .createJobsDatabaseMigrationCheck(dslContext, jobsFlyway, jobsDatabaseMinimumFlywayMigrationVersion, + jobsDatabaseInitializationTimeoutMs); + } + + @Singleton + @Named("jobsDatabaseAvailabilityCheck") + public JobsDatabaseAvailabilityCheck jobsDatabaseAvailabilityCheck(@Named("config") final DSLContext dslContext) { + return new JobsDatabaseAvailabilityCheck(dslContext, DatabaseConstants.DEFAULT_ASSERT_DATABASE_TIMEOUT_MS); + } + + @Singleton + public StreamResetPersistence streamResetPersistence(@Named("configDatabase") final Database configDatabase) { + return new StreamResetPersistence(configDatabase); + } + +} diff --git a/airbyte-server/src/main/java/io/airbyte/server/config/HelperBeanFactory.java b/airbyte-server/src/main/java/io/airbyte/server/config/HelperBeanFactory.java new file mode 100644 index 000000000000..1f7b478f5acf --- /dev/null +++ b/airbyte-server/src/main/java/io/airbyte/server/config/HelperBeanFactory.java @@ -0,0 +1,21 @@ +/* + * Copyright (c) 2022 Airbyte, Inc., all rights reserved. + */ + +package io.airbyte.server.config; + +import io.airbyte.config.persistence.ConfigRepository; +import io.airbyte.persistence.job.JobPersistence; +import io.airbyte.persistence.job.WorkspaceHelper; +import io.micronaut.context.annotation.Factory; +import jakarta.inject.Singleton; + +@Factory +public class HelperBeanFactory { + + @Singleton + public WorkspaceHelper workspaceHelper(final ConfigRepository configRepository, final JobPersistence jobPersistence) { + return new WorkspaceHelper(configRepository, jobPersistence); + } + +} diff --git a/airbyte-server/src/main/java/io/airbyte/server/config/JobErrorReportingBeanFactory.java b/airbyte-server/src/main/java/io/airbyte/server/config/JobErrorReportingBeanFactory.java new file mode 100644 index 000000000000..10040e723a98 --- /dev/null +++ b/airbyte-server/src/main/java/io/airbyte/server/config/JobErrorReportingBeanFactory.java @@ -0,0 +1,61 @@ +/* + * Copyright (c) 2022 Airbyte, Inc., all rights reserved. + */ + +package io.airbyte.server.config; + +import io.airbyte.config.Configs.DeploymentMode; +import io.airbyte.config.persistence.ConfigRepository; +import io.airbyte.persistence.job.WebUrlHelper; +import io.airbyte.persistence.job.errorreporter.JobErrorReporter; +import io.airbyte.persistence.job.errorreporter.JobErrorReportingClient; +import io.airbyte.persistence.job.errorreporter.LoggingJobErrorReportingClient; +import io.airbyte.persistence.job.errorreporter.SentryExceptionHelper; +import io.airbyte.persistence.job.errorreporter.SentryJobErrorReportingClient; +import io.micronaut.context.annotation.Factory; +import io.micronaut.context.annotation.Requires; +import io.micronaut.context.annotation.Value; +import jakarta.inject.Named; +import jakarta.inject.Singleton; +import java.util.Optional; + +/** + * Micronaut bean factory for job error reporting-related singletons. + */ +@Factory +@SuppressWarnings("PMD.AvoidDuplicateLiterals") +public class JobErrorReportingBeanFactory { + + @Singleton + @Requires(property = "airbyte.worker.job.error-reporting.strategy", + pattern = "(?i)^sentry$") + @Named("jobErrorReportingClient") + public JobErrorReportingClient sentryJobErrorReportingClient( + @Value("${airbyte.worker.job.error-reporting.sentry.dsn}") final String sentryDsn) { + return new SentryJobErrorReportingClient(sentryDsn, new SentryExceptionHelper()); + } + + @Singleton + @Requires(property = "airbyte.worker.job.error-reporting.strategy", + pattern = "(?i)^logging$") + @Named("jobErrorReportingClient") + public JobErrorReportingClient loggingJobErrorReportingClient() { + return new LoggingJobErrorReportingClient(); + } + + @Singleton + public JobErrorReporter jobErrorReporter( + @Value("${airbyte.version}") final String airbyteVersion, + final ConfigRepository configRepository, + final DeploymentMode deploymentMode, + @Named("jobErrorReportingClient") final Optional jobErrorReportingClient, + final WebUrlHelper webUrlHelper) { + return new JobErrorReporter( + configRepository, + deploymentMode, + airbyteVersion, + webUrlHelper, + jobErrorReportingClient.orElseGet(() -> new LoggingJobErrorReportingClient())); + } + +} diff --git a/airbyte-server/src/main/java/io/airbyte/server/config/SecretPersistenceBeanFactory.java b/airbyte-server/src/main/java/io/airbyte/server/config/SecretPersistenceBeanFactory.java new file mode 100644 index 000000000000..2b1965189a8d --- /dev/null +++ b/airbyte-server/src/main/java/io/airbyte/server/config/SecretPersistenceBeanFactory.java @@ -0,0 +1,114 @@ +/* + * Copyright (c) 2022 Airbyte, Inc., all rights reserved. + */ + +package io.airbyte.server.config; + +import io.airbyte.config.persistence.ConfigRepository; +import io.airbyte.config.persistence.SecretsRepositoryReader; +import io.airbyte.config.persistence.SecretsRepositoryWriter; +import io.airbyte.config.persistence.split_secrets.GoogleSecretManagerPersistence; +import io.airbyte.config.persistence.split_secrets.LocalTestingSecretPersistence; +import io.airbyte.config.persistence.split_secrets.RealSecretsHydrator; +import io.airbyte.config.persistence.split_secrets.SecretPersistence; +import io.airbyte.config.persistence.split_secrets.SecretsHydrator; +import io.airbyte.config.persistence.split_secrets.VaultSecretPersistence; +import io.airbyte.db.Database; +import io.micronaut.context.annotation.Factory; +import io.micronaut.context.annotation.Requires; +import io.micronaut.context.annotation.Value; +import jakarta.inject.Named; +import jakarta.inject.Singleton; +import java.util.Optional; + +/** + * Micronaut bean factory for secret persistence-related singletons. + */ +@Factory +@SuppressWarnings("PMD.AvoidDuplicateLiterals") +public class SecretPersistenceBeanFactory { + + @Singleton + @Requires(property = "airbyte.secret.persistence", + pattern = "(?i)^(?!testing_config_db_table).*") + @Requires(property = "airbyte.secret.persistence", + pattern = "(?i)^(?!google_secret_manager).*") + @Requires(property = "airbyte.secret.persistence", + pattern = "(?i)^(?!vault).*") + @Named("secretPersistence") + public SecretPersistence defaultSecretPersistence(@Named("configDatabase") final Database configDatabase) { + return localTestingSecretPersistence(configDatabase); + } + + @Singleton + @Requires(property = "airbyte.secret.persistence", + pattern = "(?i)^testing_config_db_table$") + @Named("secretPersistence") + public SecretPersistence localTestingSecretPersistence(@Named("configDatabase") final Database configDatabase) { + return new LocalTestingSecretPersistence(configDatabase); + } + + @Singleton + @Requires(property = "airbyte.secret.persistence", + pattern = "(?i)^testing_config_db_table$") + @Named("ephemeralSecretPersistence") + public SecretPersistence ephemeralLocalTestingSecretPersistence(@Named("configDatabase") final Database configDatabase) { + return new LocalTestingSecretPersistence(configDatabase); + } + + @Singleton + @Requires(property = "airbyte.secret.persistence", + pattern = "(?i)^google_secret_manager$") + @Named("secretPersistence") + public SecretPersistence googleSecretPersistence(@Value("${airbyte.secret.store.gcp.credentials}") final String credentials, + @Value("${airbyte.secret.store.gcp.project-id}") final String projectId) { + return GoogleSecretManagerPersistence.getLongLived(projectId, credentials); + } + + @Singleton + @Requires(property = "airbyte.secret.persistence", + pattern = "(?i)^google_secret_manager$") + @Named("ephemeralSecretPersistence") + public SecretPersistence ephemeralGoogleSecretPersistence(@Value("${airbyte.secret.store.gcp.credentials}") final String credentials, + @Value("${airbyte.secret.store.gcp.project-id}") final String projectId) { + return GoogleSecretManagerPersistence.getEphemeral(projectId, credentials); + } + + @Singleton + @Requires(property = "airbyte.secret.persistence", + pattern = "(?i)^vault$") + @Named("secretPersistence") + public SecretPersistence vaultSecretPersistence(@Value("${airbyte.secret.store.vault.address}") final String address, + @Value("${airbyte.secret.store.vault.prefix}") final String prefix, + @Value("${airbyte.secret.store.vault.token}") final String token) { + return new VaultSecretPersistence(address, prefix, token); + } + + @Singleton + @Requires(property = "airbyte.secret.persistence", + pattern = "(?i)^vault$") + @Named("ephemeralSecretPersistence") + public SecretPersistence ephemeralVaultSecretPersistence(@Value("${airbyte.secret.store.vault.address}") final String address, + @Value("${airbyte.secret.store.vault.prefix}") final String prefix, + @Value("${airbyte.secret.store.vault.token}") final String token) { + return new VaultSecretPersistence(address, prefix, token); + } + + @Singleton + public SecretsHydrator secretsHydrator(@Named("secretPersistence") final SecretPersistence secretPersistence) { + return new RealSecretsHydrator(secretPersistence); + } + + @Singleton + public SecretsRepositoryReader secretsRepositoryReader(final ConfigRepository configRepository, final SecretsHydrator secretsHydrator) { + return new SecretsRepositoryReader(configRepository, secretsHydrator); + } + + @Singleton + public SecretsRepositoryWriter secretsRepositoryWriter(final ConfigRepository configRepository, + @Named("secretPersistence") final Optional secretPersistence, + @Named("ephemeralSecretPersistence") final Optional ephemeralSecretPersistence) { + return new SecretsRepositoryWriter(configRepository, secretPersistence, ephemeralSecretPersistence); + } + +} diff --git a/airbyte-server/src/main/java/io/airbyte/server/config/TemporalBeanFactory.java b/airbyte-server/src/main/java/io/airbyte/server/config/TemporalBeanFactory.java new file mode 100644 index 000000000000..069a3853193f --- /dev/null +++ b/airbyte-server/src/main/java/io/airbyte/server/config/TemporalBeanFactory.java @@ -0,0 +1,67 @@ +/* + * Copyright (c) 2022 Airbyte, Inc., all rights reserved. + */ + +package io.airbyte.server.config; + +import io.airbyte.analytics.Deployment; +import io.airbyte.analytics.TrackingClient; +import io.airbyte.analytics.TrackingClientSingleton; +import io.airbyte.commons.temporal.TemporalClient; +import io.airbyte.commons.version.AirbyteVersion; +import io.airbyte.config.Configs.DeploymentMode; +import io.airbyte.config.Configs.TrackingStrategy; +import io.airbyte.config.Configs.WorkerEnvironment; +import io.airbyte.config.persistence.ConfigRepository; +import io.airbyte.persistence.job.JobPersistence; +import io.airbyte.persistence.job.errorreporter.JobErrorReporter; +import io.airbyte.persistence.job.factory.OAuthConfigSupplier; +import io.airbyte.persistence.job.tracker.JobTracker; +import io.airbyte.server.scheduler.DefaultSynchronousSchedulerClient; +import io.airbyte.server.scheduler.SynchronousSchedulerClient; +import io.micronaut.context.annotation.Factory; +import io.micronaut.context.annotation.Value; +import jakarta.inject.Singleton; +import java.io.IOException; + +/** + * Micronaut bean factory for Temporal-related singletons. + */ +@Factory +public class TemporalBeanFactory { + + @Singleton + public TrackingClient trackingClient(final TrackingStrategy trackingStrategy, + final DeploymentMode deploymentMode, + final JobPersistence jobPersistence, + final WorkerEnvironment workerEnvironment, + @Value("${airbyte.role}") final String airbyteRole, + final AirbyteVersion airbyteVersion, + final ConfigRepository configRepository) + throws IOException { + + TrackingClientSingleton.initialize( + trackingStrategy, + new Deployment(deploymentMode, jobPersistence.getDeployment().orElseThrow(), + workerEnvironment), + airbyteRole, + airbyteVersion, + configRepository); + + return TrackingClientSingleton.get(); + } + + @Singleton + public OAuthConfigSupplier oAuthConfigSupplier(final ConfigRepository configRepository, final TrackingClient trackingClient) { + return new OAuthConfigSupplier(configRepository, trackingClient); + } + + @Singleton + public SynchronousSchedulerClient synchronousSchedulerClient(final TemporalClient temporalClient, + final JobTracker jobTracker, + final JobErrorReporter jobErrorReporter, + final OAuthConfigSupplier oAuthConfigSupplier) { + return new DefaultSynchronousSchedulerClient(temporalClient, jobTracker, jobErrorReporter, oAuthConfigSupplier); + } + +} diff --git a/airbyte-server/src/main/java/io/airbyte/server/converters/ConfigurationUpdate.java b/airbyte-server/src/main/java/io/airbyte/server/converters/ConfigurationUpdate.java index f4ac339423cd..8eaf31178ac7 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/converters/ConfigurationUpdate.java +++ b/airbyte-server/src/main/java/io/airbyte/server/converters/ConfigurationUpdate.java @@ -16,9 +16,11 @@ import io.airbyte.config.persistence.split_secrets.JsonSecretsProcessor; import io.airbyte.protocol.models.ConnectorSpecification; import io.airbyte.validation.json.JsonValidationException; +import jakarta.inject.Singleton; import java.io.IOException; import java.util.UUID; +@Singleton public class ConfigurationUpdate { private final ConfigRepository configRepository; diff --git a/airbyte-server/src/main/java/io/airbyte/server/converters/JobConverter.java b/airbyte-server/src/main/java/io/airbyte/server/converters/JobConverter.java index 52c28f3640f1..c1a19d84ffcb 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/converters/JobConverter.java +++ b/airbyte-server/src/main/java/io/airbyte/server/converters/JobConverter.java @@ -44,14 +44,14 @@ import io.airbyte.server.scheduler.SynchronousJobMetadata; import io.airbyte.server.scheduler.SynchronousResponse; import io.airbyte.workers.helper.ProtocolConverters; +import jakarta.inject.Singleton; import java.io.IOException; import java.nio.file.Path; import java.util.List; import java.util.Optional; import java.util.stream.Collectors; -import lombok.extern.slf4j.Slf4j; -@Slf4j +@Singleton public class JobConverter { private final WorkerEnvironment workerEnvironment; @@ -243,7 +243,7 @@ public SynchronousJobRead getSynchronousJobRead(final SynchronousJobMetadata met } public static AttemptNormalizationStatusRead convertAttemptNormalizationStatus( - AttemptNormalizationStatus databaseStatus) { + final AttemptNormalizationStatus databaseStatus) { return new AttemptNormalizationStatusRead() .attemptNumber(databaseStatus.attemptNumber()) .hasRecordsCommitted(!databaseStatus.recordsCommitted().isEmpty()) diff --git a/airbyte-server/src/main/java/io/airbyte/server/converters/WorkflowStateConverter.java b/airbyte-server/src/main/java/io/airbyte/server/converters/WorkflowStateConverter.java index ff68bb6b1dd2..76bd67ab2067 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/converters/WorkflowStateConverter.java +++ b/airbyte-server/src/main/java/io/airbyte/server/converters/WorkflowStateConverter.java @@ -6,7 +6,9 @@ import io.airbyte.api.model.generated.WorkflowStateRead; import io.airbyte.commons.temporal.scheduling.state.WorkflowState; +import jakarta.inject.Singleton; +@Singleton public class WorkflowStateConverter { public WorkflowStateRead getWorkflowStateRead(final WorkflowState workflowState) { diff --git a/airbyte-server/src/main/java/io/airbyte/server/errors/IdNotFoundExceptionMapper.java b/airbyte-server/src/main/java/io/airbyte/server/errors/IdNotFoundExceptionMapper.java new file mode 100644 index 000000000000..4cd9b709cf04 --- /dev/null +++ b/airbyte-server/src/main/java/io/airbyte/server/errors/IdNotFoundExceptionMapper.java @@ -0,0 +1,35 @@ +/* + * Copyright (c) 2022 Airbyte, Inc., all rights reserved. + */ + +package io.airbyte.server.errors; + +import io.micronaut.context.annotation.Requires; +import io.micronaut.http.HttpRequest; +import io.micronaut.http.HttpResponse; +import io.micronaut.http.HttpStatus; +import io.micronaut.http.MediaType; +import io.micronaut.http.annotation.Produces; +import io.micronaut.http.server.exceptions.ExceptionHandler; +import jakarta.inject.Singleton; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +@Produces +@Singleton +@Requires(classes = IdNotFoundKnownException.class) +public class IdNotFoundExceptionMapper implements ExceptionHandler { + + private static final Logger LOGGER = LoggerFactory.getLogger(IdNotFoundExceptionMapper.class); + + @Override + public HttpResponse handle(final HttpRequest request, final IdNotFoundKnownException exception) { + final IdNotFoundKnownException idnf = new IdNotFoundKnownException("Id not found: " + exception.getMessage(), exception); + LOGGER.error("Not found exception", idnf.getNotFoundKnownExceptionInfo()); + + return HttpResponse.status(HttpStatus.NOT_FOUND) + .body(KnownException.infoFromThrowableWithMessage(exception, "Internal Server Error: " + exception.getMessage())) + .contentType(MediaType.APPLICATION_JSON); + } + +} diff --git a/airbyte-server/src/main/java/io/airbyte/server/errors/InvalidInputExceptionMapper.java b/airbyte-server/src/main/java/io/airbyte/server/errors/InvalidInputExceptionMapper.java index 44629340c4ac..26a3c9ca2947 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/errors/InvalidInputExceptionMapper.java +++ b/airbyte-server/src/main/java/io/airbyte/server/errors/InvalidInputExceptionMapper.java @@ -7,19 +7,26 @@ import io.airbyte.api.model.generated.InvalidInputExceptionInfo; import io.airbyte.api.model.generated.InvalidInputProperty; import io.airbyte.commons.json.Jsons; +import io.micronaut.context.annotation.Requires; +import io.micronaut.http.HttpRequest; +import io.micronaut.http.HttpResponse; +import io.micronaut.http.HttpStatus; +import io.micronaut.http.MediaType; +import io.micronaut.http.annotation.Produces; +import io.micronaut.http.server.exceptions.ExceptionHandler; +import jakarta.inject.Singleton; import java.util.ArrayList; import java.util.List; import javax.validation.ConstraintViolation; import javax.validation.ConstraintViolationException; -import javax.ws.rs.core.Response; -import javax.ws.rs.ext.ExceptionMapper; -import javax.ws.rs.ext.Provider; import org.apache.logging.log4j.core.util.Throwables; // https://www.baeldung.com/jersey-bean-validation#custom-exception-handler // handles exceptions related to the request body not matching the openapi config. -@Provider -public class InvalidInputExceptionMapper implements ExceptionMapper { +@Produces +@Singleton +@Requires(classes = ConstraintViolationException.class) +public class InvalidInputExceptionMapper implements ExceptionHandler { public static InvalidInputExceptionInfo infoFromConstraints(final ConstraintViolationException cve) { final InvalidInputExceptionInfo exceptionInfo = new InvalidInputExceptionInfo() @@ -39,11 +46,10 @@ public static InvalidInputExceptionInfo infoFromConstraints(final ConstraintViol } @Override - public Response toResponse(final ConstraintViolationException e) { - return Response.status(Response.Status.BAD_REQUEST) - .entity(Jsons.serialize(InvalidInputExceptionMapper.infoFromConstraints(e))) - .type("application/json") - .build(); + public HttpResponse handle(final HttpRequest request, final ConstraintViolationException exception) { + return HttpResponse.status(HttpStatus.BAD_REQUEST) + .body(Jsons.serialize(InvalidInputExceptionMapper.infoFromConstraints(exception))) + .contentType(MediaType.APPLICATION_JSON_TYPE); } } diff --git a/airbyte-server/src/main/java/io/airbyte/server/errors/InvalidJsonExceptionMapper.java b/airbyte-server/src/main/java/io/airbyte/server/errors/InvalidJsonExceptionMapper.java index c126f1299325..c1e88694958a 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/errors/InvalidJsonExceptionMapper.java +++ b/airbyte-server/src/main/java/io/airbyte/server/errors/InvalidJsonExceptionMapper.java @@ -5,19 +5,26 @@ package io.airbyte.server.errors; import com.fasterxml.jackson.core.JsonParseException; -import javax.ws.rs.core.Response; -import javax.ws.rs.ext.ExceptionMapper; -import javax.ws.rs.ext.Provider; +import io.micronaut.context.annotation.Requires; +import io.micronaut.http.HttpRequest; +import io.micronaut.http.HttpResponse; +import io.micronaut.http.HttpStatus; +import io.micronaut.http.MediaType; +import io.micronaut.http.annotation.Produces; +import io.micronaut.http.server.exceptions.ExceptionHandler; +import jakarta.inject.Singleton; -@Provider -public class InvalidJsonExceptionMapper implements ExceptionMapper { +@Produces +@Singleton +@Requires(classes = JsonParseException.class) +public class InvalidJsonExceptionMapper implements ExceptionHandler { @Override - public Response toResponse(final JsonParseException e) { - return Response.status(422) - .entity(KnownException.infoFromThrowableWithMessage(e, "Invalid json. " + e.getMessage() + " " + e.getOriginalMessage())) - .type("application/json") - .build(); + public HttpResponse handle(final HttpRequest request, final JsonParseException exception) { + return HttpResponse.status(HttpStatus.UNPROCESSABLE_ENTITY) + .body( + KnownException.infoFromThrowableWithMessage(exception, "Invalid json. " + exception.getMessage() + " " + exception.getOriginalMessage())) + .contentType(MediaType.APPLICATION_JSON_TYPE); } } diff --git a/airbyte-server/src/main/java/io/airbyte/server/errors/InvalidJsonInputExceptionMapper.java b/airbyte-server/src/main/java/io/airbyte/server/errors/InvalidJsonInputExceptionMapper.java index 90700f771782..06e917f9119c 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/errors/InvalidJsonInputExceptionMapper.java +++ b/airbyte-server/src/main/java/io/airbyte/server/errors/InvalidJsonInputExceptionMapper.java @@ -5,21 +5,26 @@ package io.airbyte.server.errors; import com.fasterxml.jackson.databind.JsonMappingException; -import io.airbyte.commons.json.Jsons; -import javax.ws.rs.core.Response; -import javax.ws.rs.ext.ExceptionMapper; -import javax.ws.rs.ext.Provider; +import io.micronaut.context.annotation.Requires; +import io.micronaut.http.HttpRequest; +import io.micronaut.http.HttpResponse; +import io.micronaut.http.HttpStatus; +import io.micronaut.http.MediaType; +import io.micronaut.http.annotation.Produces; +import io.micronaut.http.server.exceptions.ExceptionHandler; +import jakarta.inject.Singleton; -@Provider -public class InvalidJsonInputExceptionMapper implements ExceptionMapper { +@Produces +@Singleton +@Requires(classes = JsonMappingException.class) +public class InvalidJsonInputExceptionMapper implements ExceptionHandler { @Override - public Response toResponse(final JsonMappingException e) { - return Response.status(422) - .entity( - Jsons.serialize(KnownException.infoFromThrowableWithMessage(e, "Invalid json input. " + e.getMessage() + " " + e.getOriginalMessage()))) - .type("application/json") - .build(); + public HttpResponse handle(final HttpRequest request, final JsonMappingException exception) { + return HttpResponse.status(HttpStatus.UNPROCESSABLE_ENTITY) + .body(KnownException.infoFromThrowableWithMessage(exception, + "Invalid json input. " + exception.getMessage() + " " + exception.getOriginalMessage())) + .contentType(MediaType.APPLICATION_JSON_TYPE); } } diff --git a/airbyte-server/src/main/java/io/airbyte/server/errors/KnownExceptionMapper.java b/airbyte-server/src/main/java/io/airbyte/server/errors/KnownExceptionMapper.java index b849e30d5298..9d81a3fc7174 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/errors/KnownExceptionMapper.java +++ b/airbyte-server/src/main/java/io/airbyte/server/errors/KnownExceptionMapper.java @@ -5,13 +5,17 @@ package io.airbyte.server.errors; import io.airbyte.commons.json.Jsons; +import io.micronaut.context.annotation.Requires; +import io.micronaut.http.annotation.Produces; +import jakarta.inject.Singleton; import javax.ws.rs.core.Response; import javax.ws.rs.ext.ExceptionMapper; -import javax.ws.rs.ext.Provider; import org.slf4j.Logger; import org.slf4j.LoggerFactory; -@Provider +@Produces +@Singleton +@Requires(classes = KnownException.class) public class KnownExceptionMapper implements ExceptionMapper { private static final Logger LOGGER = LoggerFactory.getLogger(KnownExceptionMapper.class); diff --git a/airbyte-server/src/main/java/io/airbyte/server/errors/NotFoundExceptionMapper.java b/airbyte-server/src/main/java/io/airbyte/server/errors/NotFoundExceptionMapper.java index cda326f8856f..28f223d64114 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/errors/NotFoundExceptionMapper.java +++ b/airbyte-server/src/main/java/io/airbyte/server/errors/NotFoundExceptionMapper.java @@ -4,27 +4,33 @@ package io.airbyte.server.errors; -import io.airbyte.commons.json.Jsons; +import io.micronaut.context.annotation.Requires; +import io.micronaut.http.HttpRequest; +import io.micronaut.http.HttpResponse; +import io.micronaut.http.HttpStatus; +import io.micronaut.http.MediaType; +import io.micronaut.http.annotation.Produces; +import io.micronaut.http.server.exceptions.ExceptionHandler; +import jakarta.inject.Singleton; import javax.ws.rs.NotFoundException; -import javax.ws.rs.core.Response; -import javax.ws.rs.ext.ExceptionMapper; import org.slf4j.Logger; import org.slf4j.LoggerFactory; -public class NotFoundExceptionMapper implements ExceptionMapper { +@Produces +@Singleton +@Requires(classes = NotFoundException.class) +public class NotFoundExceptionMapper implements ExceptionHandler { private static final Logger LOGGER = LoggerFactory.getLogger(NotFoundExceptionMapper.class); @Override - public Response toResponse(final NotFoundException e) { - // Would like to send the id along but we don't have access to the http request anymore to fetch it - // from. TODO: Come back to this with issue #4189 - final IdNotFoundKnownException idnf = new IdNotFoundKnownException("Object not found. " + e.getMessage(), e); + public HttpResponse handle(final HttpRequest request, final NotFoundException exception) { + final IdNotFoundKnownException idnf = new IdNotFoundKnownException("Object not found. " + exception.getMessage(), exception); LOGGER.error("Not found exception", idnf.getNotFoundKnownExceptionInfo()); - return Response.status(404) - .entity(Jsons.serialize(idnf.getNotFoundKnownExceptionInfo())) - .type("application/json") - .build(); + + return HttpResponse.status(HttpStatus.NOT_FOUND) + .body(KnownException.infoFromThrowableWithMessage(exception, "Internal Server Error: " + exception.getMessage())) + .contentType(MediaType.APPLICATION_JSON); } } diff --git a/airbyte-server/src/main/java/io/airbyte/server/errors/UncaughtExceptionMapper.java b/airbyte-server/src/main/java/io/airbyte/server/errors/UncaughtExceptionMapper.java index 3aac5522570e..a9e181d9a6b0 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/errors/UncaughtExceptionMapper.java +++ b/airbyte-server/src/main/java/io/airbyte/server/errors/UncaughtExceptionMapper.java @@ -4,27 +4,30 @@ package io.airbyte.server.errors; -import io.airbyte.api.model.generated.KnownExceptionInfo; -import io.airbyte.commons.json.Jsons; -import javax.ws.rs.core.Response; -import javax.ws.rs.ext.ExceptionMapper; -import javax.ws.rs.ext.Provider; +import io.micronaut.context.annotation.Requires; +import io.micronaut.http.HttpRequest; +import io.micronaut.http.HttpResponse; +import io.micronaut.http.HttpStatus; +import io.micronaut.http.MediaType; +import io.micronaut.http.annotation.Produces; +import io.micronaut.http.server.exceptions.ExceptionHandler; +import jakarta.inject.Singleton; import org.slf4j.Logger; import org.slf4j.LoggerFactory; -@Provider -public class UncaughtExceptionMapper implements ExceptionMapper { +@Produces +@Singleton +@Requires(classes = Throwable.class) +public class UncaughtExceptionMapper implements ExceptionHandler { private static final Logger LOGGER = LoggerFactory.getLogger(UncaughtExceptionMapper.class); @Override - public Response toResponse(final Throwable e) { - LOGGER.error("Uncaught exception", e); - final KnownExceptionInfo exceptionInfo = KnownException.infoFromThrowableWithMessage(e, "Internal Server Error: " + e.getMessage()); - return Response.status(500) - .entity(Jsons.serialize(exceptionInfo)) - .type("application/json") - .build(); + public HttpResponse handle(final HttpRequest request, final Throwable exception) { + LOGGER.error("Uncaught exception", exception); + return HttpResponse.status(HttpStatus.INTERNAL_SERVER_ERROR) + .body(KnownException.infoFromThrowableWithMessage(exception, "Internal Server Error: " + exception.getMessage())) + .contentType(MediaType.APPLICATION_JSON); } } diff --git a/airbyte-server/src/main/java/io/airbyte/server/filter/CorsFilter.java b/airbyte-server/src/main/java/io/airbyte/server/filter/CorsFilter.java new file mode 100644 index 000000000000..d589b1ba27a5 --- /dev/null +++ b/airbyte-server/src/main/java/io/airbyte/server/filter/CorsFilter.java @@ -0,0 +1,35 @@ +/* + * Copyright (c) 2022 Airbyte, Inc., all rights reserved. + */ + +package io.airbyte.server.filter; + +import com.google.common.net.HttpHeaders; +import io.micronaut.http.HttpRequest; +import io.micronaut.http.MutableHttpResponse; +import io.micronaut.http.filter.HttpServerFilter; +import io.micronaut.http.filter.ServerFilterChain; +import java.util.Map; +import lombok.extern.slf4j.Slf4j; +import org.reactivestreams.Publisher; + +// https://medium.com/@Leejjon_net/how-to-allow-cross-origin-requests-in-a-jax-rs-microservice-d2a6aa2df484 +// @Filter("/**") +@Slf4j +public class CorsFilter implements HttpServerFilter { + + public static final Map MAP = Map.of( + HttpHeaders.ACCESS_CONTROL_ALLOW_ORIGIN, "*", + HttpHeaders.ACCESS_CONTROL_ALLOW_HEADERS, "Origin, Content-Type, Accept, Content-Encoding", + HttpHeaders.ACCESS_CONTROL_ALLOW_METHODS, "GET, POST, PUT, DELETE, OPTIONS, HEAD"); + + @Override + public Publisher> doFilter(final HttpRequest request, final ServerFilterChain chain) { + for (final Map.Entry entry : MAP.entrySet()) { + request.setAttribute(entry.getKey(), entry.getValue()); + } + + return chain.proceed(request); + } + +} diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/AttemptHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/AttemptHandler.java index c041e8ed5824..6a07ff304408 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/AttemptHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/AttemptHandler.java @@ -10,26 +10,28 @@ import io.airbyte.config.StreamSyncStats; import io.airbyte.config.SyncStats; import io.airbyte.persistence.job.JobPersistence; +import jakarta.inject.Singleton; import java.io.IOException; import java.util.stream.Collectors; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +@Singleton public class AttemptHandler { private static final Logger LOGGER = LoggerFactory.getLogger(AttemptHandler.class); private final JobPersistence jobPersistence; - public AttemptHandler(JobPersistence jobPersistence) { + public AttemptHandler(final JobPersistence jobPersistence) { this.jobPersistence = jobPersistence; } - public InternalOperationResult setWorkflowInAttempt(SetWorkflowInAttemptRequestBody requestBody) { + public InternalOperationResult setWorkflowInAttempt(final SetWorkflowInAttemptRequestBody requestBody) { try { jobPersistence.setAttemptTemporalWorkflowInfo(requestBody.getJobId(), requestBody.getAttemptNumber(), requestBody.getWorkflowId(), requestBody.getProcessingTaskQueue()); - } catch (IOException ioe) { + } catch (final IOException ioe) { LOGGER.error("IOException when setting temporal workflow in attempt;", ioe); return new InternalOperationResult().succeeded(false); } diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/ConnectionsHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/ConnectionsHandler.java index 03e408be39e4..ba32bfeb380e 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/ConnectionsHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/ConnectionsHandler.java @@ -56,6 +56,7 @@ import io.airbyte.server.scheduler.EventRunner; import io.airbyte.validation.json.JsonValidationException; import io.airbyte.workers.helper.ConnectionHelper; +import jakarta.inject.Singleton; import java.io.IOException; import java.util.Collections; import java.util.HashSet; @@ -70,6 +71,7 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; +@Singleton public class ConnectionsHandler { private static final Logger LOGGER = LoggerFactory.getLogger(ConnectionsHandler.class); @@ -96,6 +98,7 @@ public class ConnectionsHandler { this.connectionHelper = connectionHelper; } + @Deprecated(forRemoval = true) public ConnectionsHandler(final ConfigRepository configRepository, final WorkspaceHelper workspaceHelper, final TrackingClient trackingClient, diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/DestinationDefinitionsHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/DestinationDefinitionsHandler.java index 4b07f3bda272..6b8dfe56accb 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/DestinationDefinitionsHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/DestinationDefinitionsHandler.java @@ -27,8 +27,6 @@ import io.airbyte.commons.version.Version; import io.airbyte.config.ActorDefinitionResourceRequirements; import io.airbyte.config.ActorType; -import io.airbyte.config.Configs; -import io.airbyte.config.EnvConfigs; import io.airbyte.config.StandardDestinationDefinition; import io.airbyte.config.persistence.ConfigNotFoundException; import io.airbyte.config.persistence.ConfigRepository; @@ -42,6 +40,7 @@ import io.airbyte.server.scheduler.SynchronousSchedulerClient; import io.airbyte.server.services.AirbyteGithubStore; import io.airbyte.validation.json.JsonValidationException; +import jakarta.inject.Singleton; import java.io.IOException; import java.net.URI; import java.net.URISyntaxException; @@ -53,6 +52,7 @@ import java.util.stream.Collectors; @SuppressWarnings("PMD.AvoidCatchingNPE") +@Singleton public class DestinationDefinitionsHandler { private final ConfigRepository configRepository; @@ -62,27 +62,19 @@ public class DestinationDefinitionsHandler { private final DestinationHandler destinationHandler; private final AirbyteProtocolVersionRange protocolVersionRange; - public DestinationDefinitionsHandler(final ConfigRepository configRepository, - final SynchronousSchedulerClient schedulerSynchronousClient, - final DestinationHandler destinationHandler) { - this(configRepository, UUID::randomUUID, schedulerSynchronousClient, AirbyteGithubStore.production(), destinationHandler); - } - @VisibleForTesting public DestinationDefinitionsHandler(final ConfigRepository configRepository, final Supplier uuidSupplier, final SynchronousSchedulerClient schedulerSynchronousClient, final AirbyteGithubStore githubStore, - final DestinationHandler destinationHandler) { + final DestinationHandler destinationHandler, + final AirbyteProtocolVersionRange protocolVersionRange) { this.configRepository = configRepository; this.uuidSupplier = uuidSupplier; this.schedulerSynchronousClient = schedulerSynchronousClient; this.githubStore = githubStore; this.destinationHandler = destinationHandler; - - // TODO inject protocol min and max once this handler is being converted to micronaut - final Configs configs = new EnvConfigs(); - protocolVersionRange = new AirbyteProtocolVersionRange(configs.getAirbyteProtocolVersionMin(), configs.getAirbyteProtocolVersionMax()); + this.protocolVersionRange = protocolVersionRange; } @VisibleForTesting @@ -285,7 +277,8 @@ public void deleteDestinationDefinition(final DestinationDefinitionIdRequestBody configRepository.writeStandardDestinationDefinition(persistedDestinationDefinition); } - private ConnectorSpecification getSpecForImage(final String dockerRepository, final String imageTag, boolean isCustomConnector) throws IOException { + private ConnectorSpecification getSpecForImage(final String dockerRepository, final String imageTag, final boolean isCustomConnector) + throws IOException { final String imageName = DockerUtils.getTaggedImageName(dockerRepository, imageTag); final SynchronousResponse getSpecResponse = schedulerSynchronousClient.createGetSpecJob(imageName, isCustomConnector); return SpecFetcher.getSpecFromJob(getSpecResponse); diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/DestinationHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/DestinationHandler.java index e0d815203b52..d105a514e0ec 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/DestinationHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/DestinationHandler.java @@ -32,11 +32,14 @@ import io.airbyte.server.converters.ConfigurationUpdate; import io.airbyte.validation.json.JsonSchemaValidator; import io.airbyte.validation.json.JsonValidationException; +import jakarta.inject.Inject; +import jakarta.inject.Singleton; import java.io.IOException; import java.util.List; import java.util.UUID; import java.util.function.Supplier; +@Singleton public class DestinationHandler { private final ConnectionsHandler connectionsHandler; @@ -70,6 +73,7 @@ public class DestinationHandler { this.oAuthConfigSupplier = oAuthConfigSupplier; } + @Inject public DestinationHandler(final ConfigRepository configRepository, final SecretsRepositoryReader secretsRepositoryReader, final SecretsRepositoryWriter secretsRepositoryWriter, diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/HealthCheckHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/HealthCheckHandler.java index 5ec16023e3ac..2843fbc2c09d 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/HealthCheckHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/HealthCheckHandler.java @@ -6,12 +6,15 @@ import io.airbyte.api.model.generated.HealthCheckRead; import io.airbyte.config.persistence.ConfigRepository; +import jakarta.inject.Named; +import jakarta.inject.Singleton; +@Singleton public class HealthCheckHandler { private final ConfigRepository repository; - public HealthCheckHandler(final ConfigRepository repository) { + public HealthCheckHandler(@Named("configRepository") final ConfigRepository repository) { this.repository = repository; } diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/JobHistoryHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/JobHistoryHandler.java index f61476a18600..72422d9c00f9 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/JobHistoryHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/JobHistoryHandler.java @@ -38,6 +38,7 @@ import io.airbyte.server.converters.JobConverter; import io.airbyte.server.converters.WorkflowStateConverter; import io.airbyte.validation.json.JsonValidationException; +import jakarta.inject.Singleton; import java.io.IOException; import java.util.Collections; import java.util.List; @@ -46,6 +47,7 @@ import java.util.UUID; import java.util.stream.Collectors; +@Singleton public class JobHistoryHandler { private final ConnectionsHandler connectionsHandler; diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/LogsHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/LogsHandler.java index eacb4760bfb3..b9ac2378aca4 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/LogsHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/LogsHandler.java @@ -6,27 +6,47 @@ import io.airbyte.api.model.generated.LogsRequestBody; import io.airbyte.config.Configs; +import io.airbyte.config.Configs.WorkerEnvironment; import io.airbyte.config.helpers.LogClientSingleton; +import io.airbyte.config.helpers.LogConfigs; +import jakarta.inject.Inject; +import jakarta.inject.Named; +import jakarta.inject.Singleton; import java.io.File; -import lombok.AllArgsConstructor; +import java.nio.file.Path; /** * This handler is only responsible for server and scheduler logs. Jobs logs paths are determined by * the submitJob function in the JobSubmitter class in the airbyte-server module. */ -@AllArgsConstructor +@Singleton public class LogsHandler { - private final Configs configs; + private final Path workspaceRoot; + private final WorkerEnvironment workerEnvironment; + private final LogConfigs logConfigs; + + @Deprecated(forRemoval = true) + public LogsHandler(final Configs configs) { + this(configs.getWorkspaceRoot(), configs.getWorkerEnvironment(), configs.getLogConfigs()); + } + + @Inject + public LogsHandler(@Named("workspaceRoot") final Path workspaceRoot, + final WorkerEnvironment workerEnvironment, + final LogConfigs logConfigs) { + this.workspaceRoot = workspaceRoot; + this.workerEnvironment = workerEnvironment; + this.logConfigs = logConfigs; + } public File getLogs(final LogsRequestBody logsRequestBody) { switch (logsRequestBody.getLogType()) { case SERVER -> { - return LogClientSingleton.getInstance().getServerLogFile(configs.getWorkspaceRoot(), configs.getWorkerEnvironment(), configs.getLogConfigs()); + return LogClientSingleton.getInstance().getServerLogFile(workspaceRoot, workerEnvironment, logConfigs); } case SCHEDULER -> { - return LogClientSingleton.getInstance().getSchedulerLogFile(configs.getWorkspaceRoot(), configs.getWorkerEnvironment(), - configs.getLogConfigs()); + return LogClientSingleton.getInstance().getSchedulerLogFile(workspaceRoot, workerEnvironment, logConfigs); } default -> throw new IllegalStateException("Unexpected value: " + logsRequestBody.getLogType()); } diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/OAuthHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/OAuthHandler.java index e7e367e73d6b..6823d6d5a8f6 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/OAuthHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/OAuthHandler.java @@ -39,6 +39,7 @@ import io.airbyte.protocol.models.ConnectorSpecification; import io.airbyte.server.handlers.helpers.OAuthPathExtractor; import io.airbyte.validation.json.JsonValidationException; +import jakarta.inject.Singleton; import java.io.IOException; import java.net.http.HttpClient; import java.util.HashMap; @@ -50,6 +51,7 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; +@Singleton public class OAuthHandler { private static final Logger LOGGER = LoggerFactory.getLogger(OAuthHandler.class); diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/OpenApiConfigHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/OpenApiConfigHandler.java index a1f572f0c6e7..e9d4b3614ec8 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/OpenApiConfigHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/OpenApiConfigHandler.java @@ -5,10 +5,12 @@ package io.airbyte.server.handlers; import io.airbyte.commons.resources.MoreResources; +import jakarta.inject.Singleton; import java.io.File; import java.io.IOException; import java.nio.file.Files; +@Singleton public class OpenApiConfigHandler { private static final File TMP_FILE; diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/OperationsHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/OperationsHandler.java index 749a6ebf0e83..7e6c45140508 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/OperationsHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/OperationsHandler.java @@ -25,6 +25,8 @@ import io.airbyte.config.persistence.ConfigRepository; import io.airbyte.server.converters.OperationsConverter; import io.airbyte.validation.json.JsonValidationException; +import jakarta.inject.Inject; +import jakarta.inject.Singleton; import java.io.IOException; import java.util.ArrayList; import java.util.HashSet; @@ -32,21 +34,23 @@ import java.util.UUID; import java.util.function.Supplier; +@Singleton public class OperationsHandler { private final ConfigRepository configRepository; private final Supplier uuidGenerator; + @Inject + public OperationsHandler(final ConfigRepository configRepository) { + this(configRepository, UUID::randomUUID); + } + @VisibleForTesting OperationsHandler(final ConfigRepository configRepository, final Supplier uuidGenerator) { this.configRepository = configRepository; this.uuidGenerator = uuidGenerator; } - public OperationsHandler(final ConfigRepository configRepository) { - this(configRepository, UUID::randomUUID); - } - public CheckOperationRead checkOperation(final OperatorConfiguration operationCheck) { try { validateOperation(operationCheck); diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/SchedulerHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/SchedulerHandler.java index f0041e0c4d86..a212e4d52dad 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/SchedulerHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/SchedulerHandler.java @@ -43,7 +43,7 @@ import io.airbyte.api.model.generated.SynchronousJobRead; import io.airbyte.commons.docker.DockerUtils; import io.airbyte.commons.enums.Enums; -import io.airbyte.commons.features.EnvVariableFeatureFlags; +import io.airbyte.commons.features.FeatureFlags; import io.airbyte.commons.json.Jsons; import io.airbyte.commons.temporal.ErrorCode; import io.airbyte.commons.temporal.TemporalClient.ManualOperationResult; @@ -76,14 +76,14 @@ import io.airbyte.server.scheduler.SynchronousSchedulerClient; import io.airbyte.validation.json.JsonSchemaValidator; import io.airbyte.validation.json.JsonValidationException; +import jakarta.inject.Singleton; import java.io.IOException; import java.util.ArrayList; import java.util.Optional; import java.util.UUID; import javax.validation.constraints.NotNull; -import lombok.extern.slf4j.Slf4j; -@Slf4j +@Singleton public class SchedulerHandler { private static final HashFunction HASH_FUNCTION = Hashing.md5(); @@ -100,8 +100,9 @@ public class SchedulerHandler { private final JobPersistence jobPersistence; private final JobConverter jobConverter; private final EventRunner eventRunner; - private final EnvVariableFeatureFlags envVariableFeatureFlags; + private final FeatureFlags envVariableFeatureFlags; + // TODO: Convert to be fully using micronaut public SchedulerHandler(final ConfigRepository configRepository, final SecretsRepositoryReader secretsRepositoryReader, final SecretsRepositoryWriter secretsRepositoryWriter, @@ -111,7 +112,7 @@ public SchedulerHandler(final ConfigRepository configRepository, final LogConfigs logConfigs, final EventRunner eventRunner, final ConnectionsHandler connectionsHandler, - final EnvVariableFeatureFlags envVariableFeatureFlags) { + final FeatureFlags envVariableFeatureFlags) { this( configRepository, secretsRepositoryWriter, @@ -135,7 +136,7 @@ public SchedulerHandler(final ConfigRepository configRepository, final EventRunner eventRunner, final JobConverter jobConverter, final ConnectionsHandler connectionsHandler, - final EnvVariableFeatureFlags envVariableFeatureFlags) { + final FeatureFlags envVariableFeatureFlags) { this.configRepository = configRepository; this.secretsRepositoryWriter = secretsRepositoryWriter; this.synchronousSchedulerClient = synchronousSchedulerClient; @@ -390,8 +391,8 @@ public JobInfoRead cancelJob(final JobIdRequestBody jobIdRequestBody) throws IOE // wants the connection disabled when non-breaking changes are detected. If so, disable that // connection. Modify the current discoveredSchema object to add a CatalogDiff, // containsBreakingChange paramter, and connectionStatus parameter. - private void generateCatalogDiffsAndDisableConnectionsIfNeeded(SourceDiscoverSchemaRead discoveredSchema, - SourceDiscoverSchemaRequestBody discoverSchemaRequestBody) + private void generateCatalogDiffsAndDisableConnectionsIfNeeded(final SourceDiscoverSchemaRead discoveredSchema, + final SourceDiscoverSchemaRequestBody discoverSchemaRequestBody) throws JsonValidationException, ConfigNotFoundException, IOException { final ConnectionReadList connectionsForSource = connectionsHandler.listConnectionsForSource(discoverSchemaRequestBody.getSourceId(), false); for (final ConnectionRead connectionRead : connectionsForSource.getConnections()) { @@ -399,12 +400,13 @@ private void generateCatalogDiffsAndDisableConnectionsIfNeeded(SourceDiscoverSch .getConnectionAirbyteCatalog(connectionRead.getConnectionId()); final io.airbyte.api.model.generated.@NotNull AirbyteCatalog currentAirbyteCatalog = connectionRead.getSyncCatalog(); - CatalogDiff diff = connectionsHandler.getDiff(catalogUsedToMakeConfiguredCatalog.orElse(currentAirbyteCatalog), discoveredSchema.getCatalog(), - CatalogConverter.toProtocol(currentAirbyteCatalog)); - boolean containsBreakingChange = containsBreakingChange(diff); - ConnectionUpdate updateObject = + final CatalogDiff diff = + connectionsHandler.getDiff(catalogUsedToMakeConfiguredCatalog.orElse(currentAirbyteCatalog), discoveredSchema.getCatalog(), + CatalogConverter.toProtocol(currentAirbyteCatalog)); + final boolean containsBreakingChange = containsBreakingChange(diff); + final ConnectionUpdate updateObject = new ConnectionUpdate().breakingChange(containsBreakingChange).connectionId(connectionRead.getConnectionId()); - ConnectionStatus connectionStatus; + final ConnectionStatus connectionStatus; if (shouldDisableConnection(containsBreakingChange, connectionRead.getNonBreakingChangesPreference(), diff)) { connectionStatus = ConnectionStatus.INACTIVE; } else { diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/SourceDefinitionsHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/SourceDefinitionsHandler.java index 00300be1b04e..9eb2f77df630 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/SourceDefinitionsHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/SourceDefinitionsHandler.java @@ -28,8 +28,6 @@ import io.airbyte.commons.version.Version; import io.airbyte.config.ActorDefinitionResourceRequirements; import io.airbyte.config.ActorType; -import io.airbyte.config.Configs; -import io.airbyte.config.EnvConfigs; import io.airbyte.config.StandardSourceDefinition; import io.airbyte.config.persistence.ConfigNotFoundException; import io.airbyte.config.persistence.ConfigRepository; @@ -43,6 +41,8 @@ import io.airbyte.server.scheduler.SynchronousSchedulerClient; import io.airbyte.server.services.AirbyteGithubStore; import io.airbyte.validation.json.JsonValidationException; +import jakarta.inject.Inject; +import jakarta.inject.Singleton; import java.io.IOException; import java.net.URI; import java.net.URISyntaxException; @@ -54,6 +54,7 @@ import java.util.stream.Collectors; @SuppressWarnings("PMD.AvoidCatchingNPE") +@Singleton public class SourceDefinitionsHandler { private final ConfigRepository configRepository; @@ -63,26 +64,19 @@ public class SourceDefinitionsHandler { private final SourceHandler sourceHandler; private final AirbyteProtocolVersionRange protocolVersionRange; - public SourceDefinitionsHandler(final ConfigRepository configRepository, - final SynchronousSchedulerClient schedulerSynchronousClient, - final SourceHandler sourceHandler) { - this(configRepository, UUID::randomUUID, schedulerSynchronousClient, AirbyteGithubStore.production(), sourceHandler); - } - + @Inject public SourceDefinitionsHandler(final ConfigRepository configRepository, final Supplier uuidSupplier, final SynchronousSchedulerClient schedulerSynchronousClient, final AirbyteGithubStore githubStore, - final SourceHandler sourceHandler) { + final SourceHandler sourceHandler, + final AirbyteProtocolVersionRange protocolVersionRange) { this.configRepository = configRepository; this.uuidSupplier = uuidSupplier; this.schedulerSynchronousClient = schedulerSynchronousClient; this.githubStore = githubStore; this.sourceHandler = sourceHandler; - - // TODO inject protocol min and max once this handler is being converted to micronaut - final Configs configs = new EnvConfigs(); - protocolVersionRange = new AirbyteProtocolVersionRange(configs.getAirbyteProtocolVersionMin(), configs.getAirbyteProtocolVersionMax()); + this.protocolVersionRange = protocolVersionRange; } @VisibleForTesting diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/SourceHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/SourceHandler.java index ba9f33569ffe..cfb0fb278b8c 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/SourceHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/SourceHandler.java @@ -31,12 +31,15 @@ import io.airbyte.server.converters.ConfigurationUpdate; import io.airbyte.validation.json.JsonSchemaValidator; import io.airbyte.validation.json.JsonValidationException; +import jakarta.inject.Inject; +import jakarta.inject.Singleton; import java.io.IOException; import java.util.List; import java.util.Optional; import java.util.UUID; import java.util.function.Supplier; +@Singleton public class SourceHandler { private final Supplier uuidGenerator; @@ -49,6 +52,7 @@ public class SourceHandler { private final JsonSecretsProcessor secretsProcessor; private final OAuthConfigSupplier oAuthConfigSupplier; + @Inject SourceHandler(final ConfigRepository configRepository, final SecretsRepositoryReader secretsRepositoryReader, final SecretsRepositoryWriter secretsRepositoryWriter, diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/StateHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/StateHandler.java index c9a42e9e3b93..977f95813aad 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/StateHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/StateHandler.java @@ -10,10 +10,12 @@ import io.airbyte.config.StateWrapper; import io.airbyte.config.persistence.StatePersistence; import io.airbyte.workers.helper.StateConverter; +import jakarta.inject.Singleton; import java.io.IOException; import java.util.Optional; import java.util.UUID; +@Singleton public class StateHandler { private final StatePersistence statePersistence; diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/WebBackendCheckUpdatesHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/WebBackendCheckUpdatesHandler.java index 1884a8d39244..7c51e26c5494 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/WebBackendCheckUpdatesHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/WebBackendCheckUpdatesHandler.java @@ -9,17 +9,17 @@ import io.airbyte.config.StandardSourceDefinition; import io.airbyte.config.persistence.ConfigRepository; import io.airbyte.server.services.AirbyteGithubStore; +import jakarta.inject.Singleton; import java.io.IOException; import java.util.List; import java.util.Map; import java.util.Map.Entry; import java.util.UUID; import java.util.stream.Collectors; -import lombok.AllArgsConstructor; import lombok.extern.slf4j.Slf4j; -@AllArgsConstructor @Slf4j +@Singleton public class WebBackendCheckUpdatesHandler { private static final int NO_CHANGES_FOUND = 0; @@ -27,6 +27,11 @@ public class WebBackendCheckUpdatesHandler { final ConfigRepository configRepository; final AirbyteGithubStore githubStore; + public WebBackendCheckUpdatesHandler(final ConfigRepository configRepository, final AirbyteGithubStore githubStore) { + this.configRepository = configRepository; + this.githubStore = githubStore; + } + public WebBackendCheckUpdatesRead checkUpdates() { final int destinationDiffCount = getDestinationDiffCount(); diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/WebBackendConnectionsHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/WebBackendConnectionsHandler.java index 362afe51c35b..99303a29f615 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/WebBackendConnectionsHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/WebBackendConnectionsHandler.java @@ -61,6 +61,7 @@ import io.airbyte.server.scheduler.EventRunner; import io.airbyte.validation.json.JsonValidationException; import io.airbyte.workers.helper.ProtocolConverters; +import jakarta.inject.Singleton; import java.io.IOException; import java.util.ArrayList; import java.util.Collections; @@ -72,16 +73,10 @@ import java.util.UUID; import java.util.function.Function; import java.util.stream.Collectors; -import lombok.AllArgsConstructor; -import lombok.extern.slf4j.Slf4j; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -@AllArgsConstructor -@Slf4j +@Singleton public class WebBackendConnectionsHandler { - private static final Logger LOGGER = LoggerFactory.getLogger(WebBackendConnectionsHandler.class); private final ConnectionsHandler connectionsHandler; private final StateHandler stateHandler; private final SourceHandler sourceHandler; @@ -93,6 +88,26 @@ public class WebBackendConnectionsHandler { // todo (cgardens) - this handler should NOT have access to the db. only access via handler. private final ConfigRepository configRepository; + public WebBackendConnectionsHandler(final ConnectionsHandler connectionsHandler, + final StateHandler stateHandler, + final SourceHandler sourceHandler, + final DestinationHandler destinationHandler, + final JobHistoryHandler jobHistoryHandler, + final SchedulerHandler schedulerHandler, + final OperationsHandler operationsHandler, + final EventRunner eventRunner, + final ConfigRepository configRepository) { + this.connectionsHandler = connectionsHandler; + this.stateHandler = stateHandler; + this.sourceHandler = sourceHandler; + this.destinationHandler = destinationHandler; + this.jobHistoryHandler = jobHistoryHandler; + this.schedulerHandler = schedulerHandler; + this.operationsHandler = operationsHandler; + this.eventRunner = eventRunner; + this.configRepository = configRepository; + } + public WebBackendWorkspaceStateResult getWorkspaceState(final WebBackendWorkspaceState webBackendWorkspaceState) throws IOException { final var workspaceId = webBackendWorkspaceState.getWorkspaceId(); final var connectionCount = configRepository.countConnectionsForWorkspace(workspaceId); @@ -380,7 +395,8 @@ public WebBackendConnectionRead webBackendGetConnection(final WebBackendConnecti return buildWebBackendConnectionRead(connection, currentSourceCatalogId).catalogDiff(diff); } - private AirbyteCatalog updateSchemaWithOriginalDiscoveredCatalog(AirbyteCatalog configuredCatalog, AirbyteCatalog originalDiscoveredCatalog) { + private AirbyteCatalog updateSchemaWithOriginalDiscoveredCatalog(final AirbyteCatalog configuredCatalog, + final AirbyteCatalog originalDiscoveredCatalog) { // We pass the original discovered catalog in as the "new" discovered catalog. return updateSchemaWithRefreshedDiscoveredCatalog(configuredCatalog, originalDiscoveredCatalog, originalDiscoveredCatalog); } @@ -391,7 +407,7 @@ private Optional getRefreshedSchema(final UUID sourceI .sourceId(sourceId) .disableCache(true) .connectionId(connectionId); - SourceDiscoverSchemaRead schemaRead = schedulerHandler.discoverSchemaForSourceFromSourceId(discoverSchemaReadReq); + final SourceDiscoverSchemaRead schemaRead = schedulerHandler.discoverSchemaForSourceFromSourceId(discoverSchemaReadReq); return Optional.ofNullable(schemaRead); } @@ -409,7 +425,7 @@ private Optional getRefreshedSchema(final UUID sourceI */ @VisibleForTesting protected static AirbyteCatalog updateSchemaWithRefreshedDiscoveredCatalog(final AirbyteCatalog originalConfigured, - AirbyteCatalog originalDiscovered, + final AirbyteCatalog originalDiscovered, final AirbyteCatalog discovered) { /* * We can't directly use s.getStream() as the key, because it contains a bunch of other fields, so diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/WebBackendGeographiesHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/WebBackendGeographiesHandler.java index d8493a744d35..3d39209098f1 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/WebBackendGeographiesHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/WebBackendGeographiesHandler.java @@ -6,13 +6,11 @@ import io.airbyte.api.model.generated.Geography; import io.airbyte.api.model.generated.WebBackendGeographiesListResult; +import jakarta.inject.Singleton; import java.util.Arrays; import java.util.Collections; -import lombok.AllArgsConstructor; -import lombok.extern.slf4j.Slf4j; -@AllArgsConstructor -@Slf4j +@Singleton public class WebBackendGeographiesHandler { public WebBackendGeographiesListResult listGeographiesOSS() { diff --git a/airbyte-server/src/main/java/io/airbyte/server/handlers/WorkspacesHandler.java b/airbyte-server/src/main/java/io/airbyte/server/handlers/WorkspacesHandler.java index 63e88b7bbfef..6829747a3014 100644 --- a/airbyte-server/src/main/java/io/airbyte/server/handlers/WorkspacesHandler.java +++ b/airbyte-server/src/main/java/io/airbyte/server/handlers/WorkspacesHandler.java @@ -38,6 +38,8 @@ import io.airbyte.server.errors.InternalServerKnownException; import io.airbyte.server.errors.ValueConflictKnownException; import io.airbyte.validation.json.JsonValidationException; +import jakarta.inject.Inject; +import jakarta.inject.Singleton; import java.io.IOException; import java.util.List; import java.util.UUID; @@ -47,6 +49,7 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; +@Singleton public class WorkspacesHandler { private static final Logger LOGGER = LoggerFactory.getLogger(WorkspacesHandler.class); @@ -58,6 +61,7 @@ public class WorkspacesHandler { private final Supplier uuidSupplier; private final Slugify slugify; + @Inject public WorkspacesHandler(final ConfigRepository configRepository, final SecretsRepositoryWriter secretsRepositoryWriter, final ConnectionsHandler connectionsHandler, diff --git a/airbyte-server/src/main/resources/application.yml b/airbyte-server/src/main/resources/application.yml new file mode 100644 index 000000000000..9a574859829d --- /dev/null +++ b/airbyte-server/src/main/resources/application.yml @@ -0,0 +1,130 @@ +micronaut: + application: + name: airbyte-server + security: + intercept-url-map: + - pattern: /** + httpMethod: GET + access: + - isAnonymous() + - pattern: /** + httpMethod: POST + access: + - isAnonymous() + - pattern: /** + httpMethod: HEAD + access: + - isAnonymous() + server: + port: 8001 +airbyte: + cloud: + storage: + logs: + type: ${WORKER_LOGS_STORAGE_TYPE:} + gcs: + application-credentials: ${GOOGLE_APPLICATION_CREDENTIALS:} + bucket: ${GCS_LOG_BUCKET:} + minio: + access-key: ${AWS_ACCESS_KEY_ID:} + bucket: ${S3_LOG_BUCKET:} + endpoint: ${S3_MINIO_ENDPOINT:} + secret-access-key: ${AWS_SECRET_ACCESS_KEY:} + s3: + access-key: ${AWS_ACCESS_KEY_ID:} + bucket: ${S3_LOG_BUCKET:} + region: ${S3_LOG_BUCKET_REGION:} + secret-access-key: ${AWS_SECRET_ACCESS_KEY:} + state: + type: ${WORKER_STATE_STORAGE_TYPE:} + gcs: + application-credentials: ${STATE_STORAGE_GCS_APPLICATION_CREDENTIALS:} + bucket: ${STATE_STORAGE_GCS_BUCKET_NAME:} + minio: + access-key: ${STATE_STORAGE_MINIO_ACCESS_KEY:} + bucket: ${STATE_STORAGE_MINIO_BUCKET_NAME:} + endpoint: ${STATE_STORAGE_MINIO_ENDPOINT:} + secret-access-key: ${STATE_STORAGE_MINIO_SECRET_ACCESS_KEY:} + s3: + access-key: ${STATE_STORAGE_S3_ACCESS_KEY:} + bucket: ${STATE_STORAGE_S3_BUCKET_NAME:} + region: ${STATE_STORAGE_S3_BUCKET_REGION:} + secret-access-key: ${STATE_STORAGE_S3_SECRET_ACCESS_KEY:} + deployment-mode: ${DEPLOYMENT_MODE:OSS} + flyway: + configs: + initialization-timeout-ms: ${CONFIGS_DATABASE_INITIALIZATION_TIMEOUT_MS:60000} + minimum-migration-version: ${CONFIGS_DATABASE_MINIMUM_FLYWAY_MIGRATION_VERSION} + jobs: + initialization-timeout-ms: ${JOBS_DATABASE_INITIALIZATION_TIMEOUT_MS:60000} + minimum-migration-version: ${JOBS_DATABASE_MINIMUM_FLYWAY_MIGRATION_VERSION} + version: ${AIRBYTE_VERSION} + secret: + persistence: ${SECRET_PERSISTENCE:TESTING_CONFIG_DB_TABLE} + store: + gcp: + credentials: ${SECRET_STORE_GCP_CREDENTIALS:} + project-id: ${SECRET_STORE_GCP_PROJECT_ID:} + vault: + address: ${VAULT_ADDRESS:} + prefix: ${VAULT_PREFIX:} + token: ${VAULT_AUTH_TOKEN:} + role: ${AIRBYTE_ROLE:dev} + tracking-strategy: ${TRACKING_STRATEGY:LOGGING} + worker: + job: + error-reporting: + sentry: + dsn: ${JOB_ERROR_REPORTING_SENTRY_DSN} + strategy: ${JOB_ERROR_REPORTING_STRATEGY:LOGGING} + web-app: + url: ${WEBAPP_URL:} + workspace: + root: ${WORKSPACE_ROOT} + protocol: + min-version: ${AIRBYTE_PROTOCOL_VERSION_MIN:0.0.0} + max-version: ${AIRBYTE_PROTOCOL_VERSION_MAX:0.3.0} + +temporal: + cloud: + client: + cert: ${TEMPORAL_CLOUD_CLIENT_CERT:} + key: ${TEMPORAL_CLOUD_CLIENT_KEY:} + enabled: ${TEMPORAL_CLOUD_ENABLED:false} + host: ${TEMPORAL_CLOUD_HOST:} + namespace: ${TEMPORAL_CLOUD_NAMESPACE:} + host: ${TEMPORAL_HOST:`airbyte-temporal:7233`} + retention: ${TEMPORAL_HISTORY_RETENTION_IN_DAYS:30} + +datasources: + config: + connection-test-query: SELECT 1 + connection-timeout: 30000 + maximum-pool-size: 10 + minimum-idle: 0 + idle-timeout: 600000 + url: ${DATABASE_URL} + driverClassName: org.postgresql.Driver + username: ${DATABASE_USER} + password: ${DATABASE_PASSWORD} + +flyway: + enabled: true + datasources: + config: + enabled: false + locations: + - 'classpath:io/airbyte/db/instance/configs/migrations' + jobs: + enabled: false + locations: + - 'classpath:io/airbyte/db/instance/jobs/migrations' + +jooq: + datasources: + config: + jackson-converter-enabled: true + sql-dialect: POSTGRES + jobs: + jackson-converter-enabled: true + sql-dialect: POSTGRES diff --git a/airbyte-server/src/main/resources/banner/banner.txt b/airbyte-server/src/main/resources/micronaut-banner.txt similarity index 54% rename from airbyte-server/src/main/resources/banner/banner.txt rename to airbyte-server/src/main/resources/micronaut-banner.txt index 0c432cdb9496..83ea519237a6 100644 --- a/airbyte-server/src/main/resources/banner/banner.txt +++ b/airbyte-server/src/main/resources/micronaut-banner.txt @@ -5,6 +5,7 @@ / ___ |/ / / / /_/ / /_/ / /_/ __/ /_/ |_/_/_/ /_.___/\__, /\__/\___/ /____/ --------------------------------------- - Now ready at http://localhost:8000/ --------------------------------------- + : airbyte-server : +-------------------------------------------------- + Will soon be available at http://localhost:8000/ +-------------------------------------------------- diff --git a/airbyte-server/src/test/java/io/airbyte/server/handlers/DestinationDefinitionsHandlerTest.java b/airbyte-server/src/test/java/io/airbyte/server/handlers/DestinationDefinitionsHandlerTest.java index 32ba33b311bd..6d61c0f44efe 100644 --- a/airbyte-server/src/test/java/io/airbyte/server/handlers/DestinationDefinitionsHandlerTest.java +++ b/airbyte-server/src/test/java/io/airbyte/server/handlers/DestinationDefinitionsHandlerTest.java @@ -32,6 +32,7 @@ import io.airbyte.commons.docker.DockerUtils; import io.airbyte.commons.json.Jsons; import io.airbyte.commons.version.AirbyteProtocolVersionRange; +import io.airbyte.commons.version.Version; import io.airbyte.config.ActorDefinitionResourceRequirements; import io.airbyte.config.ActorType; import io.airbyte.config.Configs; @@ -92,7 +93,8 @@ void setUp() { uuidSupplier, schedulerSynchronousClient, githubStore, - destinationHandler); + destinationHandler, + new AirbyteProtocolVersionRange(new Version("0.0.0"), new Version("0.3.0"))); } private StandardDestinationDefinition generateDestinationDefinition() { diff --git a/airbyte-server/src/test/java/io/airbyte/server/handlers/SchedulerHandlerTest.java b/airbyte-server/src/test/java/io/airbyte/server/handlers/SchedulerHandlerTest.java index 957f80de8885..a0335429dabe 100644 --- a/airbyte-server/src/test/java/io/airbyte/server/handlers/SchedulerHandlerTest.java +++ b/airbyte-server/src/test/java/io/airbyte/server/handlers/SchedulerHandlerTest.java @@ -159,6 +159,11 @@ class SchedulerHandlerTest { void setup() { completedJob = mock(Job.class, RETURNS_DEEP_STUBS); jobResponse = mock(SynchronousResponse.class, RETURNS_DEEP_STUBS); + final SynchronousJobMetadata synchronousJobMetadata = mock(SynchronousJobMetadata.class); + when(synchronousJobMetadata.getConfigType()) + .thenReturn(ConfigType.SYNC); + when(jobResponse.getMetadata()) + .thenReturn(synchronousJobMetadata); configurationUpdate = mock(ConfigurationUpdate.class); jsonSchemaValidator = mock(JsonSchemaValidator.class); when(completedJob.getStatus()).thenReturn(JobStatus.SUCCEEDED); @@ -952,9 +957,9 @@ void testDiscoverSchemaForSourceMultipleConnectionsFeatureFlagOn() throws IOExce assertEquals(expectedActorCatalog, actual.getCatalog()); assertEquals(ConnectionStatus.ACTIVE, actual.getConnectionStatus()); - ArgumentCaptor expectedArgumentCaptor = ArgumentCaptor.forClass(ConnectionUpdate.class); + final ArgumentCaptor expectedArgumentCaptor = ArgumentCaptor.forClass(ConnectionUpdate.class); verify(connectionsHandler, times(3)).updateConnection(expectedArgumentCaptor.capture()); - List connectionUpdateValues = expectedArgumentCaptor.getAllValues(); + final List connectionUpdateValues = expectedArgumentCaptor.getAllValues(); assertEquals(ConnectionStatus.ACTIVE, connectionUpdateValues.get(0).getStatus()); assertEquals(ConnectionStatus.ACTIVE, connectionUpdateValues.get(1).getStatus()); assertEquals(ConnectionStatus.INACTIVE, connectionUpdateValues.get(2).getStatus()); diff --git a/airbyte-server/src/test/java/io/airbyte/server/handlers/SourceDefinitionsHandlerTest.java b/airbyte-server/src/test/java/io/airbyte/server/handlers/SourceDefinitionsHandlerTest.java index e29293240538..a4899f78c44c 100644 --- a/airbyte-server/src/test/java/io/airbyte/server/handlers/SourceDefinitionsHandlerTest.java +++ b/airbyte-server/src/test/java/io/airbyte/server/handlers/SourceDefinitionsHandlerTest.java @@ -34,6 +34,7 @@ import io.airbyte.commons.docker.DockerUtils; import io.airbyte.commons.json.Jsons; import io.airbyte.commons.version.AirbyteProtocolVersionRange; +import io.airbyte.commons.version.Version; import io.airbyte.config.ActorDefinitionResourceRequirements; import io.airbyte.config.ActorType; import io.airbyte.config.Configs; @@ -77,6 +78,7 @@ class SourceDefinitionsHandlerTest { private AirbyteGithubStore githubStore; private SourceHandler sourceHandler; private UUID workspaceId; + private AirbyteProtocolVersionRange protocolVersionRange; @SuppressWarnings("unchecked") @BeforeEach @@ -90,7 +92,10 @@ void setUp() { sourceDefinition = generateSourceDefinition(); - sourceDefinitionsHandler = new SourceDefinitionsHandler(configRepository, uuidSupplier, schedulerSynchronousClient, githubStore, sourceHandler); + protocolVersionRange = new AirbyteProtocolVersionRange(new Version("0.0.0"), new Version("0.3.0")); + + sourceDefinitionsHandler = new SourceDefinitionsHandler(configRepository, uuidSupplier, schedulerSynchronousClient, githubStore, sourceHandler, + protocolVersionRange); } private StandardSourceDefinition generateSourceDefinition() { diff --git a/airbyte-tests/src/acceptanceTests/java/io/airbyte/test/acceptance/AdvancedAcceptanceTests.java b/airbyte-tests/src/acceptanceTests/java/io/airbyte/test/acceptance/AdvancedAcceptanceTests.java index 216159195f1c..e1f46d17090d 100644 --- a/airbyte-tests/src/acceptanceTests/java/io/airbyte/test/acceptance/AdvancedAcceptanceTests.java +++ b/airbyte-tests/src/acceptanceTests/java/io/airbyte/test/acceptance/AdvancedAcceptanceTests.java @@ -5,12 +5,10 @@ package io.airbyte.test.acceptance; import static io.airbyte.test.utils.AirbyteAcceptanceTestHarness.COLUMN_ID; -import static io.airbyte.test.utils.AirbyteAcceptanceTestHarness.SOURCE_PASSWORD; import static io.airbyte.test.utils.AirbyteAcceptanceTestHarness.waitForConnectionState; import static io.airbyte.test.utils.AirbyteAcceptanceTestHarness.waitForSuccessfulJob; import static io.airbyte.test.utils.AirbyteAcceptanceTestHarness.waitWhileJobHasStatus; import static org.junit.jupiter.api.Assertions.assertEquals; -import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertNotNull; import static org.junit.jupiter.api.Assertions.assertTrue; @@ -34,8 +32,6 @@ import io.airbyte.api.client.model.generated.JobInfoRead; import io.airbyte.api.client.model.generated.JobRead; import io.airbyte.api.client.model.generated.JobStatus; -import io.airbyte.api.client.model.generated.LogType; -import io.airbyte.api.client.model.generated.LogsRequestBody; import io.airbyte.api.client.model.generated.SourceDefinitionIdRequestBody; import io.airbyte.api.client.model.generated.SourceDefinitionRead; import io.airbyte.api.client.model.generated.SourceRead; @@ -45,7 +41,6 @@ import io.airbyte.test.utils.AirbyteAcceptanceTestHarness; import java.io.IOException; import java.net.URISyntaxException; -import java.nio.charset.Charset; import java.sql.SQLException; import java.util.Collections; import java.util.List; @@ -217,29 +212,6 @@ void testCheckpointing() throws Exception { assertEquals(0, connectionState.getState().get(COLUMN1).asInt() % 5); } - @RetryingTest(3) - @Order(3) - void testRedactionOfSensitiveRequestBodies() throws Exception { - // check that the source password is not present in the logs - final List serverLogLines = java.nio.file.Files.readAllLines( - apiClient.getLogsApi().getLogs(new LogsRequestBody().logType(LogType.SERVER)).toPath(), - Charset.defaultCharset()); - - assertFalse(serverLogLines.isEmpty()); - - boolean hasRedacted = false; - - for (final String line : serverLogLines) { - assertFalse(line.contains(SOURCE_PASSWORD)); - - if (line.contains("REDACTED")) { - hasRedacted = true; - } - } - - assertTrue(hasRedacted); - } - // verify that when the worker uses backpressure from pipes that no records are lost. @RetryingTest(3) @Order(4) diff --git a/airbyte-tests/src/acceptanceTests/java/io/airbyte/test/acceptance/BasicAcceptanceTests.java b/airbyte-tests/src/acceptanceTests/java/io/airbyte/test/acceptance/BasicAcceptanceTests.java index 16c1de7fa20d..92f8f53ce8eb 100644 --- a/airbyte-tests/src/acceptanceTests/java/io/airbyte/test/acceptance/BasicAcceptanceTests.java +++ b/airbyte-tests/src/acceptanceTests/java/io/airbyte/test/acceptance/BasicAcceptanceTests.java @@ -276,7 +276,7 @@ void testDestinationCheckConnection() throws ApiException { @Test @Order(3) - void testCreateSource() throws ApiException { + void wtestCreateSource() throws ApiException { final String dbName = "acc-test-db"; final UUID postgresSourceDefinitionId = testHarness.getPostgresSourceDefinitionId(); final JsonNode sourceDbConfig = testHarness.getSourceDbConfig(); diff --git a/airbyte-workers/src/main/java/io/airbyte/workers/config/TemporalBeanFactory.java b/airbyte-workers/src/main/java/io/airbyte/workers/config/TemporalBeanFactory.java index 01f6127de110..a4ad537526fc 100644 --- a/airbyte-workers/src/main/java/io/airbyte/workers/config/TemporalBeanFactory.java +++ b/airbyte-workers/src/main/java/io/airbyte/workers/config/TemporalBeanFactory.java @@ -33,7 +33,6 @@ import jakarta.inject.Singleton; import java.io.IOException; import java.nio.file.Path; -import java.util.Optional; /** * Micronaut bean factory for Temporal-related singletons. @@ -43,22 +42,22 @@ public class TemporalBeanFactory { @Singleton @Requires(env = WorkerMode.CONTROL_PLANE) - public TrackingClient trackingClient(final Optional trackingStrategy, + public TrackingClient trackingClient(final TrackingStrategy trackingStrategy, final DeploymentMode deploymentMode, - final Optional jobPersistence, + final JobPersistence jobPersistence, final WorkerEnvironment workerEnvironment, @Value("${airbyte.role}") final String airbyteRole, final AirbyteVersion airbyteVersion, - final Optional configRepository) + final ConfigRepository configRepository) throws IOException { TrackingClientSingleton.initialize( - trackingStrategy.orElseThrow(), - new Deployment(deploymentMode, jobPersistence.orElseThrow().getDeployment().orElseThrow(), + trackingStrategy, + new Deployment(deploymentMode, jobPersistence.getDeployment().orElseThrow(), workerEnvironment), airbyteRole, airbyteVersion, - configRepository.orElseThrow()); + configRepository); return TrackingClientSingleton.get(); } diff --git a/charts/airbyte-server/templates/deployment.yaml b/charts/airbyte-server/templates/deployment.yaml index 111d4b7570b2..4d97a1b9f222 100644 --- a/charts/airbyte-server/templates/deployment.yaml +++ b/charts/airbyte-server/templates/deployment.yaml @@ -175,6 +175,26 @@ spec: configMapKeyRef: name: {{ .Release.Name }}-airbyte-env key: S3_MINIO_ENDPOINT + - name: STATE_STORAGE_MINIO_BUCKET_NAME + valueFrom: + configMapKeyRef: + name: {{ .Release.Name }}-airbyte-env + key: STATE_STORAGE_MINIO_BUCKET_NAME + - name: STATE_STORAGE_MINIO_ACCESS_KEY + valueFrom: + secretKeyRef: + name: {{ .Release.Name }}-airbyte-secrets + key: STATE_STORAGE_MINIO_ACCESS_KEY + - name: STATE_STORAGE_MINIO_SECRET_ACCESS_KEY + valueFrom: + secretKeyRef: + name: {{ .Release.Name }}-airbyte-secrets + key: STATE_STORAGE_MINIO_SECRET_ACCESS_KEY + - name: STATE_STORAGE_MINIO_ENDPOINT + valueFrom: + configMapKeyRef: + name: {{ .Release.Name }}-airbyte-env + key: STATE_STORAGE_MINIO_ENDPOINT - name: S3_PATH_STYLE_ACCESS valueFrom: configMapKeyRef: @@ -200,6 +220,16 @@ spec: configMapKeyRef: name: {{ .Release.Name }}-airbyte-env key: JOBS_DATABASE_MINIMUM_FLYWAY_MIGRATION_VERSION + - name: WORKER_LOGS_STORAGE_TYPE + valueFrom: + configMapKeyRef: + name: {{ .Release.Name }}-airbyte-env + key: WORKER_LOGS_STORAGE_TYPE + - name: WORKER_STATE_STORAGE_TYPE + valueFrom: + configMapKeyRef: + name: {{ .Release.Name }}-airbyte-env + key: WORKER_STATE_STORAGE_TYPE {{- end }} # Values from secret {{- if .Values.secrets }} diff --git a/deps.toml b/deps.toml index e3eff5d61cb4..88000fce699c 100644 --- a/deps.toml +++ b/deps.toml @@ -111,6 +111,7 @@ hibernate-types = { module = "com.vladmihalcea:hibernate-types-52", version = "2 jakarta-inject = { module = "jakarta.annotation:jakarta.annotation-api", version = "2.1.1" } javax-transaction = { module = "javax.transaction:javax.transaction-api", version = "1.3" } micronaut-bom = { module = "io.micronaut:micronaut-bom", version.ref = "micronaut" } +micronaut-data-tx = { module = "io.micronaut.data:micronaut-data-tx", version = "3.8.1" } micronaut-data-processor = { module = "io.micronaut.data:micronaut-data-processor", version = "3.8.1" } micronaut-flyway = { module = "io.micronaut.flyway:micronaut-flyway", version = "5.4.1" } micronaut-inject = { module = "io.micronaut:micronaut-inject" } @@ -119,6 +120,7 @@ micronaut-http-server-netty = { module = "io.micronaut:micronaut-http-server-net micronaut-inject-java = { module = "io.micronaut:micronaut-inject-java", version.ref = "micronaut" } micronaut-jaxrs-processor = { module = "io.micronaut.jaxrs:micronaut-jaxrs-processor", version = "3.4.0" } micronaut-jaxrs-server = { module = "io.micronaut.jaxrs:micronaut-jaxrs-server", version = "3.4.0" } +micronaut-jdbc = { module = "io.micronaut.sql:micronaut-jdbc", version = "4.7.2" } micronaut-jdbc-hikari = { module = "io.micronaut.sql:micronaut-jdbc-hikari" } micronaut-jooq = { module = "io.micronaut.sql:micronaut-jooq" } micronaut-management = { module = "io.micronaut:micronaut-management" } @@ -137,6 +139,7 @@ log4j = ["log4j-api", "log4j-core", "log4j-impl", "log4j-web"] micronaut = ["jakarta-inject", "javax-transaction", "micronaut-http-server-netty", "micronaut-http-client", "micronaut-inject", "micronaut-validation", "micronaut-runtime", "micronaut-management", "micronaut-security", "micronaut-jaxrs-server", "micronaut-flyway", "micronaut-jdbc-hikari", "micronaut-jooq"] micronaut-annotation = ["jakarta-inject", "micronaut-inject-java"] micronaut-annotation-processor = ["micronaut-inject-java", "micronaut-management", "micronaut-validation", "micronaut-data-processor", "micronaut-jaxrs-processor"] +micronaut-server = ["micronaut-jaxrs-processor", "micronaut-jaxrs-server"] micronaut-test = ["micronaut-test-core", "micronaut-test-junit5", "h2-database"] micronaut-test-annotation-processor = ["micronaut-inject-java"] slf4j = ["jul-to-slf4j", "jcl-over-slf4j", "log4j-over-slf4j"] diff --git a/docker-compose.yaml b/docker-compose.yaml index 095f67bc94c6..264c4a492fc8 100644 --- a/docker-compose.yaml +++ b/docker-compose.yaml @@ -150,6 +150,7 @@ services: - WORKER_ENVIRONMENT=${WORKER_ENVIRONMENT} - WORKSPACE_ROOT=${WORKSPACE_ROOT} - GITHUB_STORE_BRANCH=${GITHUB_STORE_BRANCH} + - MICRONAUT_ENVIRONMENTS=${WORKERS_MICRONAUT_ENVIRONMENTS} - AUTO_DETECT_SCHEMA=${AUTO_DETECT_SCHEMA} ports: - 8001 diff --git a/kube/resources/server.yaml b/kube/resources/server.yaml index 2892560a6345..85b956e1ae9c 100644 --- a/kube/resources/server.yaml +++ b/kube/resources/server.yaml @@ -158,6 +158,36 @@ spec: configMapKeyRef: name: airbyte-env key: AUTO_DETECT_SCHEMA + - name: WORKER_LOGS_STORAGE_TYPE + valueFrom: + configMapKeyRef: + name: airbyte-env + key: WORKER_LOGS_STORAGE_TYPE + - name: WORKER_STATE_STORAGE_TYPE + valueFrom: + configMapKeyRef: + name: airbyte-env + key: WORKER_STATE_STORAGE_TYPE + - name: STATE_STORAGE_MINIO_BUCKET_NAME + valueFrom: + configMapKeyRef: + name: airbyte-env + key: STATE_STORAGE_MINIO_BUCKET_NAME + - name: STATE_STORAGE_MINIO_ENDPOINT + valueFrom: + configMapKeyRef: + name: airbyte-env + key: STATE_STORAGE_MINIO_ENDPOINT + - name: STATE_STORAGE_MINIO_ACCESS_KEY + valueFrom: + secretKeyRef: + name: airbyte-secrets + key: STATE_STORAGE_MINIO_ACCESS_KEY + - name: STATE_STORAGE_MINIO_SECRET_ACCESS_KEY + valueFrom: + secretKeyRef: + name: airbyte-secrets + key: STATE_STORAGE_MINIO_SECRET_ACCESS_KEY ports: - containerPort: 8001 volumeMounts: