feat: merge m3 (#426)

* docs: Add CatalystDataGatewayRepository docs (#388) * test: Fix tests after cat-gateway update. * docs: Add `CatalystDataGatewayRepository` usage examples. * docs: Move docs from README to code comment. * test: Update `CatalystDataGatewayRepository` tests. * test: Use Fake instead of Mock. * chore: Fix Markdown errors. * chore: Explicit use of `HttpStatus` codes. * adds one more worker and compression for catgateway logs (#400) * feat: Collect flutter code coverage (#404) * test: Fix tests after cat-gateway update. * docs: Add `CatalystDataGatewayRepository` usage examples. * docs: Move docs from README to code comment. * test: Update `CatalystDataGatewayRepository` tests. * test: Use Fake instead of Mock. * chore: Fix Markdown errors. * chore: Explicit use of `HttpStatus` codes. * chore(deps-dev): bump vite in /utilities/wallet-tester (#397) Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 5.1.6 to 5.1.7. - [Release notes](https://github.com/vitejs/vite/releases) - [Changelog](https://github.com/vitejs/vite/blob/v5.1.7/packages/vite/CHANGELOG.md) - [Commits](https://github.com/vitejs/vite/commits/v5.1.7/packages/vite) --- updated-dependencies: - dependency-name: vite dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Apisit Ritreungroj <38898766+apskhem@users.noreply.github.com> * feat: Get slot number by provided date-time endpoint (#387) * fix CardanoStakeAddress error handling * refactor, add sync_state_get endpoint * refactor types * refactor * add block_hash validation * wip * wip * wip * wip * add check_network fn * fix * fix schematisis test * try * wip * try * try * try * try * wip * try * try * fix * update Network * add test_utxo test * try * fix * try * fix * wip * fix * fix docket-compose.yml file * try * try * fix * try * try * try * try * wip * fix * wip * try * try * wip * try * try * revert * wip * wip * wip * fix * fix * fix * remove mithril_snapshot loader * wip * wip * wip * wip * wip * wip * wip * add stake addr bech32 encode utils function * wip * wip * update indexing of the utxo data * fix spelling * wip * wip * finish utxo test * fix deny * fix check * fix * fix * update earthly builder versions * wip * ignore test_utxo.py in CI * dont ignore tests * add date_time_to_slot_number_get endpoint * add sql queries * fix * update slot info, fix follower indexing block time issue * add previous slot info field * fix * refactor * fix sync_state_get * wip * fix check * try * fix * finish slot_info test, fix queries * fix * cleanup * wip * wip * wip * feat: RBAC Documentation Drafting (#332) * chore: wip * Update 0005-flutter-app.md * Update 0005-flutter-app.md * Update 0005-flutter-app.md * Update 0005-flutter-app.md * Update 0005-flutter-app.md * Update 0005-flutter-app.md * Update 0005-flutter-app.md * Update 0005-flutter-app.md * Update 0005-flutter-app.md * Update 0005-flutter-app.md * Update 0005-flutter-app.md * docs(docs): Use latest docs builders and fix concepts page * docs(cips): Start drafting the CIPS for milestone 2 * docs(cips): More text for RBAC metadata draft * docs(cips): WIP updates to draft cip for role registration * docs(cips): define draft specification for a ULID cbor tag * docs(cips): Further WIP edits to RBAC * docs(cips): fix ulid spec binary encoding reference * docs(cips): Add a tag to the epoch time. * docs(cips): Add CBOR tag cip for ED25519-BIP32 Keys, Derivation paths and Signatures * docs(cips): Properly define the field tags to use where known, and clean up Stake Address specification. * docs(cips): Fix nonce so its reliable without needing blockchain data * docs(cips): updates * docs(docs): Add CDDL definition for POC x509 envelope metadata * fix(vscode): update vscode extension recommendations * docs(cips): rbac x509 envelope fix * docs(cips): wip updates to high level docs * docs(cips): Add overview of cardano transaction processign and data * docs(cips): update cardano block to be complete for clarity * docs(cips): fix layout engine * docs(cips): wip cddl for envelope metadata * docs(cips): Add cddl specs and diagrams for x509 rbac registration work * docs(cips): Add full transaction/metadata relationship diagram * refactor(cips): reorganize documentation ready for drafting descriptive prose about the formats and uses * docs(cips): add cip draft for catalyst roles using the x509-rbac standard * docs(cips): Add c509 cddl with restrictions and enhancements for plutus usage * docs(cips): Metadata envelope specification draft complete * Update docs/src/catalyst-standards/draft-cips/c509-plutus-restricted-certificate/c509-cert-plutus-restricted.cddl Co-authored-by: bkioshn <35752733+bkioshn@users.noreply.github.com> * Update docs/src/catalyst-standards/draft-cips/c509-plutus-restricted-certificate/c509-cert-plutus-restricted.cddl Co-authored-by: bkioshn <35752733+bkioshn@users.noreply.github.com> * Update docs/src/catalyst-standards/draft-cips/x509-role-registration-metadata/x509-roles.cddl Co-authored-by: bkioshn <35752733+bkioshn@users.noreply.github.com> * Update docs/src/catalyst-standards/draft-cips/x509-role-registration-metadata/x509-roles.cddl Co-authored-by: bkioshn <35752733+bkioshn@users.noreply.github.com> * Update docs/src/catalyst-standards/draft-cips/x509-envelope-metadata/x509-envelope.cddl Co-authored-by: bkioshn <35752733+bkioshn@users.noreply.github.com> * Update docs/src/catalyst-standards/draft-cips/x509-envelope-metadata/x509-envelope.cddl Co-authored-by: bkioshn <35752733+bkioshn@users.noreply.github.com> * Update docs/src/catalyst-standards/draft-cips/x509-envelope-metadata/x509-envelope.cddl Co-authored-by: bkioshn <35752733+bkioshn@users.noreply.github.com> * Update docs/src/catalyst-standards/draft-cips/c509-plutus-restricted-certificate/c509-cert-plutus-restricted.cddl Co-authored-by: bkioshn <35752733+bkioshn@users.noreply.github.com> * docs(cips): Fix time and algorithm comments * build(frontend): Relax flutter/dart version requirements to last minor release * docs(cips): wip * fix(cips): rename draft x509 envelope CIP so its easier to identify * docs(cips): WIP updates to x509 roles * fix(cips): rename RBAC definition CIP draft so its easier to identify * docs(cips): x509 certificate registration format fully defined * docs(cips): Document the restricted plutus subset. * docs(cips): Add document detailing how CIP-30 is used to sign the transaction * fix(cips): remove trailing spaces * fix(cips): Fix line lengths * fix(cips): Correct spelling * fix(cips): spelling * fix(frontend): revert changes to flutter/dart versions * fix(frontend): more flutter/dart version corrections * fix(frontend): Revert flutter files to same as main branch * fix(frontend): revert more flutter .yml files to those in main * fix(cips): Fix links between files * docs(cips): Add catalyst specific role registration documentation * docs(spelling): fix spelling --------- Co-authored-by: minikin <djminikin@gmail.com> Co-authored-by: bkioshn <35752733+bkioshn@users.noreply.github.com> * ci: configure static analysis & code formatting check * style: formatting * fix: revert browser installation scripts * style: format code * ci: optimize directions include in repo-catalyst-voices-all artifact to include only needed ones * refactor: remove empty tests * ci: add melos script to generate test reports * ci: melos script to generate test report * style: revert previously generated files formatting * style: format files * ci: update melos to exclude generated code form code coverage * ci: cleanup build script * ci: generate multiple junit test reports and save them at /test_reports * ci: depend on melos analyze instead of custom command * docs: improve melos docs * ci: remove unused melos scripts * ci: format files in test & integration_test directories * ci: break code to make sure CI will report failure for demonstration purposes * style: fix lint issues * ci: change WORKDIR after creating the user to make sure it will be owned by that user * ci: restore root user * Revert "Merge branch 'main' into feat/collect-flutter-code-coverage" This reverts commit d0f66b2, reversing changes made to 39ce401. * style: format code * ci: revert test-unit target name --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Lucio Baglione <coire1@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Apisit Ritreungroj <38898766+apskhem@users.noreply.github.com> Co-authored-by: Alex Pozhylenkov <leshiy12345678@gmail.com> Co-authored-by: Steven Johnson <stevenj@users.noreply.github.com> Co-authored-by: minikin <djminikin@gmail.com> Co-authored-by: bkioshn <35752733+bkioshn@users.noreply.github.com> * refactor: update schema_validation check (#414) * update schema_validation check * wip * fix --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Lucio Baglione <coire1@gmail.com> Co-authored-by: Stefano Cunego <93382903+kukkok3@users.noreply.github.com> Co-authored-by: Dominik Toton <166132265+dtscalac@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Apisit Ritreungroj <38898766+apskhem@users.noreply.github.com> Co-authored-by: Alex Pozhylenkov <leshiy12345678@gmail.com> Co-authored-by: Steven Johnson <stevenj@users.noreply.github.com> Co-authored-by: bkioshn <35752733+bkioshn@users.noreply.github.com>
input-output-hk · Apr 16, 2024 · 3db8f17 · 3db8f17
1 parent 7ca47b1
commit 3db8f17
Show file tree

Hide file tree

Showing 42 changed files with 651 additions and 875 deletions.
diff --git a/.github/workflows/generate-allure-report.yml b/.github/workflows/generate-allure-report.yml
@@ -67,7 +67,7 @@ jobs:
         with:
             earthfile: ./catalyst_voices/
             flags:
-            targets: test-unit
+            targets: test-report
             target_flags:
             runner_address: ${{ secrets.EARTHLY_SATELLITE_ADDRESS }}
             artifact: "false"

diff --git a/Earthfile b/Earthfile
@@ -44,4 +44,14 @@ repo-catalyst-voices-packages:
     WORKDIR /repo
     COPY --dir catalyst_voices_packages .
 
+    SAVE ARTIFACT /repo repo
+
+# repo-catalyst-voices-all - Creates artifacts of all configuration files,
+# packages and folders related to catalyst_voices frontend.
+repo-catalyst-voices-all:
+    FROM scratch
+
+    WORKDIR /repo
+    COPY --dir catalyst_voices catalyst_voices_packages utilities melos.yaml pubspec.yaml  .
+
     SAVE ARTIFACT /repo repo
diff --git a/catalyst-gateway/bin/src/cli.rs b/catalyst-gateway/bin/src/cli.rs
@@ -58,7 +58,7 @@ impl Cli {
                 let machine_id = settings.follower_settings.machine_uid;
 
                 let state = Arc::new(State::new(Some(settings.database_url)).await?);
-                let event_db = state.event_db()?;
+                let event_db = state.event_db();
 
                 tokio::spawn(async move {
                     match service::run(&settings.docs_settings, state.clone()).await {

diff --git a/catalyst-gateway/bin/src/event_db/error.rs b/catalyst-gateway/bin/src/event_db/error.rs
@@ -1,5 +1,4 @@
 //! Database Errors
-use std::env::VarError;
 
 use bb8::RunError;
 
@@ -26,9 +25,6 @@ pub(crate) enum Error {
     /// Unknown error
     #[error("error: {0}")]
     Unknown(String),
-    /// Variable error
-    #[error(transparent)]
-    VarErr(#[from] VarError),
     /// No config
     #[error("No config")]
     NoConfig,

diff --git a/catalyst-gateway/bin/src/event_db/mod.rs b/catalyst-gateway/bin/src/event_db/mod.rs
@@ -54,9 +54,7 @@ pub(crate) struct EventDB {
 ///
 /// The env var "`DATABASE_URL`" can be set directly as an anv var, or in a
 /// `.env` file.
-pub(crate) async fn establish_connection(
-    url: Option<String>, do_schema_check: bool,
-) -> Result<EventDB, Error> {
+pub(crate) async fn establish_connection(url: Option<String>) -> Result<EventDB, Error> {
     // Support env vars in a `.env` file,  doesn't need to exist.
     dotenv().ok();
 
@@ -72,11 +70,5 @@ pub(crate) async fn establish_connection(
 
     let pool = Pool::builder().build(pg_mgr).await?;
 
-    let db = EventDB { pool };
-
-    if do_schema_check {
-        db.schema_version_check().await?;
-    }
-
-    Ok(db)
+    Ok(EventDB { pool })
 }
diff --git a/catalyst-gateway/bin/src/service/api/cardano/date_time_to_slot_number_get.rs b/catalyst-gateway/bin/src/service/api/cardano/date_time_to_slot_number_get.rs
@@ -1,13 +1,9 @@
 //! Implementation of the GET `/date_time_to_slot_number` endpoint
 
-use poem_extensions::{
-    response,
-    UniResponse::{T200, T503},
-};
+use poem_extensions::{response, UniResponse::T200};
 use poem_openapi::payload::Json;
 
 use crate::{
-    cli::Error,
     event_db::{
         error::Error as DBError,
         follower::{DateTime, SlotInfoQueryType},
@@ -23,7 +19,7 @@ use crate::{
             resp_5xx::{server_error_response, ServerError, ServiceUnavailable},
         },
     },
-    state::{SchemaVersionStatus, State},
+    state::State,
 };
 
 /// # All Responses
@@ -39,19 +35,7 @@ pub(crate) type AllResponses = response! {
 pub(crate) async fn endpoint(
     state: &State, date_time: Option<DateTime>, network: Option<Network>,
 ) -> AllResponses {
-    let event_db = match state.event_db() {
-        Ok(event_db) => event_db,
-        Err(Error::EventDb(DBError::MismatchedSchema { was, expected })) => {
-            tracing::error!(
-                expected = expected,
-                current = was,
-                "DB schema version status mismatch"
-            );
-            state.set_schema_version_status(SchemaVersionStatus::Mismatch);
-            return T503(ServiceUnavailable);
-        },
-        Err(err) => return server_error_response!("{err}"),
-    };
+    let event_db = state.event_db();
 
     let date_time = date_time.unwrap_or_else(chrono::Utc::now);
     let network = network.unwrap_or(Network::Mainnet);

diff --git a/catalyst-gateway/bin/src/service/api/cardano/registration_get.rs b/catalyst-gateway/bin/src/service/api/cardano/registration_get.rs
@@ -2,12 +2,11 @@
 
 use poem_extensions::{
     response,
-    UniResponse::{T200, T400, T404, T503},
+    UniResponse::{T200, T400, T404},
 };
 use poem_openapi::payload::Json;
 
 use crate::{
-    cli::Error,
     event_db::{error::Error as DBError, follower::SlotNumber},
     service::{
         common::{
@@ -22,7 +21,7 @@ use crate::{
         },
         utilities::check_network,
     },
-    state::{SchemaVersionStatus, State},
+    state::State,
 };
 
 /// # All Responses
@@ -40,19 +39,8 @@ pub(crate) async fn endpoint(
     state: &State, stake_address: StakeAddress, provided_network: Option<Network>,
     slot_num: Option<SlotNumber>,
 ) -> AllResponses {
-    let event_db = match state.event_db() {
-        Ok(event_db) => event_db,
-        Err(Error::EventDb(DBError::MismatchedSchema { was, expected })) => {
-            tracing::error!(
-                expected = expected,
-                current = was,
-                "DB schema version status mismatch"
-            );
-            state.set_schema_version_status(SchemaVersionStatus::Mismatch);
-            return T503(ServiceUnavailable);
-        },
-        Err(err) => return server_error_response!("{err}"),
-    };
+    let event_db = state.event_db();
+
     let date_time = slot_num.unwrap_or(SlotNumber::MAX);
     let stake_credential = stake_address.payload().as_hash().to_vec();
     let network = match check_network(stake_address.network(), provided_network) {

diff --git a/catalyst-gateway/bin/src/service/api/cardano/staked_ada_get.rs b/catalyst-gateway/bin/src/service/api/cardano/staked_ada_get.rs
@@ -2,12 +2,11 @@
 
 use poem_extensions::{
     response,
-    UniResponse::{T200, T400, T404, T503},
+    UniResponse::{T200, T400, T404},
 };
 use poem_openapi::payload::Json;
 
 use crate::{
-    cli::Error,
     event_db::{error::Error as DBError, follower::SlotNumber},
     service::{
         common::{
@@ -22,7 +21,7 @@ use crate::{
         },
         utilities::check_network,
     },
-    state::{SchemaVersionStatus, State},
+    state::State,
 };
 
 /// # All Responses
@@ -39,19 +38,7 @@ pub(crate) async fn endpoint(
     state: &State, stake_address: StakeAddress, provided_network: Option<Network>,
     slot_num: Option<SlotNumber>,
 ) -> AllResponses {
-    let event_db = match state.event_db() {
-        Ok(event_db) => event_db,
-        Err(Error::EventDb(DBError::MismatchedSchema { was, expected })) => {
-            tracing::error!(
-                expected = expected,
-                current = was,
-                "DB schema version status mismatch"
-            );
-            state.set_schema_version_status(SchemaVersionStatus::Mismatch);
-            return T503(ServiceUnavailable);
-        },
-        Err(err) => return server_error_response!("{err}"),
-    };
+    let event_db = state.event_db();
 
     let date_time = slot_num.unwrap_or(SlotNumber::MAX);
     let stake_credential = stake_address.payload().as_hash().to_vec();

diff --git a/catalyst-gateway/bin/src/service/api/cardano/sync_state_get.rs b/catalyst-gateway/bin/src/service/api/cardano/sync_state_get.rs
@@ -2,12 +2,11 @@
 
 use poem_extensions::{
     response,
-    UniResponse::{T200, T404, T503},
+    UniResponse::{T200, T404},
 };
 use poem_openapi::payload::Json;
 
 use crate::{
-    cli::Error,
     event_db::error::Error as DBError,
     service::common::{
         objects::cardano::{network::Network, sync_state::SyncState},
@@ -17,7 +16,7 @@ use crate::{
             resp_5xx::{server_error_response, ServerError, ServiceUnavailable},
         },
     },
-    state::{SchemaVersionStatus, State},
+    state::State,
 };
 
 /// # All Responses
@@ -32,19 +31,7 @@ pub(crate) type AllResponses = response! {
 /// # GET `/sync_state`
 #[allow(clippy::unused_async)]
 pub(crate) async fn endpoint(state: &State, network: Option<Network>) -> AllResponses {
-    let event_db = match state.event_db() {
-        Ok(event_db) => event_db,
-        Err(Error::EventDb(DBError::MismatchedSchema { was, expected })) => {
-            tracing::error!(
-                expected = expected,
-                current = was,
-                "DB schema version status mismatch"
-            );
-            state.set_schema_version_status(SchemaVersionStatus::Mismatch);
-            return T503(ServiceUnavailable);
-        },
-        Err(err) => return server_error_response!("{err}"),
-    };
+    let event_db = state.event_db();
 
     let network = network.unwrap_or(Network::Mainnet);
 

diff --git a/catalyst-gateway/bin/src/service/api/health/ready_get.rs b/catalyst-gateway/bin/src/service/api/health/ready_get.rs
@@ -9,14 +9,13 @@ use poem_extensions::{
 };
 
 use crate::{
-    cli::Error,
     event_db::error::Error as DBError,
     service::common::responses::{
         resp_2xx::NoContent,
         resp_4xx::ApiValidationError,
         resp_5xx::{server_error_response, ServerError, ServiceUnavailable},
     },
-    state::{SchemaVersionStatus, State},
+    state::State,
 };
 
 /// All responses
@@ -53,22 +52,20 @@ pub(crate) type AllResponses = response! {
 ///   but unlikely)
 /// * 503 Service Unavailable - Service is not ready, do not send other requests.
 pub(crate) async fn endpoint(state: Data<&Arc<State>>) -> AllResponses {
-    match state.schema_version_check().await {
+    match state.event_db().schema_version_check().await {
         Ok(_) => {
             tracing::debug!("DB schema version status ok");
-            state.set_schema_version_status(SchemaVersionStatus::Ok);
             T204(NoContent)
         },
-        Err(Error::EventDb(DBError::MismatchedSchema { was, expected })) => {
+        Err(DBError::MismatchedSchema { was, expected }) => {
             tracing::error!(
                 expected = expected,
                 current = was,
                 "DB schema version status mismatch"
             );
-            state.set_schema_version_status(SchemaVersionStatus::Mismatch);
             T503(ServiceUnavailable)
         },
-        Err(Error::EventDb(DBError::TimedOut)) => T503(ServiceUnavailable),
+        Err(DBError::TimedOut) => T503(ServiceUnavailable),
         Err(err) => server_error_response!("{err}"),
     }
 }
diff --git a/catalyst-gateway/bin/src/service/mod.rs b/catalyst-gateway/bin/src/service/mod.rs
@@ -23,9 +23,6 @@ pub(crate) enum Error {
     /// An IO error has occurred
     #[error(transparent)]
     Io(#[from] std::io::Error),
-    /// A mismatch in the expected `EventDB` schema version
-    #[error("expected schema version mismatch")]
-    SchemaVersionMismatch,
 }
 
 /// # Run Catalyst Gateway Service.

diff --git a/catalyst-gateway/bin/src/service/utilities/middleware/schema_validation.rs b/catalyst-gateway/bin/src/service/utilities/middleware/schema_validation.rs
@@ -11,10 +11,7 @@ use std::sync::Arc;
 
 use poem::{web::Data, Endpoint, EndpointExt, Middleware, Request, Result};
 
-use crate::{
-    service::common::responses::resp_5xx::ServiceUnavailable,
-    state::{SchemaVersionStatus, State},
-};
+use crate::{service::common::responses::resp_5xx::ServiceUnavailable, state::State};
 
 /// A middleware that raises an error  with `ServiceUnavailable` and 503 status code
 /// if a DB schema version mismatch is found the existing `State`.
@@ -44,7 +41,7 @@ impl<E: Endpoint> Endpoint for SchemaVersionValidationImpl<E> {
             // if so, return the `ServiceUnavailable` error, which implements
             // `ResponseError`, with status code `503`.
             // Otherwise, return the endpoint as usual.
-            if state.is_schema_version_status(&SchemaVersionStatus::Mismatch) {
+            if state.event_db().schema_version_check().await.is_err() {
                 return Err(ServiceUnavailable.into());
             }
         }