New http api to nomos-node integration #490

al8n · 2023-10-30T02:36:14Z

Integrate new http api crate to nomos-node.

…omos-node into http-api-axum-backend

…into metrics-api

bacv · 2023-11-01T09:40:33Z

nodes/nomos-node-api/src/http/backend/axum.rs

-async fn add_cert(State(store): State<OverwatchHandle>, Json(cert): Json<Certificate>) -> Response {
+async fn add_cert(
+    State(handle): State<OverwatchHandle>,
+    Json(cert): Json<Certificate>,


This handle should expect Cert that is serialized with wire::serialize. At least mempool-da/add had it in such way.
Expecting bytes and then deserializing it with wire::deserialize::<Certificate>(&payload).unwrap() works, but I think this would ultimately need it's own extractor.

Ah, thanks! I think this should be the reason why the CI continues failing.

tbh, if all other API are json we should be consistent and use json here as well unless there's a specific reason not to do so

tbh, if all other API are json we should be consistent and use json here as well unless there's a specific reason not to do so

Yes, make sense, I will try to make the CI works by using JSON args.

danielSanchezQ · 2023-11-01T12:36:23Z

tests/Cargo.toml

@@ -23,7 +23,7 @@ mixnet-topology = { path = "../mixnet/topology" }
 # Using older versions, since `mixnet-*` crates depend on `rand` v0.7.3.
 rand = "0.7.3"
 once_cell = "1"
-secp256k1 = { version = "0.26", features = ["rand"] }
+secp256k1 = { version = "0.28", features = ["rand"] }


Why do we update this here? Are we using something not available in other versions? Otherwise it would be better if it has its own pr.

This should be updated by vscode plugin. I will change it back.

danielSanchezQ · 2023-11-01T12:39:19Z

nodes/nomos-node-api/src/lib.rs

-        Ok(Self { settings })
+        Ok(Self {
+            settings,
+            handle: service_state.overwatch_handle,
+        })
    }

    /// Service main loop
    async fn run(mut self) -> Result<(), DynError> {
        let endpoint = B::new(self.settings.backend_settings).await?;
-        endpoint.serve().await?;
+        endpoint.serve(self.handle).await?;
        Ok(())
    }
 }


This probably needs to be updated. There are things to do here. Usually you would want the service to hold the whole service state. Lifecycle is ready as well. Not to be done now, but lets open an issue to handle the service here. I can probably implement that myself.

Created at #500.

danielSanchezQ · 2023-11-01T12:40:52Z

nodes/nomos-node-api/src/http/backend/axum.rs

+#[derive(Debug, Clone, serde::Deserialize, serde::Serialize)]
+#[cfg_attr(feature = "clap", derive(clap::Args))]
 pub struct AxumBackendSettings {
-    pub addr: SocketAddr,
-    pub handle: OverwatchHandle,
+    /// Socket where the server will be listening on for incoming requests.
+    #[cfg_attr(feature = "clap", arg(
+        short, long = "http-addr",
+        default_value_t = std::net::SocketAddr::new(
+            std::net::IpAddr::V4(std::net::Ipv4Addr::new(127, 0, 0, 1)),
+            8080,
+        ),
+        env = "HTTP_BIND_ADDRESS"
+    ))]
+    pub address: std::net::SocketAddr,
+    /// Allowed origins for this server deployment requests.
+    #[cfg_attr(feature = "clap", arg(long = "http-cors-origin"))]
+    pub cors_origins: Vec<String>,


Why do we have clap here? Are we using it as standalone somewhere?

In the old HTTP service, we have clap, so I just keep them in the new HTTP service. I have not seen we use it standalone.

I think we should remove it if we do not use it.

nodes/nomos-node-api/tests/todo.rs

danielSanchezQ

Looking good! and CI is 🟢 !
Just left a tiny comment.

danielSanchezQ · 2023-11-06T08:28:10Z

nodes/nomos-node/src/main.rs

-use nomos_mempool::network::adapters::libp2p::{Libp2pAdapter, Settings as AdapterSettings};
-use nomos_network::backends::libp2p::Libp2p;
+
+use nomos_mempool::network::adapters::libp2p::Settings as AdapterSettings;


Why is this change here? This is related to mempool right?

Yes, we need this setting to construct a mempool service.

But why the adapter and the backend is gone now?

In the previous, we had type aliases for creating bridges.

* Add raw api module to nomos-cli Add a separate module which contains all raw api calls and make them reuse the same reqwest client for increased efficiency * Use json instead of wire format in mempool add api Uniform all http api to use json as the encoding

Put a hard limit of 512 blocks in the response returned by GetBlocks to avoid slowing things down. This number was chosen rather arbitrarily. We might want to do some more fine tuning.

* Readd docker build context * Configurable da protocol voter * Do not use private key naming for da voter

* Add option to provide custom writer to the logger backend * fmt

* Do not exit the process during dissemination This is now library code and we should not exit the process if an error is encountered * clippy happy

add HTTP requests header

al8n added 30 commits October 17, 2023 16:57

axum backend

04652c3

Add metrics API

c11977a

Merge branch 'master' into http-api-axum-backend

078ff64

Fix tests

8feb6a0

Merge branch 'http-api-axum-backend' of https://github.com/logos-co/n…

683a664

…omos-node into http-api-axum-backend

Merge branch 'http-api-axum-backend' into metrics-api

ab70f86

Add imports

b5c9874

fix tests

bafb133

remove unused dependencies

925e89d

remove unused dependencies

ee4f912

Merge branch 'master' into http-api-axum-backend

179872e

Merge branch 'http-api-axum-backend' into metrics-api

eb425d5

fix CI

7f51104

Merge branch 'http-api-axum-backend' of https://github.com/logos-co/n…

8b16f49

…omos-node into http-api-axum-backend

Fix test

5f593f4

Merge branch 'http-api-axum-backend' into metrics-api

f74cbfd

fix fmt

8020526

remove unused dependencies

613ba12

remove unused dependency

934c776

Make cl generic

acc8221

Merge branch 'http-api-axum-backend' into metrics-api

2f99091

remove left-over

cf9c3b6

Merge branch 'metrics-api' of https://github.com/logos-co/nomos-node …

68eda42

…into metrics-api

fix CI

51599c6

Merge branch 'master' into metrics-api

d92ba77

move crate

935ec05

fix fmt

1e57334

Fix CI

a87baf8

Merge branch 'master' into metrics-api

2069a49

Merge branch 'master' into metrics-api

f1184ab

al8n added 2 commits November 1, 2023 17:21

Merge branch 'master' into integrate-new-http

fdc03ae

fix CI

bcd02e6

bacv reviewed Nov 1, 2023

View reviewed changes

danielSanchezQ reviewed Nov 1, 2023

View reviewed changes

youngjoon-lee reviewed Nov 1, 2023

View reviewed changes

nodes/nomos-node-api/tests/todo.rs Show resolved Hide resolved

al8n mentioned this pull request Nov 1, 2023

Handle service state update events in nomos-node-api #500

Open

al8n added 3 commits November 1, 2023 21:05

remove clap

ae6715b

fix CI

6492162

Merge branch 'master' into integrate-new-http

4eba6b8

al8n requested review from danielSanchezQ, youngjoon-lee and bacv November 6, 2023 02:33

al8n added 2 commits November 6, 2023 10:33

Merge branch 'master' into integrate-new-http

3da3b93

Add missing feature

3446d2a

danielSanchezQ approved these changes Nov 6, 2023

View reviewed changes

al8n and others added 9 commits November 8, 2023 13:40

merge

17b6fb3

Add block_info api for new HTTP service (#509)

aac71bd

Store logs from failed CI as artifacts (#508)

4c25718

Limit GetBlocks response size (#515)

b8f0f3e

Put a hard limit of 512 blocks in the response returned by GetBlocks to avoid slowing things down. This number was chosen rather arbitrarily. We might want to do some more fine tuning.

Da voter config for docker compose (#513)

90b2268

* Readd docker build context * Configurable da protocol voter * Do not use private key naming for da voter

Add option to provide custom writer to the logger backend (#518)

44c7509

* Add option to provide custom writer to the logger backend * fmt

Do not exit the process during dissemination (#516)

ee50585

* Do not exit the process during dissemination This is now library code and we should not exit the process if an error is encountered * clippy happy

do not use prefix in serialization (#514)

c6f9590

add HTTP requests header

al8n force-pushed the integrate-new-http branch from 23c023d to c6f9590 Compare November 8, 2023 06:25

al8n added 2 commits November 8, 2023 14:29

fix conflicts

4541f58

remove open api derive on Voter alias

eda0e4e

al8n force-pushed the integrate-new-http branch from 888d233 to eda0e4e Compare November 8, 2023 07:16

al8n merged commit c3422c1 into master Nov 8, 2023
7 checks passed

al8n deleted the integrate-new-http branch November 8, 2023 08:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New http api to nomos-node integration #490

New http api to nomos-node integration #490

al8n commented Oct 30, 2023

bacv Nov 1, 2023

al8n Nov 1, 2023

zeegomo Nov 2, 2023

al8n Nov 2, 2023

danielSanchezQ Nov 1, 2023

al8n Nov 1, 2023

danielSanchezQ Nov 1, 2023

al8n Nov 1, 2023

danielSanchezQ Nov 1, 2023

al8n Nov 1, 2023

danielSanchezQ Nov 1, 2023

danielSanchezQ left a comment

danielSanchezQ Nov 6, 2023

al8n Nov 6, 2023

danielSanchezQ Nov 6, 2023

al8n Nov 6, 2023

New http api to nomos-node integration #490

New http api to nomos-node integration #490

Conversation

al8n commented Oct 30, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danielSanchezQ left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment