Skip to content
This repository has been archived by the owner on Mar 1, 2024. It is now read-only.

Combine sdk and cli #290

Closed
wants to merge 2 commits into from
Closed

Combine sdk and cli #290

wants to merge 2 commits into from

Conversation

ParthDesai
Copy link
Contributor

@ParthDesai ParthDesai commented Jan 23, 2024

Description

This PR combines subspace-sdk into this repository.

What is changed?

First commit

First commit combines sdk and cli and some changes to make it compile and test successfully. The integration test and examples of sdk shifted to the root of the project so that they can be recognized by the rust ecosystem.

Second commit

Second commit updates the test CI. The pulsar CI was running format, test and clippy in one job but due to addition of sdk tests which are long running, we need to have different jobs. The new jobs are basically copy of SDK's original jobs.

Copy link
Member

@nazar-pc nazar-pc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that this is a straightforward conversion, but this is not how it should generally be done. First, the repository should be converted into workspace, with pulsar being not the root package, but rather one of workspace packages instead. Then SDK should be added as another workspace member that Pulsar can use, that way you have one lock file and don't need such a messy CI and cd sdk in strategic places.

Comment on lines +2 to +4
# Since, SDK and pulsar both uses async heavily, we need to disallow below type to prevent OS thread block
# This is imported from SDK
disallowed-types = ["std::sync::Mutex", "std::sync::RwLock", "std::sync::Once", "std::sync::Condvar"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly it doesn't make a lot of sense to me. There is nothing wrong with these types if they are use properly and clippy will already warn when potentially used incorrectly.

Comment on lines +46 to +58
anyhow = "1"
clap = { version = "4", features = ["derive"] }
derive_builder = "0.12"
derive_more = "0.99"
fdlimit = "0.2"
futures = "0.3"
serde_json = "1"
subspace-farmer-components = { git = "https://github.com/subspace/subspace", rev = "bd435100200b3dcce6d6f50534d52e3cd039ca8e" }
tempfile = "3"
tokio = { version = "1.34.0", features = ["rt-multi-thread", "macros"] }
tracing = "0.1"
tracing-futures = "0.2"
tracing-subscriber = "0.3"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Examples and integrations tests should be under sdk, not main package.

@ParthDesai
Copy link
Contributor Author

ParthDesai commented Jan 23, 2024

I understand that this is a straightforward conversion, but this is not how it should generally be done. First, the repository should be converted into workspace, with pulsar being not the root package, but rather one of workspace packages instead. Then SDK should be added as another workspace member that Pulsar can use, that way you have one lock file and don't need such a messy CI and cd sdk in strategic places.

If we want to go with workspace route, we cannot have sdk as member, since it is itself a workspace and nested workspaces are not allowed (yet). In this case, we will need to put members of sdk workspace as part of the root workspace with examples and integration tests in root.

@nazar-pc
Copy link
Member

In this case, we will need to put members of sdk workspace as part of the root workspace

Yes

with examples and integration tests in root.

As I explained above, we don't need anything at the root. The root should be just a workspace, see how monorepo is organized.

@nazar-pc
Copy link
Member

I cancelled extra workflow for now to open CI time for other jobs, the manually triggered workflow is still running


jobs:
clippy-fmt-test:
fmt:
runs-on: ${{ fromJson(github.repository_owner == 'subspace' && '["self-hosted", "ubuntu-20.04-x86-64"]' || 'ubuntu-22.04') }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
runs-on: ${{ fromJson(github.repository_owner == 'subspace' && '["self-hosted", "ubuntu-20.04-x86-64"]' || 'ubuntu-22.04') }}
runs-on: ${{ fromJson(github.repository_owner == 'subspace' && '["self-hosted", "ubuntu-20.04-x86-64"]' || '"ubuntu-22.04"') }}

@ParthDesai
Copy link
Contributor Author

Closing this in favor of #291

@ParthDesai ParthDesai closed this Jan 25, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants