Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

subxt-core crate #1466

Merged
merged 67 commits into from
Mar 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
28f6ceb
start migrating, broken
tadeohepperle Feb 28, 2024
75caf56
first iteration of updating
tadeohepperle Feb 29, 2024
7907468
fmt and clippy
tadeohepperle Feb 29, 2024
33af807
Merge branch 'master' into tadeohepperle/scale-type-resolver-integration
tadeohepperle Feb 29, 2024
1b822f8
Merge branch 'master' into tadeohepperle/scale-type-resolver-integration
tadeohepperle Mar 1, 2024
03127c7
Merge branch 'master' into tadeohepperle/scale-type-resolver-integration
tadeohepperle Mar 5, 2024
f07813f
add Composite<u32> decoding via scale value patch
tadeohepperle Mar 5, 2024
3f6fa39
bump scale type gen versions
tadeohepperle Mar 5, 2024
fd8458b
Merge branch 'master' into tadeohepperle/scale-type-resolver-integration
tadeohepperle Mar 6, 2024
400f379
fix decoding with new scale decode
tadeohepperle Mar 7, 2024
022892f
compiling with changed deps
tadeohepperle Mar 7, 2024
0922672
core utils, condig, client, metadata
tadeohepperle Mar 7, 2024
37ee03e
core crate compiling
tadeohepperle Mar 7, 2024
78bc368
signer crate no once lock
tadeohepperle Mar 7, 2024
0cf6f57
add core to no-std-tests, change imports
tadeohepperle Mar 7, 2024
9c541e0
broken commit, start pulling everything together in subxt
tadeohepperle Mar 7, 2024
95e6711
port more things to subxt
tadeohepperle Mar 11, 2024
1773021
events in core crate, extrinsics sadly much more difficult
tadeohepperle Mar 11, 2024
2be1d7a
almost all examples pass again
tadeohepperle Mar 11, 2024
535c28b
dynamic values fix in examples
tadeohepperle Mar 12, 2024
5e8f2f4
fix no std issue and fmt
tadeohepperle Mar 12, 2024
0c4bbc0
remove unused dependencies
tadeohepperle Mar 12, 2024
b499239
fix lightclient impl
tadeohepperle Mar 12, 2024
116b800
Merge branch 'master' into tadeohepperle/subxt-core-3
tadeohepperle Mar 13, 2024
45bb612
runtime version refactor
tadeohepperle Mar 13, 2024
50640cf
formatting and addressing nits
tadeohepperle Mar 13, 2024
2ba9752
more comments addressed
tadeohepperle Mar 13, 2024
46ec496
update wasm example and no-std-signer tests
tadeohepperle Mar 13, 2024
42ac561
Merge branch 'master' into tadeohepperle/subxt-core-3
tadeohepperle Mar 13, 2024
d326733
other nits and error impl on signer errors
tadeohepperle Mar 13, 2024
d3524e6
Merge branch 'master' into tadeohepperle/subxt-core-3
tadeohepperle Mar 14, 2024
09802e0
fix feature flag
tadeohepperle Mar 14, 2024
ba1f2f4
fix runtime version refactor
tadeohepperle Mar 14, 2024
13d0ef7
fix doc links
tadeohepperle Mar 14, 2024
59a289d
fix integration tests
tadeohepperle Mar 14, 2024
e1eabb7
fix feature flag gated client state
tadeohepperle Mar 15, 2024
8ab7ee8
Merge branch 'master' into tadeohepperle/subxt-core-3
tadeohepperle Mar 15, 2024
48a32be
fix native feature in CI
tadeohepperle Mar 15, 2024
6addf45
fix lightclient utils
tadeohepperle Mar 18, 2024
314a760
Merge branch 'master' into tadeohepperle/subxt-core-3
tadeohepperle Mar 18, 2024
091385a
make imports more lean in subxt-core
tadeohepperle Mar 18, 2024
739b3cb
integrate changes from subxt-core imports into subxt
tadeohepperle Mar 18, 2024
323e3ab
other changes in subxt simplify imports more
tadeohepperle Mar 18, 2024
1739f85
fix and docs
tadeohepperle Mar 18, 2024
c1a7a52
doc false for cli
tadeohepperle Mar 18, 2024
eeca559
fix clippy
tadeohepperle Mar 18, 2024
6b228da
Merge branch 'master' into tadeohepperle/subxt-core-3
tadeohepperle Mar 18, 2024
70962f1
remove events block hash in tests
tadeohepperle Mar 19, 2024
ac23528
codegen no-std support in generated code
tadeohepperle Mar 19, 2024
8992b95
export alloc crate for no-std codegen
tadeohepperle Mar 20, 2024
c71825f
fix doc test
tadeohepperle Mar 20, 2024
06cfb6d
implement James comments
tadeohepperle Mar 20, 2024
5caed4a
remove std traits, use core traits instead
tadeohepperle Mar 20, 2024
6cd17e6
address nits
tadeohepperle Mar 21, 2024
1ad09f4
remove unusued dep in no-std tests
tadeohepperle Mar 21, 2024
c1a2e3f
fix Box import in no_std
tadeohepperle Mar 21, 2024
3051462
Merge branch 'master' into tadeohepperle/subxt-core-3
tadeohepperle Mar 22, 2024
e273363
sp-crypto-hashing instead of sp-core-hashing
tadeohepperle Mar 22, 2024
1ba372a
bump scale-typegen, add no std codegen tests
tadeohepperle Mar 22, 2024
b2aa59e
fix some things
tadeohepperle Mar 22, 2024
95ef002
Merge branch 'master' into tadeohepperle/subxt-core-3
jsdw Mar 25, 2024
88c3cee
replace unmaintained derivative with derive_where to remove non-canon…
jsdw Mar 26, 2024
0aebf5e
fmt
jsdw Mar 26, 2024
c36c018
remove unused dep
jsdw Mar 26, 2024
60ef594
fix deps
jsdw Mar 26, 2024
519a107
update artifacts to fix type ID mismatches
jsdw Mar 26, 2024
7fb423d
bump to latest scale-typegen
jsdw Mar 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -187,8 +187,8 @@ jobs:
- name: Cargo check subxt-signer
run: |
cargo check -p subxt-signer
cargo check -p subxt-signer --no-default-features --features sr25519,native
cargo check -p subxt-signer --no-default-features --features ecdsa,native
cargo check -p subxt-signer --no-default-features --features sr25519
cargo check -p subxt-signer --no-default-features --features ecdsa

# We can't enable web features here, so no cargo hack.
- name: Cargo check subxt-lightclient
Expand Down
67 changes: 54 additions & 13 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 5 additions & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
members = [
"cli",
"codegen",
"core",
"lightclient",
"testing/substrate-runner",
"testing/test-runtime",
Expand Down Expand Up @@ -70,7 +71,7 @@ codec = { package = "parity-scale-codec", version = "3.6.9", default-features =
color-eyre = "0.6.3"
console_error_panic_hook = "0.1.7"
darling = "0.20.8"
derivative = "2.2.0"
derive-where = "1.2.7"
derive_more = "0.99.17"
either = { version = "1.10.0", default-features = false }
frame-metadata = { version = "16.0.0", default-features = false }
Expand All @@ -93,6 +94,8 @@ scale-value = { version = "0.14.1", default-features = false }
scale-bits = { version = "0.5.0", default-features = false }
scale-decode = { version = "0.11.1", default-features = false }
scale-encode = { version = "0.6.0", default-features = false }
scale-typegen = "0.4.2"
scale-typegen-description = "0.4.2"
serde = { version = "1.0.197", default-features = false, features = ["derive"] }
serde_json = { version = "1.0.114", default-features = false }
syn = { version = "2.0.15", features = ["full", "extra-traits"] }
Expand All @@ -106,8 +109,6 @@ url = "2.5.0"
wabt = "0.10.0"
wasm-bindgen-test = "0.3.24"
which = "5.0.0"
scale-typegen-description = "0.2.0"
scale-typegen = "0.2.0"
strip-ansi-escapes = "0.2.0"

# Light client support:
Expand Down Expand Up @@ -136,6 +137,7 @@ sp-keyring = "34.0.0"

# Subxt workspace crates:
subxt = { version = "0.35.0", path = "subxt", default-features = false }
subxt-core = { version = "0.35.0", path = "core", default-features = false }
subxt-macro = { version = "0.35.0", path = "macro" }
subxt-metadata = { version = "0.35.0", path = "metadata", default-features = false }
subxt-codegen = { version = "0.35.0", path = "codegen" }
Expand Down
12 changes: 6 additions & 6 deletions artifacts/demo_chain_specs/polkadot.json

Large diffs are not rendered by default.

Binary file modified artifacts/polkadot_metadata_full.scale
Binary file not shown.
Binary file modified artifacts/polkadot_metadata_small.scale
Binary file not shown.
Binary file modified artifacts/polkadot_metadata_tiny.scale
Binary file not shown.
2 changes: 1 addition & 1 deletion cli/src/commands/codegen.rs
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ pub struct Opts {
#[clap(long = "substitute-type", value_parser = substitute_type_parser)]
substitute_types: Vec<(String, String)>,
/// The `subxt` crate access path in the generated code.
/// Defaults to `::subxt`.
/// Defaults to `::subxt::ext::subxt_core`.
#[clap(long = "crate")]
crate_path: Option<String>,
/// Do not generate documentation for the runtime API code.
Expand Down
5 changes: 1 addition & 4 deletions cli/src/commands/explore/pallets/calls.rs
Original file line number Diff line number Diff line change
Expand Up @@ -154,10 +154,7 @@ fn mocked_offline_client(metadata: Metadata) -> OfflineClient<SubstrateConfig> {
H256::from_str("91b171bb158e2d3848fa23a9f1c25182fb8e20313b2c1eb49219da7a70ce90c3")
.expect("Valid hash; qed");

let runtime_version = subxt::backend::RuntimeVersion {
spec_version: 9370,
transaction_version: 20,
};
let runtime_version = subxt::client::RuntimeVersion::new(9370, 20);

OfflineClient::<SubstrateConfig>::new(genesis_hash, runtime_version, metadata)
}
Expand Down
15 changes: 10 additions & 5 deletions codegen/src/api/calls.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ use super::CodegenError;
use heck::{ToSnakeCase as _, ToUpperCamelCase as _};
use proc_macro2::TokenStream as TokenStream2;
use quote::{format_ident, quote};
use scale_typegen::typegen::ir::ToTokensWithSettings;
use scale_typegen::{typegen::ir::type_ir::CompositeIRKind, TypeGenerator};
use subxt_metadata::PalletMetadata;

Expand All @@ -16,7 +17,7 @@ use subxt_metadata::PalletMetadata;
///
/// - `type_gen` - [`scale_typegen::TypeGenerator`] that contains settings and all types from the runtime metadata.
/// - `pallet` - Pallet metadata from which the calls are generated.
/// - `crate_path` - The crate path under which subxt is located, e.g. `::subxt` when using subxt as a dependency.
/// - `crate_path` - The crate path under which the `subxt-core` crate is located, e.g. `::subxt::ext::subxt_core` when using subxt as a dependency.
pub fn generate_calls(
type_gen: &TypeGenerator,
pallet: &PalletMetadata,
Expand All @@ -41,9 +42,9 @@ pub fn generate_calls(
.iter()
.map(|(name, field)| {
// Note: fn_arg_type this is relative the type path of the type alias when prefixed with `types::`, e.g. `set_max_code_size::New`
let fn_arg_type = &field.type_path;
let fn_arg_type = field.type_path.to_token_stream(type_gen.settings());
let call_arg = if field.is_boxed {
quote! { #name: ::std::boxed::Box::new(#name) }
quote! { #name: #crate_path::alloc::boxed::Box::new(#name) }
} else {
quote! { #name }
};
Expand Down Expand Up @@ -71,7 +72,9 @@ pub fn generate_calls(
let docs = &var.composite.docs;

// this converts the composite into a full struct type. No Type Parameters needed here.
let struct_def = type_gen.upcast_composite(&var.composite);
let struct_def = type_gen
.upcast_composite(&var.composite)
.to_token_stream(type_gen.settings());
let alias_mod = var.type_alias_mod;
// The call structure's documentation was stripped above.
let call_struct = quote! {
Expand Down Expand Up @@ -105,7 +108,9 @@ pub fn generate_calls(
.into_iter()
.unzip();

let call_type = type_gen.resolve_type_path(call_ty)?;
let call_type = type_gen
.resolve_type_path(call_ty)?
.to_token_stream(type_gen.settings());
let call_ty = type_gen.resolve_type(call_ty)?;
let docs = type_gen.docs_from_scale_info(&call_ty.docs);

Expand Down
7 changes: 5 additions & 2 deletions codegen/src/api/constants.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
use heck::ToSnakeCase as _;
use proc_macro2::TokenStream as TokenStream2;
use quote::{format_ident, quote};
use scale_typegen::typegen::ir::ToTokensWithSettings;
use scale_typegen::TypeGenerator;
use subxt_metadata::PalletMetadata;

Expand All @@ -31,7 +32,7 @@ use super::CodegenError;
///
/// - `type_gen` - [`scale_typegen::TypeGenerator`] that contains settings and all types from the runtime metadata.
/// - `pallet` - Pallet metadata from which the constants are generated.
/// - `crate_path` - The crate path under which subxt is located, e.g. `::subxt` when using subxt as a dependency.
/// - `crate_path` - The crate path under which the `subxt-core` crate is located, e.g. `::subxt::ext::subxt_core` when using subxt as a dependency.
pub fn generate_constants(
type_gen: &TypeGenerator,
pallet: &PalletMetadata,
Expand All @@ -55,7 +56,9 @@ pub fn generate_constants(
));
};

let return_ty = type_gen.resolve_type_path(constant.ty())?;
let return_ty = type_gen
.resolve_type_path(constant.ty())?
.to_token_stream(type_gen.settings());
let docs = constant.docs();
let docs = type_gen
.settings()
Expand Down
7 changes: 4 additions & 3 deletions codegen/src/api/custom_values.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,13 @@
// see LICENSE for license details.

use heck::ToSnakeCase as _;
use scale_typegen::typegen::ir::ToTokensWithSettings;
use scale_typegen::TypeGenerator;
use std::collections::HashSet;
use subxt_metadata::{CustomValueMetadata, Metadata};

use proc_macro2::TokenStream as TokenStream2;
use quote::{quote, ToTokens};
use quote::quote;

/// Generate the custom values mod, if there are any custom values in the metadata. Else returns None.
pub fn generate_custom_values(
Expand Down Expand Up @@ -60,8 +61,8 @@ fn generate_custom_value_fn(
let return_ty = type_gen
.resolve_type_path(custom_value.type_id())
.expect("type is in metadata; qed")
.to_token_stream();
let decodable = quote!(#crate_path::custom_values::Yes);
.to_token_stream(type_gen.settings());
let decodable = quote!(#crate_path::utils::Yes);
(return_ty, decodable)
} else {
// if type registry does not contain the type, we can just return the Encoded scale bytes.
Expand Down
5 changes: 4 additions & 1 deletion codegen/src/api/errors.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ use scale_typegen::TypeGenerator;
use subxt_metadata::PalletMetadata;

use super::CodegenError;
use scale_typegen::typegen::ir::ToTokensWithSettings;

/// Generate error type alias from the provided pallet metadata.
pub fn generate_error_type_alias(
Expand All @@ -18,7 +19,9 @@ pub fn generate_error_type_alias(
return Ok(quote!());
};

let error_type = type_gen.resolve_type_path(error_ty)?;
let error_type = type_gen
.resolve_type_path(error_ty)?
.to_token_stream(type_gen.settings());
let error_ty = type_gen.resolve_type(error_ty)?;
let docs = &error_ty.docs;
let docs = type_gen
Expand Down
14 changes: 9 additions & 5 deletions codegen/src/api/events.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.

use super::CodegenError;
use proc_macro2::TokenStream as TokenStream2;
use quote::quote;
use scale_typegen::typegen::ir::ToTokensWithSettings;
use scale_typegen::TypeGenerator;
use subxt_metadata::PalletMetadata;

use super::CodegenError;

/// Generate events from the provided pallet metadata.
///
/// The function creates a new module named `events` under the pallet's module.
Expand Down Expand Up @@ -37,7 +37,7 @@ use super::CodegenError;
///
/// - `type_gen` - [`scale_typegen::TypeGenerator`] that contains settings and all types from the runtime metadata.
/// - `pallet` - Pallet metadata from which the events are generated.
/// - `crate_path` - The crate path under which subxt is located, e.g. `::subxt` when using subxt as a dependency.
/// - `crate_path` - The crate path under which the `subxt-core` crate is located, e.g. `::subxt::ext::subxt_core` when using subxt as a dependency.
pub fn generate_events(
type_gen: &TypeGenerator,
pallet: &PalletMetadata,
Expand All @@ -56,7 +56,9 @@ pub fn generate_events(
let event_struct_name = &var.composite.name;
let event_name = var.variant_name;
let alias_mod = var.type_alias_mod;
let struct_def = type_gen.upcast_composite(&var.composite);
let struct_def = type_gen
.upcast_composite(&var.composite)
.to_token_stream(type_gen.settings());
quote! {
#struct_def
#alias_mod
Expand All @@ -68,7 +70,9 @@ pub fn generate_events(
}
});

let event_type = type_gen.resolve_type_path(event_ty)?;
let event_type = type_gen
.resolve_type_path(event_ty)?
.to_token_stream(type_gen.settings());
let event_ty = type_gen.resolve_type(event_ty)?;
let docs = &event_ty.docs;
let docs = type_gen
Expand Down
Loading
Loading