Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions .github/workflows/api_ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ jobs:
needs: oas_lint_and_test
strategy:
matrix:
client: [algod, indexer]
client: [algod, indexer, kmd]
include:
- client: algod
output_dir: crates/algod_client
Expand Down Expand Up @@ -77,7 +77,7 @@ jobs:
needs: oas_lint_and_test
strategy:
matrix:
client: [algod, indexer]
client: [algod, indexer, kmd]
include:
- client: algod
package_dir: packages/typescript/algod_client
Expand All @@ -89,6 +89,11 @@ jobs:
package_subdir: indexer_client
workspace: "@algorandfoundation/indexer-client"
algokit_utils_test_filter: tests/indexer
- client: kmd
package_dir: packages/typescript/kmd_client
package_subdir: kmd_client
workspace: "@algorandfoundation/kmd-client"
algokit_utils_test_filter: tests/kmd
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup-api-tools
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/api_openapi_sync.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
spec: [algod, indexer]
spec: [algod, indexer, kmd]
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup-api-tools
Expand Down
21 changes: 21 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ members = [
"crates/ffi_macros",
"crates/algod_client",
"crates/indexer_client",
"crates/kmd_client",
"tools/build_pkgs",
"crates/uniffi-bindgen",
"docs",
Expand Down
29 changes: 23 additions & 6 deletions api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ npm install

### Convert OpenAPI 2.0 to OpenAPI 3.0

Converts both Algod and Indexer OpenAPI 2.0 specs to OpenAPI 3.0:
Converts the Algod, Indexer, and KMD OpenAPI 2.0 specs to OpenAPI 3.0:

```bash
cargo api convert-openapi
Expand All @@ -39,18 +39,22 @@ Convert individual specifications:
# Convert only algod spec
cargo api convert-algod

# Convert only indexer spec
# Convert only indexer spec
cargo api convert-indexer

# Convert only kmd spec
cargo api convert-kmd
```

The converted specs will be available at:

- `specs/algod.oas3.json`
- `specs/indexer.oas3.json`
- `specs/kmd.oas3.json`

### Generate Rust API Clients

Generate both Rust API clients using the custom Jinja2-based generator:
Generate all Rust API clients using the custom Jinja2-based generator:

```bash
cargo api generate-all
Expand All @@ -64,16 +68,20 @@ cargo api generate-algod

# Generate indexer client only
cargo api generate-indexer

# Generate kmd client only
cargo api generate-kmd
```

The generated Rust clients will be available at:

- `../crates/algod_client/`
- `../crates/indexer_client/`
- `../crates/kmd_client/`

### Generate TypeScript API Clients

Generate both TypeScript API clients using the TypeScript generator:
Generate all TypeScript API clients using the TypeScript generator:

```bash
cargo api generate-ts-all
Expand All @@ -87,12 +95,16 @@ cargo api generate-ts-algod

# Generate indexer client only
cargo api generate-ts-indexer

# Generate kmd client only
cargo api generate-ts-kmd
```

The generated TypeScript clients will be available at:

- `../packages/typescript/algod_client/`
- `../packages/typescript/indexer_client/`
- `../packages/typescript/kmd_client/`

### Development Scripts

Expand All @@ -109,6 +121,7 @@ cargo api lint-oas
# Format generated Rust code
cargo api format-algod
cargo api format-indexer
cargo api format-kmd
```

## Custom Rust OAS Generator
Expand All @@ -128,7 +141,7 @@ The project uses a custom Jinja2-based generator located in `oas_generator/` tha
The generator creates complete Rust crates with the following structure:

```
crates/{algod_client,indexer_client}/
crates/{algod_client,indexer_client,kmd_client}/
├── Cargo.toml
├── README.md
└── src/
Expand All @@ -152,13 +165,17 @@ The `algod.oas2.json` is taken directly from [go-algorand](https://github.com/al

The `indexer.oas2.json` is taken directly from [indexer](https://github.com/algorand/indexer/blob/master/api/indexer.oas2.json). To convert the spec to OpenAPI 3.0, use `cargo api convert-indexer` which runs the same TypeScript conversion script.

### KMD

The KMD Swagger 2.0 specification is sourced from [go-algorand](https://github.com/algorand/go-algorand/blob/master/daemon/kmd/api/swagger.json). Convert it to OpenAPI 3.0 with `cargo api convert-kmd` which invokes [scripts/convert-openapi.ts](scripts/convert-openapi.ts).

The current approach is to manually edit and tweak the OAS2 specs fixing known issues from the source repositories, then use the custom Rust OAS generator to generate clients from the v3 specs. OpenAPI v3 is preferred for client generation as it offers enhanced schema features, better component reusability, and improved type definitions compared to v2.

## Generator Configuration

The custom Rust generator is configured with:

- **Package names**: `algod_client`, `indexer_client`
- **Package names**: `algod_client`, `indexer_client`, `kmd_client`
- **Msgpack detection**: Automatic handling of binary-encoded fields
- **Algorand extensions**: Support for signed transaction via a vendor extension
- **Type safety**: Complete OpenAPI to Rust type mapping
Expand Down
2 changes: 2 additions & 0 deletions api/oas_generator/rust_oas_generator/generator/filters.py
Original file line number Diff line number Diff line change
Expand Up @@ -284,6 +284,8 @@ def detect_client_type(spec_title: str) -> str:
return "Algod"
if "indexer" in title_lower:
return "Indexer"
if "kmd" in title_lower:
return "Kmd"

# Fallback: extract first word and capitalize
first_word = spec_title.split()[0] if spec_title.split() else "Api"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -361,6 +361,7 @@ def _register_globals(self) -> None:
"get_request_body_name": lambda op: "request" if op.request_body else None,
"is_request_body_required": lambda op: bool(op.request_body and op.request_body.get("required", False)),
"should_import_request_body_type": type_analyzer.should_import_request_body_type,
"is_empty_request_body": lambda op: self._is_empty_request_body(op),
# Client type detection
"get_client_type": lambda spec: self._detect_client_type_from_spec(spec),
}
Expand Down Expand Up @@ -398,6 +399,34 @@ def _rust_vec(rust_type: str) -> str:
"""Wrap Rust type in Vec."""
return f"Vec<{rust_type}>"

def _is_empty_request_body(self, operation: Operation) -> bool:
"""Check if a request body has no meaningful content (empty object)."""
if not operation.request_body:
return True

# Check the content schemas
content = operation.request_body.get("content", {})
for _, media_obj in content.items():
schema = media_obj.get("schema", {})

# If there's a $ref, we need to resolve it
if "$ref" in schema:
ref_name = schema["$ref"].split("/")[-1]
# Look up the schema in components
if hasattr(self, "spec") and self.spec:
components = self.spec.get("components", {})
schemas = components.get("schemas", {})
schema = schemas.get(ref_name, {})

# Check if schema has any properties
properties = schema.get("properties", {})
required = schema.get("required", [])

if properties or required:
return False

return True

def _detect_client_type_from_spec(self, spec: ParsedSpec | dict[str, Any]) -> str:
"""Detect client type from the OpenAPI specification.

Expand Down
32 changes: 31 additions & 1 deletion api/oas_generator/rust_oas_generator/parser/oas_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@
None: "u64",
"int32": "u32",
"int64": "u64",
"uint8": "u8",
"uint16": "u16",
"uint32": "u32",
"uint64": "u64",
},
"number": {
Expand Down Expand Up @@ -887,18 +890,45 @@ def _parse_schemas(self) -> dict[str, Schema]:

def _parse_schema(self, name: str, schema_data: dict[str, Any]) -> Schema | None:
"""Parse a single schema."""
vendor_extensions = self._extract_vendor_extensions(schema_data)

# Handle pure $ref schemas as aliases to other types
if "$ref" in schema_data and not schema_data.get("properties"):
alias_rust_type = rust_type_from_openapi(schema_data, self.schemas, set())
return Schema(
name=name,
schema_type="alias",
description=schema_data.get("description"),
properties=[],
required_fields=[],
vendor_extensions=vendor_extensions,
underlying_rust_type=alias_rust_type,
enum_values=[],
)

schema_type = schema_data.get("type", "object")
properties_data = schema_data.get("properties", {})
required_fields = schema_data.get("required", [])

vendor_extensions = self._extract_vendor_extensions(schema_data)
enum_values = self._extract_enum_values(schema_type, schema_data)

underlying_rust_type = None
properties = []

if schema_type == "array":
underlying_rust_type = self._handle_array_schema(schema_data)
elif schema_type in {"string", "integer", "number", "boolean"} and not enum_values:
underlying_rust_type = rust_type_from_openapi(schema_data, self.schemas, set())
return Schema(
name=name,
schema_type="alias",
description=schema_data.get("description"),
properties=[],
required_fields=[],
vendor_extensions=vendor_extensions,
underlying_rust_type=underlying_rust_type,
enum_values=enum_values,
)
else:
properties = self._parse_properties(properties_data, required_fields)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ impl {{ client_type }}Client {
#[cfg_attr(feature = "ffi_uniffi", uniffi::constructor)]
pub fn testnet() -> Self {
let http_client = Arc::new(DefaultHttpClient::new(
{% if client_type == "Indexer" %}"https://testnet-idx.4160.nodely.dev"{% else %}"https://testnet-api.4160.nodely.dev"{% endif %}
{% if client_type == "Indexer" %}"https://testnet-idx.4160.nodely.dev"{% elif client_type == "Kmd" %}"http://localhost:7833"{% else %}"https://testnet-api.4160.nodely.dev"{% endif %}
));
Self::new(http_client)
}
Expand All @@ -87,7 +87,7 @@ impl {{ client_type }}Client {
#[cfg_attr(feature = "ffi_uniffi", uniffi::constructor)]
pub fn mainnet() -> Self {
let http_client = Arc::new(DefaultHttpClient::new(
{% if client_type == "Indexer" %}"https://mainnet-idx.4160.nodely.dev"{% else %}"https://mainnet-api.4160.nodely.dev"{% endif %}
{% if client_type == "Indexer" %}"https://mainnet-idx.4160.nodely.dev"{% elif client_type == "Kmd" %}"http://localhost:7833"{% else %}"https://mainnet-api.4160.nodely.dev"{% endif %}
));
Self::new(http_client)
}
Expand All @@ -97,8 +97,8 @@ impl {{ client_type }}Client {
#[cfg_attr(feature = "ffi_uniffi", uniffi::constructor)]
pub fn localnet() -> Self {
let http_client = Arc::new(DefaultHttpClient::with_header(
{% if client_type == "Indexer" %}"http://localhost:8980"{% else %}"http://localhost:4001"{% endif %},
{% if client_type == "Indexer" %}"X-Indexer-API-Token"{% else %}"X-Algo-API-Token"{% endif %},
{% if client_type == "Indexer" %}"http://localhost:8980"{% elif client_type == "Kmd" %}"http://localhost:4002"{% else %}"http://localhost:4001"{% endif %},
{% if client_type == "Indexer" %}"X-Indexer-API-Token"{% elif client_type == "Kmd" %}"X-KMD-API-Token"{% else %}"X-Algo-API-Token"{% endif %},
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
).expect("Failed to create HTTP client with API token header"));
Self::new(http_client)
Expand All @@ -116,7 +116,7 @@ impl {{ client_type }}Client {
{% endif %}
pub async fn {{ operation.rust_function_name }}(
&self,
{% if has_request_body(operation) %}
{% if has_request_body(operation) and operation.method.upper() not in ['GET', 'HEAD', 'DELETE'] %}
{% set request_body_name = get_request_body_name(operation) %}
{% set request_body_type = get_request_body_type(operation) %}
{% if is_request_body_required(operation) %}{{ request_body_name }}: {{ request_body_type }},
Expand All @@ -139,7 +139,7 @@ impl {{ client_type }}Client {
get_success_response_type(operation) }}{% endif %}{% else %}(){% endif %}, Error> {
let result = super::{{ operation.rust_function_name }}::{{ operation.rust_function_name }}(
self.http_client.as_ref(),
{% if has_request_body(operation) %}
{% if has_request_body(operation) and operation.method.upper() not in ['GET', 'HEAD', 'DELETE'] %}
{{ get_request_body_name(operation) }},
{% endif %}
{% for param in operation.parameters %}
Expand Down
Loading
Loading