Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -1 +1,5 @@
blank_issues_enabled: true
blank_issues_enabled: false
contact_links:
- name: Discussions
url: https://github.com/CogitatorTech/infera/discussions
about: Please ask and answer general questions here
3 changes: 3 additions & 0 deletions .github/workflows/dist_pipeline.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
name: Build Extension Binaries
on:
workflow_dispatch:
pull_request:
branches:
- main
push:
tags:
- 'v*'
Expand Down
11 changes: 8 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -61,12 +61,17 @@ rust-coverage: ## Generate code coverage report for Infera crate
.PHONY: rust-lint
rust-lint: rust-format ## Run linter checks on Rust files
@echo "Linting Rust files..."
@cargo clippy --manifest-path infera/Cargo.toml --features "tract" -- -D warnings
@cargo clippy --manifest-path infera/Cargo.toml --features "tract" -- -D warnings -D clippy::unwrap_used -D clippy::expect_used

.PHONY: rust-fix-lint
rust-fix-lint: ## Fix Rust linter warnings
@echo "Fixing linter warnings..."
@cargo clippy --fix --allow-dirty --allow-staged --manifest-path infera/Cargo.toml --features "tract" -- -D warnings
@cargo clippy --fix --allow-dirty --allow-staged --manifest-path infera/Cargo.toml --features "tract" -- -D warnings -D clippy::unwrap_used -D clippy::expect_used

.PHONY: rust-careful
careful: ## Run security checks on Rust code
@echo "Running security checks..."
@cargo careful

.PHONY: rust-clean
rust-clean: ## Clean Rust build artifacts
Expand Down Expand Up @@ -110,7 +115,7 @@ install-deps: ## Set up development environment (for Debian-based systems)
@echo "Setting up development environment..."
@sudo apt-get install -y cmake clang-format snap python3-pip
@sudo snap install rustup --classic
@cargo install cargo-tarpaulin cbindgen cargo-edit cargo-audit cargo-outdated
@cargo install cargo-tarpaulin cbindgen cargo-edit cargo-audit cargo-outdated cargo-careful
@cd infera && cargo check --features "tract"
@git submodule update --init --recursive
@pip install --user --upgrade pip uv
Expand Down
46 changes: 29 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,21 @@ See the [ROADMAP.md](ROADMAP.md) for the list of implemented and planned feature

### Quickstart

#### Install from Community Extensions Repository

You can install and load Infera from
the [DuckDB community extensions](https://duckdb.org/community_extensions/extensions/infera) repository by running the
following SQL commands in the DuckDB shell:

```sql
install infera from community;
load infera;
```

#### Build from Source

Alternatively, you can build Infera from source and use it by following these steps:

1. Clone the repository and build the Infera extension from source:

```bash
Expand All @@ -71,13 +86,21 @@ make release
./build/release/duckdb
```

3. Run the following SQL commands in the shell to try Infera out:
> [!NOTE]
> After building from source, the Infera binary will be `build/release/extension/infera/infera.duckdb_extension`.
> You can load it using the `load 'build/release/extension/infera/infera.duckdb_extension';` in the DuckDB shell.
> Note that the extension binary will only work with the DuckDB version that it was built against.
> You can download the pre-built binaries from the [releases page](https://github.com/CogitatorTech/infera/releases) for
> your platform.


#### Trying Infera

```sql
-- Normally, we need to load the extension first,
-- but the `duckdb` binary that we built in the previous step
-- already has Infera statically linked to it.
-- So, we don't need to load the extension explicitly.
-- 0. Install and load Infera
-- Skip this step if you built from source and ran `./build/release/duckdb`
install infera from community;
load infera;

-- 1. Load a simple linear model from a remote URL
select infera_load_model('linear_model',
Expand All @@ -93,21 +116,10 @@ select infera_unload_model('linear_model');

-- 4. Check the Infera version
select infera_get_version();
````
```

[![Simple Demo 1](https://asciinema.org/a/745806.svg)](https://asciinema.org/a/745806)

> [!NOTE]
> After building from source, the Infera binary will be `build/release/extension/infera/infera.duckdb_extension`.
> You can load it using the `load 'build/release/extension/infera/infera.duckdb_extension';` in the DuckDB shell.
> Note that the extension binary will only work with the DuckDB version that it was built against.
> At the moment, Infera is not available as
> a [DuckDB community extension](https://duckdb.org/community_extensions/list_of_extensions).
> Nevertheless, you can still use Infera by building it from source yourself, or downloading pre-built binaries from
> the [releases page](https://github.com/CogitatorTech/infera/releases) for your platform.
> Please check the [this page](https://duckdb.org/docs/stable/extensions/installing_extensions.html) for more details on
> how to use extensions in DuckDB.

---

### Documentation
Expand Down
28 changes: 14 additions & 14 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,21 @@

The table below includes the information about all SQL functions exposed by Infera.

| # | Function | Return Type | Description |
|----|:-------------------------------------------------------------|:-----------------|:--------------------------------------------------------------------------------------------------------------------------------------------|
| 1 | `infera_load_model(name VARCHAR, path_or_url VARCHAR)` | `BOOLEAN` | Loads an ONNX model from a local file path or a remote URL and assigns it a unique name. Returns `true` on success. |
| 2 | `infera_unload_model(name VARCHAR)` | `BOOLEAN` | Unloads a model, freeing its associated resources. Returns `true` on success. |
| 3 | `infera_set_autoload_dir(path VARCHAR)` | `VARCHAR (JSON)` | Scans a directory for `.onnx` files, loads them automatically, and returns a JSON report of loaded models and any errors. |
| 4 | `infera_get_loaded_models()` | `VARCHAR (JSON)` | Returns a JSON array containing the names of all currently loaded models. |
| # | Function | Return Type | Description |
|----|:-------------------------------------------------------------|:-----------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1 | `infera_load_model(name VARCHAR, path_or_url VARCHAR)` | `BOOLEAN` | Loads an ONNX model from a local file path or a remote URL and assigns it a unique name. Returns `true` on success. |
| 2 | `infera_unload_model(name VARCHAR)` | `BOOLEAN` | Unloads a model, freeing its associated resources. Returns `true` on success. |
| 3 | `infera_set_autoload_dir(path VARCHAR)` | `VARCHAR (JSON)` | Scans a directory for `.onnx` files, loads them automatically, and returns a JSON report of loaded models and any errors. |
| 4 | `infera_get_loaded_models()` | `VARCHAR (JSON)` | Returns a JSON array containing the names of all currently loaded models. |
| 5 | `infera_get_model_info(name VARCHAR)` | `VARCHAR (JSON)` | Returns a JSON object with metadata about a specific loaded model (name, input/output shapes). If the model is not loaded, this function raises an error. |
| 6 | `infera_predict(name VARCHAR, features... FLOAT)` | `FLOAT` | Performs inference on a batch of data, returning a single float value for each input row. |
| 7 | `infera_predict_multi(name VARCHAR, features... FLOAT)` | `VARCHAR (JSON)` | Performs inference and returns all outputs as a JSON-encoded array. This is useful for models that produce multiple predictions per sample. |
| 8 | `infera_predict_multi_list(name VARCHAR, features... FLOAT)` | `LIST[FLOAT]` | Performs inference and returns all outputs as a typed list of floats. Useful for multi-output models without JSON parsing. |
| 9 | `infera_predict_from_blob(name VARCHAR, data BLOB)` | `LIST[FLOAT]` | Performs inference on raw `BLOB` data (for example, used for an image tensor), returning the result as a list of floats. |
| 10 | `infera_is_model_loaded(name VARCHAR)` | `BOOLEAN` | Returns `true` if the given model is currently loaded, otherwise `false`. |
| 11 | `infera_get_version()` | `VARCHAR (JSON)` | Returns a JSON object with version and build information for the Infera extension. |
| 12 | `infera_clear_cache()` | `BOOLEAN` | Clears the entire model cache directory, freeing up disk space. Returns `true` on success. |
| 13 | `infera_get_cache_info()` | `VARCHAR (JSON)` | Returns cache statistics including directory path, total size in bytes, file count, and configured size limit. |
| 6 | `infera_predict(name VARCHAR, features... FLOAT)` | `FLOAT` | Performs inference on a batch of data, returning a single float value for each input row. |
| 7 | `infera_predict_multi(name VARCHAR, features... FLOAT)` | `VARCHAR (JSON)` | Performs inference and returns all outputs as a JSON-encoded array. This is useful for models that produce multiple predictions per sample. |
| 8 | `infera_predict_multi_list(name VARCHAR, features... FLOAT)` | `LIST[FLOAT]` | Performs inference and returns all outputs as a typed list of floats. Useful for multi-output models without JSON parsing. |
| 9 | `infera_predict_from_blob(name VARCHAR, data BLOB)` | `LIST[FLOAT]` | Performs inference on raw `BLOB` data (for example, used for an image tensor), returning the result as a list of floats. |
| 10 | `infera_is_model_loaded(name VARCHAR)` | `BOOLEAN` | Returns `true` if the given model is currently loaded, otherwise `false`. |
| 11 | `infera_get_version()` | `VARCHAR (JSON)` | Returns a JSON object with version and build information for the Infera extension. |
| 12 | `infera_clear_cache()` | `BOOLEAN` | Clears the entire model cache directory, freeing up disk space. Returns `true` on success. |
| 13 | `infera_get_cache_info()` | `VARCHAR (JSON)` | Returns cache statistics including directory path, total size in bytes, file count, and configured size limit. |

> [!NOTE]
> The `features...` arguments accept `FLOAT` as well as values from `DOUBLE`, `INTEGER`, `BIGINT`, and `DECIMAL`
Expand Down
2 changes: 1 addition & 1 deletion infera/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "infera"
version = "0.2.0"
version = "0.3.0"
edition = "2021"
publish = false

Expand Down
2 changes: 1 addition & 1 deletion infera/bindings/infera_extension.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -527,7 +527,7 @@ static void LoadInternal(ExtensionLoader &loader) {

void InferaExtension::Load(ExtensionLoader &loader) { LoadInternal(loader); }
std::string InferaExtension::Name() { return "infera"; }
std::string InferaExtension::Version() const { return "v0.2.0"; }
std::string InferaExtension::Version() const { return "v0.3.0"; }

} // namespace duckdb

Expand Down
1 change: 0 additions & 1 deletion infera/src/config.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
// src/config.rs
// Centralized configuration management for Infera

use once_cell::sync::Lazy;
Expand Down
7 changes: 5 additions & 2 deletions infera/src/engine.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
// src/engine.rs
// Contains the core ONNX inference logic using the Tract library.

use crate::error::InferaError;
Expand Down Expand Up @@ -213,7 +212,11 @@ pub(crate) fn run_inference_blob_impl(
let blob_bytes = unsafe { std::slice::from_raw_parts(blob_data, blob_len) };
let float_vec: Vec<f32> = blob_bytes
.chunks_exact(4)
.map(|chunk| f32::from_ne_bytes(chunk.try_into().unwrap()))
.map(|chunk| {
// SAFETY: chunks_exact(4) guarantees exactly 4 bytes, so this conversion cannot fail
let array: [u8; 4] = chunk.try_into().unwrap_or_default();
f32::from_ne_bytes(array)
})
.collect();
let expected_elements: usize = model
.input_shape
Expand Down
1 change: 0 additions & 1 deletion infera/src/error.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
// src/error.rs
// Contains the InferaError enum and thread-local error handling logic.

use std::cell::RefCell;
Expand Down
1 change: 0 additions & 1 deletion infera/src/ffi_utils.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
// src/ffi_utils.rs
// Contains C-compatible structs and memory management functions for the FFI boundary.

use std::ffi::{c_char, CString};
Expand Down
1 change: 0 additions & 1 deletion infera/src/http.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
// src/http.rs
// Handles downloading and caching of remote models.

use crate::config::{LogLevel, CONFIG};
Expand Down
12 changes: 10 additions & 2 deletions infera/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
// src/lib.rs
// The public C API layer and module declarations.

use serde_json::json;
Expand Down Expand Up @@ -248,7 +247,16 @@ pub extern "C" fn infera_get_loaded_models() -> *mut c_char {
let models = model::MODELS.read();
let list: Vec<String> = models.keys().cloned().collect();
let joined = serde_json::to_string(&list).unwrap_or_else(|_| "[]".to_string());
CString::new(joined).unwrap().into_raw()
match CString::new(joined) {
Ok(cstr) => cstr.into_raw(),
Err(_) => {
// Fallback to empty JSON array if string contains null bytes
match CString::new("[]") {
Ok(cstr) => cstr.into_raw(),
Err(_) => std::ptr::null_mut(), // This should never happen with "[]"
}
}
}
}

/// Returns a JSON string with version and build information about the Infera library.
Expand Down
1 change: 0 additions & 1 deletion infera/src/model.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
// src/model.rs
// Defines the internal representation of a model and the global model store.

use once_cell::sync::Lazy;
Expand Down
2 changes: 1 addition & 1 deletion vcpkg.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "infera",
"version": "0.2.0",
"version": "0.3.0",
"description": "Infera DuckDB Extension",
"dependencies": [
{
Expand Down
Loading