Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@ See [dockerfiles/README.md](./dockerfiles/README.md) for more options, including
# 📚 Documentation

- [Writing Hub kernels](./docs/writing-kernels.md)
- [Building kernels with Docker](./docs/docker.md)
- [Building kernels with Nix](./docs/nix.md)
- [Building kernels with Docker](./docs/docker.md) (for systems without Nix)
- [Local kernel development](docs/local-dev.md) (IDE integration)
- [Why Nix?](./docs/why-nix.md)

Expand Down
17 changes: 12 additions & 5 deletions docs/docker.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Using the kernel builder with Docker

<!-- toc -->

- [Using the kernel builder with Docker](#using-the-kernel-builder-with-docker)
- [Quick Start](#quick-start)
- [CLI Interface](#cli-interface)
Expand All @@ -16,7 +17,12 @@
- [Building from URL](#building-from-url)
- [Available Docker Images](#available-docker-images)
- [Development](#development)
<!-- tocstop -->
<!-- tocstop -->

**Warning**: we strongly recommend [building kernels with Nix](nix.md).
Using Nix directly makes it easier to cache all dependencies and is more
robust. We provide a Docker image for systems where Nix cannot be
installed.

## Quick Start

Expand Down Expand Up @@ -204,12 +210,13 @@ docker run --rm \

The kernel-builder is available in different variants with specific tags:

| Tag | Description |
| --- | ----------- |
| `[SHA]` | Specific commit hash version (example: `ghcr.io/huggingface/kernel-builder:abc123`) |
| `user-[SHA]` | Non root user variant (use when specific permissions are needed) |
| Tag | Description |
| ------------ | ----------------------------------------------------------------------------------- |
| `[SHA]` | Specific commit hash version (example: `ghcr.io/huggingface/kernel-builder:abc123`) |
| `user-[SHA]` | Non root user variant (use when specific permissions are needed) |

All images are available from the GitHub Container Registry:

```
ghcr.io/huggingface/kernel-builder
```
Expand Down
44 changes: 38 additions & 6 deletions docs/nix.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
# Using the kernel builder with Nix

The kernel builder uses Nix for building kernels. You can build or
run the kernels directly if you have [Nix installed](https://nixos.org/download/)
on your system. On systems without Nix you can use the [Docker](./docker.md)
image, which is a wrapper around Nix.
run the kernels directly if you have Nix installed on your system.
We recommend installing Nix in the following way:

- Linux: use the [official Nix installer](https://nixos.org/download/).
- macOS: use the [Determinate Nix installer](https://docs.determinate.systems/determinate-nix/).

## Getting started

Expand Down Expand Up @@ -35,9 +37,6 @@ cd examples/activation
nix build . -L
```

You can put this `flake.nix` in your own kernel's root directory to
get add Nix support to your kernel.

## Shell for local development

`kernel-builder` provides shells for developing kernels. In such a shell,
Expand Down Expand Up @@ -82,6 +81,39 @@ nix develop -L .#test
python -m pytest tests
```

## Adding test dependencies to development shells

You can add test dependencies to a development or testing shell. Adapt
the kernel's `flake.nix` to use the `pythonCheckInputs` option:

```nix
{
description = "Flake for my kernel";

inputs = {
kernel-builder.url = "github:huggingface/kernel-builder";
};

outputs =
{
self,
kernel-builder,
}:
kernel-builder.lib.genFlakeOutputs {
path = ./.;
rev = self.shortRev or self.dirtyShortRev or self.lastModifiedDate;

# The einops and numpy test dependencies are added here:
pythonCheckInputs = pkgs: with pkgs; [ einops numpy ];
};
}
```

The available packages can be found on [search.nixos.org](https://search.nixos.org/packages?channel=25.05&query=python312Packages).

Keep in mind that these additional dependencies will only be available to
the Nix shells, not the final kernel uploaded to the Hub.

## Building a kernel without `flake.nix`

If a kernels source directory does not have a `flake.nix` file, you can build the
Expand Down