Skip to content

Releases: kornia/kornia

v0.9.0rc1 — compile-first, deployable augmentations

Choose a tag to compare

@edgarriba edgarriba released this 19 Jul 16:58
3695153

kornia 0.9.0rc1 — headline: compile-first, deployable augmentations.

Release candidate. Install with pip install --pre kornia (won't be picked up by a plain pip install kornia). Please try torch.export / torch.compile on your pipelines and report regressions before the final 0.9.0.

🚀 Highlights

  • torch.export support — deterministic transforms (Normalize/Denormalize/Resize/CenterCrop/PadTo) and AugmentationSequential pipelines now capture cleanly and match eager, on torch 2.9.1 and 2.10 (#3856, #3858, #3861).
  • torch.compile fullgraph sweep — ~20 ops across augmentation / enhance / color / filters / losses / geometry made graph-break-free, with a PR-time dynamo CI job on the CI torch version.
  • Performance — equalize 18×, elastic_transform2d 40×, RandomChannelShuffle 12×, posterize 16× (GPU); lazy transform-matrix (up to 2.9×); RNG-device propagation through container .to()/.cuda(); the compiled pipeline beats torchvision v2.

✨ New Features

  • Multispectral (N-channel) RandomGrayscale (#3853)
  • PatchMix augmentation (#3664)
  • Deterministic ColorJitter order, fullgraph-safe (#3799)
  • correlate2d / convolve2d / correlate3d / convolve3d aliases (#3604)
  • Optional image shape-mismatch support (#3600)

🐛 Fixes

  • cusolver-free warps + closed-form 3×3 inverse — geometric ops run on the Jetson wheel (#3834, #3840)
  • transform_points returns empty unchanged for 0-point input (#3855)
  • Container torch.export: resample-mode crop after a resize no longer uses a stale shape (#3861)
  • ONNX-exportability regressions (#3779); sharpness device/dtype (#3830); median_blur B=0; LightGlue empty keypoints (#3584); BgrToRgba (#3673)

🧹 Internal

  • _commit_state state-write chokepoint (#3857); _blend_by_prob dispatch DRY (#3844); dead-override + dead-code removal (#3843, #3862)

📖 Docs

  • Honest benchmark framing + [0,1] input footgun note (#3850); custom-augmentation guide fix (#3852); base-class hierarchy diagram (#3860); torch.export/compile/ONNX deployment section (#3859)

🤖 Generated with Claude Code

v0.8.3

Choose a tag to compare

@edgarriba edgarriba released this 19 May 20:21
d6bb4bf

Summary

Kornia v0.8.3 is a patch release rolling up ~175 commits since v0.8.2. Highlights:
new feature detectors and descriptors (XFeat, ALIKED), VisualPrompter batched
inference, half-precision (float16/bfloat16) support and test infrastructure,
performance improvements to core geometry ops (depth_to_normals ~3x, transform
points, denormalize_points_with_intrinsics ~2x), MPS device fixes, ONNX export
fixes, and broad documentation/docstring coverage. Several deprecated utilities
in kornia.utils begin emitting DeprecationWarning targeting future removal.

Features

  • add missing docstrings to color module (#3712) (9ffbca8)
  • feat: enable batched inference and add tests for VisualPrompter (#3671) (bbb581f)
  • Add XFeat, add new weights to (#3654) (67e6c12)
  • Add missing docstrings to kornia/color/sepia.py (#3628) (45c0b8e)
  • Add small sr test suite (#3557) (5af8116)
  • Add unit tests and improve RGB↔YUV conversion accuracy (#3539) (dd1568e)
  • feat: add AdaptiveQuadInterp3d and scale-space detector speedups (#3623) (398665c)
  • feat: add ALIKED local feature detector and descriptor (#3617) (42e4f78)
  • feat: add iterative_quad_interp3d with Cramer's rule (#3616) (1940cf5)
  • feat(image): add KORNIA_CHECK_IMAGE_LAYOUT for layout validation (#3549) (af4419f)
  • feat(geometry): implement closed-form solve_quartic (#3536) (4b3f216)
  • Enable grouping for GitHub Actions in Dependabot (#3506) (0a6d910)
  • Add GitHub Actions updates to Dependabot config (#3497) (4c5a843)
  • Add SmolVLM2 model scaffold (#3475) (74d3354)
  • feat: add rectangular and gaussian kernels for mutual information loss (#3478) (8856d82)
  • feat(contrib): add 3D volume support to distance_transform (#3470) (e816450)
  • feat: add initial skeleton for paligemma mode (#3477) (a38cf3d)
  • feat(sam3): Implement Phase 2 architecture (PromptEncoder + MaskDecoder) (#3433) (e912db6)
  • Add gradcheck for RandomVerticalFlip (#3476) (b0900e4)
  • feat: Add KimiVL Vision-Language Model implementation (#3448) (ee23688)
  • New features: Mutual information losses and normalized mutual information losses. (#3398) (feb79cf)
  • add more instructions and development protocol (#3438) (3982489)
  • Implement Siglip2 in kornia.models (#3412) (5c04bfc)
  • implement ai policy for contributors (#3435) (240c078)
  • implement custom errors (#3430) (e68ac75)
  • Add GitHub Copilot repository instructions (#3425) (d416945)
  • Add tests for augmentation parameter validation utilities (#3417) (8522699)
  • Add parametrized_test decorator and framework files (3a98a57)
  • feat: automate simple test cases with parametrized_tester decorator (#3367) (4002382)
  • feat: support padding_mode="fill" and fill_value in RandomAffine (#3391) (c809fbd)

Bug Fixes

  • fix(tests): verify gradients through distCoeff and K in distort_point… (#3685) (430ecc4)
  • fix(geometry): harmonise dtypes in transform_points bmm (#3707) (1a3af2a)
  • fix: propagate device and dtype in get_shear_matrix2d and get_shear_matrix3d (#3652) (631a082)
  • Fix BgrToRgba calling wrong conversion function (#3677) (d5bb50c)
  • fix xfeat dynamo (#3659) (6f03716)
  • fix(onnx): fix io_name_conversion bug and update variadic type hints (#3644) (35f7816)
  • fix: skip MPS-unsupported tests and fix MPS-specific runtime errors (#3649) (8501790)
  • fix: correct logic bugs in augmentation and crop validation (#3605) (5fa1208)
  • Fix typo in index.rst (out → our) (#3633) (f9535c9)
  • fix: resolve two CI failures in coverage workflow (#3639) (bf56558)
  • fix doctest warnings (#3632) (37ac967)
  • fix: fix doctest failures for ALIKED and fft_conv (#3630) (4c9c0b6)
  • Fix/rgba png loading (#3608) (9019330)
  • fix: use StrEnum instead of string and enum (#3573) (d66489b)
  • fix(siglip2): clamp logit_scale to avoid overflow (#3480) (f4ee06c)
  • fix: correct Thin Plate Spline warp normalization for grid_sample (#3468) (3330795)
  • Fix RandomThinPlateSpline same_on_batch behaviour and add unit tests (#3465) (f99683d)
  • fix(augmentation): resolve device mismatch in RandomGaussianIllumination (#3457) (db0447b)
  • Fix AugmentationSequential crash with mix-only augmentations (#3424) (745c6f1)
  • Fix doctest failures and build-docs task (#3456) (bc7b3e3)
  • fix: preserve align_corners from extra_args in inverse_masks (fixes #3400) (#3450) (e71e50c)
  • fix doctests in ci (#3402) (dff89bd)
  • fix(filters): validate positive sigma values in gaussian_blur2d (#3397) (d5e46fc)
  • Fix smoke_inputs in morphology tests to include both tensor and kernel arguments (0441264)
  • fix: resolve broadcasting error in affine transform for multi-channel masks (#3388) (8f4561c)
  • fix(docs): replace deprecated torch.range with torch.arange (#3381) (de52fdf)
  • fix(core): enable weights_only=False for TensorWrapper serialization (04b0723)
  • Fix Bug #2876: normalize_min_max now supports 3D tensors (C, H, W) (#3352) (eae117b)
  • Fix homography between unit square and scaled version (#3335) (8ca3e6b)

Performance

  • perf(geometry): speed up depth_to_normals (~3×) and denormalize_points_with_intrinsics (~2×) (#3656) (78496b7)
  • perf: vectorize xyz conversions (5.6x speedup) with safety checks (#3482) (f6339e0)
  • optimize(color): use addcmul for rgb_to_grayscale speedup (#3474) (bf0f8a1)
  • Perf: optimized unsharp by using lerp (#3418) (d4347d3)
  • Optimize render_gaussian2d (~18x speedup) and fix PDF variance bug (#3366) (2b6d466)

Documentation

  • Docs/contrib docstrings (#3715) (860747e)
  • Docs/core docstrings (#3716) (f0c6553)
  • docs: add missing docstrings to planar_tracker (D101, D102, D103) (#3709) (97ac706)
  • docs: add missing docstrings to augmentation module (#3666) (c6c108f)
  • docs: refine shapes and clarify logits in DiceLoss docstrings (39b4d79)
  • docs: add structural docstrings to kornia.losses.dice (9f370b0)
  • docs: fix canonical URL by setting html_baseurl to /en/latest/ (#3587) (#3589) (245f578)
  • docs: add CLAUDE.md with dev environment and architecture guide (bcd6a38)
  • docs: fix output shape comment in RawToRgb example (#3563) (e4bea59)
  • docs: add interactive image equalization demo reference (#3509) (efc676a)
  • docs: add missing docstrings for public classes (D101) (#3447) (32aec76)
  • Docs: add module docstrings(D100) for sample and draw utilities (#3390) (15c8327)
  • docs: add missing package docstrings (D104) (d76c2a5)
  • Docs: fix remaining D401 docstring violations (#3385) (c78f19c)
  • Docs: fix D401 imperative mood in adalam utils (#3384) (466040a)
  • Docs: fix docstring imperative mood (D401) (2acccd1)
  • docs: Fix RST formatting errors in augmentation.container (#3373) (e68c3f0)
  • docs: remove deprecated slack link (188ff49)
  • docs: update community invite link (78eddb1)
  • Docs: clarified difference between ImageSequential and AugmentationSequential (#3369) (bd37294)

Tests

  • tests: improve branch coverage and remove dead test stubs (#3651) (7bbae17)
  • test : add missing exception and dynamo tests for FaceDetector (#3520) (af0be4f)
  • test(image): migrate image to BaseTester (#3504) (6770ba5)
  • tests(tracking): migrate tracking to BaseTester (#3505) (56bfd3d)
  • tests(geometry): migrate geometry module to use BaseTester. (#3469) (a0d32de)
  • tests(integration): migrate integration to BaseTester (#3431) (55d9665)
  • tests(image): migrate test_image to BaseTester (#3407) (29f8261)

CI

  • ci: allow tutorials to fail to unblock merges (#3591) (16401e1)
  • ci: add path filters to skip CI on docs/non-code changes (#3445) (c90e3d7)

Refactor

  • remove too-many-positional-arguments ignore from ty config (#3543) (c161d43)
  • refactor(color): vectorize yuv with device-aware dispatch and safety … (#3514) (d27a9b7)
  • Remove deprecated reviewers from Dependabot config (#3493) (7049f29)
  • remove unused testing code (#3429) (b2705ae)
  • Refactor kornia.models across repo (#3426) (0b78c77)
  • remove old testing code (#3423) (ece6360)
  • refactor(tests): Add parametrization to smoke, shape, and gradcheck tests (#3419) (c627572)
  • refactor(tests/losses): Add @pytest.mark.grad() decorator and dtype parameter to test_gradcheck methods (#3414) (791c978)
  • remove grad_estimator, nerf and x modules (#3401) (4d6ca7b)
  • Refactor morphology tests using @parametrized_test decorator (c2cd3b4)
  • Refactor: Move NeRF helper utilities to kornia.geometry.camera.utils (#3365) (30eec36)

Dependencies

  • build(deps): bump urllib3 from 2.6.3 to 2.7.0 (#3719) (33cf66d)
  • ⬆️ bump actions/github-script in the github-actions group (#3711) (d398fcf)
  • build(deps): bump pytest from 9.0.2 to 9.0.3 (#3701) (3e49eff)
  • build(deps): bump pillow from 12.1.1 to 12.2.0 (#3700) (d726543)
  • build(deps-dev): update transformers requirement from <5.2 to <5.6 (#3695) (f3f6272)
  • build(deps): bump cryptography from 46.0.5 to 46.0.7 (#3699) (7d699f1)
  • ⬆️ bump the github-actions group with 4 updates (#3691) (7352ca3)
  • build(deps): bump transformers from 4.56.2 to 5.0.0rc3 (#3697) (9c4c4d4)
  • build(deps): bump onnx from 1.20.0 to 1.21.0 (#3692) (07d4816)
  • build(deps): bump pillow from 12.0.0 to 12.1.1 (#3569) (0f8d197)
  • build(deps): bump filelock from 3.20.1 to 3.20.3 (#3570) (1d0fd0a)
  • build(deps): bump cryptography from 46.0.3 to 46.0.5 (#3571) (b63296a)
  • build(deps-dev): update transformers requirement from <4.58 to <5.2 (#3561) (5179c95)
  • build(deps): bump protobuf from 6.33.2 to 6.33.5 (#3546) (d64263c)
  • build(deps): bump urllib3 from 2.6.2 to 2.6.3 (#3545) (6b36b77)
  • build(deps): bump virtualenv from 20.35.4 to 20.36.1 (#3544) (bb2d17e)
  • ⬆️ Bump prefix-dev/setup-pixi in the github-actions group (#3542) (fdeb503)
  • ⬆️ Bump the github-actions group with 4 updates (#3507) (8fa47b2)
  • ⬆️ bump actions/github-script from 7 to 8 (#3499) (c27ca0d)
  • ⬆️ bump actions/cache from 4 to 5 (#3500) (258638e)
  • ⬆️ bump actions/download-artifact from 4 to 7 (#3501) (d849854)
  • ⬆️ bump codecov/codecov-action from 3.1.6 to 5.5.2 (#3502) (775ca1...
Read more

v0.8.2

Choose a tag to compare

@edgarriba edgarriba released this 08 Nov 12:07
856fd1a

What's Changed

Read more

v0.8.1

Choose a tag to compare

@edgarriba edgarriba released this 08 May 10:49
5348564

What's Changed

New Contributors

Full Changelog: v0.8.0...v0.8.1

v0.8.0

Choose a tag to compare

@edgarriba edgarriba released this 11 Jan 05:12
921ba45

What's Changed

New Contributors

Full Changelog: v0.7.4...v0.8.0

v0.7.4

Choose a tag to compare

@ducha-aiki ducha-aiki released this 05 Nov 09:27
b2edb53

What's Changed

New Contributors

Full Changelog: v0.7.3...v0.7.4

v0.7.3

Choose a tag to compare

@edgarriba edgarriba released this 28 Jun 15:15

What's Changed

New Contributors

Full Changelog: v0.7.2...v0.7.3

v0.7.2

Choose a tag to compare

@edgarriba edgarriba released this 14 Mar 09:13
55f3dbb

What's Changed

New Contributors

Full Changelog: https://github.com/kornia/kornia/commits/v0.7.2

v0.7.1

Choose a tag to compare

@edgarriba edgarriba released this 27 Dec 10:47
0a7d027

What's Changed

Read more

v0.7.0 Image API, RT-DETR and Object Detection API, LightGlue Matcher, MobileSam, new Sensors API and many more

Choose a tag to compare

@edgarriba edgarriba released this 02 Aug 09:57
29e4f96

Highlights

Image API

In this release we have added a new Image API as placeholder to support a more generic multibackend api. You can export/import from files, numpy and dlapck.

>>> # from a torch.tensor
>>> data = torch.randint(0, 255, (3, 4, 5), dtype=torch.uint8)  # CxHxW
>>> pixel_format = PixelFormat(
...     color_space=ColorSpace.RGB,
...     bit_depth=8,
... )
>>> layout = ImageLayout(
...     image_size=ImageSize(4, 5),
...     channels=3,
...     channels_order=ChannelsOrder.CHANNELS_FIRST,
... )
>>> img = Image(data, pixel_format, layout)
>>> assert img.channels == 3

Object Detection API

We have added the ObjectDetector that includes by default the RT-DETR model. The detection pipeline is fully configurable by supplying a pre-processor, a model, and a post-processor. Example usage is shown below.

from io import BytesIO

import cv2
import numpy as np
import requests
import torch
from PIL import Image
import matplotlib.pyplot as plt

from kornia.contrib.models.rt_detr import RTDETR, DETRPostProcessor, RTDETRConfig
from kornia.contrib.object_detection import ObjectDetector, ResizePreProcessor

model_type = "hgnetv2_x"  # also available: resnet18d, resnet34d, resnet50d, resnet101d, hgnetv2_l
checkpoint = f"https://github.com/kornia/kornia/releases/download/v0.7.0/rtdetr_{model_type}.ckpt"
config = RTDETRConfig(model_type, 80, checkpoint=checkpoint)
model = RTDETR.from_config(config).eval()

detector = ObjectDetector(model, ResizePreProcessor(640), DETRPostProcessor(0.3))

url = "https://github.com/kornia/data/raw/main/soccer.jpg"
img = Image.open(BytesIO(requests.get(url).content))
img = np.asarray(img, dtype=np.float32) / 255
img_pt = torch.from_numpy(img).permute(2, 0, 1)
detection = detector.predict([img_pt])

for cls_score_xywh in detection[0].numpy():
    class_id = int(cls_score_xywh[0])
    score = cls_score_xywh[1]
    x, y, w, h = cls_score_xywh[2:].round().astype(int)
    cv2.rectangle(img, (x, y, w, h), (255, 0, 0), 3)

    text = f"{class_id}, {score:.2f}"
    font = cv2.FONT_HERSHEY_SIMPLEX
    (text_width, text_height), _ = cv2.getTextSize(text, font, 1, 2)
    cv2.rectangle(img, (x, y - text_height, text_width, text_height), (255, 0, 0), cv2.FILLED)
    cv2.putText(img, text, (x, y), font, 1, (255, 255, 255), 2)

plt.imshow(img)
plt.show()

img

Deep Models

As part of the kornia.contrib module, we started building a models module where Deep Learning models for Computer Vision (Semantic Segmentation, Object Detection, etc.) will exist.

From an abstract base class ModelBase, we will implement and make available these deep learning models (eg Segment anything). Similarly, we provide standard structures to be used with the results of these models such as SegmentationResults.

The idea is that we can abstract and standardize how these models will behave with our High level APIs. Like for example interacting with the Visual Prompter backend (today Segment Anything is available).

ModelBase provides methods for loading checkpoints (load_checkpoint), and compiling itself via the torch.compile API. And we plan to increase it according to the needs of the community.

Within this release, we are also making other models available to be used like RT_DETR and tiny_vit.

Example of using these abstractions to implement a model:

# Each model should be a submodule inside the `kornia.contrib.models`, and the Model class itself will be exposed under this
# `models` module.

from kornia.contrib.models.base import ModelBase
from dataclasses import dataclass
from kornia.contrib.models.structures import SegmentationResults
from enum import Enum

class MyModelType(Enum):
    """Map the model types."""
    a = 0
    ...

@dataclass
class MyModelConfig:
    model_type: str | int | SamModelType | None = None
    checkpoint: str | None = None
    ...

class MyModel(ModelBase[MyModelConfig]):
    def __init__(...) -> None:
        ...

    @staticmethod
    def from_config(config: MyModelConfig) -> MyModel:
        """Build the model based on the config"""
        ...

    def forward(...) -> SegmentationResults:
        ...

RT-DETR

In most object detection models, non-maximum suppression (NMS) is necessary to remove overlapping and similar bounding boxes. This post-processing algorithm has high latency, preventing object detectors from reaching real-time speed. DETR is a new class of detectors that eliminate NMS step by using transformer decoder to directly predict bounding boxes. RT-DETR enhances Deformable DETR to achieve real-time speed on server-class GPUs by using an efficient backbone. More details can be seen here

TinyViT

TinyViT is an efficient and high-performing transformer model for images. It achieves a top-1 accuracy of 84.8% on ImageNet-1k with only 21M parameters. See TinyViT for more information.

MobileSAM

MobileSAM replaces the heavy ViT-H backbone in the original SAM with TinyViT, which is more than 100 times smaller in terms of parameters and around 40 times faster in terms of inference speed. See MobileSAM for more details.

To use MobileSAM, simply specify "mobile_sam" in the SamConfig:

from kornia.contrib.visual_prompter import VisualPrompter
from kornia.contrib.models.sam import SamConfig

prompter = VisualPrompter(SamConfig("mobile_sam", pretrained=True))

LightGlue matcher

Added the LightGlue LightGlue-based matcher in kornia API. This is based on the original code from paper “LightGlue: Local Feature Matching at Light Speed”. See [LSP23] for more details.

The LightGlue algorithm won a money prize in the Image Matching Challenge 2023 @ CVPR23: https://www.kaggle.com/competitions/image-matching-challenge-2023/overview

See a working example integrating with COLMAP: #2469
image

New Sensors API

New kornia.sensors module to interface with sensors like Camera, IMU, GNSS etc.

We added CameraModel , PinholeModel , CameraModelBase for now.

Usage example:

Define a CameraModel

>>> # Pinhole Camera Model
>>> cam = CameraModel(ImageSize(480, 640), CameraModelType.PINHOLE, torch.Tensor([328., 328., 320., 240.]))
>>> # Brown Conrady Camera Model
>>> cam = CameraModel(ImageSize(480, 640), CameraModelType.BROWN_CONRADY, torch.Tensor([1.0, 1.0, 1.0, 1.0,
... 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]))
>>> # Kannala Brandt K3 Camera Model
>>> cam = CameraModel(ImageSize(480, 640), CameraModelType.KANNALA_BRANDT_K3, torch.Tensor([1.0, 1.0, 1.0,
... 1.0, 1.0, 1.0, 1.0, 1.0]))
>>> # Orthographic Camera Model
>>> cam = CameraModel(ImageSize(480, 640), CameraModelType.ORTHOGRAPHIC, torch.Tensor([328., 328., 320., 240.]))
>>> cam.params
tensor([328., 328., 320., 240.])

Added kornia.geometry.solvers submodule

New module for geometric vision solvers that include the following:

This is part of an upgrade of the find_fundamental to support the 7POINT algorithm.

Image terminal printing

Added kornia.utils.print_image API for printing any given image tensors or image path to terminal.

>>> kornia.utils.print_image("panda.jpg")

Screenshot 2023-07-26 at 11 39 00 PM

What's Changed

  • fix skipped tests for cuda TestColorJiggleGen by @johnnv1 in #2341
  • remove inplace operation by @Parskatt in #2346
  • Replace bandit, flake8, isort, pyupgrade, and yesqa with ruff by @cclauss in #2292
  • fix unused import on geometry.conversions by @johnnv1 in #2357
  • [pre-commit.ci] pre-commit suggestions by @pre...
Read more