Releases: kornia/kornia
v0.7.2
What's Changed
- rename missing main by @edgarriba in #2723
- Bump pytest from 7.4.3 to 7.4.4 by @dependabot in #2712
- Replace docs source link from
viewcode
to github by @johnnv1 in #2727 - Bump readthedocs python version by @johnnv1 in #2729
- Bump accelerate from 0.25.0 to 0.26.1 by @dependabot in #2730
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2734
- Fixes
extract_tensor_patches
to work with partial patches cases by @johnnv1 in #2735 - [SAM] document the image (tensor) should be scaled between [0,1] by @scott-vsi in #2738
- add ProjectionZ1, Orthographic, Affine, KannalaBrandt by @edgarriba in #2728
- bump torch
2.1.2
by @johnnv1 in #2742 - remove retrigger CI on PR's when labeled by @johnnv1 in #2744
- rename directory:
test
->tests
by @johnnv1 in #2743 - [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2747
- Depreciates
kornia.testing
by @johnnv1 in #2745 - Add salt and pepper noise with docs and tests by @vgilabert94 in #2746
- alphabetical order augmentations docs and add salt pepper by @edgarriba in #2757
- Added DogHardNet LightGlue by @ducha-aiki in #2758
- fix F841 by @johnnv1 in #2759
- feat: differentiable jpeg by @johnnv1 in #2760
- [CI] split coverage into multiple jobs by @johnnv1 in #2756
- Ci: fix old torch install by @johnnv1 in #2763
- RandomSaltAndPepperNoise: Update algorithm to use indexing by @vgilabert94 in #2762
- add dedode descriptor B weights for lightglue by @ducha-aiki in #2769
- skip unsupported tests cases for torch==1.9.1 by @johnnv1 in #2770
- remove typing report by @johnnv1 in #2765
- Bump pytest from 7.4.4 to 8.0.0 by @dependabot in #2766
- Ensure support to
torch==2.2.0
by @johnnv1 in #2772 - Resize compile by @edgarriba in #2774
- [FIX] ColorJiggle works on non-3-channel images by @shijianjian in #2767
- Introduces
benchmarks/
by @johnnv1 in #2777 - Add [*, 3, H, W] support to Diff. JPEG by @ChristophReich1996 in #2776
- [test suite] add slow marker vit by @johnnv1 in #2779
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2783
- Fix
ruff 0.2.1
config by @johnnv1 in #2784 - force solve with torch.float64 by @edgarriba in #2785
- Fix robots for web crawlers by @edgarriba in #2790
- Remove device from get_planckian_coeffs and register self.pl as a buffer by @Modexus in #2792
- ViT: Load Jax augreg weights by @gau-nernst in #2786
- docs: add not-found option by @johnnv1 in #2796
- Feat: Support list of masks in
AugmentationSequential
by @johnnv1 in #2740 - test suite: skip fp64 canny dynamo test by @johnnv1 in #2797
- Test suite: separate losses tests into individual files by @fleventy-5 in #2801
- Test suite: separate contrib tests into individual files by @fleventy-5 in #2802
- Test suite: separate metrics tests into individual files by @fleventy-5 in #2805
- Feat: add Gradient Illumination augmentations (gaussian) by @vgilabert94 in #2780
- add missing docs for RandomGaussianIllumination by @edgarriba in #2806
- feat: Add RandomJPEG Augmentation by @ChristophReich1996 in #2803
- Update augmentation.module.rst by @edgarriba in #2808
- Add: differentiable clipping, floor and rounding functions by @jeffin07 in #2795
- [feat] KMeans implementation by @avinashselvam in #2804
- Bump pytest from 8.0.0 to 8.0.1 by @dependabot in #2810
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2811
- Aug: Add simple support to dict for AugSequential by @johnnv1 in #2799
- Bump pytest from 8.0.1 to 8.0.2 by @dependabot in #2815
- Fix: Typo auto rand augment by @harsh1504660 in #2817
- Fix Typo: Corrected SUBPLOLICY_CONFIG to SUBPOLICY_CONFIG by @harsh1504660 in #2819
- remove image prompter in favor of visual prompter by @johnnv1 in #2814
- Upgrade model checkpointing by @machineko in #2820
- Update torch.inverse to torch.linalg.inv by @wangshuai09 in #2824
- Refactor and add more palettes of colormaps by @vgilabert94 in #2794
- feat: RandomLinearIllumination and RandomLinearCornerIllumination by @vgilabert94 in #2823
- fix (docs): add
RandomLinearIllumination
and `RandomLinearCornerIll… by @johnnv1 in #2827 - ColorMap tests coverage by @johnnv1 in #2825
- Bump pytest from 8.0.2 to 8.1.1 by @dependabot in #2829
- feat: augmentation sequential supports mask (B, H, W) with images (B, C, H, W) by @johnnv1 in #2800
- fix(docs): remove Jina AI QAbot from Kornia documentation by @nan-wang in #2831
- update kornia_rs and make sure contiguous data by @edgarriba in #2828
- CI: ensure pytorch 2.2.1 support by @johnnv1 in #2833
- fix visual prompter test in dynamo by @edgarriba in #2834
- Fix of possible CPU and GPU device error by @Fleyderer in #2838
- Add DeDoDe (clean version) by @ducha-aiki in #2835
- bump version 0.7.2 by @edgarriba in #2832
New Contributors
- @scott-vsi made their first contribution in #2738
- @vgilabert94 made their first contribution in #2746
- @Modexus made their first contribution in #2792
- @fleventy-5 made their first contribution in #2801
- @avinashselvam made their first contribution in #2804
- @harsh1504660 made their first contribution in #2817
- @machineko made their first contribution in #2820
- @wangshuai09 made their first contribution in #2824
- @nan-wang made their first contribution in #2831
- @Fleyderer made their first contribution in #2838
Full Changelog: https://github.com/kornia/kornia/commits/v0.7.2
v0.7.1
What's Changed
- Lie groups docs update by @cjpurackal in #2495
- Fixed RandomJigsaw #2494 by @shijianjian in #2504
- Fixed video batching bug #2410 #2497 by @shijianjian in #2502
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2508
- LG-test by @ducha-aiki in #2510
- Fix docs build -
sphinx==7.0.1
by @johnnv1 in #2518 - [feat] add right and left jacobian for So3 by @cjpurackal in #2509
- Fix: update random_rain.py by @f-amerehi in #2514
- Fix docs: Unpin sphinx by @johnnv1 in #2527
- Fix tutorials links to use the github page by @johnnv1 in #2529
- Bump accelerate from 0.21.0 to 0.22.0 by @dependabot in #2530
- Update precommit hooks by @johnnv1 in #2521
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2528
- Added kornia resize in object detection by @jeffin07 in #2526
- [feat] Add transplantation augmentation by @JanSellner in #2523
- Use
grid_sample
from F without importing it. by @antoinebrl in #2532 - Clarify documentation to get shear matrices by @priba in #2533
- Bump pytest from 7.4.0 to 7.4.1 by @dependabot in #2537
- Add support for class data keys in AugmentationSequential by @miquelmarti in #2536
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2538
- fix readme badges by @johnnv1 in #2539
- Bump pytest from 7.4.1 to 7.4.2 by @dependabot in #2542
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2543
- Bump accelerate from 0.22.0 to 0.23.0 by @dependabot in #2547
- [feature] rt-detr onnx by @edgarriba in #2548
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2550
- Make transforms Spawn Multiprocessing Context Friendly by @NielsPichon in #2499
- Fix TestEqualization by @johnnv1 in #2553
- add coverage CI by @johnnv1 in #2551
- fixes zero division in depth_from_disparity by @edgarriba in #2556
- Fix improper
importlib.util
import by @Avasam in #2558 - Depth to 3d improvements by @edgarriba in #2557
- fix object detection onnx by @edgarriba in #2555
- reduce repository size (history rewriting) by @johnnv1 in #2561
- fix typing by @edgarriba in #2563
- Add missing tests (losses, io, connected components) by @johnnv1 in #2565
- allow warp_affine fill num_channels other than three by @edgarriba in #2568
- Fix depth to 3d docs by @edgarriba in #2569
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2570
- Update augmentation.rst by @Otteri in #2584
- fix bucket off-by-one-error by @Seanny123 in #2582
- update version to dev by @johnnv1 in #2586
- drop
torchvision
from docs deps by @johnnv1 in #2591 - Adds 7pt solver to RANSAC by @ducha-aiki in #2595
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2596
- [feat] Add Average Endpoint Error to Metrics by @ChristophReich1996 in #2500
- Add
torch.jit.script
support forwarp_affine
by @balbok0 in #2588 - Update readthedocs.yml by @edgarriba in #2598
- [speed up CI tests suite] Skip slow tests as default by @johnnv1 in #2587
- Fix of multi head attention implementation by @yarkoslav in #2589
- fix doctest by @edgarriba in #2604
- drop
cv2
as dev dependency by @johnnv1 in #2593 - Optimize the connected components labeling algorithm by @SonyPony in #2609
- xpass rasanc randomness tests by @johnnv1 in #2613
- #2348 Fix bug in ycbcr_to_rgb function by @juliaaz in #2610
- bump pytorch version to 2.1.0 by @johnnv1 in #2605
- add deps info to tests header by @johnnv1 in #2585
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2616
- xfail TestRandAugment::test_transform_mat based in torch version by @johnnv1 in #2617
- Ensure python 3.11 support by @johnnv1 in #2606
- Fix incorrect computation of affine shear matrix around center point by @Vihtoriaaa in #2608
- Add alias 'average_endpoint_error' for aepe method by @be-unkind in #2615
- skip dynamo test for py311 torch 2.0.x by @johnnv1 in #2621
- expose
average_endpoint_error
metric by @johnnv1 in #2620 - replace deprecated prompter test and fix skiping dynamo test by @johnnv1 in #2623
- Update docs CI by @johnnv1 in #2624
- Add differentiable 5PC (five point algorithm) for essential matrix estimation by @weitong8591 in #2580
- add RandomClahe by @michael-2956 in #2614
- [feat] Add NamedPose by @cjpurackal in #2482
- Fix visual prompter bug (#2627) by @kaftanski in #2630
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2632
- [feat] Raw to rgb conversion with downscaling by @bogdanmagometa in #2629
- feat: LightGlue-ONNX by @fabio-sim in #2631
- Drop
scipy
from dev dependencies by @johnnv1 in #2634 - fix: docs: OnnxLightGlue by @fabio-sim in #2635
- Bump pytest from 7.4.2 to 7.4.3 by @dependabot in #2641
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2644
- Bump accelerate from 0.23.0 to 0.24.1 by @dependabot in #2645
- Make use of composition for augmentation multiprocessing friendliness by @johnnv1 in #2637
- Rename variables shadowing build-in input function by @alorthius in #2628
- addopt ruff formatter by @johnnv1 in #2638
- add deprecated decorator
depth_to_3d
by @johnnv1 in #2636 - Add EfficientViT model by @edgarriba in #2607
- feat: CPU support for OnnxLightGlue by @fabio-sim in #2643
- Typo fix in
focal.py
by @omerferhatt in #2654 - [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2653
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2660
- Doc Fix: lafs2 shape in matchers by @maxc303 in #2663
- skip TestEfficientViT::test_smoke_large for torch lower than 2.0.0 by @edgarriba in #2664
- fix unused type ignore by @johnnv1 in #2666
- Bump pytorch to 2.1.1 by @johnnv1 in #2667
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2669
- Allow CombineTensorPatches to work with overlapping patches by @ashnair1 in #2650
kornia.nerf
improvements by @edgarriba in #2661- improve nerf docs by @edgarriba in #2673
- refactor kornia geometry to use kornia core rather than torch. Part 1… by @Tchaikovic in #2676
- move onnxruntime to
x
requirements by @johnnv1 in #2670 - Multiplication with
torch.ones
not needed for kernel_values by @kunaltyagi in #2678 - Bump accelerate from 0.24.1 to 0.25.0 by @dependabot in #2677
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2679
- fi...
v0.7.0 Image API, RT-DETR and Object Detection API, LightGlue Matcher, MobileSam, new Sensors API and many more
Highlights
Image API
In this release we have added a new Image
API as placeholder to support a more generic multibackend api. You can export/import from files, numpy and dlapck.
>>> # from a torch.tensor
>>> data = torch.randint(0, 255, (3, 4, 5), dtype=torch.uint8) # CxHxW
>>> pixel_format = PixelFormat(
... color_space=ColorSpace.RGB,
... bit_depth=8,
... )
>>> layout = ImageLayout(
... image_size=ImageSize(4, 5),
... channels=3,
... channels_order=ChannelsOrder.CHANNELS_FIRST,
... )
>>> img = Image(data, pixel_format, layout)
>>> assert img.channels == 3
Object Detection API
We have added the ObjectDetector
that includes by default the RT-DETR model. The detection pipeline is fully configurable by supplying a pre-processor, a model, and a post-processor. Example usage is shown below.
from io import BytesIO
import cv2
import numpy as np
import requests
import torch
from PIL import Image
import matplotlib.pyplot as plt
from kornia.contrib.models.rt_detr import RTDETR, DETRPostProcessor, RTDETRConfig
from kornia.contrib.object_detection import ObjectDetector, ResizePreProcessor
model_type = "hgnetv2_x" # also available: resnet18d, resnet34d, resnet50d, resnet101d, hgnetv2_l
checkpoint = f"https://github.com/kornia/kornia/releases/download/v0.7.0/rtdetr_{model_type}.ckpt"
config = RTDETRConfig(model_type, 80, checkpoint=checkpoint)
model = RTDETR.from_config(config).eval()
detector = ObjectDetector(model, ResizePreProcessor(640), DETRPostProcessor(0.3))
url = "https://github.com/kornia/data/raw/main/soccer.jpg"
img = Image.open(BytesIO(requests.get(url).content))
img = np.asarray(img, dtype=np.float32) / 255
img_pt = torch.from_numpy(img).permute(2, 0, 1)
detection = detector.predict([img_pt])
for cls_score_xywh in detection[0].numpy():
class_id = int(cls_score_xywh[0])
score = cls_score_xywh[1]
x, y, w, h = cls_score_xywh[2:].round().astype(int)
cv2.rectangle(img, (x, y, w, h), (255, 0, 0), 3)
text = f"{class_id}, {score:.2f}"
font = cv2.FONT_HERSHEY_SIMPLEX
(text_width, text_height), _ = cv2.getTextSize(text, font, 1, 2)
cv2.rectangle(img, (x, y - text_height, text_width, text_height), (255, 0, 0), cv2.FILLED)
cv2.putText(img, text, (x, y), font, 1, (255, 255, 255), 2)
plt.imshow(img)
plt.show()
Deep Models
As part of the kornia.contrib
module, we started building a models
module where Deep Learning models for Computer Vision (Semantic Segmentation, Object Detection, etc.) will exist.
From an abstract base class ModelBase
, we will implement and make available these deep learning models (eg Segment anything). Similarly, we provide standard structures to be used with the results of these models such as SegmentationResults
.
The idea is that we can abstract and standardize how these models will behave with our High level APIs. Like for example interacting with the Visual Prompter
backend (today Segment Anything
is available).
ModelBase
provides methods for loading checkpoints (load_checkpoint
), and compiling itself via the torch.compile
API. And we plan to increase it according to the needs of the community.
Within this release, we are also making other models available to be used like RT_DETR
and tiny_vit
.
Example of using these abstractions to implement a model:
# Each model should be a submodule inside the `kornia.contrib.models`, and the Model class itself will be exposed under this
# `models` module.
from kornia.contrib.models.base import ModelBase
from dataclasses import dataclass
from kornia.contrib.models.structures import SegmentationResults
from enum import Enum
class MyModelType(Enum):
"""Map the model types."""
a = 0
...
@dataclass
class MyModelConfig:
model_type: str | int | SamModelType | None = None
checkpoint: str | None = None
...
class MyModel(ModelBase[MyModelConfig]):
def __init__(...) -> None:
...
@staticmethod
def from_config(config: MyModelConfig) -> MyModel:
"""Build the model based on the config"""
...
def forward(...) -> SegmentationResults:
...
RT-DETR
In most object detection models, non-maximum suppression (NMS) is necessary to remove overlapping and similar bounding boxes. This post-processing algorithm has high latency, preventing object detectors from reaching real-time speed. DETR is a new class of detectors that eliminate NMS step by using transformer decoder to directly predict bounding boxes. RT-DETR enhances Deformable DETR to achieve real-time speed on server-class GPUs by using an efficient backbone. More details can be seen here
TinyViT
TinyViT is an efficient and high-performing transformer model for images. It achieves a top-1 accuracy of 84.8% on ImageNet-1k with only 21M parameters. See TinyViT for more information.
MobileSAM
MobileSAM replaces the heavy ViT-H backbone in the original SAM with TinyViT, which is more than 100 times smaller in terms of parameters and around 40 times faster in terms of inference speed. See MobileSAM for more details.
To use MobileSAM, simply specify "mobile_sam"
in the SamConfig
:
from kornia.contrib.visual_prompter import VisualPrompter
from kornia.contrib.models.sam import SamConfig
prompter = VisualPrompter(SamConfig("mobile_sam", pretrained=True))
LightGlue matcher
Added the LightGlue
LightGlue-based matcher in kornia API. This is based on the original code from paper “LightGlue: Local Feature Matching at Light Speed”. See [LSP23] for more details.
The LightGlue algorithm won a money prize in the Image Matching Challenge 2023 @ CVPR23: https://www.kaggle.com/competitions/image-matching-challenge-2023/overview
See a working example integrating with COLMAP: #2469
New Sensors API
New kornia.sensors
module to interface with sensors like Camera, IMU, GNSS etc.
We added CameraModel
, PinholeModel
, CameraModelBase
for now.
Usage example:
Define a CameraModel
>>> # Pinhole Camera Model
>>> cam = CameraModel(ImageSize(480, 640), CameraModelType.PINHOLE, torch.Tensor([328., 328., 320., 240.]))
>>> # Brown Conrady Camera Model
>>> cam = CameraModel(ImageSize(480, 640), CameraModelType.BROWN_CONRADY, torch.Tensor([1.0, 1.0, 1.0, 1.0,
... 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]))
>>> # Kannala Brandt K3 Camera Model
>>> cam = CameraModel(ImageSize(480, 640), CameraModelType.KANNALA_BRANDT_K3, torch.Tensor([1.0, 1.0, 1.0,
... 1.0, 1.0, 1.0, 1.0, 1.0]))
>>> # Orthographic Camera Model
>>> cam = CameraModel(ImageSize(480, 640), CameraModelType.ORTHOGRAPHIC, torch.Tensor([328., 328., 320., 240.]))
>>> cam.params
tensor([328., 328., 320., 240.])
Added kornia.geometry.solvers
submodule
New module for geometric vision solvers that include the following:
This is part of an upgrade of the find_fundamental
to support the 7POINT
algorithm.
Image terminal printing
Added kornia.utils.print_image
API for printing any given image tensors or image path to terminal.
>>> kornia.utils.print_image("panda.jpg")
What's Changed
- fix skipped tests for cuda
TestColorJiggleGen
by @johnnv1 in #2341 - remove inplace operation by @Parskatt in #2346
- Replace bandit, flake8, isort, pyupgrade, and yesqa with ruff by @cclauss in #2292
- fix unused import on
geometry.conversions
by @johnnv1 in #2357 - [pre-commit.ci] pre-commit suggestions by @pre...
v0.6.12 ImagePrompter API (SAM), Guided Filter and bugfixes
Highlights
ImagePrompter API
In this release we have added a new ImagePrompter
API that settles the basis as a foundational api for the task to query geometric information to images inspired by LLM. We leverage the ImagePrompter API via the Segment Anything (SAM) making the model more accessible, packaged and well maintained for industry standards.
Check the full tutorial: https://github.com/kornia/tutorials/blob/master/nbs/image_prompter.ipynb
import kornia as K
from kornia.contrib.image_prompter import ImagePrompter
from kornia.geometry.keypoints import Keypoints
from kornia.geometry.boxes import Boxes
image: Tensor = K.io.load_image("soccer.jpg", ImageLoadType.RGB32, "cuda")
# Load the prompter
prompter = ImagePrompter(config, device="cuda")
# set the image: This will preprocess the image and already generate the embeddings of it
prompter.set_image(image)
# Generate the prompts
keypoints = Keypoints(torch.tensor([[[500, 375]]], device="cuda")) # BxNx2
# For the keypoints label: 1 indicates a foreground point; 0 indicates a background point
keypoints_labels = torch.tensor([[1]], device="cuda") # BxN
boxes = Boxes(
torch.tensor([[[[425, 600], [425, 875], [700, 600], [700, 875]]]], device="cuda"), mode='xyxy'
)
# Runs the prediction with all prompts
prediction = prompter.predict(
keypoints=keypoints,
keypoints_labels=keypoints_labels,
boxes=boxes,
multimask_output=True,
)
Guided Blurring
Blur images by preserving edges via Bilateral and Guided Blurring
-> https://kornia.readthedocs.io/en/latest/filters.html#kornia.filters.guided_blur
What's Changed
- Fixed typo in face_detection.rst by @Aneesh02 in #2308
- Fix elastic transformation if only partially applied by @JanSellner in #2303
- Fix current typing errors by @johnnv1 in #2305
- remove numpy link from image utils by @johnnv1 in #2306
- Update io.rst with
ImageLoadType
by @edgarriba in #2309 - improve local feature orientation by @ducha-aiki in #2310
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2266
- fix: Fix links to CONTRIBUTING.md by @timleslie in #2312
- Fix readme links by @johnnv1 in #2311
- Bump pytest from 7.2.2 to 7.3.0 by @dependabot in #2316
- Update nerf test to use
kornia/data
by @johnnv1 in #2319 - Fix pos_weight in focal loss by @roytseng-tw in #2323
- Fix bugs in Bilateral filter tests by @gau-nernst in #2320
- Add Guided filter by @gau-nernst in #2322
- Bump pytest from 7.3.0 to 7.3.1 by @dependabot in #2327
- Fix kernel size ordering by @gau-nernst in #2326
- update url collect_env.py by @edgarriba in #2329
- Fixes LAF visualization due to kornia_moons update by @ducha-aiki in #2331
- add keypoints to the docs by @johnnv1 in #2330
- Fix missing file keypoints docs by @johnnv1 in #2332
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2318
- Ensure support to torch 2.0 by @johnnv1 in #2272
- Use separable filter2d for box filter by @gau-nernst in #2328
- [feat] add segment anything base by @johnnv1 in #2315
- make api docs more visible by @edgarriba in #2334
New Contributors
- @Aneesh02 made their first contribution in #2308
- @timleslie made their first contribution in #2312
- @roytseng-tw made their first contribution in #2323
Full Changelog: v0.6.11...v0.6.12
v0.6.11 DISK local features, new augmentations and bugfixes
Highlights
In this release we have added DISK, which is the best free local feature for 3D reconstruction. (part of winning solutions in IMC2021 together with SuperGlue).
Thanks to @jatentaki for the great work and relicensing the DISK to Apache 2!
import kornia.feature as KF
disk = KF.DISK.from_pretrained('depth').to(device)
with torch.inference_mode():
inp = torch.cat([img1, img2], dim=0)
features1, features2 = disk(inp, 2048,
pad_if_not_divisible=True)
kps1, descs1 = features1.keypoints, features1.descriptors
kps2, descs2 = features2.keypoints, features2.descriptors
dists, idxs = KF.match_smnn(descs1, descs2, 0.98)
What's Changed
- Release 0.6.10 by @edgarriba in #2212
- Update contributing guide by @johnnv1 in #2217
- Fix RandomGaussianBlur error when sigma is passed as a tensor. by @juliendenize in #2220
- Separate gradcheck for test_conversions by @ducha-aiki in #2223
- Drop JIT support for
core.check
,Boxes
, and some others by @johnnv1 in #2219 - PEP621: Move more configuration into pyproject.toml by @cclauss in #2225
- enable
disallow_incomplete_defs
on mypy by @johnnv1 in #2094 - Update tensor equality tests to use
assert_close()
by @gau-nernst in #2233 - Feat/random median blur by @DariaMinieieva in #2234
- Clean up metadata and fix requirements by @johnnv1 in #2232
- Feat/random snow by @just1ce415 in #2229
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2236
- add random snow and median blur to docs by @edgarriba in #2238
- Use more pythonic expressions in rgb to hls by @alinamuliak in #2235
- Fix typo in doc for RandAugment by @gau-nernst in #2243
- Add bilateral filter by @gau-nernst in #2242
- Feature LAF docs fix by @ducha-aiki in #2245
- Drop JIT support for
geometry.subpix
by @johnnv1 in #2253 - small cleanup on root files by @johnnv1 in #2251
- Add Joint Bilateral Filter by @gau-nernst in #2249
- Remove Re-definition found for builtin input function - Update tests by @alexg-lviv in #2255
- Bump pytest from 7.2.1 to 7.2.2 by @dependabot in #2256
- fix extract_patches and consequent bugs by @ducha-aiki in #2262
- Remove typecheck for 2D kernel size on filters by @johnnv1 in #2259
- Bump accelerate from 0.16 to 0.17.0 by @dependabot in #2265
- Feat/dice loss averaging by @ViriAldi in #2264
- Fix SOLD2 on CUDA by @ducha-aiki in #2270
- Fix Normalize with integer inputs by @adamjstewart in #2269
- Add warnings as error on documentation build by @johnnv1 in #2273
- Bump accelerate from 0.17.0 to 0.17.1 by @dependabot in #2280
- add batch_squared_norm to kornia/geometry/linalg all by @xoiga123 in #2279
- remove numpy dependency by @edgarriba in #2277
- Update release workflow by @edgarriba in #2278
- Fix/diamond square normalize by @hennels in #2283
- Add Random rain augmentation by @BohdanVey in #2268
- Add typehints for kornia.geometry.linalg.inverse_transformation by @xoiga123 in #2286
- Bump accelerate from 0.17.1 to 0.18.0 by @dependabot in #2289
- [WIP] Integrating DISK by @jatentaki in #2285
- Add releases 0.6.6 - 0.6.10 to Changelog by @ducha-aiki in #2295
- fix the padding bug 32 -> 16 by @ducha-aiki in #2294
- expose DISKFeatures structure by @ducha-aiki in #2297
- added conv-mode to filter2d by @ducha-aiki in #2271
- Disk features docs by @jatentaki in #2299
- Remove print contrast by @juliendenize in #2291
- [bugfix] make adalam work w/o scale and ori by @ducha-aiki in #2300
- Bump version for 0.6.11 release by @ducha-aiki in #2296
New Contributors
- @gau-nernst made their first contribution in #2233
- @DariaMinieieva made their first contribution in #2234
- @just1ce415 made their first contribution in #2229
- @alinamuliak made their first contribution in #2235
- @alexg-lviv made their first contribution in #2255
- @ViriAldi made their first contribution in #2264
- @xoiga123 made their first contribution in #2279
- @hennels made their first contribution in #2283
- @BohdanVey made their first contribution in #2268
Full Changelog: v0.6.10...v0.6.11
v0.6.10 Auto Augmentations module, apply_colormap, disparity utitlities, start to support torchdynamo and MUCH more
What's Changed
- add
depth_from_disparity
function by @pri1311 in #2096 - update config tests by @johnnv1 in #2107
- update CI nightly by @johnnv1 in #2106
- Fix AugmentationSequential to return list of boxes by @johnnv1 in #2114
- Bump pytorch version by @johnnv1 in #2100
- add
PadTo
to docs by @johnnv1 in #2122 - add colormap and
apply_ColorMap
for integer tensor by @johnnv1 in #1996 - Fix numerical stability for binary focal loss by @zimka in #2125
- Add RandomGaussianBlur with instance-level gaussian kernel generation by @juliendenize in #1663
- add transparent pad to
CenterCrop
docs example by @johnnv1 in #2124 - Fix support for (*, 3, H, W) tensors in yuv by @ChristophReich1996 in #2108
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2113
- Ensure support to Python 3.9 and 3.10 by @johnnv1 in #2025
- Add Vector2 by @cjpurackal in #2134
- Add 3D-SSIM loss by @pri1311 in #2130
- Fix different warning and errors when building doc by @johnnv1 in #2127
- Add Common Regression Losses by @ChristophReich1996 in #2109
- fix TensorWrapper serialization by @edgarriba in #2132
- Run nightly CI just with labeled PR by @johnnv1 in #2128
- Split the half precision tests workflow by @johnnv1 in #2118
- move deps to
setup.py
by @johnnv1 in #2137 - move tests workflow to reusable workflow by @johnnv1 in #2133
- rename to have the right readme at front page by @johnnv1 in #2141
- Fixe DoG accuracy, add
upscale_double
by @vicsyl in #2105 - Update
nightly
labeled condition by @johnnv1 in #2140 - improve
TestUpscaleDouble
by @johnnv1 in #2147 - fix adalam tests for py310 by @johnnv1 in #2145
- add
fail-fast:false
as default on tests workflow by @johnnv1 in #2146 - Added Face detection Interactive demo by @jeffin07 in #2142
- Bump pytest from 7.2.0 to 7.2.1 by @dependabot in #2148
- add SSIM3D and
depth_from_disparity
to docs by @pri1311 in #2150 - Explicitly cast output to input type to avoid type mismatch errors by @JanSellner in #1842
- Fix params computation for
LongestMaxSize
andSmallestMaxSize
by @johnnv1 in #2131 - torch_version_geq -> torch_version_ge according to todo by @ducha-aiki in #2157
- fix doc build -
sphinx-autodoc-typehints==1.21.3
by @johnnv1 in #2159 - ScaleSpaceDetector -> Fast ScaleSpaceDetector by @ducha-aiki in #2154
- Improve losses tests, add
TestSSIM3d
, andBaseTester.gradcheck
by @johnnv1 in #2152 - modify comments of rgb and lab conversion by @gravitychen in #2153
- add repr and getitem to vector by @cjpurackal in #2163
- Augmentation Base Refactor by @shijianjian in #2117
- unpin
sphinx-autodoc-typehints
by @johnnv1 in #2166 - Fix adalam-config by @ducha-aiki in #2170
- Fix docs of
boxes
,MultiResolutionDetector
.apply colormap
,AugmentationSequential
by @johnnv1 in #2167 - add exception test for se2 + small bug fix by @cjpurackal in #2160
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2174
- Fix MobileViT by @chinhsuanwu in #2172
- Fix output types of augmentations on autocast regions by @johnnv1 in #2168
- Fix planckian jitter for cuda by @johnnv1 in #2177
- [enhance] improve flipping and cropping speed by @shijianjian in #2179
- Replace jit test method in favor of dynamo in
BaseTester
by @johnnv1 in #2120 - downgrade docformatter to 1.5.1 by @edgarriba in #2176
- Fix: resample method None default missing for inverse masks by @miquelmarti in #2185
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2194
- Move padding_size to device in pad for boxes by @miquelmarti in #2197
- Return boxes tensor directly if no boxes by @miquelmarti in #2196
- The isinstance checks order are inverted by @csaybar in #2192
- Skip
x
tests fortorch=1.12.1
andaccelerate
not available by @johnnv1 in #2178 - Drop some ci job by @johnnv1 in #2191
- Make value an attribute of RandomErasing instances again by @miquelmarti in #2195
- TensorWrapper bug fix + add radd, rmul, rsub by @cjpurackal in #2190
- Run CI to just test fp64 on ubuntu and nightly run dynamo tests on PR's by @johnnv1 in #2199
- Small refactor on
filters
module: Dropping JIT support by @johnnv1 in #2187 - Bump accelerate from 0.15.0 to 0.16.0 by @dependabot in #2202
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2203
- Add
integral_image
andintegral_tensor
by @AnimeshMaheshwari22 in #1779 - move kornia check api to kornia.core.check by @edgarriba in #2143
- Fix/repr bug by @neyazbasheer in #2207
- Replace
assert_allclose
byassert_close
by @johnnv1 in #2210 - Fix random crop for keypoints on CUDA device by @johnnv1 in #2209
- Remove outdated augmentation example by @johnnv1 in #2206
- DataKey: add 'image' as alias of 'input' by @adamjstewart in #2193
- Remove py 3.7 for nightly CI by @johnnv1 in #2204
- [Feat] Initiate AutoAugment modules by @shijianjian in #2181
- Fix CUDA failling tests of same device on
Augmentations
by @johnnv1 in #2215
New Contributors
- @pri1311 made their first contribution in #2096
- @zimka made their first contribution in #2125
- @vicsyl made their first contribution in #2105
- @jeffin07 made their first contribution in #2142
- @gravitychen made their first contribution in #2153
- @csaybar made their first contribution in #2192
- @AnimeshMaheshwari22 made their first contribution in #1779
- @neyazbasheer made their first contribution in #2207
- @adamjstewart made their first contribution in #2193
Full Changelog: v0.6.9...v0.6.10
v0.6.9 Revamp kornia.geometry: Hyperplane, Ray, Quaternion, liegroup; restructure CI and typing robustness
What's Changed
- Quaternion pow bug fix (div by zero) by @cjpurackal in #1946
- fix cuda init by @ducha-aiki in #1953
- Bump accelerate from 0.13.1 to 0.13.2 by @dependabot in #1957
- add kornia.testing api in docs by @edgarriba in #1954
- Fix line numbers of included examples. by @colllin in #1950
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #1949
- Feat/randombrightness contrast saturation hue by @duc12111 in #1955
- Normalize with intrinsics by @ducha-aiki in #1727
- Liegroups by @edgarriba in #1887
- Add sepia by @johnnv1 in #1947
- fix doctest in
kornia.geometry.liegroup
by @edgarriba in #1960 - minor improvements to So3 by @cjpurackal in #1966
- Documentation: proper Sørensen–Dice coefficient by @sergiev in #1961
- use torch lts in doctest ci by @edgarriba in #1968
- Add
Hyperplane
andRay
API by @edgarriba in #1963 - Bump pytest from 7.1.3 to 7.2.0 by @dependabot in #1972
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #1975
- drop python 3.6 by @johnnv1 in #1971
- Add some ortho tests for so3 by @stevenlovegrove in #1970
- fix some typing annotations by @johnnv1 in #1967
- ZCA Whiteing demo by @marianna13 in #1932
- doctest to minimal python 3.8 by @edgarriba in #1974
- fix import in assert_close helper by @pmeier in #1982
- Remove unnecessary configs by @johnnv1 in #1984
- Remove
mypy
from running on tests by @johnnv1 in #1983 - Remove some
# type: ignore
fromkornia.feature
by @johnnv1 in #1995 - add quaternion to euler conversion by @edgarriba in #1994
- Update google analytics is for G4 property by @edgarriba in #1999
- implement
kornia.geometry.linalg.euclidean_distance
by @edgarriba in #2000 - quaternion, so3 and se3 as non batched by @edgarriba in #1997
- Bump accelerate from 0.13.2 to 0.14.0 by @dependabot in #2004
- Remove unused
type: ignore
by @johnnv1 in #1998 - Bump pytest-mypy from 0.10.0 to 0.10.1 by @dependabot in #2005
- Join the gh-actions for docs by @johnnv1 in #2003
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2010
- [feat] liegroup so2 by @cjpurackal in #1973
- add rotation and translation classmethods in se3 and so3 by @edgarriba in #2001
- [feat] implement adjoint for liegroups by @cjpurackal in #2007
- Fix typing errors by @johnnv1 in #2012
- remove unused deepsource by @johnnv1 in #2016
- So2 bug fix by @cjpurackal in #2015
- use resample instead of mode argument in RandomElasticTransform per default by @JanSellner in #2017
- Add reusable workflow to env setup and update CI's by @johnnv1 in #2009
- Remove redudant casts by @johnnv1 in #2022
- Fix type annotation for torch 1.13.0 by @johnnv1 in #2023
- Drop pytorch 1.8 (LTS) support by @johnnv1 in #2024
- Fix an error in
match_smnn
by @anstadnik in #2020 - Remove deprecated code in
kornia.augmentation
by @johnnv1 in #2028 - so2 tests update and cleanup by @cjpurackal in #2029
- Fix PR action trigger by @johnnv1 in #2026
- Set equal_nan to False in assert_close by @edgarriba in #1986
- drop flake8 dependency by @johnnv1 in #2032
- Improves performance of the slowest CPU tests by @johnnv1 in #2036
- add default python and update pre-commit hooks by @johnnv1 in #2040
- facedetector now returns a list of tensors containing the boxes x image by @lferraz in #2034
- add random for liegroups by @cjpurackal in #2041
- Add/ensure support for pytorch 1.13.0 by @johnnv1 in #2035
- Update constants to be able to inherit types from
get
method by @johnnv1 in #2047 - Pass along data_keys or extra_args in *ApplyInverse with containers by @miquelmarti in #2046
- update mul for so2 by @cjpurackal in #2051
- None for align_corners arg of resize op with nearest mode by @miquelmarti in #2049
- Remove type ignore from the codebase by @johnnv1 in #2030
- making
RandomGaussianNoise
play nicely on GPU by @nitaifingerhut in #2050 - fix pep561 and remove deprecated
license_file
by @johnnv1 in #2057 - Set padding mode to zeros for inverse of resize aug via crop by @miquelmarti in #2054
- replacing .repeat(...) with .expand(...) by @nitaifingerhut in #2059
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2065
- Bump accelerate from 0.14.0 to 0.15.0 by @dependabot in #2058
- Fix GHA macos
queued
and coverage upload by @johnnv1 in #2038 - Fix F401 by @johnnv1 in #2067
- enable
fast_mode
on grandchecks by @johnnv1 in #2069 - BUGFIX:
RandomMotionBlur
is not deterministic when usingself._params
by @nitaifingerhut in #2068 - Motion blur by @nitaifingerhut in #2075
- bugfix: KORNIA_CHECK_SHAPE by @nitaifingerhut in #2076
- [feat] Implement se2 by @cjpurackal in #2019
- Fix f401 by @johnnv1 in #2077
- remove type ignore by @johnnv1 in #2078
- bug fix for getitem in liegroups and quaternion by @cjpurackal in #2079
- Remove deprecated code in
kornia.augmentation
by @johnnv1 in #2052 - [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2081
- Disable
fail-fast
on CI by @johnnv1 in #2085 - enable
check_untyped_defs
on mypy by @johnnv1 in #2086 - enable
disallow_any_generics
on mypy by @johnnv1 in #2092 - [feat] add vee to so2, se2 by @cjpurackal in #2091
- Fix padding for random crops by @miquelmarti in #2087
- fix failing tests related to
solve_cast
on torch 1.9 by @johnnv1 in #2066 - Add
TensorWrapper
,Vector3
,Scalar
and improvements infit_plane
by @edgarriba in #1987 - fix parameters generator to be reproducible (3D) by @johnnv1 in #2088
- Fix tests on CUDA by @johnnv1 in #2098
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #2099
- [feat] adjoint for se2, so2 by @cjpurackal in #2101
- add trans, trans_x, trans_y + minor changes se2 by @cjpurackal in #2103
New Contributors
- @colllin made their first contribution in #1950
- @sergiev made their first contribution in #1961
- @stevenlovegrove made their first contribution in #1970
- @JanSellner made their first contribution in #2017
- @anstadnik made their first contribution in #2020
Full Changelog: v0.6.8...v0.6.9
v0.6.8 Experimental Nerf, Deep Edge Detection, community demos, Quaternion slerp, augmentations dispatcher, image matching and other bug fixes
Highlights
NeRF API
In this release in we include an experimental kornia.nerf
submodule with a high level API that implements a vanilla Neural Radiance Field (NeRF). Read more about the roadmap of this project: #1936 // contribution done by @YanivHollander
from kornia.nerf import NerfSolver
from kornia.geomtry.camera import PinholeCamera
camera: PinholeCamera = create_one_camera(5, 9, device, dtype)
img = create_red_images_for_cameras(camera, device)
nerf_obj = NerfSolver(device=device, dtype=dtype)
num_img_rays = 15
nerf_obj.init_training(camera, 1.0, 3.0, False, img, num_img_rays, batch_size=5, num_ray_points=10, lr=1e-2)
nerf_obj.run(num_epochs=10)
img_rendered = nerf_obj.render_views(camera)[0].permute(2, 0, 1)
Improvements, docs and tutorials soon!
Edge Detection
Added kornia.contrib.EdgeDetection
API that implements dexined
: https://github.com/xavysp/DexiNed
import kornia as K
from kornia.contrib import EdgeDetection
edge_detection = EdgeDetector().to(device)
# preprocess
img = K.image_to_tensor(frame, keepdim=False).to(device)
img = K.color.bgr_to_rgb(img.float())
# detect !
with torch.no_grad():
edges = edge_detection(img)
img_vis = K.tensor_to_image(edges.byte())
Image matching bugfixes:
After testing kornia LoFTR and AdaLAM under big load, our users and we have experiences some bugs in corners cases, such as big images or no input correspondences, which caused pipeline to crash. Not anymore!
- Fixes typo bug that influences LoFTR training by @georg-bn in #1854
- Enlargen LoFTR positional encoding map if large images are input by @georg-bn in #1853
- Make AdaLAM output match confidence by @ducha-aiki in #1862
- fix AdaLAM crash by @ducha-aiki in #1881
- Adalam fix2 by @ducha-aiki in #1888
- No crash in local feature matching if empty tensor output by @ducha-aiki in #1890
- Fix warning in AdaLAM by @skydes in #1925
Various kornia demos in gradio by community:
See demos in our HuggingFace space: https://huggingface.co/kornia
- Added gradio Image Stitching demo link by @kadirnar in #1871
- edge detection demo by @p-mishra1 in #1876
- Added Hugging Face edge detection demo link by @ramon-rd in #1874
- [docs] add gradio app html and embeddings in filters by @lappemic in #1883
- Geometry image transform demo by @dvando in #1922
- add spaces demo by @johko in #1905
- add space demo for homography warping by @johko in #1924
- created resize_antialias.html file by @gauthamk28 in #1877
- I added html file for module Line Fitting by @kadirnar in #1886
- Add edge detector and morphological operator demos in the rst docs files by @ramon-rd in #1884
- Image registration demo by @marianna13 in #1897
- [docs] Add total_variation_denoising gradio by @gagan3012 in #1880
- [Docs] Refactor the embedded Gradio demos by @NimaBoscarino in #1901
RANSAC improvements
We have added homography-from-line-segments solver, as well as various speed-ups. We are not yet at OpenCV RANSAC quality level, more improvements to come :) But the line-solver is pretty unique! We also have example in our tutorials https://kornia-tutorials.readthedocs.io/en/latest/line_detection_and_matching_sold2.html
- Added homography from line segment correspondences by @ducha-aiki in #1851
- RANSAC improvements by @ducha-aiki in #1435
- Add get_perpendicular and get_closest_point_on_epipolar_line by @ducha-aiki in #1915
- Fix svdvals usage by @ducha-aiki in #1926
Apple Silicon M1 support is closer, CI improvements
We are slowly working on being able to run kornia on M1. So far we have added possibility to test locally on M1 and mostly report Pytorch MPS backend crashes in various use-cases. Once this work is finished, we may provide some workarounds to have kornia-M1
- remove conv3d from spatial_gradient by @ducha-aiki in #1898
- Added possibility to run tests for mps locally by @ducha-aiki in #1716
- CI update pytorch-->-1.12.1 by @ducha-aiki in #1892
- lts is not supported on mac-os, separate it by @ducha-aiki in #1904
- Update setup to declarative metadata by @johnnv1 in #1885
- Bump pytest from 7.1.2 to 7.1.3 by @dependabot in #1860
- add concurrency cancel-in-progress in cpu workflow by @edgarriba in #1865
- Use --no-implicit-optional for type checking by @hauntsaninja in #1910
Quaternion improvements
Implemented Quaternion.slerp to interpolate between quaternions using quaternion arithmetic -- contributed by @cjpurackal
import torch
from kornia.geometry.quaternion import Quaternion
q0 = Quaternion.identity(batch_size=1)
q1 = Quaternion(torch.tensor([[1., .5, 0., 0.]]))
q2 = q0.slerp(q1, .3)
More augmentations!
- [feat] Added Jigsaw Augmentation by @shijianjian in #1852
- [Feat] Added AugmentationDispatcher by @shijianjian #191
What's Changed
- Added homography from line segment correspondences by @ducha-aiki in #1851
- Fixes typo bug that influences LoFTR training by @georg-bn in #1854
- Enlargen LoFTR positional encoding map if large images are input by @georg-bn in #1853
- docs: clarify the relation of color_jitter and color_jiggle by @kunaltyagi in #1858
- Bump pytest from 7.1.2 to 7.1.3 by @dependabot in #1860
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #1833
- Binary focal loss: Use pre-computed probs and increase readability by @klieret in #1848
- add concurrency cancel-in-progress in cpu workflow by @edgarriba in #1865
- modifying
add_weighted
to accept Tensors foralpha
/beta
/gamma
by @nitaifingerhut in #1868 - Make AdaLAM output match confidence by @ducha-aiki in #1862
- fix bugs shift_rgb by @duc12111 in #1861
- Added gradio Image Stitching demo link by @kadirnar in #1871
- edge detection demo by @p-mishra1 in #1876
- Added Hugging Face edge detection demo link by @ramon-rd in #1874
- fix AdaLAM crash by @ducha-aiki in #1881
- [feat] Added Jigsaw Augmentation by @shijianjian in #1852
- [docs] add gradio app html and embeddings in filters by @lappemic in #1883
- created resize_antialias.html file by @gauthamk28 in #1877
- Adalam fix2 by @ducha-aiki in #1888
- Add edge detector and morphological operator demos in the rst docs files by @ramon-rd in #1884
- I added html file for module Line Fitting by @kadirnar in #1886
- No crash in local feature matching if empty tensor output by @ducha-aiki in #1890
- CI update pytorch-->-1.12.1 by @ducha-aiki in #1892
- Fix fail test by @ducha-aiki in #1896
- Added possibility to run tests for mps locally by @ducha-aiki in #1716
- remove conv3d from spatial_gradient by @ducha-aiki in #1898
- lts is not supported on mac-os, separate it by @ducha-aiki in #1904
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #1900
- quaternion index bug fix by @cjpurackal in #1903
- Update setup to declarative metadata by @johnnv1 in #1885
- [pre-commi...
v0.6.7 SOLD2, AdaLAM, Quaternion, Mosaic Aug and Edge-aware Blur
Highlights
SOLD2 line segment detector & descriptor
Contributed by SOLD2 original authors
Geometry-aware matchers: AdaLAM & FGINN
Good old Lowe ratio-test is good for descriptor matching (implemented as `match_snn`, `match_smnn` in kornia, but it is often not enough: it does not take into account keypoint positions. With this version we started to add geometry aware descriptor matchers, starting with [FGINN](https://arxiv.org/abs/1503.02619) and [AdaLAM](https://arxiv.org/abs/2006.04250). Later we plan to add something like SuperGlue (but free version, ofc).AdaLAM works particularly well with kornia.feature.KeyNetAffNetHardNet
. AdaLAM is adopted from original author's implementation.
import matplotlib.pyplot as plt
import cv2
import kornia as K
import kornia.feature as KF
import numpy as np
import torch
from kornia_moons.feature import *
def load_torch_image(fname):
img = K.image_to_tensor(cv2.imread(fname), False).float() /255.
img = K.color.bgr_to_rgb(img)
return img
device = K.utils.get_cuda_device_if_available()
fname1 = 'kn_church-2.jpg'
fname2 = 'kn_church-8.jpg'
img1 = load_torch_image(fname1)
img2 = load_torch_image(fname2)
feature = KF.KeyNetAffNetHardNet(5000, True).eval().to(device)
input_dict = {"image0": K.color.rgb_to_grayscale(img1), # LofTR works on grayscale images only
"image1": K.color.rgb_to_grayscale(img2)}
hw1 = torch.tensor(img1.shape[2:])
hw2 = torch.tensor(img1.shape[2:])
adalam_config = {"device": device}
with torch.inference_mode():
lafs1, resps1, descs1 = feature(K.color.rgb_to_grayscale(img1))
lafs2, resps2, descs2 = feature(K.color.rgb_to_grayscale(img2))
dists, idxs = KF.match_adalam(descs1.squeeze(0), descs2.squeeze(0),
lafs1, lafs2, # Adalam takes into account also geometric information
config=adalam_config,
hw1=hw1, hw2=hw2) # Adalam also benefits from knowing image size
More - in our Tutorials section
Geometry conversions
Converting camera pose from (R,t) to actually pose in world coordinates can be a pain. We are relieving you from it, by implementing various conversion functions, such as camtoworld_to_worldtocam_Rt
, worldtocam_to_camtoworld_Rt
, camtoworld_graphics_to_vision_4x4
, etc. The conversions come with two variants: for (R,t)
tensor tuple, or with since extrinsics mat4x4
.
Quaternion API
More geometry-related stuff! We have added Quaternion API to make work with rotation representations easy. Checkout the PR
>>> q = Quaternion.identity(batch_size=4)
>>> q.data
Parameter containing:
tensor([[1., 0., 0., 0.],
[1., 0., 0., 0.],
[1., 0., 0., 0.],
[1., 0., 0., 0.]], requires_grad=True)
>>> q.real
tensor([[1.],
[1.],
[1.],
[1.]], grad_fn=<SliceBackward0>)
>>> q.vec
tensor([[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]], grad_fn=<SliceBackward0>)
Mosaic Augmentation
We recently included the RandomMosaic to mosaic image transforms and combine them into one output image. The output image is composed of the parts from each sub-image.
The mosaic transform steps are as follows:
- Concate selected images into a super-image.
- Crop out the outcome image according to the top-left corner and crop size.
>>> mosaic = RandomMosaic((300, 300), data_keys=["input", "bbox_xyxy"])
>>> boxes = torch.tensor([[
... [70, 5, 150, 100],
... [60, 180, 175, 220],
... ]]).repeat(8, 1, 1)
>>> input = torch.randn(8, 3, 224, 224)
>>> out = mosaic(input, boxes)
>>> out[0].shape, out[1].shape
(torch.Size([8, 3, 300, 300]), torch.Size([8, 8, 4]))
Edge-aware blurring
Thanks to @nitaifingerhut
!wget https://github.com/kornia/data/raw/main/drslump.jpg
import torch
import kornia
import cv2
import matplotlib.pyplot as plt
# read the image with OpenCV
img: np.ndarray = cv2.imread('./drslump.jpg')
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
# convert to torch tensor
data: torch.tensor = kornia.image_to_tensor(img, keepdim=False)/255. # BxCxHxW
data-=0.2*torch.rand_like(data).abs()
plt.figure(figsize=(12,8))
edge_blurred = kornia.filters.edge_aware_blur_pool2d(data, 19)
plt.imshow(kornia.tensor_to_image(torch.cat([data, edge_blurred],axis=3)))
What's Changed
- 0.8 is too strict for smnn matching, 0.95 is much better default by @ducha-aiki in #1807
- fix bug and add scale coef by @ducha-aiki in #1808
- No crash matching by @ducha-aiki in #1810
- Added FGINN matching by @ducha-aiki in #1813
- Added SOLD2 by @rpautrat #1507 #1844
- disable accelerate for macos and pytorch 1.10.2 by @edgarriba in #1811
- update flake8 to 5.0.4 and fixes by @edgarriba in #1818
- Bump accelerate from 0.10.0 to 0.11.0 by @dependabot in #1802
- Bump accelerate from 0.11.0 to 0.12.0 by @dependabot in #1820
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #1821
- Allowing more than 3/4 dims for
total_variation
+ addingreduction
by @nitaifingerhut in #1815 - edge aware blur2d by @nitaifingerhut in #1822
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #1828
- Adds conversions between graphics and vision coordinate frames by @ducha-aiki in #1823
- Add Quaternion API by @edgarriba in #1801
- fix tests float16 module losses by @MrShevan in #1809
- AdaLAM match filtering (clean) by @ducha-aiki in #1831
- [Feat] Init Mosaic Augmentation by @shijianjian in #1713
- Change embedded Gradio demo to use web component instead of iframe by @NimaBoscarino in #1835
- fix tests and warnings by @MrShevan in #1834
Full Changelog: v0.6.6...v0.6.7
v0.6.6 ParametrizedLine API and load_image macos and windows
Highlights
ParametrizedLine API
First of integrations to revamp kornia.geometry
to align with Eigen and Sophus.
Docs: https://kornia.readthedocs.io/en/latest/geometry.line.html?#kornia.geometry.line.ParametrizedLine
See: example: https://github.com/kornia/kornia/blob/master/examples/geometry/fit_line2.py
Support for macos and windows in load_image
Automated the packaging infra in kornia_rs
to handle multi architecture builds. Arm64 soon :)
See: https://github.com/kornia/kornia-rs
# load the image using the rust backend
img: Tensor = K.io.load_image(file_name, K.io.ImageLoadType.RGB32)
img = img[None] # 1xCxHxW / fp32 / [0, 1]
HuggingFacce integration
Created Kornia AI org under the HuggingFace platform.
Starting to port the tutorials under HuggingFace kornia org to rapidly show live docs and make community.
Link: https://huggingface.co/kornia
Demos:
- kornia enhance: https://kornia.readthedocs.io/en/latest/enhance.html#interactive-demo
- augmentations playground: https://huggingface.co/spaces/kornia/kornia-augmentations-tester
What's new ?
- update slack link by @edgarriba in #1719
- fixes
EarlyStoppping
condition by @edgarriba in #1718 - Fix warning:
meshgrid
needindexing argument
by @FavorMylikes in #1629 - Bump accelerate from 0.8.0 to 0.9.0 by @dependabot in #1720
- fixes for half precision in imgwarp by @edgarriba in #1723
- Fix transforms for empty boxes and keypoints inputs by @hal-314 in #1741
- few mypy fixes by @edgarriba in #1724
- Implement
project
andunproject
inPinholeCamera
by @YanivHollander in #1729 - deprecate
filter2D
filter3D
api by @edgarriba in #1725 - fixing doctest in pinhole by @edgarriba in #1743
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #1742
- Fix/crop transforms by @hal-314 in #1739
- Fix Boxes.from_tensor(boxes, mode="vertices") by @hal-314 in #1740
- adding
rgb_to_y
by @nitaifingerhut in #1734 - fix typing callable in load storage by @edgarriba in #1768
- Add rgb_to_y to all by @ashnair1 in #1762
- Fix bug preventing sample wise augmentations by @ashnair1 in #1761
- update pytorch ci matrix 1.10.2 and 1.11.0 by @edgarriba in #1771
- docs: Fix a few typos by @timgates42 in #1774
- Refactor and add tests in
get_perspective_transform
by @edgarriba in #1767 - [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #1776
- update libfacedetection url path by @edgarriba in #1780
- enable black in the precommit by @edgarriba in #1777
- Bump kornia-rs from 0.0.2 to 0.0.5 by @dependabot in #1784
- kornia io support for macos and win by @edgarriba in #1785
- deploy docs to gh-pages by @edgarriba in #1787
- update pytest 7.1.2; pytest-flake8 1.1.1; flake8 4.0.1 by @edgarriba in #1786
- adding weights to positive examples by @MrShevan in #1765
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #1789
- add
KORNIA_CHECK_SAME_DEVICES
by @MrShevan in #1788 - Bump accelerate from 0.9.0 to 0.10.0 by @dependabot in #1748
- Add
sphinxcontrib.gtagjs
to track docs by @edgarriba in #1790 - Add an interactive demo to the kornia.enhance docs by @NimaBoscarino in #1793
- Update the Gradio demo URL to point to Kornia HF org by @NimaBoscarino in #1795
- Add
ParametrizedLine
andfit_line
by @edgarriba in #1794 - add link to interactive augmentations demo by @cceyda in #1797
New Contributors
- @FavorMylikes made their first contribution in #1629
- @MrShevan made their first contribution in #1765
- @NimaBoscarino made their first contribution in #1793
Full Changelog: v0.6.5...v0.6.6