Orthogonal Subspace Projection Layers

Official code of "Layer-Centric Factors of Variation Disentanglement for Task- and Model-Agnostic Generalization" paper (ICML 2026).

• Layers: OSP-Linear, OSP-Conv, OSP-ResNet_Block, OSP-attention, OSP-MLP
• Models: OSP-ResNet-18/34/50 OSP-ViT

##Usage Let import src.osp.layers as osp, then define osp.Linear or osp.Conv2d instead nn.Linear or nn.Conv2d.

Each layers have a additional variable subspace_dim, it must be divisor of out_features or out_channels.

Here is the examples

import src.osp.layers as osp

linear1 = osp.Linear(in_features=10, out_features=100, subspace_dim=20)
conv1 = osp.Conv2d(in_channels=3, out_channels=64, subspace_dim=8)

Models

• ResNet

• src.osp.resnet.py: OSP-ResNet-18/34/50

  import .src.osp.resnet as osp resnet18 = osp.OSPResNet18(num_classes=1000) resnet34 = osp.OSPResNet18(num_classes=1000) resnet50 = osp.OSPResNet18(num_classes=1000)

• ViT

• .timm

• move all files to pytorch-image-models following same directories. Then you can use OSP-ViT as like ViT easily.

BibTeX

@inproceedings{
anonymous2026layercentric,
title={Layer-Centric Factors of Variation Disentanglement for Task- and Model-Agnostic Generalization},
author={Anonymous},
booktitle={Forty-third International Conference on Machine Learning},
year={2026},
url={https://openreview.net/forum?id=QE9b2nOWi2}
}