Neural Feature Engineering

This repo collects implementations of feature geometry, a mathematical framework designed for principled representation learning with deep neural networks as building blocks.

Neural Feature Learning in Function Space

Learning Modal Decomposition

Hirschfeld-Gebelein-Rényi (HGR) maximal correlation functions

• Nested_H_Score
• H_score_Seq
• MaxCorr_Normal: compute maximal correlation functions for normal variables

Learning With Orthogonality Constraints

Learn features uncorrelated with given features

• f_orth_cosine

Learning With Side Information (Conditioned Inference)

• SideInfo_multihead

Multimodal Learning With Missing Modalities

• mm_cosine
• mm_pair_cosine

Sequential Dependence Decomposition and Feature Learning

Sequential Dependence Decomposition

• SEQ: decomposing sequential dependence into Markov chains of different orders.

References

[1] Xu, Xiangxiang, and Shao-Lun Huang. "Maximal correlation regression." IEEE Access 8 (2020): 26591-26601.

[2] Xu, Xiangxiang, and Lizhong Zheng. "Sequential Dependence Decomposition and Feature Learning." 2023 59th Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE, 2023.

[3] Xu, Xiangxiang, and Lizhong Zheng. "Neural Feature Learning in Function Space." Journal of Machine Learning Research (JMLR), 25(142), 2024.