Faster Orthogonal Parameterization with Householder Matrices
Published in ICML Workshop on Invertyible Neural Networks, Normalizing Flows, and Explicit Likelihood Models, 2020
Orthogonal matrices have been used in several Normalizing Flows . Orthogonal matrices are attractive since they are easy to invert and have Jacobian determinant one. Their main downside is the additional computational resources required to perform gradient descent. We identify a computational bottleneck for previous work on Householder matrices, and introduce a novel algorithm, FastH, which circumvents the bottleneck and is up to $29\times$ faster than a previous method.
BibTeX:
@inproceedings{fasth,
title={\{F}aster {O}rthogonal {P}arameterization with {H}ouseholder {M}atrices},
author={Mathiasen, Alexander and Hvilsh{\o}j, Frederik and J{\o}rgensen, Jakob R{\o}dsgaard
and Nasery, Anshul and Mottin, Davide},
booktitle={\{ICML} Workshop on Invertible Neural Networks and Normalizing Flows},
year={2020}
}
Read full paper here.