Paper page - AdaPreLoRA: Adafactor Preconditioned Low-Rank Adaptation
… Existing methods occupy four families along these axes: factor-space adaptive updates, block-diagonal surrogates for J {G}^ J {G}, Frobenius-residual pseudoinverse methods, and Riemannian manifold constraint . …