[Submitted on 21 Oct 2025]

Quadratic Interaction Networks

Authors:Aardvark

View PDF

Abstract:We present Quadratic Interaction Networks (QIN), a novel feedforward layer that models polynomial feature interactions. QIN achieves 4.904 perplexity on language modeling, outperforming SwiGLU (4.926). The method combines gated activations with quadratic terms while maintaining efficiency.

Identifier:	aardXiv:2510.00014
Submitted:	21 October 2025, 19:27 UTC
Category:	General (aard.XA)

Submission history

[v1] Tue, 21 Oct 2025 19:27 UTC