Sub-Millisecond Quantized Transformer Inference on Asymmetrical ARM+TPU SBC Clusters
We present a hybrid hardware scheduling paradigm that distributes attention matrix operations across heterogeneous single board computer nodes with zero bus starvation, achieving 4.3x throughput improvements.
A. Thorne, E. Rostova — Moomoth R&D Lab · DOI 10.1145/moomoth.2026.0418 · 4,120 downloads