A3D-MoE: Acceleration of Large Language Models With Mixture of Experts via 3-D Heterogeneous Integration https://sscs.ieee.org/wp-content/themes/movedo/images/empty/thumbnail.jpg 150 150 https://secure.gravatar.com/avatar/8fcdccb598784519a6037b6f80b02dee03caa773fc8d223c13bfce179d70f915?s=96&d=mm&r=g
Abstract:
Conventional large language models (LLMs) have large parameter sets, making inference costly and energy-intensive; mixture-of-experts (MoEs) mitigates this by activating fewer weights per token, but fine-grained MoE LLMs still face runtime workload variability, inefficient conventional scheduling, and high bandwidth memory (HBM) loading energy/bandwidth demands. A3D-MoE addresses these with 3…