large language models (LLMs)

  • No Comments

A3D-MoE: Acceleration of Large Language Models With Mixture of Experts via 3-D Heterogeneous Integration

A3D-MoE: Acceleration of Large Language Models With Mixture of Experts via 3-D Heterogeneous Integration 150 150
Author(s): Wei-Hsing Huang, Janak Sharda, Cheng-Jhih Shih, Yuyao Kong, Faaiq Waqar, Pin-Jun Chen, Yingyan Lin, Shimeng Yu

Abstract:

Conventional large language models (LLMs) have large parameter sets, making inference costly and energy-intensive; mixture-of-experts (MoEs) mitigates this by activating fewer weights per token, but fine-grained MoE LLMs still face runtime workload variability, inefficient conventional scheduling, and high bandwidth memory (HBM) loading energy/bandwidth demands. A3D-MoE addresses these with 3…

Published in: IEEE Journal on Exploratory Solid-State Computational Devices and Circuits
Page(s): 176 – 184
Year of Publication: 2026
Electronic ISSN: 2329-9231
DOI: 10.1109/JXCDC.2026.3713957
Publisher: IEEE

View on IEEE Xplore

  • No Comments

3-D Stacked HBM and Compute Accelerators for LLM: Optimizing Thermal Management and Power Delivery Efficiency

3-D Stacked HBM and Compute Accelerators for LLM: Optimizing Thermal Management and Power Delivery Efficiency 150 150
Author(s): Janak Sharda, Madison Manley, Jungyoun Kwak, Chinsung Park, Muhannad Bakir, Shimeng Yu

Abstract:

Advanced packaging is becoming essential for designing hardware accelerators for large language models (LLMs). Different architectures, such as 2.5-D integration of memory with logic, have been proposed; however, the bandwidth limits the throughput of the complete system. Recent works have proposed memory on logic systems, where high bandwidth memory (HBM) …

Published in: IEEE Journal on Exploratory Solid-State Computational Devices and Circuits
Page(s): 116 – 122
Year of Publication: 2025
Electronic ISSN: 2329-9231
DOI: 10.1109/JXCDC.2025.3617298
Publisher: IEEE

View on IEEE Xplore

Search this website