EDN

The hidden bottleneck in LLM inference and the impact on MLPerf benchmarking

Here is how the prefill versus generation split exposes GPU structural inefficiencies in AI processor designs.

MeMo's memory model lets teams upgrade their LLM without retraining it — and performance jumps 26%

Researchers' MeMo keeps AI memory separate from reasoning, so teams can upgrade their LLM without retraining it and see a 26% ...
Semiconductor Engineering

The Edge LLM Offload Story

Modern edge devices demand heterogeneous AI architectures that can mix and match subsystems to accelerate different aspects ...
Edhat

Yuheng Bu seeks a better way to ensure the trustworthiness of AI-generated text

UC Santa Barbara’s Robert Mehrabian College of Engineering, Yuheng Bu, assistant professor in the Computer Science Department, has received a prestigious Early CAREER Award from the National Science F ...
EDN

MLPerf and the rise of latency-aware LLM benchmarking

Here is a sneak peek at the evolution of the MLPerf benchmark and how generative AI forced a radical shift in AI hardware ...

Researchers automated LLM reasoning strategy design and cut token usage by 69.5%

Researchers from Meta and Google built AutoTTS to automatically discover optimal LLM reasoning strategies, cutting token ...
Tech Xplore

Making LLMs faster and more efficient across multiple languages

Large language models (LLMs), which are the artificial intelligence (AI) systems behind modern chatbots, translation tools, ...
Crypto Briefing

MIT’s MeMo framework boosts LLM performance by 26% without retraining

MIT's MeMo framework trains a compact memory model that boosts LLM performance by up to 26.73% without retraining, with major implications for crypto AI agents.
Hackaday

An LLM From “Scratch”

Reading a book about bowling is not the same as actually bowling. If that resonates with you and you want to learn more about large language models, check out the LLM From Scratch project. The ...