Edge Cases by AGI, Inc. #005 with Shuo Yang @ UC Berkeley: FreeToken

Edge Cases is AGI, Inc.'s invite-only research night on what it takes to run real intelligence on-device. Speaker Shuo Yang, PhD student in Electrical Engineering and Computer Sciences at UC Berkeley, advised by Ion Stoica. Works on efficient AI systems: LLM inference, GPU optimization and generative AI serving. Maintains FreeToken, an open-source system for running large Mixture-of-Experts models on commodity hardware. Talk FreeToken: Making Frontier MoE Models Accessible on Local Hardware Why local LLM inference matters, and what gets in the way of running large MoE models on consumer hardware: memory capacity, bandwidth limits and the bottlenecks of CPU/GPU heterogeneous inference. Shuo will walk through the FreeToken system design, including bandwidth-adaptive execution, expert management and disk streaming, then performance results, practical lessons and what this opens up for local AI agents. Details Saturday, December 12, 12:00 PM Our founders' house in San Francisco. Exact address shared on approval. 40 minute talk, off-the-record Q&A, then lunch with the group For AI researchers, PhD students and research engineers Seats are limited and registration is approval-based More about the series: theagi.company/edge-cases Want to present your on-device research at a future Edge Cases? Apply here