3

Themis: Software-Defined Hardware Prefetching
Data cache misses represent a significant portion of stall cycles in datacenter workloads. Hardware prefetchers that reduce such stalls …
Keisuke Kamahori, Neil Adit, Kan Zhu, Yuqi Mai, Victor Lee, Heiner Litz, Chris Kennelly, Snehasish Kumar, Hanna Alam, Milad Hashemi, David Li, Adrian Sampson, Baris Kasikci, Tipp Moseley, Parthasarathy Ranganathan, Akanksha Jain
PDF
TraceLab: Characterizing Coding Agent Workloads for LLM Serving
  • Collected and released real-world coding agent traces from day-to-day use of Claude Code and Codex
  • Characterized coding agent workloads and identified opportunities for optimizing LLM serving
Kan Zhu, Mathew Jacob, Chenxi Ma, Yi Pan, Stephanie Wang, Arvind Krishnamurthy, Baris Kasikci
PDF Code Project
PolyServe: Efficient Multi-SLO Serving at Scale
  • Grouped requests into bins by per-token latency requirement and scheduled each bin to a subset of servers
  • Maintained high SLO attainment while maximizing throughput for multi-SLO serving at scale
Kan Zhu, Haiyang Shi, Le Xu, Jiaxin Shan, Arvind Krishnamurthy, Baris Kasikci, Liguang Xie
PDF