1 | Kan Zhuhttps://kanzhu.netlify.app/publication-type/1/1Wowchemy (https://wowchemy.com)en-usSun, 20 Oct 2024 00:00:00 +0000https://kanzhu.netlify.app/media/icon_hu0b7a4cb9992c9ac0e91bd28ffd38dd00_9727_512x512_fill_lanczos_center_3.png1https://kanzhu.netlify.app/publication-type/1/NanoFlow: Towards Optimal Large Language Model Serving Throughputhttps://kanzhu.netlify.app/publication/nanoflow/Sun, 20 Oct 2024 00:00:00 +0000https://kanzhu.netlify.app/publication/nanoflow/<!-- <div class="alert alert-note"> <div> Click the <em>Cite</em> button above to demo the feature to enable visitors to import publication metadata into their reference management software. </div> </div> <div class="alert alert-note"> <div> Create your slides in Markdown - click the <em>Slides</em> button to check out the example. </div> </div> Supplementary notes can be added here, including [code, math, and images](https://wowchemy.com/docs/writing-markdown-latex/). -->Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencehttps://kanzhu.netlify.app/publication/quest/Sun, 20 Oct 2024 00:00:00 +0000https://kanzhu.netlify.app/publication/quest/<!-- <div class="alert alert-note"> <div> Click the <em>Cite</em> button above to demo the feature to enable visitors to import publication metadata into their reference management software. </div> </div> <div class="alert alert-note"> <div> Create your slides in Markdown - click the <em>Slides</em> button to check out the example. </div> </div> Supplementary notes can be added here, including [code, math, and images](https://wowchemy.com/docs/writing-markdown-latex/). -->Atom: Low-bit Quantization for Efficient and Accurate LLM Servinghttps://kanzhu.netlify.app/publication/atom/Sun, 20 Oct 2024 00:00:00 +0000https://kanzhu.netlify.app/publication/atom/<!-- <div class="alert alert-note"> <div> Click the <em>Cite</em> button above to demo the feature to enable visitors to import publication metadata into their reference management software. </div> </div> <div class="alert alert-note"> <div> Create your slides in Markdown - click the <em>Slides</em> button to check out the example. </div> </div> Supplementary notes can be added here, including [code, math, and images](https://wowchemy.com/docs/writing-markdown-latex/). -->From Optimal to Practical: Efficient Micro-op Cache Replacement Policies for Data Center Applicationshttps://kanzhu.netlify.app/publication/uop/Sun, 20 Oct 2024 00:00:00 +0000https://kanzhu.netlify.app/publication/uop/<!-- <div class="alert alert-note"> <div> Click the <em>Cite</em> button above to demo the feature to enable visitors to import publication metadata into their reference management software. </div> </div> <div class="alert alert-note"> <div> Create your slides in Markdown - click the <em>Slides</em> button to check out the example. </div> </div> Supplementary notes can be added here, including [code, math, and images](https://wowchemy.com/docs/writing-markdown-latex/). -->Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models.https://kanzhu.netlify.app/publication/fidder/Sun, 20 Oct 2024 00:00:00 +0000https://kanzhu.netlify.app/publication/fidder/<!-- <div class="alert alert-note"> <div> Click the <em>Cite</em> button above to demo the feature to enable visitors to import publication metadata into their reference management software. </div> </div> <div class="alert alert-note"> <div> Create your slides in Markdown - click the <em>Slides</em> button to check out the example. </div> </div> Supplementary notes can be added here, including [code, math, and images](https://wowchemy.com/docs/writing-markdown-latex/). -->