ELL3N HSU
← Projects
Personal project In progress

LLM Inference Lab

A traceable learning record for LLM inference, KV cache, and evaluation methods.

Markdown

Planned exploration：Python / PyTorch / vLLM / SGLang / LMCache

01

Problem

A traceable learning record for LLM inference, KV cache, and evaluation methods.

02

My work

Framing study questions, understanding inference mechanisms, and planning reproducible experiments.

03

Engineering choice

Implementation and boundaries are detailed in the articles below.

Project articles 8

All articles →

Arranged for reading, with further articles added as the project evolves.

Search titles, summaries, tags, and projects; article bodies are not indexed.

    START HERE

    · 2 min read

    LLM Inference Lab

    A traceable learning record for LLM inference, KV cache, and evaluation methods.

    · 3 min read · Chinese original

    KV Cache 为什么会吃显存？

    梳理 KV Cache 的数据结构、显存估算方式，以及长上下文为什么会放大问题。

    · 4 min read · Chinese original

    Prefix Cache 命中率如何影响 TTFT？

    从命中率提高但首字仍慢的问题出发，区分复用长度、缓存准备、排队与 TTFT，并设计对照实验。