Learning desk

MultiAgent EDU StackGather good sources. Teach what matters.
T5TauricResearch/TradingAgentsT5A Man Who Invented Modern AI (Before Everyone Else) – Jürgen Schmidhuber [video]T5GPT-4 finished training four years ago todayT5AI Settles a 25 Year-Old Problem We Left BehindT5What it was like working on LLMs and security at Meta (2022-2026)T5Ask HN: How do you go from writing code to deploying with agents?T5What Happened: OpenAI and HuggingFaceT5Apple says Mac users in China can connect to Alibaba's Qwen AI serviceT5Show HN: Try Benzi – A coding harness/agent beating Claude Code itself on SonnetT5The AI Apocalypse Is HereT3Auto mode is now the default in Claude Code for Pro, Max, and Team plansT5Show HN: Tura – Build agent that uses 80% less token and delivers better resultsT5TauricResearch/TradingAgentsT5A Man Who Invented Modern AI (Before Everyone Else) – Jürgen Schmidhuber [video]T5GPT-4 finished training four years ago todayT5AI Settles a 25 Year-Old Problem We Left BehindT5What it was like working on LLMs and security at Meta (2022-2026)T5Ask HN: How do you go from writing code to deploying with agents?T5What Happened: OpenAI and HuggingFaceT5Apple says Mac users in China can connect to Alibaba's Qwen AI serviceT5Show HN: Try Benzi – A coding harness/agent beating Claude Code itself on SonnetT5The AI Apocalypse Is HereT3Auto mode is now the default in Claude Code for Pro, Max, and Team plansT5Show HN: Tura – Build agent that uses 80% less token and delivers better results
← Dispatches

TELLER: Non-intrusive Cross-Layer Root-Cause Analysis for LLM Inference

Primary research

#1245

T1new
Topic
unassigned (set during synthesis)
First seen
2026-08-04 07:15:58
Last seen
2026-08-04 07:15:58

Source raw items (1)

  • arXiv2026-08-04 07:15:21
    TELLER: Non-intrusive Cross-Layer Root-Cause Analysis for LLM Inference

    Large language model (LLM) inference has evolved from an offline workload into a continuously operated software service, yet root-cause analysis remains difficult because a single request spans the inference engine, Python/C++ backend, host CUDA APIs, GPU kernels, and distributed communication. Existing profilers expose raw timelines, while log-based diagnosis often misses cross-layer execution semantics and request-level structure. We present TELLER, a non-intrusive Trace- and Log-aware LLM inference Root-cause analysis framework. TELLER first collects NVTX/CUPTI traces and service logs without modifying model binaries, then reconstructs per-request call-chain trees and aligns log lines with the corresponding execution steps. We introduce a dependency-aware causal-context slice that preserves parent-child structure, temporal order, and communication relations, and a Trace Pair Encoding (TPE) tokenizer that compresses such slices into compact structural token sequences with parent, depth, and duration attributes. On top of these representations, TELLER combines numeric candidate localization with a multimodal root-cause model that jointly predicts abnormal steps, localizes suspicious operators, and generates natural-language explanations. Experiments on multi-node GPU inference workloads show a clear compression-accuracy trade-off: a moderate TPE vocabulary reduces per-step trace length by more than 80% while achieving the best overall performance on both horizontal (cross-node communication) and vertical (within-node execution stack) views, whereas more aggressive compression substantially degrades diagnosis quality. Further analyses under low-fault priors, strengthened baselines, modality ablations, explanation-quality checks, and tracing overhead show that TELLER provides a practical triage and evidence-localization substrate for LLM inference RCA.