T3Mermaid to ASCII art (mermaid-ascii)T3Kimi K3, and what we can still learn from the pelican benchmarkT3Firefox in WebAssemblyT3Spot birds not golfT3[AINews] Kimi K3 2.8T-A50B: the largest open model ever released; Opus 4.8-class at Sonnet 5 pricingT1From physical surfaces to human-centric heat stress: LST and UTCI heat mapping reveals nonlinear effects of urban morphologyT1DualHNIE: Dual-Channel Hypergraph Learning for Node Importance Estimation in Heterogeneous Knowledge GraphsT1GenTL: A General Transfer Learning Model for Building Thermal DynamicsT1A short review on the maximum clique problem algorithms with classical, AI, and quantum methodsT1Man, Machine, and Masterpiece: Artistic Ownership in the AI EraT1HABIB_TAZ at SemEval-2026 Task 11: Disentangling Formal Logic from Content via Synthetic Training and Multi-Objective OptimizationT1How Well Does AI-Generated Feedback Work? Intrinsic and Extrinsic Evaluation across more than 20,000 EFL Essay DraftsT3Mermaid to ASCII art (mermaid-ascii)T3Kimi K3, and what we can still learn from the pelican benchmarkT3Firefox in WebAssemblyT3Spot birds not golfT3[AINews] Kimi K3 2.8T-A50B: the largest open model ever released; Opus 4.8-class at Sonnet 5 pricingT1From physical surfaces to human-centric heat stress: LST and UTCI heat mapping reveals nonlinear effects of urban morphologyT1DualHNIE: Dual-Channel Hypergraph Learning for Node Importance Estimation in Heterogeneous Knowledge GraphsT1GenTL: A General Transfer Learning Model for Building Thermal DynamicsT1A short review on the maximum clique problem algorithms with classical, AI, and quantum methodsT1Man, Machine, and Masterpiece: Artistic Ownership in the AI EraT1HABIB_TAZ at SemEval-2026 Task 11: Disentangling Formal Logic from Content via Synthetic Training and Multi-Objective OptimizationT1How Well Does AI-Generated Feedback Work? Intrinsic and Extrinsic Evaluation across more than 20,000 EFL Essay Drafts

Adversarial Attacks on LLMs

Named practitioner synthesis

#237

T3digested

Canonical URL: https://lilianweng.github.io/posts/2023-10-25-adv-attack-llm/
Topic: Safety Security
First seen: 2026-07-16 19:08:00
Last seen: 2026-07-16 19:08:00

Source raw items (1)

Blog / Newsletter2026-07-16 19:07:22
Adversarial Attacks on LLMs
<p>The use of large language models in the real world has strongly accelerated by the launch of ChatGPT. We (including my team at OpenAI, shoutout to them) have invested a lot of effort to build default safe behavior into the model during the alignment process (e.g. via <a href="https://openai.com/research/learning-to-summarize-with-human-feedback">RLHF</a>). However, adversarial attacks or jailbreak prompts could potentially trigger the model to output something undesired.</p> <p>A large body of ground work on adversarial attacks is on images, and differently it operates in the continuous, high-dimensional space. Attacks for discrete data like text have been considered to be a lot more challenging, due to lack of direct gradient signals. My past post on <a href="https://lilianweng.github.io/posts/2021-01-02-controllable-text-generation/">Controllable Text Generation</a> is quite relevant to this topic, as attacking LLMs is essentially to control the model to output a certain type of (