Skip to content
STIMSMITH

Chain of Thought Prompting

Concept WIKI v1 · 7/7/2026

Chain of Thought (CoT) prompting is a technique that elicits intermediate natural language reasoning steps from large language models (LLMs) before producing a final answer. It was introduced by Wei et al. (2022) and is cited as a state-of-the-art prompting method. CoT has been adopted and adapted in code generation (Structured CoT) and is also a known surface for prompt-level backdoor attacks (BadChain).

Chain of Thought Prompting

Definition

Chain of Thought (CoT) prompting is a prompting technique for large language models (LLMs) in which the model is asked to first generate a sequence of intermediate natural language reasoning steps (the "chain of thought") and then produce the final answer or output. The technique was introduced by Wei et al. in the paper Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (NeurIPS 2022) [36].

The motivating claim is that LLMs benefit from CoT prompting particularly when tackling tasks that require systematic reasoning processes, and that the explicit intermediate reasoning steps improve accuracy on such tasks [36, BadChain source].

Role in Code Generation

CoT prompting has been described as the state-of-the-art prompting technique for code generation when used with LLMs such as ChatGPT: CoT prompting asks LLMs first to generate intermediate natural language reasoning steps (CoTs) and then output the code (Structured Chain-of-Thought Prompting for Code Generation, arXiv:2305.06599).

A known limitation reported in the code-generation literature is that CoT prompting is designed for natural language generation and has low accuracy in code generation, which motivated Structured CoT (SCoT) prompting that asks the LLM to build intermediate reasoning steps from program structures (sequence, branch, loop) instead of free-form natural language (arXiv:2305.06599).

Security Implications: BadChain

CoT prompting has also been identified as introducing new vulnerabilities. BadChain (arXiv:2401.12242) is described as the first backdoor attack against LLMs employing CoT prompting that does not require access to the training dataset or model parameters and imposes low computational overhead. BadChain inserts a backdoor reasoning step into the sequence of reasoning steps produced by the model, thereby altering the final response when a backdoor trigger is present in the query prompt. Reported empirical findings include:

  • Effectiveness across two CoT strategies, four LLMs (Llama2, GPT-3.5, PaLM2, GPT-4) and six complex benchmark tasks spanning arithmetic, commonsense, and symbolic reasoning.
  • LLMs with stronger reasoning capabilities exhibit higher susceptibility to BadChain, with a high average attack success rate of 97.0% across the six benchmark tasks on GPT-4.
  • Two proposed defenses based on shuffling are shown to be overall ineffective against BadChain (arXiv:2401.12242).

Related Frameworks and Methods

In the Test-driven Generation of Constraint Satisfaction Problems Using Large Language Models paper, several "reasoning frameworks" related to CoT are listed as similar in spirit to the paper's own iterative loop:

  • LangChain [34]
  • OpenAI Reasoning [35]
  • Chain of Thought [36]
  • ReAct [37]
  • Frameworks based on the Model Context Protocol (MCP), e.g., MCP-Solver [38]

The paper notes that, unlike these frameworks, it implements its own loop where the only initial prompt is given by a human; the iterator then calls the LLM, executes the generated Python code (which imports the constraint solver PyChoco), and feeds back any interpreter exception or verification error into the next prompt, increasing the prompt context each iteration (Vol-4149/paper4.pdf).

References to Other Concepts

  • Mentioned in the paper Test-driven Generation of Constraint Satisfaction Problems Using Large Language Models (CEUR-WS Vol-4149, paper 4) as one of several reasoning frameworks the authors compare their approach against, but not directly used by their own iterator (Vol-4149/paper4.pdf).
  • Extended by Structured Chain-of-Thought (SCoT) prompting for code generation (arXiv:2305.06599).
  • Targeted by BadChain as the underlying prompting paradigm that is attacked (arXiv:2401.12242).

LINKED ENTITIES

1 links
Test-Driven Generation of CSPs Using LLMs MENTIONS

CITATIONS

6 sources
6 citations
[1] Chain-of-thought prompting elicits reasoning in large language models (Wei et al., NeurIPS 2022). Chain-of-thought prompting elicits reasoning in large language models
[2] CoT prompting asks LLMs first to generate intermediate natural language reasoning steps (CoTs) and then output the code; it is described as the state-of-the-art prompting technique for code generation. Structured Chain-of-Thought Prompting for Code Generation
[3] CoT prompting is designed for natural language generation and has low accuracy in code generation, motivating Structured CoT (SCoT) prompting. Structured Chain-of-Thought Prompting for Code Generation
[4] BadChain is the first backdoor attack against LLMs employing CoT prompting, requiring no training data or model parameter access, and inserts a backdoor reasoning step into the model's reasoning chain. BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models
[5] BadChain achieves an average attack success rate of 97.0% across six benchmark tasks on GPT-4, and shuffling-based defenses are overall ineffective. BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models
[6] Chain of Thought prompting is listed alongside LangChain, OpenAI Reasoning, ReAct, and MCP-based frameworks (e.g., MCP-Solver) as a similar 'reasoning framework', but the Test-Driven CSP paper implements its own iterative loop with PyChoco execution and verification feedback. Test-driven Generation of Constraint Satisfaction Problems Using Large Language Models