Advanced Prompting: Chain-of-Thought & Beyond

Chain-of-Thought (CoT) Prompting

Chain-of-thought prompting asks the model to show its reasoning step by step before arriving at a final answer. This simple technique can dramatically improve accuracy on math, logic, and multi-step reasoning tasks.

The key phrase is "think step by step" or "let us work through this." When a model reasons out loud, it catches errors it would otherwise make when jumping straight to an answer.

Research shows CoT improves accuracy on math problems from roughly 60% to 90% on benchmarks. The improvement is largest for complex, multi-step problems where the model needs to track intermediate results.

Zero-Shot CoT vs Few-Shot CoT

Zero-shot CoT simply adds "think step by step" to your prompt. It works surprisingly well and requires no examples.

Few-shot CoT provides one or two worked examples showing the reasoning process, then asks the model to apply the same approach to a new problem. This is more reliable because the model can see exactly what kind of reasoning you expect.

Use zero-shot CoT for quick tasks where you just need better accuracy. Use few-shot CoT when the reasoning pattern is unusual or domain-specific — for example, legal reasoning, medical differential diagnosis, or financial analysis where the steps follow a specific methodology.

Tree-of-Thoughts (ToT)

Tree-of-thoughts extends chain-of-thought by exploring multiple reasoning paths simultaneously. Instead of one linear chain, you ask the model to consider several approaches and evaluate which one is most promising.

This is particularly useful for problems with multiple valid approaches: "Consider three different strategies for solving this problem. For each, outline the approach, execute it, and evaluate the result. Then select the best answer."

ToT is powerful for creative problem-solving, strategy decisions, and complex optimization. The trade-off is more tokens and longer responses — reserve it for high-stakes decisions where accuracy matters more than speed.

Self-Consistency

Self-consistency runs the same prompt multiple times and takes the majority answer. This is like polling multiple experts — if 4 out of 5 runs give the same answer, it is likely correct.

You can simulate self-consistency in a single prompt: "Solve this problem using three different methods. If all three give the same answer, report it as final. If they disagree, explain the discrepancy."

This technique is most valuable for math, factual questions, and any task with a single correct answer. It is less useful for creative or subjective tasks.

Step-Back Prompting

Step-back prompting asks the model to first identify the general principle behind a question before answering the specific question. It is like an expert who pauses to think about the bigger picture before diving into details.

Instead of directly asking "What happens when you increase the learning rate in gradient descent?", first ask "What are the general principles governing learning rate selection in optimization?" The model establishes the theoretical framework, then applies it to the specific question.

Step-back prompting improves accuracy on science, engineering, and technical questions where understanding the underlying principle leads to better specific answers.

When to Use Which Technique

Simple factual questions — zero-shot is fine, no advanced techniques needed.

Math and logic problems — chain-of-thought is the default. Add self-consistency for critical calculations.

Complex decisions with trade-offs — tree-of-thoughts to explore multiple options.

Technical or scientific questions — step-back prompting to ground the answer in principles.

High-stakes outputs — combine CoT with self-consistency for maximum reliability.

The overhead of advanced techniques is only worth it when accuracy matters. For casual conversation or creative tasks, these techniques add complexity without benefit.