Skip to content

🏏 Plan and Solve Prompting

Abstract

This section covers "Plan and Solve Prompting".

✒️ Overview

CoT prompting considerably enhances the reasoning abilities of LLMs. However, both few-shot CoT and zero-shot prompting have their own drawbacks. Few-shot CoT enhances the reasoning abilities by providing examples with reasoning chains, however crafting reasoning chains manually is laborious.

Zero-shot CoT enhances reasoning abilities just by appending the magic phrase “Let’s think step by step”. Although zero-shot CoT avoids manually crafting examples, zero-shot CoT is prone to errors like missing steps or misunderstanding.

Plan-and-Solve (PS) Prompting technique involves dividing tasks into sub-tasks and carrying them out according to a plan. This approach is further enhanced with more detailed instructions, creating "PS+ Prompting." PS prompting consistently outperforms Zero-shot-CoT and performs comparably or better than other prompting methods like few-shot CoT.

✒️ How it works

PS prompting consists of two components: first, devising a plan to divide the entire task into smaller subtasks, and then carrying out the sub-tasks according to the plan.

  1. Plan formulation - The LLM is instructed to analyze the problem and create a plan, breaking it down into smaller, more manageable subtasks. Here the phrase “Let’s first understand the problem and devise a plan to solve the problem. Then, let’s carry out the plan and solve the problem step by step” is added to the original prompt.
  2. Plan execution -  With the plan established, the LLM works through each subtask in sequence, performing necessary calculations, applying appropriate reasoning, and reaching the final solution.

To further improve the quality of generated reasoning steps, more detailed instructions are added to PS prompting. Specifically, the instructions “extract relevant variables and their corresponding numerals” and “calculate intermediate results (pay attention to calculation and commonsense)” are added. This new prompting variant is referred to as “PS+ prompting” strategy.

✒️ Pros

  • Enhanced reasoning - Guides LLMs through structured logic, improving multi-step task accuracy.
  • Increased transparency - Step-by-step approach clarifies reasoning process, facilitating understanding and debugging.
  • Reduced manual effort - No need for pre-made examples, only the problem statement, making it "zero-shot."

✒️ Cons

  • Potential for misinterpretation - LLM freedom in plan formulation may lead to errors or irrelevant steps.
  • Increased computational cost - Additional reasoning and planning demands more resources compared to simpler methods.
  • Limited task applicability - Structured approach may not be effective for all tasks, particularly those not requiring planning.

To summarize, PS prompting addresses the issues with CoT prompting and enhances the reasoning abilities of LLMs by dividing tasks into sub-tasks and carrying them out according to a plan.

📝 Note: The above images are from "Plan and Solve Prompting" Paper.