Skip to content

🍓Prompt Engineering Steps

Abstract

This section covers prompt engineering steps in details.

🦜 Video lecture for this chapter - Link

Prompt Engineering is the art and science of writing and refining prompts iteratively depending on the generated outputs to finally arrive at the desired output. Here's a typical workflow:

1️⃣ Understand the task

A good understanding of the task forms the foundation for crafting effective prompts that guide the model towards accurate and relevant responses.

Key aspects of understanding the task include:

  • Clearly defining the goal - Articulate exactly what you want the model to do or produce.

  • Identifying the inputs - Determine the information the model will need to process (e.g., text, code, tables).

  • Specifying the desired outputs - Describe the format and content of the expected results.

  • Considering nuances and constraints - Understand any specific requirements or limitations of the task.

2️⃣ Write the prompt

Craft a prompt that provides clear instructions to the LLM about the desired output. The prompt should be specific, concise, and unambiguous. It can include context, instructions, examples, and constraints to guide the LLM's behaviour.

Key elements of a well-written prompt include:

  • Clear task description - Explicitly state what you want the LLM to do or generate.

  • Relevant context and examples - Provide necessary background information and relevant examples to help the model understand the task better.

  • Appropriate tone and style - Employ a tone and style that align with the task and desired response (e.g., formal, informal, creative).

3️⃣ Test and Evaluate the prompt

Pass the prompt to the LLM, evaluate the generated output on various criteria and then analyze any shortcomings in the responses and identify potential areas for prompt improvement.

Evaluating the responses based on various criteria, such as:

  • Relevance - Do the responses directly address the task and prompt?
  • Accuracy - Are the facts and information in the responses correct?
  • Completeness - Do the responses cover all aspects of the prompt?
  • Creativity - Are the responses original and engaging?
  • Fluency - Are the responses well-written and easy to understand?

4️⃣ Refine the prompt

Based on the prompt output evaluation, refine the prompt by including context, instructions, examples, or constraints to guide the LLM's behaviour. Here, the goal is to create a prompt that produces relevant and accurate outputs.

Key strategies for refining prompts include

Addressing shortcomings

  • If responses lack relevance, provide more specific instructions or context.
  • If responses are inaccurate, correct any errors or biases in the prompt.
  • If responses are incomplete, add missing information or examples.

Adjusting language and structure

  • Use clearer and simpler language to enhance understanding.
  • Break down complex tasks into smaller, more manageable steps.
  • Emphasize key points and desired outcomes.
  • Experiment with different sentence structures and word choices.

Incorporating feedback

  • Gather feedback from users or experts to identify areas for improvement.
  • Incorporate suggestions for clarity, relevance, or creativity.

5️⃣ Iterate

After refining the prompt, iterate testing, evaluating and refining the prompt until you get the desired output.

Key elements of the iteration step include

  • Testing the prompt - Use the prompt to generate responses from the language model.

  • Evaluating the responses - Assess the quality of the responses based on factors like relevance, accuracy, completeness, creativity, and fluency.

  • Identifying areas for improvement - Analyze any shortcomings in the responses and pinpoint potential areas for prompt refinement.

  • Refining the prompt - Make adjustments to the prompt's language, structure, examples, or instructions to better guide the model towards desired outputs.

  • Repeating the cycle - Continue testing, evaluating, and refining the prompt until the responses consistently meet the quality standards and achieve the desired outcomes.

To summarize, prompt engineering is an iterative process that involves experimentation, refinement, and evaluation to achieve the desired outputs.