Skip to content

🪈 Batch Prompting

Abstract

This section covers "Batch Prompting".

🦜 Video lecture for this chapter - Link

✒️ Overview

Batch prompting in the context of LLMs refers to the process of submitting multiple prompts to the model in a single batch, rather than individually. This approach is often used when dealing with a large volume of similar tasks or queries. This speeds up the response time and also ensures consistency in the generated responses, as the model processes all prompts in the batch under the same context and state of knowledge.

✒️ How it works

Instead of one question, you have a list of questions or tasks. Batch all prompts and give them at once to the LLM. The LLM processes all prompts and generates responses for each prompt in one batch.

🍿 Example

Assume that you have to generate product descriptions for different items. Instead of using individual prompts which will consume a lot of time, you can use batch prompting having multiple prompts.

Prompt 

- Describe a blue cotton t-shirt with a graphic print.
- Describe a pair of black leather boots with zippers.
- Describe a silver metal necklace with a pendant.
- Describe a wooden coffee table with a glass top.
- Describe a pink floral dress with ruffles.

Constraints: Make sure that the description is short and consists of a maximum of 20 words.

Output 

1. Blue cotton t-shirt featuring a vibrant, colorful graphic print on the front.
2. Black leather boots, ankle-high with stylish side zippers and a sleek finish.
3. Elegant silver metal necklace with a delicate, intricately designed pendant.
4. Wooden coffee table with a modern, sleek glass top and sturdy legs.
5. Pink floral dress adorned with ruffles, light and airy, perfect for spring.

✒️ Pros

  • Speed - Faster than handling prompts one by one, especially for similar tasks.
  • Scalability - Can handle large volumes of prompts simultaneously.
  • Insights - Easier to compare and analyze outputs.

✒️ Cons

  • Less control -  Generic prompts may lack nuance for individual cases.
  • Error amplification - Shared errors can affect the outputs of all the prompts.
  • Limited tasks - Suitable only for tasks which are similar in nature.

To summarize, if you have a bunch of similar tasks, you can use batch prompting to send multiple prompts to an LLM in one go which will save you a lot of time.