Skip to content

🦅 Raw and Chat LLMs

Abstract

This section covers "Raw and Chat LLMs".

🌅 Raw LLMs

Raw LLMs refer to the LLMs which are trained on a large corpus of text data without any additional fine-tuning or customization for specific tasks or domains. During training, raw LLMs learn to predict the next word in a sequence. They are the foundation on which other, more specialized LLMs like Chat LLMs are built.

Example - Suppose you have a Raw LLM trained on a large corpus of English text. For the input phrase "The sky is", the Raw LLM might generate words like "blue", "clear", "cloudy", etc., based on the patterns it has learned in the training data.

Pros

  • Highly creative and flexible output.
  • Can handle open-ended tasks.

Cons

  • Prone to producing false or nonsensical information because they prioritize sounding plausible over being truthful.
  • Output can be unpredictable and require refining.
  • Struggle to follow the instructions

🌄Chat LLMs

Chat LLMs are built on the top of raw LLMs. They are specifically fine-tuned for conversational tasks such as responding to user queries or engaging in dialogue. They focus on making conversations natural, informative and engaging.

Example - You ask a chat LLM, "What's the weather like today?". It will likely respond with a relevant answer like "It's sunny and warm. Perfect day for a picnic!" rather than a generic or creative response.

Pros

  • Designed for more focused and helpful responses.
  • Better at staying on topic and following instructions.
  • Often incorporate safety mechanisms to reduce harmful output.

Cons

  • Requires additional training (fine-tuning).
  • Still susceptible to occasional factual errors.

In summary, while both raw and chat language models are based on the same underlying transformer model, Raw LLMs are great for open-ended text generation while Chat LLMs are customized and fine-tuned for conversational tasks, allowing them to generate more contextually relevant and engaging responses in conversations.