🧱 LLM Pretraining and Meta-Training

Abstract

This section covers LLM training phases: "Pretraining" and "Meta-Training".

Large Language Models (LLMs), the recent buzz in Artificial Intelligence, have garnered a lot of attention in both academic and industry circles with their remarkable performances in most of the natural language processing (NLP) tasks. These models are essentially deep learning models, specifically transformer-based, pretrained on large volumes of text data and then aligned to human preferences using meta-training.

LLM Training.

Pretraining

Pretraining is a fundamental concept in the context of Large Language Models (LLMs). It involves training the model on a massive amount of text data. Pretraining is what gives these models their ability to generate human-like text.

Meta-Training

Pretraining provides LLMs with knowledge, and the ability to generate human-like text. However, they are not explicitly trained to follow the instructions. Meta-training which explicitly involves instruction following training provides LLMs with the ability to understand the user instructions and accordingly generate the desired output.

Here's an intuitive example to explain pretraining and mea-training:

Imagine you want to teach a robot to be a helpful assistant in your kitchen. You could use pretraining and meta-training to achieve this:

1. Pretraining :

   First, you expose the robot to a vast library of cookbooks, cooking blogs, and recipes. It reads and learns from this large volumes of text, gaining knowledge about various ingredients, cooking techniques, and the language used in cooking. This step is similar to how LLMs are pretrained on a diverse range of text from various sources like Wikipedia, Books, websites, social media platforms etc.

2. Meta-training :

   Once the robot has been pretrained, you train it to follow the instructions i.e., understand the given instructions and act strictly according to them. Meta-training involves giving incentives when it acts according to the instructions and penalties when it doesn’t follow the instructions. At the end of meta-training the robot learns to understand the given instructions and act accordingly.