Introduction to Fine-Tuning in Machine Learning

As AI becomes more useful to businesses and AI agents in particular take on more complex workflows, AI fine-tuning will be an important part of the story. Those agents can answer questions about data and take actions on our behalf—writing code, managing calendars, presenting analytic insights, and so much more. Along the way, AI models will have to work together to investigate data and document stores, collect information, double-check their accuracy, trigger the right actions, and communicate them. The sophisticated AI models involved will be well-versed in general patterns of language, but they’ll also need to be optimized for specific domains with their own parlance, knowledge base, and more. How can the fine-tuning process help an AI model, and future AI agents, get the nuances and details right? We’ll explore aspects of AI model fine-tuning below.

What Is Fine-Tuning?

AI model fine-tuning is the process that data scientists and machine learning (ML) engineers use to adapt a trained ML model to perform better at a specific task. Fine-tuning, for example, might be used to take a general-purpose large language model (LLM), such as Cohere’s Command or Meta’s Llama 2, and make it more conversant in a healthcare setting or a customer service role.

Those general-purpose models, sometimes called “foundation models,” are trained on large, general data sets—such as data from nearly the entire open internet. This helps them learn and understand a broad range of features and patterns. Some have a good handle on language; others can recognize and manipulate multimodal data, such as images, video, and audio as well as text.

Fine-tuning takes this sophisticated model and continues its training on a smaller data set drawn from a specific task or business domain. This gives the model an understanding of the taxonomy, context, and even specialized knowledge in that field.

The idea is to keep the vast knowledge from the model’s original training regimen while giving it a better understanding of the nuances and details of the domain where the AI model will be working. For example, a general-purpose AI model can be fine-tuned to read and discuss medical imaging, or an LLM-driven chatbot that’s become more fluent in local dialects can be fine-tuned to improve its role in customer service. The race to construct highly capable AI agents in a wide range of domains will often depend on fine-tuned models. Here’s a deeper discussion of the methods and mechanics of fine-tuning.

Key Takeaways

• AI fine-tuning is the process of training a general-purpose AI model to give it more context and knowledge for a specific task, such as sentiment analysis, language generation, image detection, and many other workloads.
• Fine-tuning is one of several methods of improving an LLM’s outputs for applications, including AI agents.
• Much smaller training data sets are required for fine-tuning versus training an AI model from scratch.
• After fine-tuning, an AI model can be more conversant and accurate in a specific field, such as medicine, manufacturing, or customer service.

Benefits and Challenges of Fine-Tuning

A primary challenge teams face when undertaking AI fine-tuning is the availability of high-quality, labeled data relevant to their use cases—acquiring or creating such data can be expensive and time-consuming. For example, labeling data can mean hours of noting areas in images that an AI should learn to recognize, or labeling sections of text to highlight key information. Still, fine-tuning offers compelling advantages that make it a crucial technique in modern AI development. Let’s look at some pros and cons.

Pros

• Saves time and money versus training an AI model from scratch: Fine-tuning a foundation model offers several advantages, including considerably reduced training time and lower computational costs, as foundation models have already had extensive general training and need less time and resources to adapt to a new task. Additionally, fine-tuning can be effective even with smaller, task-specific data sets, which can reduce the time and cost associated with data collection and preprocessing. The reduced training time and computational requirements of fine-tuning can also result in lower energy consumption, which reduces costs.
• Delivers better performance for specific tasks: Fine-tuning foundation models can lead to improved accuracy and faster convergence, with convergence being the point where an ML model’s error rate becomes constant, indicating that it won’t continue to improve its accuracy with more training on the same dataset. Foundation models can be quickly adapted to the nuances of a new task with less data and fewer training iterations. This combination of general knowledge and task-specific learning helps the model perform well when it encounters new data.

Cons

• Overfitting: Overfitting in AI fine-tuning happens when the training rounds on new data are too thorough and end up overriding layers that help the model with general knowledge. That general knowledge and language flexibility of the foundation model is critical to helping it perform well when it encounters new data. With overfitting, the model starts to memorize the training data rather than learning the underlying patterns that enable it to generalize so it can handle new examples.
• Cost: The computational resources and the expertise required for fine-tuning AI models can be costly. The process relies on high performance GPUs or TPUs (tensor processing units), which can be expensive to purchase or rent from a cloud provider. It can also be expensive to hire the needed experts in machine learning and natural language processing.
• Time: Fine-tuning can be time-consuming in a couple of ways. First there’s time to prepare the data, which can include collection, cleaning, annotation, and formatting. Once the fine-tuning process is underway, fine-tuning can take hours, days, or even weeks, depending on the size and complexity of the task. More time translates to higher costs for compute time.

How Fine-Tuning Works

AI fine-tuning works by using existing models as starting points. These foundation models, such as Meta’s Llama 2 or Cohere’s Command, are commonly available from cloud providers. Many organizations will bring other models to their data science platforms from centralized repositories, such as Hugging Face, TensorFlow Hub, and PyTorch Hub, that host pretrained machine learning models.

Start with a Pretrained Model

Choose a model that’s well suited to your task, whether that’s classifying text, analyzing sentiment, answering questions, writing articles, generating code, detecting objects, or any other job that requires AI or ML.

Fine-tuning an AI model requires three basic ingredients: a collection of properly formatted data, the right foundation model, and an infrastructure that provides the neural networks for deep learning and the GPUs to power the training regimen. These resources are often assembled in a data science platform or, more recently, a generative AI cloud service.

As often happens with technology, the AI fine-tuning process has gotten easier over time, thanks to new tools and infrastructure services from open source projects and cloud-provider development teams. These tools and services are helping automate fine-tuning—including complex tasks we’ll discuss, such as hyperparameter optimization, model selection, and data preprocessing. That’s making the process more accessible to nonexperts.

In fact, industry observers have noted that tools have gotten so good at abstracting away the data science minutia of fine-tuning that the hardest part is now collecting and formatting the best possible data set. Here, too, there are libraries of data ready-made for various domains, such as healthcare and finance, and capabilities, such as computer vision, sentiment analysis, or anomaly detection. In fact, a growing trend is to use a model for your use case that’s already been fine-tuned for that task using these libraries. From there, the organization might further fine-tune using a smaller data set and, perhaps, use a retrieval-augmented generation, or RAG, architecture to further improve AI outputs.

Companies have found ways to supplement traditional fine-tuning methods with a technique called reinforcement learning. This allows AI models to learn through trial and error and self-improvement rather than using a separate process of data set labeling and supervised fine-tuning.

How to Fine-Tune an AI Model in Four Steps

1. Use a pretrained model: The first step is to select a foundation model that’s appropriate to the task. There are popular models for natural language processing, computer vision, text generation, and other areas.

2. Add new data: Next, gather and prepare a task-specific data set. These might consist of labeled customer reviews or sample questions and answers in the domain your model is meant to address.

3. Adjust: Step three is to adjust the model as needed. Adjustments might include freezing layers to preserve the model’s previous learning; adjusting the learning rate, which can also help preserve the model’s existing knowledge; and adding layers where completely new tasks are learned, such as a classification layer for a text classification or a regression layer for predictions.

4. Train the model: This process involves feeding the new data through the model and updating the model’s parameters. The goal is to refine the model’s performance of your task while keeping the general knowledge from its initial training.

Techniques and Methods in Fine-Tuning

When fine-tuning a model, there are a range of techniques available to choose from. The first decision is whether you need full or selective fine-tuning.

Full Fine-Tuning

Full fine-tuning is a process where all the layers and parameters of a foundational model are updated during the training process. This is a good choice when you have a large and diverse data set that can adequately update the model’s parameters without danger of overfitting.

Selective Fine-Tuning

Selective fine-tuning involves updating only a subset of the model’s layers or parameters using a smaller data set. This method is good for preserving the general knowledge of the foundation model and lowering the time and compute cost of the training regime. Here are examples of techniques for selective fine-tuning.

• Data augmentation: This involves generating additional training data from your existing data set by applying transformations. For images, transformations might include rotation, scaling, cropping, or adding noise. This can improve fine-tuning performance when a task-specific data set is small.
• Early stopping: This technique is used to monitor and stop training when performance on a validation set stops improving. This helps with efficiency and can help prevent overfitting.
• Ensemble methods: This method combines several fine-tuned models in an attempt to reduce variance in model outputs.
• Fine-tuning specific layers: This technique makes fine-tuning more efficient by adjusting only the shallower layers of a deep neural network while locking or freezing deeper layers.
• Hyperparameter tuning: This includes various techniques to adjust features of a fine-tuning process, such as learning rate or batch size.
• Layer freezing: Here, layers in the foundation model that are responsible for basic and universal understanding are locked. This helps prevent overfitting, where the model has essentially memorized the training data rather than learning generalizable patterns that allow it to correctly handle new data.
• Learning rate scheduling: A slower learning rate can allow for more subtle and precise adjustments to the foundation model’s parameters. The process gradually reduces the learning rate, or step size, over time, helping the model learn new material more effectively by taking smaller steps as it approaches the optimal solution.
• Regularization techniques: These are used specifically to prevent overfitting.
• Transfer learning: This involves further training a foundation model on a smaller data set that’s specific to a certain task.

Fine-Tuning vs. Training from Scratch

In some use cases, it makes sense to design and train your AI model from scratch. In most cases, however, an organization can get the desired result by fine-tuning a foundation model.

Training an AI model from scratch can be the better choice in a few instances. One example is when the domain you’re working in is very exacting, such as a niche medical application with perhaps very specific image data. Training from scratch requires you to assemble a large data set and run long rounds of training on AI-specific infrastructure. This can be expensive, requiring up to thousands of GPUs and millions of dollars. It’s also a task where the expertise of data scientists and ML engineers will be needed.

Fine-tuning an AI model, on the other hand, involves taking a foundation model and adapting it to a specific task using a smaller, task-specific data set. This process is often faster and more efficient because the model starts with a strong foundation in language and general knowledge and only needs to adjust to the nuances of the new task. This can improve the AI model’s performance for your needs with less work assembling and preparing data and many fewer rounds of training versus training from scratch. See more on the benefits of fine-tuning below.

Common Use Cases for Fine-Tuning

A wide range of companies across industries, from startups to multinational corporations, are fine-tuning pretrained models. In fact, it’s becoming a standard practice for any organization looking to use AI because it lets them deploy systems tailored to their specific needs without the massive investment required for full-scale model training. Here are some use case examples.

Real-World Examples

Organizations working in finance, logistics, healthcare, and many other domains are bringing fine-tuned generative AI to their daily operations. These real-world stories will help you explore a wide range of live use cases. Here are three specific examples:

• Customer support: An industrial-safety solutions company implemented an AI-powered tool that immediately improved its customer support operations. The team uses fine-tuned Cohere Command R+ LLM to process customer inquiries and generate accurate responses.
• Healthcare: University researchers are using AI to take a deeper look at patient data to help design new cancer therapies. They’re using cloud infrastructure to host, run, and fine-tune LLMs at scale.
• Service industry: An applied AI firm in Brazil fine-tunes LLMs to assist with reminiscence therapy, a proven psychological treatment that helps patients suffering from cognitive decline.

What’s Next in Fine-Tuning

AI fine-tuning is evolving rapidly, especially as more AI agents depend on fine-tuned models. The future promises more automation, new techniques, and a wider array of model choices that can help organizations adapt AI models to their needs.

These innovations include improved ML services that automate fine-tuning, including hyperparameter optimization, model selection, and data preprocessing. We expect that context-aware data augmentation techniques will help the model learn more relevant features more quickly, and dynamic learning will allow a model to adjust its learning rate on the fly. And look for ML and foundation model builders to continue to release more versatile and powerful models that can transfer knowledge across different modalities and can be fine-tuned to perform tasks that require understanding multiple types of data. The trick will be to build a data infrastructure that’s versatile enough to take advantage of these new innovations when they arrive.

Enhance Your AI Models with Generative AI Solutions

Did you know that Oracle Cloud Infrastructure (OCI) gives you access to what you need to fine-tune machine learning and foundation models? Oracle provides the physical infrastructure, data processing infrastructure, data science platform, and generative AI services in fully managed services. OCI’s generative AI service, for example, offers a simple and seamless integration with versatile LLMs in an easy-to-use service. Use it to fine-tune models for a wide range of use cases, including writing assistance, summarization, analysis, and chat.

Meanwhile, your data scientists and ML engineers can leverage Oracle’s data science platform to collaborate on building, training, and deploying ML models using Python and open source tools. A JupyterLab-based environment provides all the resources needed to experiment, develop models, and scale up model training with NVIDIA GPUs and distributed training. Take models into production and keep them healthy with ML operations capabilities, such as automated pipelines, model deployments, and model monitoring.

Fine-Tuning FAQs

How is fine-tuning different from other types of model training?

Fine-tuning AI models is different from training AI models from scratch. It’s the process of adding rounds of training to a sophisticated foundation model, leading to more context-relevant outputs in a specific task. Fine-tuning often requires less data and is less time-consuming and costly than building and training a model from scratch.

Can fine-tuning be used with any type of model?

AI fine-tuning can be used with foundation models in a wide range of use cases, whether that’s image recognition, text classification, language generation, audio outputs, or other outputs.