What Is Semi-Supervised Learning?

Semi-supervised learning is a form of machine learning that involves both labeled and unlabeled training data sets. As inferred by its name, this method incorporates elements of both supervised learning and unsupervised learning. Semi-supervised learning uses a two-step process. First, a project’s algorithm is initially trained using a labeled data set, as in supervised learning. After that, the algorithm moves forward by training with an unlabeled data set.

Semi-supervised learning is ideal when projects have a lot of training data, but most or all of it is unlabeled. In the case of projects with only unlabeled data available, semi-supervised learning can get projects up and running by doing initial training with manually labeled data before switching to solely unlabeled training data. With projects using this approach, teams must take care when manually labeling data because it becomes the foundation on which the rest of the project is built.

The decision to use semi-supervised learning often comes down to the available data sets. In the big data era, unlabeled data is far more available and accessible than labeled data, and depending on the source, it will cost less to obtain.

Still, a project may have to forge ahead with only unlabeled data. When this happens, teams must decide whether it’s useful to employ the exploratory nature of unsupervised learning versus spending the time and money to label part of the data set as a means of initial algorithm training.

What Is Semi-Supervised Learning?

Semi-supervised learning is a machine learning technique that sits between supervised learning and unsupervised learning. It uses both labeled and unlabeled data to train algorithms and may deliver better results than using labeled data alone.

To decide if semi-supervised learning is appropriate for a project, teams should ask questions including the following:

• What data sets are available to us for this project?
• Are any of these data sets labeled? An example of labels for a financial data set might be transaction data with labels indicating whether a transaction is fraudulent or legitimate.
• If data sets are all unlabeled, does the team have the resources to label at least some data?
• Are the project’s goals more achievable via supervised or unsupervised learning? Factors to weigh here include a mix of practical and technical, including compute resources, budget, deadlines, and desired outcomes.
• Is our labeled data set sufficient to teach the model the patterns and characteristics of, for example, fraudulent and legitimate transactions?

The answers to these questions will determine feasibility. Once the decision is made to go with semi-supervised learning, the next step is to prepare two training data sets. The first is generally a small labeled data set to anchor the project’s foundational training. The second training data set is larger—often much larger—and unlabeled. When the system processes the unlabeled data set, it generates pseudo-labels using what it learned from the labeled set. This process then iterates to refine the algorithm and optimize performance.

The most common types of semi-supervised learning are:

• Self-training: With self-training, the process uses the labeled data set to train the algorithm, then subsequent training generates high-confidence (more than 99% probability) pseudo-labels for the unlabeled data set such that all records have labels. Then, the system trains on the expanded data set featuring the original labeled training data concatenated with the unlabeled data set using pseudo-labels, allowing for training on greater volumes of data compared to the original labeled data set.
• Co-training: With co-training, the process takes a small labeled data set and approaches it with two distinct views (feature groups) focusing on complementary and independent information. Each group trains a separate algorithm, then proceeds to make predictions on an unlabeled data set to classify pseudo-labels for each resulting model. Each pseudo-label generated by a classifier (an algorithm that predicts a label) comes with a probability score, and the pseudo-label with the higher probability score is then added into the other training data set.

For example, a weather forecasting model may start with a data set using labels on recorded metrics, such as wind speed, atmospheric pressure, and humidity, while the other model uses more generalized data, such as geographic location, date/time, and recorded average precipitation. Both models generate pseudo-labels, and when the metrics model has a higher probability score than the general model, that pseudo-label is applied to the general model, and vice versa.

Each method continues training to refine areas with low-probability outcomes until a comprehensive final model is produced.

Semi-Supervised Learning Pros and Cons

Semi-supervised machine learning combines the structure of launching a project using supervised learning with the benefits of unsupervised learning, such as advanced anomaly detection and the ability to uncover hidden patterns and structures within unlabeled data. While not appropriate for every situation, its inherent flexibility makes it a feasible option for a wide spectrum of project needs and goals.

Semi-Supervised Learning FAQs

In what situations is semi-supervised learning typically used?

Semi-supervised learning works best when projects have access to only or mostly unlabeled data. In those circumstances, teams can manually label a subset of data to create the training data set for the first step, then allow the model to explore the unlabeled data set.

What’s the difference between semi-supervised and unsupervised learning?

Unsupervised learning allows models to explore unlabeled data sets with the goal of discovering patterns and relationships between inputs and outputs on its own. Semi-supervised learning uses this method, but with a precursor step of training the algorithm on a small labeled data set to build a foundational direction for the project.

What are some pros and cons of semi-supervised learning?

Pros of semi-supervised learning include:

• It uses both labeled and unlabeled data sets.
• There are better capabilities for unstructured data, such as heavy volumes of text, video, or audio.
• It uses more readily accessible and less expensive unlabeled data sets.
• Improved model performance, especially with limited data.

Cons of semi-supervised learning include:

• It may require time and money to manually label a training data set.
• There’s potentially lower accuracy and transparency compared with supervised learning with quality labeled data sets.
• It’s unsuitable for some types of projects, such as those with strict guidelines or that require high accuracy standards for safety.
• Not well-suited to complex, diverse data sets.