How to Create a Powerful Knowledge Base Chatbot with Unstructured Data

Introduction

Chatbots aren’t new, but better chatbots are now available, thanks to AI services. Searching documents with these better chatbots is greatly enhanced with vector search instead of traditional keyword searches. And while training models on unstructured data may seem daunting, using retrieval-augmented generation can help to create a savvier chatbot that’s trained with new information.

This tutorial demonstrates how to train a chatbot on a rich knowledge base of several types of documents, PDF files, and data stored in database tables. The tutorial will show you how to transform raw documents containing unstructured data into structured data, store them in an Oracle Cloud Infrastructure (OCI) Object Storage bucket, and utilize advanced AI models to generate contextual responses from natural language queries. Best of all, it’s highly modular, so you can use a variety of models.

Demo

Prerequisites and setup

1. Oracle Cloud account—sign-up page
2. Oracle Cloud Infrastructure—documentation
3. Oracle Database 23ai—documentation
4. OCI Generative AI—documentation

Getting started

• Detailed steps and sample code on GitHub