What Is Weaviate? A Semantic Search Database

Just as generative AI is remaking how users interact with computers, it’s placing new demands on the databases that store and retrieve information underlying the systems’ answers. Vector databases, designed for developers working with very large unstructured or semi-structured data sets, store the embeddings, or positions in mathematical space of potentially billions of vectors calculated from the raw data. The databases can return answers that enhance AI models’ knowledge by rapidly computing relationships among vectors.

Unlike traditional databases, which excel at finding exact query matches, vector databases, such as Weaviate (as well as the likes of Milvus, Pinecone, Chroma, and Qdrant), represent similarity among stored objects by their distance from one another in a multidimensional vector space. The databases can find related information quickly and return answers based on what models presume a user means, even if they’re approximately close and not an exact match for a query.

That makes Weaviate and other vector databases powerful natural language processing tools for applications such as recommendations for ecommerce and streaming services, semantic or similarity searches, fraud detection, and retrieval-augmented generation (RAG).

What Is Weaviate?

Weaviate, owned by the Dutch startup of the same name, is a specialized vector database that excels at rapid searches across billions of vectors. Its search architecture uses database sharding, a technique that spreads information across many servers, to increase scalability and performance.

Weaviate uses different types of database indexes to improve performance depending on the size of the data set. Its flat index stores vector locations and is suited for small data sets. Its HNSW (hierarchical navigable small world) index is slower to build but a better fit for large data sets.

Weaviate emphasizes developer productivity by offering a range of tools and interfaces, including client libraries in Python, JavaScript, Go, and Java, and an API for the GraphQL query language. The database is distributed under the open source BSD-3-Clause license. Its strength is approximate nearest neighbors (ANN) searches, whose results enhance generative AI queries.

Customers can run Weaviate on their own computers, in a public cloud service in a serverless model, or through the company’s managed cloud service with dedicated hardware.

How Oracle Outperforms Weaviate in Availability and Scalability

Oracle Database 23ai also provides AI vector search, which lets it store and search vectors as Weaviate does, with subsecond response times. However, instead of storing and retrieving data from a single-purpose vector database, development teams using Oracle Database 23ai can store and search relational, graph, spatial, JSON, and Internet of Things data—along with vector search. With the Oracle database, IT groups don’t need to manage multiple databases and copies of data, leading to less complexity.

While Weaviate supports clustering for improved database availability, Oracle Database 23ai offers a wider range of high-availability options, including Oracle Real Application Clusters, which let database instances run on separate servers for fault tolerance and load balancing; Oracle Active Data Guard, for immediate replication to a standby database so no data gets lost during an unplanned outage; and Oracle GoldenGate, for data consistency across multiple sites. For scalability, Oracle Globally Distributed Database supports sharding for vector searches; routing queries; and update, insert, and delete operations to the shards.

Speciality vector databases can rely on periodic batch updates for fast-changing data, which can lead to inconsistent data sets. With Oracle Database 23ai, the RAG pipeline and underlying data sources stay in synch. When it comes to queries, Oracle Autonomous Database Select AI turns natural language queries into SQL commands, using LLMs from Anthropic, Cohere, Google, Meta, Microsoft, and OpenAI. Oracle Database 23ai also lets developers tap AI services from other large cloud providers, including Google’s Gemini models, Microsoft’s Azure AI services, and OpenAI’s LLMs, for use with enterprise data stored in Oracle.

Weaviate FAQs

How does Weaviate handle semantic and vector search?

The Weaviate database enables low-latency searches of vector embeddings constructed from unstructured data for a variety of applications. Those include recommendation engines and RAG, as well as hybrid searches that combine semantic understanding of relationships among text, images, audio, and video with conventional keyword searching.

What is the difference between Pinecone and Weaviate?

Weaviate ships with a BSD-3-Clause license and can run on-premises or in a public cloud service. Pinecone is distributed under a proprietary license and runs in the cloud (as well as via a link to an AWS private cloud). Pinecone supports a wide range of data types, while Weaviate can be customized because of its open source license.

Can Weaviate be used with NLP models?

Yes, Weaviate lets users run and scale their own natural language processing models, in addition to using models that ship with the software.

What sets Weaviate apart from other vector databases?

Weaviate focuses on attracting developers to its product through support for a variety of programming languages and AI tools. Its architecture is built for horizontal scalability through data sharding.