Oracle Private AI Services Container FAQ

The first release of the container enables vectors to be created via REST calls using the OpenAI API. Later releases are planned to enable vector index offload with NVIDIA GPUs, and to run open weight Large Language Models powered by vLLM.

You use REST APIs to communicate with the container.

The OpenAI API (REST API) is used to create vector embeddings and list models.

The first release of the container does not require a GPU to run.

Later releases will optionally be able to use GPUs.

The 3rd party licenses for the Private AI Services Container can be found here.

Information about logging can be found here.

The standard port for HTTP is 8080.

The standard port for SSL/TLS is 8443.

You can configure alternative ports for HTTP and TLS.

The config.json config file is optional. The config.json file is needed for more advanced configurations. These environment variables can also be used with the config.json file.

The list of the vector embedding models that are included with the container can be found here.

The container can work with any Oracle ONNX Pipeline model. The ONNX Pipeline enables embedding models based on the Transformers or Sentence Transformer Libraries to be converted into the ONNX format.  The ONNX Pipeline models can be run using either the Oracle AI Database 26ai, or the Oracle Private AI Services Container.  More information about creating pretrained models in ONNX Pipeline format can be found here.

A list of the embedding models, including text, image, and reranker models, that are known to work in the container can be found here.

There is no best vector embedding model. The model that you decide on will depend on your use case and your data.

Consider the following factors when choosing your embedding model(s): Are you processing English text only? Are you processing text for non-English languages? Are you processing image data? Are you processing video data? Are you processing DNA data? Are you using text to search for images? Are you using images to search for text? Is the quality of the search the most important factor? Is the speed of the search the most important factor? Do you need a good blend of quality and performance? Do you have a corporate standard embedding model that needs to be used?

A list of compatible runtimes can be found here.

The REST requests used to create vector embeddings are stateless, so you can use as many containers as you want. Multiple machines can be used.  Also multiple containers can be run on the same host as long as each container listens on a different TCP port. A load balancer can be used to route the REST requests to a container. The Oracle Database Operator for Kubernetes 2.1 and later enables deployments with multiple pods and a load balancer.

Using multiple containers with a load balancer makes high availability transparent to the REST clients. If a load balancer is not used, high availability is still possible, but it will not be transparent to the client as you will need to know the relevant TCP ports to route the REST requests to.

The container should not run on the same machine as the Oracle AI Database Server.   The container should run on a Linux machine that is close to the Oracle AI Database Server. The container is designed to accelerate resource intensive tasks such as creating vectors or vector index creation to machines other than the Oracle AI Database. This enables the Oracle AI Database to have low latency and high throughput without being burdened with these resource intensive AI infrastructure tasks.

The container uses the following security:

• TLS 1.3 for SSL
• PKCS12 keystores for encrypted passwords
• API Keys for authentication
• SELinux in enforcing mode is supported