Kubernetes Engine (OKE)

OKE powers OCI AI services

Kubernetes is the go-to platform to deploy AI workloads. OKE powers Oracle Cloud Infrastructure (OCI) AI services.

AI model building

– The initial build stage of an AI project involves defining the problem and preparing data to create models.

– Kubernetes clusters can significantly improve efficiency by granting shared access to expensive and often limited GPU resources while providing secure and centrally managed environments.

– Kubeflow, a Kubernetes-related open source project, provides a comprehensive framework designed to streamline the building, training, and deployment of models.

OKE for AI model building

OKE is built on top of OCI, offering a complete stack of high performance infrastructure designed for AI/ML workloads such as:

– The full range of NVIDIA GPUs including H100, A100, A10, etc.

– Ultrafast RDMA networks

Using OKE self-managed nodes, you can run AI/ML building workloads on your Kubernetes clusters.

OKE powers OCI AI services

Kubernetes is the go-to platform to deploy AI workloads. OKE powers OCI AI services.

AI model training

– In model training, data scientists select an algorithm and initiate training jobs using prepared data. This stage requires sophisticated scheduling systems to handle the jobs efficiently.

– Kubernetes projects such as Volcano and Kueue help handle such requirements and make efficient use of compute resources.

– Large-scale distributed training requires low-latency internode communications in the cluster. This is where a specialized ultrafast network with remote direct memory access (RDMA) is needed. It enables data to be moved directly to or from an application’s memory, bypassing the CPU to reduce latency.

OKE for AI model training

OKE is built on top of OCI, offering a complete stack of high performance infrastructure designed for AI/ML workloads such as:

– The full range of NVIDIA GPUs including H100, A100, A10, etc.

– Low-latency, ultra-high performance RDMA networks

Using OKE self-managed nodes, you can run AI/ML training on your Kubernetes clusters.

OKE powers OCI AI services

Kubernetes is the go-to platform to deploy AI workloads. OKE powers OCI AI services.

AI model inferencing (serving)

– AI model inferencing is where Kubernetes really shines. Kubernetes can automatically scale the number of inference pods up or down based on demand, ensuring efficient use of resources.

– Kubernetes provides sophisticated resource management, including the ability to specify CPU and memory limits for containers.

OKE for AI model inference

OKE is designed with resilience at its core, leveraging Kubernetes’ built-in pod autoscaling to scale worker nodes based on usage. Worker nodes can be distributed across multiple fault and/or availability domains for high availability.

OKE virtual nodes provide a serverless Kubernetes experience. They only need to scale at the pod level, without ever scaling worker nodes. This allows for quicker scaling and more economical management since service fees are based solely on the pods in use.

Virtual nodes are well-suited for inference workloads and can use Arm processors, which are becoming a much more attractive option for AI inference—especially when GPUs are in short supply.

Existing applications can benefit by migrating to OCI and OKE

OKE offers lower total cost of ownership and improved time to market.

OKE simplifies operations at scale in the following ways:

• Lift and shift; there’s no need to rearchitect
• Reduce operations burden with automation
• Save time on infrastructure management
• Increase resource utilization and efficiency
• Improve agility, flexibility, uptime, and resilience
• Reduce compliance risk and enhance security

Microservices offer many advantages over monolithic applications

Future-proof your applications with an OKE-centric microservices architecture.

• Architecture modernization
• Faster pace of innovation
• Deployment automation
• Parallel development
• Easier scalability
• Higher reliability
• More flexibility
• Greater agility