Administration on OME

Model Agent Administration

Tue, 14 Mar 2023 00:00:00 +0000

The Model Agent is the core component responsible for downloading, managing, and distributing models across your OME cluster. This guide provides comprehensive information for cluster administrators who need to configure, monitor, and troubleshoot the Model Agent in production environments.

Architecture Overview

DaemonSet Deployment

The Model Agent is deployed as a Kubernetes DaemonSet, ensuring it runs on every node in your cluster. This distributed architecture provides several benefits:

Controller Configuration

Mon, 01 Jan 0001 00:00:00 +0000

The OME controller manager (ome-manager) is configured through command-line flags passed to its container. This page documents those flags and how to change them on a running cluster.

Setting flags

The flags are parsed once at startup, so a change takes effect when the controller pod restarts. No image rebuild is required.

Ingress Administration

Mon, 01 Jan 0001 00:00:00 +0000

OME (Open Model Engine) provides flexible ingress configuration to support various Kubernetes ingress controllers and deployment scenarios. This guide covers the configuration options available to cluster administrators.

Supported Ingress Controllers

OME supports the following ingress solutions:

Standard Kubernetes Ingress Controllers

NGINX Ingress Controller - Installation Guide
Traefik - Installation Guide
Kong Ingress Controller - Installation Guide
HAProxy Ingress - Installation Guide
Contour - Installation Guide

Service Mesh Ingress

Istio (VirtualService + Gateway) - Installation Guide

Gateway API

Gateway API Controllers - Installation Guide

OME Ingress Configuration

OME ingress behavior is configured through the inferenceservice-config ConfigMap in the OME controller namespace.