Published on November 4, 2026

We are excited to announce the release of KServe v0.21. This release focuses on autoscaling flexibility, operational hardening, and deeper llm-d integration, with key highlights including:

• Direct KEDA autoscaling for LLMInferenceService, with true scale-to-zero
• Kernel Cache initial code donation with image signature verification
• Standalone vLLM tokenizer deployment replacing the UDS tokenizer sidecar
• Canary traffic splitting on RawDeployment HTTPRoutes
• Rollout strategy controls for GPU-constrained rolling updates
• LocalModelCache support for LoRA adapters
• oci+fetch:// KServe-side OCI image pull and ModelScope (modelscope://) storage support
• TLS hardening: cluster TLS profile flags, hot reload of rotated certificates, and TLS for transformers
• Python 3.13 support and Go 1.26 upgrade

Published on August 6, 2026

We are excited to announce the release of KServe v0.20. This release delivers major new capabilities across the platform with key highlights including:

• Confidential model serving with TEE-based encrypted model decryption
• Traffic splitting API for progressive LLMInferenceService deployments
• Canary rollout support for InferenceService in RawDeployment mode
• KV cache offloading with CPU memory tiering for vLLM workloads
• Distributed tracing API for LLMInferenceService components
• Managed DRA (Dynamic Resource Allocation) for GPU provisioning
• Anthropic Messages API routing support
• Native OCI ImageVolume mounting (oci+native://) for model storage
• vLLM as a standalone runtime for InferenceService
• AutoGluon server with time series inference support

Published on June 14, 2026

We are excited to announce the release of KServe v0.19. This release focuses heavily on LLMInferenceService maturity with key highlights including:

• Static LoRA adapter support for efficient fine-tuned model serving
• Model name-based routing for shared-gateway deployments
• Graceful pod shutdown with vLLM drain to prevent in-flight request drops
• HPA/KEDA scaling status surfaced directly in service conditions
• LocalModelCache integration for LLMInferenceService
• Dual-protocol (REST/gRPC) routing for InferenceService Standard mode
• Seamless llm-d v0.6 to v0.7 migration with version-gated controller logic

Published on April 29, 2026

We are excited to announce the release of KServe v0.18. This release brings multi-node inference support without Ray, LeaderWorkerSet (LWS)-based autoscaling for multi-node workloads, OpenAI Responses API routing, namespace-scoped ModelCache, vLLM upgrade to v0.19.0, llm-d v0.6 integration, enhanced security hardening, and GKE Gateway compatibility improvements.

Everyone is racing to run Large Language Models (LLMs), in the cloud, on-prem, and even on edge devices. The real challenge, however, isn't the first deployment; it's scaling, managing, and maintaining hundreds of LLMs efficiently. We initially approached this challenge with a straightforward vLLM deployment wrapped in a Kubernetes StatefulSet.

Published on March 13, 2026

We are excited to announce the release of KServe v0.17, a landmark release that brings LLMInferenceService to production readiness with a GenAI-first architecture built on the llm-d framework. This release introduces KV-cache aware intelligent routing, disaggregated prefill-decode, distributed inference with tensor/data/expert parallelism, Envoy AI Gateway integration with token-based rate limiting, and a completely restructured modular Helm chart architecture.

Enterprises today seek to integrate generative AI (GenAI) capabilities into their applications. However, scaling large AI models introduces complexity: managing high-volume traffic from large language models (LLMs), optimizing inference performance, maintaining predictable latency, and controlling infrastructure costs.

Platform engineering leaders require more than just model deployment capabilities. They need a robust, Kubernetes-native infrastructure that supports:

• Efficient GPU utilization
• Intelligent request routing
• Distributed inference patterns
• Cost-aware autoscaling
• Production-grade governance

This article demonstrates how two open-source solutions, KServe and llm-d, can be combined to address these challenges.

We explore the role of each solution, illustrate their integration architecture, and provide practical guidance for AI platform teams, with deeper focus on KServe's LLMInferenceService, available since KServe v0.16.

Published on May 27, 2025

We are thrilled to announce the release of KServe v0.15, marking a significant leap forward in serving both predictive and generative AI models. This release introduces enhanced support for generative AI workloads, including advanced features for serving large language models (LLMs), improved model and KV caching mechanisms, and integration with Envoy AI Gateway.

Published on December 23, 2024

We are excited to announce KServe v0.14. In this release we are introducing a new Python client designed for KServe, and a new model cache feature; we are promoting OCI storage for models as a stable feature; and we added support for deploying models directly from Hugging Face.

Published on May 15, 2024

We are excited to unveil KServe v0.13, marking a significant leap forward in evolving cloud native model serving to meet the demands of Generative AI inference. This release is highlighted by three pivotal updates: enhanced Hugging Face runtime, robust vLLM backend support for Generative Models, and the integration of OpenAI protocol standards.