API Gateway
Home › Technical Guides › Deployment
Deployment

Deploying High-Availability API Gateways on Kubernetes and Bare Metal

Production gateway reliability depends on declarative Kubernetes Gateway API resources, graceful pod draining, and tuned Linux kernel socket parameters.

Moving from Legacy Ingress to Kubernetes Gateway API

Legacy Kubernetes `Ingress` forced teams into vendor-specific YAML annotations for timeouts, rewrites, and canary splits. Kubernetes Gateway API (v1 GA) introduces role-oriented resources: `GatewayClass`, `Gateway` (listeners and TLS), and `HTTPRoute` / `GRPCRoute`.

Platform engineers manage certificates and ports on `Gateway` while app teams self-serve `HTTPRoute` rules in their namespaces, bounded safely by cross-namespace `ReferenceGrant` permissions.

  • Role-oriented separation across `GatewayClass`, `Gateway`, and `HTTPRoute`
  • Standardized weighted splitting, header matching, and mirroring
  • Typed CRDs catch routing mistakes via admission webhooks at apply time

Zero-Downtime Helm Upgrades and Graceful Draining

Upgrading gateway pods abruptly drops keepalive traffic and causes HTTP 502s. Pair `maxUnavailable: 0` rolling updates with `PodDisruptionBudgets` and multi-zone `topologySpreadConstraints`.

Add a `preStop` sleep (5–10 s) before `SIGTERM` so cloud load balancers finish removing the pod IP from EndpointSlices while active sockets drain cleanly.

  • Enforce `PodDisruptionBudget` and multi-AZ pod spread constraints
  • Coordinate `preStop` drain delays with `terminationGracePeriodSeconds`
  • Gate `/readyz` probes on full route and TLS snapshot sync

Bare-Metal and Linux Kernel Network Tuning

High-concurrency gateways open thousands of client and upstream sockets simultaneously, often hitting OS file-descriptor (`RLIMIT_NOFILE`) or ephemeral port limits before CPU saturates.

Raise `nofile` limits (e.g., 1,048,576), expand `net.ipv4.ip_local_port_range`, tune `net.core.somaxconn` for burst SYN queues, and enable `SO_REUSEPORT` across worker cores.

  • Raise `ulimit -n` (`nofile`) above peak client + upstream socket counts
  • Increase `net.core.somaxconn` for burst TLS accept queues
  • Enable upstream HTTP keepalive pooling to conserve ephemeral ports

Infrastructure-as-Code Validation in CI/CD

Run `helm template`, schema linting, and Gateway API conflict checks in CI before merging changes, catching duplicate hostnames or missing TLS secrets pre-deploy.

Kubernetes & Bare-Metal Gateway Tuning Parameters

Parameter / ResourceDefault RiskProduction PracticeLayerPrimary Benefit
`PodDisruptionBudget`Simultaneous pod evictions`minAvailable: 2` across 3 AZsKubernetesProtects ingress availability during node maintenance
`preStop` + `terminationGracePeriod`Immediate SIGTERM drops live HTTP5–10 s `preStop` + >= 60 s graceKubernetesZero-502 rolling upgrades & LB deregistration
`nofile` (`RLIMIT_NOFILE`)1024/4096 FD cap causes EMFILERaise to 262,144–1,048,576Linux OSSupports high concurrent client + upstream sockets
`net.core.somaxconn`128/4096 drops burst SYNsTune to 16,384+ matching backlogLinux KernelAbsorbs sudden TLS connection storms cleanly
`net.ipv4.ip_local_port_range`Narrow port pool exhaustionSet `1024 65535` + keepalive reuseLinux KernelPrevents outbound ephemeral port exhaustion

Apply kernel `sysctl` adjustments thoughtfully in DaemonSets or host bootstrap scripts according to node sizing.

Worked Illustration: Configuring Zero-Downtime Gateway Rollouts in Helm

This worked illustration configures a 4-replica Kubernetes gateway Deployment to survive rolling Helm upgrades with zero dropped connections.

  1. Step 1 (Zone Spread & PDB): Set `topologySpreadConstraints` across availability zones (`maxSkew: 1`) and attach a PDB with `minAvailable: 3`.
  2. Step 2 (Readiness Gate): Point `readinessProbe` to `/readyz`, returning HTTP 200 only after initial `HTTPRoute` and TLS sync completes.
  3. Step 3 (Graceful Drain): Add an 8-second `preStop` sleep before `SIGTERM` so cloud load balancers stop routing new traffic prior to shutdown.
  4. Step 4 (Verification): Run a rolling Helm upgrade under 2,000 req/s steady load and confirm zero TCP resets or HTTP 502 responses.

Engineering Illustration Summary: Coordinating readiness gates, `preStop` draining, and PDBs eliminates transient 502s during gateway upgrades.

Kubernetes & Bare-Metal Gateway Deployment Checklist

  • Prefer Kubernetes Gateway API (`Gateway` + `HTTPRoute`) over vendor-specific Ingress annotations.
  • Spread gateway replicas across availability zones with active `PodDisruptionBudgets`.
  • Configure `preStop` drain delays and `terminationGracePeriodSeconds` for zero-502 rollouts.
  • Verify `RLIMIT_NOFILE`, `somaxconn`, and ephemeral port ranges on proxy nodes.
  • Gate readiness probes on full route and TLS certificate snapshot synchronization.
  • Automate Helm lint, dry-run diff, and route conflict validation in CI/CD.

Frequently Asked Questions

Why does Kubernetes Gateway API separate `Gateway` from `HTTPRoute`?

Separating them lets platform teams govern listeners and TLS centrally while app teams manage path routing in their own namespaces.

Why do HTTP 502s spike during Kubernetes deployments even with readiness probes?

Without a `preStop` drain delay, containers exit before cloud load balancers finish removing their pod IP from active pools.

When should I run an API gateway on bare metal instead of Kubernetes?

Bare metal suits ultra-high-throughput edge ingress needing NUMA/NIC pinning or routing across both VMs and clusters.

How do you avoid ephemeral port exhaustion to upstreams?

Enable HTTP/1.1 or HTTP/2 keepalive connection pooling so requests reuse persistent sockets instead of opening new source ports.

Primary Standards & Specifications

  • Kubernetes Gateway API v1 Documentation (Kubernetes SIG Network)
  • Helm Chart Best Practices Guide (CNCF Helm)
  • Linux Kernel IP Sysctl Networking Documentation (kernel.org)

Related Technical Guides