Skip to main content

Ingress and Workloads

Cosmonic Control uses Envoy as its HTTP ingress proxy and a set of Kubernetes CRDs for declaring workloads. This page explains how to configure ingress and deploy WebAssembly workloads.

What are Cosmonic Control deployment manifests?​

Deploying a workload to Cosmonic Control is the same as deploying any other resource to Kubernetes: declarative Custom Resource Definition (CRD) manifests written in YAML.

Workload manifests can be applied manually with kubectl and managed in GitOps pipelines with tools like Argo CD. Manifests are composed according to the runtime.wasmcloud.dev/v1alpha1 API for workloads and the control.cosmonic.io/v1alpha2 API for HTTP routing.

How ingress works​

Cosmonic Control routes external HTTP traffic to Wasm workloads in two stages:

  1. An edge proxy terminates external connections. Traefik is deployed by default; Istio is an alternative; operators can also bypass the edge proxy and expose Envoy directly.
  2. Envoy sits behind the edge proxy on a ClusterIP Kubernetes Service named ingress. Envoy matches the request's Host header against the XDS routing configuration and forwards the request to the Wasm workload.

Envoy's routing table is pushed over gRPC from the XDS cache whenever workloads or HTTPTriggers change. If a workload runs on multiple hosts, requests are load balanced round robin. If a host crashes, it drops out of the rotation and the workload continues to serve from the remaining hosts.

Ingress architecture diagram

Ingress request flow diagram

Traefik (default)​

Installing the chart with no overrides deploys Traefik as the edge proxy and creates a Traefik IngressClass marked as the cluster default. Traefik's Service is a NodePort on ports 30080 (http) and 30443 (https):

helm install cosmonic-control oci://ghcr.io/cosmonic/cosmonic-control \
  --version 0.14.0 \
  --namespace cosmonic-system \
  --create-namespace

For a local Kind cluster, forward host ports 80 and 443 to those NodePorts:

# kind-config.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
    extraPortMappings:
      - containerPort: 30080
        hostPort: 80
        protocol: TCP
      - containerPort: 30443
        hostPort: 443
        protocol: TCP

The chart auto-creates an Ingress for the Perses dashboard at perses.localhost.cosmonic.sh. To auto-create Ingress resources for additional hosts at install time, list them under ingress.hosts:

# values.yaml
ingress:
  hosts:
    - host: "welcome-tour.localhost.cosmonic.sh"
    - host: "hello.localhost.cosmonic.sh"

For workloads deployed after the install, add a standard Kubernetes Ingress that points at the ingress Service (Envoy) on port 80. Envoy resolves the request to the right workload based on the Host header:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: welcome-tour
  namespace: cosmonic-system
  annotations:
    # "web" for http, "websecure" for https. List both to accept both.
    traefik.ingress.kubernetes.io/router.entrypoints: web,websecure
    traefik.ingress.kubernetes.io/router.observability.accesslogs: "true"
    traefik.ingress.kubernetes.io/router.observability.metrics: "true"
    traefik.ingress.kubernetes.io/router.observability.tracing: "true"
spec:
  ingressClassName: traefik
  rules:
    - host: welcome-tour.localhost.cosmonic.sh
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: ingress
                port:
                  number: 80

Istio​

To route through an existing Istio IngressGateway instead, set ingress.provider=istio. The chart creates a Gateway (cosmonic-gateway by default, configurable via ingress.istio.gatewayName) and a VirtualService for each entry under ingress.hosts, plus a VirtualService for the Perses dashboard:

helm install cosmonic-control oci://ghcr.io/cosmonic/cosmonic-control \
  --version 0.14.0 \
  --namespace cosmonic-system \
  --create-namespace \
  --set ingress.provider=istio \
  --set 'ingress.hosts[0].host=*.localhost.cosmonic.sh'

If the IngressGateway pods use labels other than istio: ingressgateway, override ingress.istio.ingressGatewaySelector. Istio itself (control plane and IngressGateway) must be installed before the Cosmonic Control chart; istioctl install --set profile=minimal plus a separate components.ingressGateways[0] install covers the baseline setup.

Direct Envoy exposure​

To bypass the edge proxy altogether and expose Envoy directly, set ingress.enabled=false and set envoy.service.type to the appropriate service type. Traefik is not deployed in this mode, and the Perses Ingress is not auto-created.

On a cloud cluster, expose Envoy as a LoadBalancer:

helm install cosmonic-control oci://ghcr.io/cosmonic/cosmonic-control \
  --version 0.14.0 \
  --namespace cosmonic-system \
  --create-namespace \
  --set ingress.enabled=false \
  --set envoy.service.type=LoadBalancer

Cloud-specific annotations go on envoy.service.annotations. For example, to pin AWS to an internet-facing NLB:

  --set-json 'envoy.service.annotations={"service.beta.kubernetes.io/aws-load-balancer-type":"nlb","service.beta.kubernetes.io/aws-load-balancer-scheme":"internet-facing"}'

On a bare-metal cluster, expose a NodePort and front it with your own load balancer or proxy:

  --set ingress.enabled=false \
  --set envoy.service.type=NodePort \
  --set envoy.service.httpNodePort=30950

TLS​

Envoy handles plain HTTP. TLS terminates at the edge:

  • Traefik: set ingress.traefik.tls.enabled: true to serve HTTPS on the websecure entrypoint for the Perses host and every ingress.hosts entry, with a certificate from cert-manager or an existing Secret in cosmonic-system (v0.13.0 and later; see HTTPS on the bundled Traefik ingress for the values). For Ingress objects you add yourself, configure TLS with annotations such as traefik.ingress.kubernetes.io/router.tls: "true" and a certresolver, or supply Secrets via spec.tls. See the Traefik TLS routing docs.
  • Istio: configure TLS on the Gateway resource.
  • Direct Envoy exposure: terminate TLS upstream, for example via Cloudflare or a cloud load balancer, and forward plain HTTP to the ingress Service.

How to deploy WebAssembly workloads with ingress​

An HTTP workload is two resources: a WorkloadDeployment that runs the component, and an HTTPTrigger that routes requests to it. The minimum required fields per component are:

  • name: Name of the component
  • image: OCI address for the component image

The HTTPTrigger Helm chart renders both resources from one values file (chart 0.2.0 and later, for Cosmonic Control v0.13.0 and later). Additional interfaces (beyond wasi:http) and the ingress host are typically supplied there:

components:
  - name: blobby
    image: ghcr.io/cosmonic-labs/components/blobby:0.2.0

ingress:
  host: 'blobby.wasmworkloads.io'

hostInterfaces:
  - namespace: wasi
    package: blobstore
    version: 0.2.0-draft
    interfaces:
      - blobstore
  - namespace: wasi
    package: logging
    version: 0.1.0-draft
    interfaces:
      - logging

In an Argo CD deployment:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: demo-hello
  namespace: argocd
spec:
  project: default
  source:
    path: .
    repoURL: oci://ghcr.io/cosmonic-labs/charts/http-trigger
    targetRevision: 0.2.0
    helm:
      values: |
        components:
          - name: http
            image: ghcr.io/cosmonic-labs/control-demos/hello-world:0.1.2
        ingress:
          host: "hello.localhost.cosmonic.sh"
  destination:
    name: in-cluster
    namespace: hello
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - ServerSideApply=true
      - CreateNamespace=true
      - RespectIgnoreDifferences=true
    retry:
      limit: -1
      backoff:
        duration: 30s
        factor: 2
        maxDuration: 5m

The chart renders a WorkloadDeployment and an HTTPTrigger with the release name. Written by hand, the same workload is:

apiVersion: runtime.wasmcloud.dev/v1alpha1
kind: WorkloadDeployment
metadata:
  name: demo-hello
  namespace: hello
spec:
  replicas: 1
  template:
    spec:
      hostInterfaces:
        - namespace: wasi
          package: http
          interfaces:
            - incoming-handler
          config:
            host: hello.localhost.cosmonic.sh
      components:
        - name: http
          image: ghcr.io/cosmonic-labs/control-demos/hello-world:0.1.2
---
apiVersion: control.cosmonic.io/v1alpha2
kind: HTTPTrigger
metadata:
  name: demo-hello
  namespace: hello
spec:
  hostnames:
    - hello.localhost.cosmonic.sh
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: demo-hello

Declare the inbound wasi:http interface on the WorkloadDeployment (incoming-handler for a WASI P2 component, handler for a WASI P3 component), along with any other capability imports, under spec.template.spec.hostInterfaces. config.host names the hostname the workload serves. Components that fetch over the network add wasi:http/outgoing-handler and scope outbound destinations with localResources.allowedHosts:

apiVersion: runtime.wasmcloud.dev/v1alpha1
kind: WorkloadDeployment
metadata:
  name: api
spec:
  replicas: 1
  template:
    spec:
      hostInterfaces:
        - namespace: wasi
          package: http
          interfaces:
            - incoming-handler
            - outgoing-handler
          config:
            host: api.example.com
        - namespace: wasi
          package: logging
          version: 0.1.0-draft
          interfaces:
            - logging
      components:
        - name: api
          image: ghcr.io/example/api:0.1.0
          localResources:
            allowedHosts:
              - 'https://upstream.example.com'

The inbound entry needs no version. Without it, the workload still reports ready and the trigger still reports Programmed, but the host has nothing to dispatch the request to and every request gets a 404. If a newly deployed HTTP workload answers only 404, check this entry first.

The WorkloadDeployment manages WorkloadReplicaSet and Workload resources. The HTTPTrigger only routes: several triggers can share a hostname with different paths, several triggers can point at the same WorkloadDeployment, and deleting a trigger never deletes the workload. Before v0.13.0 the trigger embedded the workload and created the WorkloadDeployment itself; see Upgrading from v0.12.2 to v0.13.0.

Request timeouts​

To bound how long Envoy waits for a response, set timeouts on the trigger rule:

spec:
  rules:
    - backendRefs:
        - name: api
      timeouts:
        request: 300s          # whole request, default 15s
        backendRequest: 60s    # each try against the host

Durations use the Gateway API format: up to four of h, m, s and ms, such as 300s or 5m. backendRequest cannot be longer than request. When Envoy gives up it returns upstream request timeout.

Splitting traffic​

A rule can list several backendRefs with a weight each, for blue/green and canary rollouts. Weights are relative, a missing weight counts as 1, and 0 sends a backend nothing:

spec:
  hostnames:
    - hello.example.com
  rules:
    - backendRefs:
        - name: hello-v1
          weight: 90
        - name: hello-v2
          weight: 10

A backend whose WorkloadDeployment does not exist answers its share with a 500, and a backend with no ready workloads answers its share with a 503 rather than shifting it to the other backends, so a broken canary shows up at its own weight. Backends in one rule that set different config.host values cannot share the rule.

Path matching​

Each rule lists matches, and a request that satisfies any one of them uses the rule. A match is a path with a type and a value. A rule with no matches matches every path (PathPrefix /). Two path types are routed, following the Gateway API HTTPPathMatch:

Exact: the request path must match exactly.

matches:
  - path:
      type: Exact
      value: /api/users

/api/users matches only /api/users. /api/users/123 does not match.

PathPrefix: the request path must begin with the prefix, compared by whole path segments.

matches:
  - path:
      type: PathPrefix
      value: /api

/api matches /api, /api/users, and /api/v1/users. A trailing slash on a prefix is ignored, so /api/ and /api are the same match.

note

The schema also accepts RegularExpression, but Control does not route it yet: a match that uses it is dropped and reported in the trigger's status. Paths must start with /, be at most 1024 characters, and must not contain //, /./, /../, %2f or #.

Conflict detection

When two triggers claim the same hostname and path match, the oldest trigger keeps it, following HTTPRoute precedence. The newer trigger reports the lost match as Conflicted in its status and keeps serving its other rules (PartiallyInvalid), or reports Accepted: False if nothing is left. Within a trigger, an earlier rule wins over a later one. Envoy checks exact matches before prefixes, and longer prefixes before shorter ones. Before v0.13.0 the newest trigger won instead.

Trigger status​

kubectl get httptriggers prints HOSTNAMES, ACCEPTED, RESOLVED and PROGRAMMED:

  • ACCEPTED is False when no rule can be routed.
  • RESOLVED is False when a backendRefs entry names a WorkloadDeployment that does not exist.
  • PROGRAMMED is True once every Envoy replica has accepted the configuration containing the trigger's current routes.

status.backends lists each referenced WorkloadDeployment with its weight and readyWorkloads. Use kubectl describe httptrigger <name> to read the condition messages.

Non-HTTP workloads​

Workloads that do not serve inbound HTTP are deployed directly as a WorkloadDeployment, without an HTTPTrigger. The most common shape is a NATS-driven worker that exports wasmcloud:messaging/handler@0.2.0 and replies on the reply_to subject.

apiVersion: runtime.wasmcloud.dev/v1alpha1
kind: WorkloadDeployment
metadata:
  name: task-worker
  namespace: default
spec:
  replicas: 1
  template:
    spec:
      hostInterfaces:
        - namespace: wasi
          package: logging
          version: 0.1.0-draft
          interfaces:
            - logging
        - namespace: wasmcloud
          package: messaging
          version: '0.2.0'
          interfaces:
            - handler
            - consumer
            - types
          config:
            subscriptions: tasks.worker
      components:
        - name: task-worker
          image: ghcr.io/example/task-worker:0.1.0
          poolSize: 1

config.subscriptions on the messaging interface is the NATS subject the host subscribes to on the component's behalf. Inbound messages on that subject invoke the component's wasmcloud:messaging/handler.handle_message export.

A typical two-component application pairs an HTTP workload (a WorkloadDeployment behind an HTTPTrigger) with one of these WorkloadDeployments: the HTTP workload publishes a request via wasmcloud:messaging/consumer.request, and the worker receives it on the subscribed subject and publishes a reply.

Service-plus-component workloads​

For workloads that need a long-running, in-memory state alongside per-request component invocations (a parsed corpus, a search index, a cache), use template.spec.service to colocate a service with the components on the same host. The service exports wasi:cli/run and stays resident; components make loopback TCP calls to it, typically on 127.0.0.1.

spec:
  template:
    spec:
      volumes:
        - name: data
          ephemeral: {}
      service:
        image: ghcr.io/example/index-service:0.1.0
        localResources:
          volumeMounts:
            - name: data
              mountPath: /data
      components:
        - name: api
          image: ghcr.io/example/api:0.1.0
          localResources:
            volumeMounts:
              - name: data
                mountPath: /data

The shared ephemeral volume is the canonical way to hand snapshots from the service side to the component side without reaching out to external storage. volumeMounts defaults to read-write; the readOnly field accepts only true or omission.

hostInterfaces is a scheduling filter​

spec.template.spec.hostInterfaces is matched against the set of interfaces the host advertises. If a workload declares an interface the host does not advertise, the workload fails placement with WORKLOAD_STATE_ERROR rather than failing at link time inside a host pod.

Two consequences worth knowing:

  • A workload that imports wasi:webgpu/webgpu@0.0.1 requires --set gpu=true on the hostgroup chart so the host advertises WebGPU. Without it, the workload sticks at WORKLOAD_STATE_ERROR and never schedules.
  • If a workload declares only the interfaces it actually imports — and the host advertises at least those — the workload will place. Declaring extra interfaces that the host does not provide will block placement, even if the component does not actually use them.

Warm instance pools​

A component can keep a pool of warm instances ready so requests avoid a cold start. Set poolSize on the component (spec.template.spec.components[].poolSize) in a WorkloadDeployment; the pool grows to that size under load. Warm pools benefit a WASI p3 handler component; a p2 incoming-handler accepts poolSize but ignores it and serves every request from a fresh instance.

spec:
  template:
    spec:
      components:
        - name: api
          poolSize: 8               # warm instances kept ready
          reclaimWindowSeconds: 300 # retire instances an idle sweep finds unused
          reclaimMinInstances: 2    # never sweep below this floor

Two optional fields, added in Cosmonic Control v0.9.3, tune how an idle pool shrinks:

  • reclaimWindowSeconds: how long the pool watches its own peak concurrency before retiring the warm instances that peak did not need. A pool grows to poolSize under load and, left unset, stays there, so a spike's high-water mark outlives the spike. Set this and the pool sweeps every window, keeps the instances its measured peak actually needed, and drains the rest. Unset, warm instances are never reclaimed for idleness. Only meaningful alongside poolSize.
  • reclaimMinInstances: a floor the idle sweep never retires below, capped at poolSize. It is a floor on reclaim, not a target to grow to: instances are still built only when a call needs one. Unset lets a fully idle pool empty out, so the first call after a quiet spell starts cold. Only meaningful alongside reclaimWindowSeconds.

Further reading​

note

Many existing wasmCloud applications include OAM-formatted YAML manifests designed for vanilla wasmCloud deployments. These OAM manifests are not Kubernetes-native and are not to be confused with the CRD manifests used by Cosmonic Control. (OAM manifests can be identified by the API version core.oam.dev/v1beta1.)

Last updated on by github-actions[bot]