Long-running service

A long-running service is a Deployment that runs indefinitely, backed by a Service for in-cluster routing and an Ingress for external HTTPS access. This is the standard shape for web apps, APIs, and any workload that serves traffic continuously.

Preconditions

Before using this recipe, confirm:

• Your tenant is provisioned and you have completed Your first deployment end-to-end.
• Your workload repo is set up per Your repo, your workloads — ArgoCD is pointed at your repo and syncing.
• You know your tenant name (the <your-tenant> prefix used in namespace names).
• You have requested a DNS record for your service hostname from RCS — records for tenant hostnames are currently created manually via ticket. You can carry on without it; the URL just will not resolve until the record exists.

Use case

Use this recipe when your workload:

• Runs continuously (does not exit on its own).
• Serves HTTP/HTTPS traffic from outside the cluster.
• Needs a stable hostname like <app>.<your-tenant>.kestrel.arbutus.cloud.

If your workload runs to completion and exits, see Jobs & CronJobs. If you need a short-lived interactive environment, see Dev pod.

nginx is a stand-in for your real service

This recipe uses nginxinc/nginx-unprivileged as a working example — continuity with the first-deployment walkthrough. Replace the image, ports, and resource requests with your own application. The securityContext block and Ingress shape stay the same regardless of what you deploy.

Adapting this for research workloads

If you are deploying a research application (e.g. a model-serving API, a Shiny dashboard, an RStudio Server instance), the recipe is identical — swap the container image and adjust resource requests to match your workload’s profile. The PSS-compliant securityContext, Ingress hostname shape, and ArgoCD sync flow apply to every long-running service on Kestrel, not just nginx.

YAML

All manifests go in your workload repo under a directory that your ArgoCD Application’s spec.source.path points at (e.g. manifests/my-service/). Replace every <your-tenant> placeholder with your real tenant name.

Deployment

Minimal

The minimal variant ships one replica with no probes, no anti-affinity, and uber-user-significant priority. Good for a first deploy or a low-stakes internal service.

manifests/my-service/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-service
  namespace: <your-tenant>-prod
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-service
  template:
    metadata:
      labels:
        app: my-service
    spec:
      priorityClassName: uber-user-significant
      securityContext:
        runAsNonRoot: true
        runAsUser: 101
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: nginx
          image: nginxinc/nginx-unprivileged:1.27-alpine
          ports:
            - containerPort: 8080
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: [ALL]
          resources:
            requests:
              cpu: 100m
              memory: 64Mi
            limits:
              cpu: 200m
              memory: 128Mi
          volumeMounts:
            - name: tmp
              mountPath: /tmp
            - name: cache
              mountPath: /var/cache/nginx
            - name: run
              mountPath: /var/run
      volumes:
        - name: tmp
          emptyDir: {}
        - name: cache
          emptyDir: {}
        - name: run
          emptyDir: {}

Recommended

The recommended variant adds 2 replicas, liveness/readiness probes, pod anti-affinity (spread across nodes), and uber-user-preempt-high priority. Use this for production workloads backing tenant-facing Ingress routes.

manifests/my-service/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-service
  namespace: <your-tenant>-prod
spec:
  replicas: 2
  selector:
    matchLabels:
      app: my-service
  template:
    metadata:
      labels:
        app: my-service
    spec:
      priorityClassName: uber-user-preempt-high
      securityContext:
        runAsNonRoot: true
        runAsUser: 101
        seccompProfile:
          type: RuntimeDefault
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                labelSelector:
                  matchLabels:
                    app: my-service
                topologyKey: kubernetes.io/hostname
      containers:
        - name: nginx
          image: nginxinc/nginx-unprivileged:1.27-alpine
          ports:
            - containerPort: 8080
          livenessProbe:
            httpGet:
              path: /
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /
              port: 8080
            initialDelaySeconds: 3
            periodSeconds: 5
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: [ALL]
          resources:
            requests:
              cpu: 100m
              memory: 64Mi
            limits:
              cpu: 200m
              memory: 128Mi
          volumeMounts:
            - name: tmp
              mountPath: /tmp
            - name: cache
              mountPath: /var/cache/nginx
            - name: run
              mountPath: /var/run
      volumes:
        - name: tmp
          emptyDir: {}
        - name: cache
          emptyDir: {}
        - name: run
          emptyDir: {}

Why all this securityContext boilerplate?

Kestrel enforces Kubernetes Pod Security Standard restricted cluster-wide via Kyverno. Every Pod must run as non-root, use the runtime-default seccomp profile, drop every Linux capability, and disable privilege escalation. If you copy-paste this manifest verbatim (changing only <your-tenant>), you are compliant. If you cut corners, Kyverno rejects the Pod at admission with a specific error naming the missing field. See Known limitations for the full rule.

Service and Ingress

The Service is ClusterIP — LoadBalancer and NodePort are blocked on Kestrel (see Known limitations). The Ingress uses ingressClassName: traefik and a tenant-scoped hostname.

DNS records are currently created manually

Automated DNS for tenant hostnames is not set up yet — creating an Ingress does not create the matching DNS record. Until automation ships, open a ticket with RCS asking for my-service.<your-tenant>.kestrel.arbutus.cloud (and any other hostnames you plan to use). Without the record, the TLS certificate stays READY: False and the URL in the Verify step will not resolve. You can request it now and carry on — the rest of the recipe does not depend on it until Verify.

manifests/my-service/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: my-service
  namespace: <your-tenant>-prod
spec:
  type: ClusterIP
  selector:
    app: my-service
  ports:
    - port: 80
      targetPort: 8080
---
# manifests/my-service/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-service
  namespace: <your-tenant>-prod
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
    traefik.ingress.kubernetes.io/router.tls: "true"
spec:
  ingressClassName: traefik
  tls:
    - hosts:
        - my-service.<your-tenant>.kestrel.arbutus.cloud
      secretName: my-service-tls
  rules:
    - host: my-service.<your-tenant>.kestrel.arbutus.cloud
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-service
                port:
                  number: 80

Replace my-service in the hostname with your application name. The resulting URL is https://my-service.<your-tenant>.kestrel.arbutus.cloud/. TLS is handled by cert-manager and Let’s Encrypt production — no manual certificate management needed, but the certificate stays READY: False until the DNS record for the hostname exists (see the caution above).

Dry-run

Before committing, validate your manifests locally:

kubectl apply --dry-run=client -f manifests/my-service/deployment.yaml
kubectl apply --dry-run=client -f manifests/my-service/service.yaml
kubectl apply --dry-run=client -f manifests/my-service/ingress.yaml

All three should render without errors. This catches YAML syntax issues and missing required fields before ArgoCD tries to apply them.

Verify

After pushing to your repo and letting ArgoCD sync:

kubectl get pods -n <your-tenant>-prod
kubectl get svc -n <your-tenant>-prod
kubectl get ingress -n <your-tenant>-prod
kubectl logs -n <your-tenant>-prod deployment/my-service

Expected:

• Pod(s) are Running (2 pods for the recommended variant, 1 for minimal).
• The Service shows ClusterIP with port 80 → 8080.
• The Ingress shows my-service.<your-tenant>.kestrel.arbutus.cloud as the hostname.
• Logs show your application’s startup messages.

Open https://my-service.<your-tenant>.kestrel.arbutus.cloud/ in a browser. For the nginx example, you should see the default nginx welcome page with a valid TLS certificate. If the hostname does not resolve, the DNS record probably has not been created yet — see the caution in Service and Ingress and the troubleshooting entry below.

If something goes wrong

• Pod stuck in Pending. Check kubectl describe pod -n <your-tenant>-prod <pod-name> — the Events section shows the reason, usually cluster capacity. (The recommended variant’s anti-affinity is a soft preferred rule and never blocks scheduling — it can only cause Pending if you changed it to requiredDuringSchedulingIgnoredDuringExecution.) If instead no new pod appears at all, check the ReplicaSet’s events (kubectl describe rs -n <your-tenant>-prod) for forbidden: exceeded quota — the pod was rejected at creation and never reaches Pending. Free capacity or request more via RCS ticket.
• No pods appear and the ReplicaSet shows a Kyverno admission error. Kyverno rejects non-compliant pods at admission, so the pod is never created — the denial shows up in the ReplicaSet’s events (kubectl describe rs -n <your-tenant>-prod). You modified the securityContext block — revert to the verbatim YAML above. See Known limitations for the full PSS rule.
• Namespace rejected with forceTenantPrefix. The namespace name does not start with your tenant name. All namespaces must be <your-tenant>-<suffix>. See Known limitations.
• Ingress returns 404 or certificate warning. cert-manager takes ~30 seconds to issue the first certificate. Wait and retry. If it persists, check kubectl describe ingress -n <your-tenant>-prod my-service and kubectl describe certificate -n <your-tenant>-prod my-service-tls.
• Hostname does not resolve (NXDOMAIN), or certificate stuck at READY: False for more than a few minutes. The DNS record for my-service.<your-tenant>.kestrel.arbutus.cloud probably does not exist yet — DNS records are currently created manually (see the caution in Service and Ingress). Check with kubectl get certificate -n <your-tenant>-prod, and open a ticket with RCS to have the hostname created. Once the record exists, cert-manager retries and issues the certificate without any action on your side.
• ArgoCD shows OutOfSync but no errors. ArgoCD polls on a 3-minute interval. Click Sync in the ArgoCD UI for an immediate reconcile.
• Need to make a temporary change outside GitOps? Use the escape hatch workflow — but remember that ArgoCD reverts any mutation on the next sync.