Known limitations

This page is the canonical list of things Kestrel does not let you do and the sanctioned path for each. When another page says “this is blocked — see Known limitations”, it links here to the matching anchor. Every entry names the enforcement layer (Capsule admission webhook, Kyverno ClusterPolicy, kube-apiserver, etc.) so you know why a kubectl error came from where it did.

LoadBalancer blocked

Blocked by: Capsule Tenant.spec.serviceOptions.allowedServices.loadBalancer: false on the shipped tenant chart.

Submitting a Service manifest with type: LoadBalancer is rejected at admission with a Capsule denial. Kestrel does not provision cloud load balancers per workload — the cluster sits behind a single shared edge, not a cloud LB pool, so the resource shape is disabled at the tenant boundary.

Use instead: expose HTTP and HTTPS workloads through an Ingress with ingressClassName: traefik; the shared Traefik ingress controller handles TLS termination and routing. See Ingress on Kestrel for the concrete recipe, and Network model for where Traefik sits in the traffic path.

NodePort blocked

Blocked by: Capsule Tenant.spec.serviceOptions.allowedServices.nodePort: false.

Submitting a Service manifest with type: NodePort is rejected at admission. Node ports would pin traffic to individual cluster nodes and bypass the shared edge, so the service type is disabled at the tenant boundary.

Use instead: the same Ingress path as above — external HTTP/HTTPS goes through Traefik, and pod-to-pod traffic uses in-cluster ClusterIP services. See Ingress on Kestrel.

ExternalName blocked

Blocked by: Capsule Tenant.spec.serviceOptions.allowedServices.externalName: false.

Submitting a Service manifest with type: ExternalName is rejected at admission. Capsule restricts tenants to ClusterIP services; ExternalName is a CNAME-style aliasing Service type that falls outside the allowed set, so the tenant boundary disables it along with LoadBalancer and NodePort.

Use instead: put the external hostname in a ConfigMap and inject it into the workload through an env var, or use a Pod HostAlias entry for the in-pod /etc/hosts override. Both shapes keep the dependency visible in the manifest you own.

Namespace prefix

Blocked by: Capsule Tenant.spec.forceTenantPrefix: true.

Creating a namespace whose name does not start with your tenant name is rejected at kubectl apply time with a Capsule admission error. Even a cluster-admin running from a CI pipeline hits the same rule — the admission webhook does not care where the request came from, only what it says.

Use instead: name every namespace <your-tenant>-* — for example demo-experiments, demo-prod, demo-playground. See Tenancy model for the reasoning behind the rule.

Wildcard hostnames

Blocked by: Capsule Tenant.spec.ingressOptions.allowWildcardHostnames: false.

An Ingress with a wildcard host (e.g. host: "*.<your-tenant>.kestrel.arbutus.cloud") is rejected at admission. Wildcard hostnames would let one tenant shadow every subdomain under a parent, which breaks the per-host routing guarantees the shared edge depends on.

Use instead: one Ingress (or one Ingress rule) per concrete hostname. If you have a legitimate need for a wildcard range — for example, a platform-style tenant that legitimately owns a subtree — open a ticket with RCS and they can add an explicit allow.

Hostname collision scope

Blocked by: Capsule Tenant.spec.ingressOptions.hostnameCollisionScope: Tenant.

An Ingress whose host already exists on another Ingress in the same tenant is rejected at admission. The collision check is scoped within a tenant — two different tenants cannot shadow each other because cross-tenant routing is already walled off, but inside your tenant you cannot accidentally double-book the same hostname across two namespaces.

Use instead: pick a unique hostname per Ingress rule. If you are migrating traffic from one namespace to another, delete the old Ingress first (or temporarily switch the hostname on the stale copy) before creating the new one.

Priority class allowlist

Blocked by: Capsule Tenant.spec.priorityClasses.allowed, which enumerates exactly three values: uber-user-significant, uber-user-preempt-medium, and uber-user-preempt-high.

A Pod that sets priorityClassName to anything outside that allowlist is rejected at admission. Other priority classes exist on the cluster — including uber-user-important and the uber-infra-* classes — but they are not in the tenant allowlist, so a tenant Pod that requests one is rejected at Capsule admission. Never recommend uber-user-important for a tenant workload; it is not allowed.

Use instead: pick one of the three user-facing classes that matches your workload’s preemption appetite. Use uber-user-preempt-high for production long-running services. The tradeoffs between the three (which preempts what, which gets preempted first) are covered in Priority classes. If none fits, open a ticket with RCS and describe the workload profile.

Pod Security restricted

Blocked by: Kyverno ClusterPolicy psa applying the Pod Security Standard restricted profile at failureAction: Enforce to every Pod cluster-wide. The enforcement is Kyverno, not the in-tree PodSecurity admission controller — the error message in kubectl output names Kyverno, so that is what you grep for when debugging.

A Pod is rejected if it runs as root, mounts a hostPath volume, uses hostNetwork, sets privileged: true, lacks runAsNonRoot: true, lacks a seccomp profile (seccompProfile.type: RuntimeDefault), or does not drop all Linux capabilities (capabilities.drop: [ALL]). The symptom is a Kyverno admission denial with the specific rule name, not a generic “forbidden”.

Use instead: set securityContext.runAsNonRoot: true, seccompProfile.type: RuntimeDefault, allowPrivilegeEscalation: false, capabilities.drop: [ALL], and readOnlyRootFilesystem: true wherever the workload tolerates it. The first-deployment walkthrough ships a complete compliant securityContext block that you can copy and adapt.

Cross-tenant pod traffic

Blocked by: the default-deny NetworkPolicy distributed via Capsule GlobalTenantResource to every namespace in every tenant, combined with an intra-tenant allow rule that selects by the capsule.clastix.io/tenant label.

A Pod in one tenant trying to reach a Pod in another tenant is dropped at packet time — the Services resolve, but the traffic does not arrive. Traffic within the same tenant (any namespace in the tenant) works out of the box.

Use instead: keep cooperating workloads inside the same tenant where possible. If two tenants legitimately need to exchange traffic, open a ticket with RCS and they will add an explicit cross-tenant allow NetworkPolicy. See Network model for the default rule set.

Shared resource pool

Blocked by: Capsule ResourcePool label selector — every namespace inside a tenant matches the same pool via capsule.clastix.io/tenant: <your-tenant>.

A runaway workload in one namespace can exhaust the CPU, memory, or storage quota for the whole tenant — a Job thrashing in <your-tenant>-experiments can starve a Deployment in <your-tenant>-prod even though the namespaces look separate. This is intentional: quota is a tenant property, not a namespace property.

Use instead: set explicit requests and limits on every Pod so the scheduler caps individual workloads, and use a Kubernetes LimitRange in each namespace if you want a per-namespace ceiling on top of the shared pool. See Resource pools and quotas for the tier numbers and the accounting model.

Quota tiers are preset

Blocked by: the tenant Helm chart’s tier helper (charts/tenant/templates/_helpers.tpl) — tier values (sandbox, standard, premium) are constants, not per-tenant overrides.

Tenants cannot pick arbitrary quota numbers at tenant-creation time; you pick one of three named tiers, or you request custom which is negotiated with RCS. The constraint is shipped-chart-wide, so moving outside the three named tiers always routes through a conversation rather than a config file.

Use instead: pick a tier when you request a tenant, and open a ticket for a tier change when your workload graduates. See Resource pools and quotas for the concrete numbers and the tier-change flow.

No GPU workloads

Blocked by: hardware availability — there is no GPU node pool in the v1 cluster, so the block is physical, not admission-layer.

A Pod requesting nvidia.com/gpu schedules but never runs — no node satisfies the resource request, so the Pod sits Pending forever. This is v1-only; GPU hardware is pending migration from the legacy environment.

Use instead: defer GPU workloads to post-v1 and run CPU-only workloads on Kestrel for now. If you have a time-sensitive GPU workflow, RCS can point you at the interim GPU environment through a ticket.

Cluster-scoped resource proxy

Blocked by: Capsule Tenant.spec.owners[*].proxySettings, which limits tenant-owner visibility on Nodes, StorageClasses, and IngressClasses to List only — no mutation, no Get on individual cluster-scoped resources outside that allowlist.

kubectl get nodes works and returns the node list. A mutating request against a cluster-scoped resource — creating a Node, editing a StorageClass, or otherwise reaching a cluster-scoped type outside the allowlist — is rejected at the Capsule proxy. Tenants can see what the cluster looks like, but cannot reshape it.

Use instead: read the cluster-scoped resources you need via kubectl get (or the equivalent client-go list call) and open a ticket with RCS for anything that requires a mutation at the cluster scope.

Tenant deletion

Blocked by: Capsule Tenant.spec.preventDeletion: true on every tenant the shipped chart produces.

A delete request against the Tenant resource (through kubectl, the Kubernetes API, or an ArgoCD prune) is rejected — by design. The guard prevents an accidental prune, a misdirected CI job, or a miscopied command from evaporating an entire research team’s scope. Tenant teardown is an RCS procedure, not a self-service action.

Use instead: when a project legitimately ends, open a ticket with RCS to request tenant teardown; they will coordinate the order-of-operations (namespace drain, PVC disposition, group deletion in Keycloak) with you. See Tenancy model for the ownership context.

Storage classes

Blocked by: Capsule Tenant.spec.storageClasses.allowed, which enumerates exactly two values: csi-cinder-sc-delete and csi-cinder-sc-retain.

Kestrel provides two OpenStack Cinder storage classes: csi-cinder-sc-delete (the default; reclaim policy Delete) and csi-cinder-sc-retain (reclaim policy Retain). Both are block volumes and support ReadWriteOnce only. A PVC that names any other storageClassName is rejected at admission and the PVC stays Pending.

ReadWriteMany (shared) volumes are not offered through a storage class — open a ticket with RCS to have a per-tenant NFS volume provisioned.

Use instead: use csi-cinder-sc-delete (or csi-cinder-sc-retain when you need the volume to survive PVC deletion) for single-writer workloads. The reclaim / backup behavior and the NFS path for shared volumes are covered in Storage classes.