Known limitations

This page is the canonical list of things Kestrel does not let you do and the sanctioned path for each. When another page says “this is blocked — see Known limitations”, it links here to the matching anchor. Every entry names the enforcement layer (Capsule admission webhook, Kyverno ClusterPolicy, kube-apiserver, etc.) so you know why a kubectl error came from where it did.

LoadBalancer blocked

Blocked by: Capsule Tenant.spec.serviceOptions.allowedServices.loadBalancer: false on the shipped tenant chart.

Submitting a Service manifest with type: LoadBalancer is rejected at admission with a Capsule denial. Kestrel does not provision cloud load balancers per workload — the cluster sits behind a single shared edge, not a cloud LB pool, so the resource shape is disabled at the tenant boundary.

Use instead: expose HTTP and HTTPS workloads through an Ingress with ingressClassName: traefik; the shared Traefik ingress controller handles TLS termination and routing. See Ingress on Kestrel for the concrete recipe, and Network model for where Traefik sits in the traffic path.

NodePort blocked

Blocked by: Capsule Tenant.spec.serviceOptions.allowedServices.nodePort: false.

Submitting a Service manifest with type: NodePort is rejected at admission. Node ports would pin traffic to individual cluster nodes and bypass the shared edge, so the service type is disabled at the tenant boundary.

Use instead: the same Ingress path as above — external HTTP/HTTPS goes through Traefik, and pod-to-pod traffic uses in-cluster ClusterIP services. See Ingress on Kestrel.

ExternalName blocked

Blocked by: Capsule Tenant.spec.serviceOptions.allowedServices.externalName: false.

Submitting a Service manifest with type: ExternalName is rejected at admission. Capsule restricts tenants to ClusterIP services; ExternalName is a CNAME-style aliasing Service type that falls outside the allowed set, so the tenant boundary disables it along with LoadBalancer and NodePort.

Use instead: put the external hostname in a ConfigMap and inject it into the workload through an env var, or use a Pod HostAlias entry for the in-pod /etc/hosts override. Both shapes keep the dependency visible in the manifest you own.

Namespace prefix

Blocked by: Capsule Tenant.spec.forceTenantPrefix: true.

Creating a namespace whose name does not start with your tenant name is rejected at kubectl apply time with a Capsule admission error. The rule is enforced at admission for every tenant identity — including CI pipelines using your tenant credentials — so a badly prefixed manifest fails no matter where it is applied from.

Use instead: name every namespace <your-tenant>-* — for example def-profname-experiments, def-profname-prod, def-profname-playground. See Tenancy model for the reasoning behind the rule.

Wildcard hostnames

Blocked by: platform policy, not an admission webhook — an Ingress with a wildcard host (e.g. host: "*.<your-tenant>.kestrel.arbutus.cloud") is not rejected by the API, but it is unsupported and will not work. Wildcard hostnames would let one tenant shadow every subdomain under a parent, which breaks the per-host routing guarantees the shared edge depends on — and because RCS creates DNS records per concrete hostname, a wildcard host would never resolve anyway.

Use instead: one Ingress (or one Ingress rule) per concrete hostname. If you have a legitimate need for a wildcard range — for example, a platform-style tenant that legitimately owns a subtree — open a ticket with RCS.

Hostname collision scope

Blocked by: nothing at admission, currently — Capsule’s hostname-collision guard is not configured on Kestrel tenants, so this is a rule you enforce yourself rather than an error the API returns.

An Ingress whose host already exists on another Ingress is accepted — there is no admission check — and Traefik’s routing between the duplicates is ambiguous: which backend answers is undefined. Nothing stops you from accidentally double-booking the same hostname across two of your namespaces except care.

Use instead: pick a unique hostname per Ingress rule. If you are migrating traffic from one namespace to another, delete the old Ingress first (or temporarily switch the hostname on the stale copy) before creating the new one.

Priority class allowlist

Blocked by: Capsule Tenant.spec.priorityClasses.allowed, which enumerates exactly three values: uber-user-significant, uber-user-preempt-medium, and uber-user-preempt-high.

A Pod that sets priorityClassName to anything outside that allowlist is rejected at admission. Other priority classes exist on the cluster — including uber-user-important and the uber-infra-* classes — but they are not in the tenant allowlist, so do not set them on tenant workloads: Capsule rejects the Pod at admission.

Use instead: pick one of the three user-facing classes that matches your workload’s preemption appetite. Use uber-user-preempt-high for production long-running services. The tradeoffs between the three (which preempts what, which gets preempted first) are covered in Priority classes. If none fits, open a ticket with RCS and describe the workload profile.

Pod Security restricted

Blocked by: Kyverno ClusterPolicy psa applying the Pod Security Standard restricted profile at failureAction: Enforce to every Pod cluster-wide. The enforcement is Kyverno, not the in-tree PodSecurity admission controller — the error message in kubectl output names Kyverno, so that is what you grep for when debugging.

A Pod is rejected if it runs as root, mounts a hostPath volume, uses hostNetwork, sets privileged: true, lacks runAsNonRoot: true, lacks a seccomp profile (seccompProfile.type: RuntimeDefault), or does not drop all Linux capabilities (capabilities.drop: [ALL]). The symptom is a Kyverno admission denial with the specific rule name, not a generic “forbidden”.

Use instead: set securityContext.runAsNonRoot: true, seccompProfile.type: RuntimeDefault, allowPrivilegeEscalation: false, capabilities.drop: [ALL], and readOnlyRootFilesystem: true wherever the workload tolerates it. The first-deployment walkthrough ships a complete compliant securityContext block that you can copy and adapt.

Cross-tenant pod traffic

Blocked by: the default-deny NetworkPolicy distributed via Capsule GlobalTenantResource to every namespace in every tenant, combined with an intra-tenant allow rule that selects by the capsule.clastix.io/tenant label.

A Pod in one tenant trying to reach a Pod in another tenant is dropped at packet time — the Services resolve, but the traffic does not arrive. Traffic within the same tenant (any namespace in the tenant) works out of the box.

Use instead: keep cooperating workloads inside the same tenant where possible. If two tenants legitimately need to exchange traffic, open a ticket with RCS and they will add an explicit cross-tenant allow NetworkPolicy. See Network model for the default rule set.

Shared resource pool

Blocked by: Capsule ResourcePool label selector — every namespace inside a tenant matches the same pool via capsule.clastix.io/tenant: <your-tenant>.

A runaway workload in one namespace can exhaust the CPU, memory, or storage quota for the whole tenant — a Job thrashing in <your-tenant>-experiments can starve a Deployment in <your-tenant>-prod even though the namespaces look separate. This is intentional: quota is a tenant property, not a namespace property.

Use instead: set explicit requests and limits on every Pod so the scheduler caps individual workloads, and use a Kubernetes LimitRange in each namespace if you want a per-namespace ceiling on top of the shared pool. See Resource pools and quotas for the tier numbers and the accounting model.

Quota tiers are preset

Blocked by: the tenant Helm chart’s tier helper (charts/tenant/templates/_helpers.tpl) — tier values (sandbox, standard, premium) are constants, not per-tenant overrides.

Tenants cannot pick arbitrary quota numbers at tenant-creation time; you pick one of three named tiers, or you request custom which is negotiated with RCS. The constraint is shipped-chart-wide, so moving outside the three named tiers always routes through a conversation rather than a config file.

Use instead: pick a tier when you request a tenant, and open a ticket for a tier change when your workload graduates. See Resource pools and quotas for the concrete numbers and the tier-change flow.

No GPU workloads

Blocked by: hardware availability — there is no GPU node pool in the v1 cluster, so the block is physical, not admission-layer.

A Pod requesting nvidia.com/gpu is accepted but never schedules — no node can satisfy the resource request, so the Pod sits Pending with FailedScheduling events. This is v1-only; GPU hardware is pending migration from the legacy environment.

Use instead: defer GPU workloads to post-v1 and run CPU-only workloads on Kestrel for now. If you have a time-sensitive GPU workflow, RCS can point you at the interim GPU environment through a ticket.

Cluster-scoped resource proxy

Blocked by: capsule-proxy — every tenant kubectl session goes through the Capsule proxy, which intercepts reads of cluster-scoped resources (Nodes, StorageClasses, IngressClasses, and similar) and only reveals what an explicit proxy rule grants. Kestrel currently grants none.

kubectl get nodes returns an empty list for a tenant — even when everything is healthy — so do not use it as a connectivity check; the right check is kubectl auth whoami showing your oidc:<your-tenant> group in the Groups row (see Install kubelogin). The same goes for kubectl get storageclasses and kubectl get ingressclasses. Mutating anything at cluster scope is forbidden outright — tenants hold no RBAC there.

Use instead: the docs are the inventory: the two storage classes are listed in Storage classes, and the ingress class is always traefik. Open a ticket with RCS for anything that needs cluster-scoped visibility or change beyond that.

Tenant deletion

Blocked by: Capsule Tenant.spec.preventDeletion: true on every tenant the shipped chart produces.

A delete request against the Tenant resource (through kubectl, the Kubernetes API, or an ArgoCD prune) is rejected — by design. The guard prevents an accidental prune, a misdirected CI job, or a miscopied command from evaporating an entire research team’s scope. Tenant teardown is an RCS procedure, not a self-service action.

Use instead: when a project legitimately ends, open a ticket with RCS to request tenant teardown; they will coordinate the order-of-operations (namespace drain, PVC disposition, RAP retirement in CCDB) with you. See Tenancy model for the ownership context.

Storage classes

Blocked by: the installed catalogue — csi-cinder-sc-delete and csi-cinder-sc-retain are the only two StorageClass resources on the cluster. There is no Capsule admission allowlist behind this; the limit is what exists.

Kestrel provides two OpenStack Cinder storage classes: csi-cinder-sc-delete (the default; reclaim policy Delete) and csi-cinder-sc-retain (reclaim policy Retain). Both are block volumes and support ReadWriteOnce only. A PVC that names any other storageClassName is accepted at admission — nothing rejects it — but no provisioner ever answers it, so it sits Pending unbound with a storageclass not found event. If a PVC hangs in Pending, check its storageClassName first.

ReadWriteMany (shared) volumes are not offered through a storage class — open a ticket with RCS to have a per-tenant NFS volume provisioned.

Use instead: use csi-cinder-sc-delete (or csi-cinder-sc-retain when you need the volume to survive PVC deletion) for single-writer workloads. The reclaim / backup behavior and the NFS path for shared volumes are covered in Storage classes.