Skip to content
Operate

Kubernetes management

Day-2 operation of EKS, GKE and self-managed clusters: upgrades, node lifecycle, autoscaling, capacity and the boring reliability work that decides whether a platform is trusted.

Day one was the easy part

A cluster is straightforward to create and expensive to keep. Version support windows expire on someone else's schedule, node pools drift away from what the workloads actually need, autoscalers make decisions nobody reviews, and the person who set it up has moved on. Nothing is broken, exactly — it is just that no one can say what would happen if a zone went away.

What the work involves

  • Version upgrades planned against the provider's support window, rehearsed on non-production first, executed without a maintenance night where the architecture allows it.
  • Node lifecycle: capacity types, instance families, consolidation policy, and disruption budgets that reflect what the workload can actually tolerate.
  • Autoscaling that is reviewed rather than assumed — both the cluster autoscaler or Karpenter layer and the HPAs sitting on top of it. what the autoscaler owns, and what it does not →
  • Capacity and headroom measured rather than guessed, including the DaemonSet overhead every node pays before a single application pod schedules.
  • Reliability hygiene: probes, pod disruption budgets, topology spread, priority classes — the settings that only matter on the day they matter.
  • During the inventory fortnight we add one thing before the plan: alert rules for node pressure and disk, pending and crash-looping pods, certificate expiry, backup job failure where a backup schedule exists, and cost anomalies from the daily billing export — on your existing Prometheus and Alertmanager or routed into the alerting you already have, delivered as a pull request you review.

Security baseline, not a security audit

  • RBAC and workload identity (IRSA or Pod Identity on EKS, Workload Identity on GKE) reviewed, least-privilege where we operate
  • network policies for the namespaces we touch, where the CNI enforces them
  • image scanning in the pipelines we maintain
  • secrets handling: nothing in git, an external secrets store where the platform has one
  • control-plane and node updates on a schedule

Where GuardDuty or Security Hub is enabled and our read-only role can read it, findings that touch the cluster appear in the monthly summary. This is not a security audit, a pentest or certification preparation — the diagnostic page says the same What the diagnostic is not →

How the work goes

  1. Inventory

    Two weeks of measurement before any change: what runs, what it requests, what it actually uses, and where the cluster has no headroom left.

  2. Stabilise

    The findings that carry risk go first — version end-of-life, single points of failure, workloads with no disruption budget, nodes that cannot survive a zone loss.

  3. Operate

    A standing arrangement: upgrades, capacity reviews and an on-call path, with a monthly written summary of what changed and what it cost.

During ongoing operation

Where alerts go
Your existing alerting — Alertmanager, PagerDuty, Opsgenie, a Slack channel — or the alert set above, installed in the first fortnight and routed into whatever you already watch.
When we answer
P1 — production down or degraded for users — is acknowledged within 30 minutes in working hours, 09:00–18:00 CET, Monday to Friday. P2 within 2 hours. Nights and weekends: P1 only, and only under a written on-call arrangement priced separately. Planned absences are announced in advance and written into the agreement.
Who answers
A senior engineer from the team working on your platform. Who that is can change — someone is ill, someone leaves — which is why the runbooks and the record of every change live in your repository, so whoever answers starts from what you can read too.
When a fix needs a change we have not agreed
We stop at the plan and ask. Urgent actions during an incident follow only a runbook agreed before, and are written up afterwards.
After an incident
A write-up in the next monthly summary: timeline, cause, what changed.
The monthly summary
Changes, incidents, capacity trend and cost. sample →
Illustrative layout · figures from the EFS case

Is this the right call?

A good fit when

  • Teams running production Kubernetes without a dedicated platform person
  • Clusters that grew organically and now nobody wants to touch
  • Companies whose auditor has started asking about upgrade policy

Not us when

  • Clusters we are not allowed to observe — we do not operate what we cannot measure
  • Teams looking for someone to take the blame rather than the work

What we need, and what we will never ask for

What we need

  • A read-only role in the billing account.
  • Our agent with read-only RBAC and no privileged containers, where you keep it.
  • A branch or fork we may open pull requests from.
  • An engineer who can answer “why is this here?” — about an hour in total during the inventory, as on the diagnostic, and whoever reviews pull requests afterwards.

What we never ask for

  • Administrator credentials in the cloud account or the cluster.
  • Merge rights on your repositories.
  • Your source code.
  • A migration onto a platform of ours.

Questions

How is ongoing operation priced?

Month by month, against a written scope, with a summary each month. Either side can end it with 30 days' written notice.

What does a month look like?

Upgrades and capacity work from the agreed plan, alerts answered inside the stated window, and anything unplanned brought back as a question rather than done quietly. It ends in a written summary of what changed, what broke and what it cost.

What access does ongoing operation need?

A read-only role, our agent where you keep it, and a branch we may open pull requests from. No administrator credentials and no merge rights — the block above lists it in full.