DevOps Engineer Interview Questions

Summary

On every commit: build once, run fast checks (lint, unit tests, static analysis), build and scan the container image, publish it with an immutable tag, run integration tests against it, then promote that same artifact through staging to production with automated deployment, progressive rollout and automatic rollback on bad metrics.

Longer Answer

A concrete shape:

1. Pull request: lint, unit tests, SAST and dependency scanning, build image, ephemeral preview environment if useful.
2. Merge to main: build image tagged with the commit SHA, generate SBOM and sign it, push to the registry.
3. Staging: deploy automatically (GitOps update of the image tag), run integration, contract and smoke tests.
4. Production: deploy the same digest with a canary or progressive rollout, watching SLIs, and roll back automatically on regression; database migrations follow expand-and-contract.

Cross-cutting: secrets via OIDC and a secret manager (no long-lived keys in CI), pipeline as code reviewed like application code, fast feedback (minutes, not hours), and DORA metrics to track improvement.

Key Points

• Build once, promote the same artifact.
• Fast checks first; scan and sign images.
• Automated staging; progressive production rollout.
• OIDC credentials; pipelines as code; measure DORA.

Deep dive: Docker interview questions

Summary

Make every migration backward compatible with the code that is running (expand and contract), run migrations as a separate, controlled pipeline step before the new code rolls out (not on application start-up across many replicas), make them idempotent and reviewable, and keep destructive changes for a later release after nothing uses the old schema.

Longer Answer

Example of renaming a column: add the new column, deploy code that writes both and reads the old, backfill in batches, switch reads to the new column, stop writing the old, then drop it in a later release. Large table changes need online schema tools or batching to avoid locks. Rollbacks then mean deploying previous code, which still works with the expanded schema; never rely on "down" migrations in production for destructive changes.

Key Points

• Expand and contract.
• Separate migration step, run once.
• Backward compatible with running code.
• Destructive changes later; batch big changes.

Summary

Avoid long-lived credentials: use OIDC federation from the CI system to the cloud (GitHub Actions, GitLab and others can assume cloud roles with short-lived tokens), fetch application secrets at runtime from a secret manager rather than passing them through the pipeline, scope CI permissions per job and environment, and protect production credentials behind approvals and protected branches.

Longer Answer

Additional controls: mask secrets in logs (but do not rely on masking), never echo or write them to artifacts, restrict which branches and workflows can access production secrets, rotate any remaining static secrets automatically, scan repositories and history for leaked secrets (gitleaks, GitHub secret scanning), and audit secret access. Pull requests from forks must never get secrets.

Key Points

• OIDC to cloud; no long-lived keys.
• Runtime secrets from a secret manager.
• Scope per job/environment; approvals for prod.
• Scan for leaks; rotate; audit.

Summary

The DORA metrics measure software delivery performance: deployment frequency, lead time for changes, change failure rate, and time to restore service (with recent research adding measures such as rework rate). They are used together because speed and stability are linked: high performers do well on both.

Longer Answer

Use them for teams to understand and improve their own system, trending over time, not to rank teams or individuals. Gaming is easy (splitting deployments, not counting incidents), so pair metrics with qualitative insight and look for bottlenecks they reveal (long review queues, manual approvals, slow tests). Measure from systems (version control, CI/CD, incident tools) rather than self-reporting.

Key Points

• Deployment frequency, lead time, change failure rate, recovery time.
• Speed and stability together.
• For improvement, not ranking.
• Collect from systems; look for bottlenecks.

Summary

The browser parses the URL, checks caches, resolves the domain through DNS (browser, OS, resolver, root, TLD and authoritative servers), opens a TCP connection (or QUIC for HTTP/3) to the IP, performs a TLS handshake and validates the certificate, sends the HTTP request, which passes through CDNs, load balancers and proxies to an application server that may call databases and caches, then receives the response and renders the page, fetching more resources.

Longer Answer

A DevOps-flavoured answer adds where things go wrong and how you would see it: DNS misconfiguration or TTL issues, expired certificates, load balancer health checks removing all targets, connection limits, slow upstream dependencies, CDN caching headers. Mention HTTP keep-alive and connection reuse, redirects (HTTP to HTTPS), cookies, and the tools you would use at each layer (dig, curl -v, openssl s_client, load-balancer logs, traces).

Key Points

• URL parse, cache, DNS resolution.
• TCP/QUIC, TLS handshake, certificate check.
• HTTP through CDN, LB, proxies to the app.
• Know failure points and tools per layer.

Summary

Trunk-based development (short-lived branches merged to main at least daily, kept releasable, with unfinished work behind feature flags) supports continuous delivery and is associated with higher delivery performance in DORA's research. GitFlow (long-lived develop and release branches) suits scheduled releases of versioned software but creates merge pain and slows feedback.

Longer Answer

Trunk-based development requires strong automated testing, fast CI, feature flags or branch-by-abstraction for incomplete work, and code review that keeps up (small pull requests). GitFlow can still fit products that ship versioned releases to customers who upgrade on their own schedule (mobile apps, on-premise software), though even there many teams use trunk-based development with release branches cut from main.

Key Points

• Trunk-based: small, frequent merges to main.
• Associated with higher delivery performance.
• Needs fast CI and feature flags.
• GitFlow for versioned, scheduled releases.

Summary

The same build artifact (a container image digest, a JAR, a package) is promoted unchanged from test to staging to production; only configuration differs per environment. You never rebuild for production.

Longer Answer

Rebuilding per environment means what you tested is not what you ship: dependencies may resolve differently, base images may have changed, and build steps may vary. Promoting a single artifact gives traceability (this digest came from this commit and passed these tests), simpler rollbacks (redeploy the previous digest) and supply-chain integrity (sign once, verify everywhere). It requires externalised configuration and environment-agnostic builds.

Key Points

• Same artifact through every environment.
• Config injected per environment.
• Traceability and easy rollback.
• Rebuilding breaks test validity.

Summary

Treat the pipeline as production infrastructure: pin third-party actions and plugins to commit SHAs, minimise and scope permissions (read-only tokens by default), isolate untrusted code (forked pull requests) from secrets, use ephemeral, hardened runners, protect branches and require reviews for pipeline changes, verify dependencies, and produce signed artifacts with provenance.

Longer Answer

Real incidents (compromised GitHub Actions, malicious npm packages, the SolarWinds and Codecov breaches) show the pipeline is a high-value target because it holds credentials and produces what runs in production. Frameworks such as SLSA define increasing levels of build integrity: scripted builds, provenance, hardened isolated builders, and two-person review. Self-hosted runners on persistent machines are risky for public repositories because a malicious job can persist on the host.

Key Points

• Pin actions by SHA; least-privilege tokens.
• Isolate untrusted PRs from secrets.
• Ephemeral hardened runners.
• Signed artifacts with provenance (SLSA).

Summary

Measure where time goes, then: cache dependencies and Docker layers, run independent jobs in parallel and split test suites across runners, run only what changed (affected-project detection in monorepos), move slow end-to-end tests out of the per-commit path into a later stage, fix or quarantine flaky tests, and use faster runners where the cost is justified.

Longer Answer

Targets: under 10 minutes for the commit stage is a common goal, because slow feedback leads to batching changes and context switching. Also check for waiting rather than working (queued jobs, insufficient runners), redundant steps (building the same thing twice), and tests that sleep. Track pipeline duration and failure rate as metrics over time.

Key Points

• Measure first.
• Cache, parallelise, shard tests.
• Run only affected work.
• Fix flaky tests; move slow tests later.

Summary

Platform engineering builds an internal developer platform: self-service, paved-road tooling (service templates, CI/CD, environments, observability, secrets, infrastructure modules) operated as a product for internal developers, so product teams can ship and run software without each becoming experts in every tool. It is a way to scale DevOps practices across many teams.

Longer Answer

"You build it, you run it" can overload product teams with infrastructure complexity; a platform team reduces cognitive load by offering good defaults and abstractions while keeping teams autonomous. Success factors: treat developers as customers (research, feedback, adoption metrics), make the golden path the easiest path rather than mandatory, document well, and avoid building a ticket-driven operations team under a new name. Developer portals such as Backstage often front the platform.

Key Points

• Internal developer platform as a product.
• Self-service golden paths.
• Reduces cognitive load on product teams.
• Measure adoption and developer experience.

Summary

A recursive resolver answers a query by asking root servers, then the TLD servers, then the domain's authoritative servers, and caches results for the record's TTL. Record types include A/AAAA (addresses), CNAME (aliases), MX, TXT, NS and SRV.

Longer Answer

Production issues: changes that take hours to propagate because of long TTLs (lower TTLs well before migrations); CNAME at the zone apex not allowed (use provider alias records); expired domains; misconfigured delegation; negative caching of NXDOMAIN; DNS resolver overload inside clusters; split-horizon DNS confusion between internal and external views; and DNS-based failover that clients ignore because they cache longer than the TTL. "It's always DNS" is a joke because DNS sits under almost everything.

Key Points

• Recursive resolution and caching by TTL.
• Common record types.
• Lower TTLs before changes; apex aliases.
• Resolver overload and caching surprises.

Summary

An L4 (transport-layer) load balancer forwards TCP/UDP connections based on IPs and ports without understanding the protocol: very fast, protocol-agnostic, preserves end-to-end TLS. An L7 (application-layer) load balancer understands HTTP: it can route by host, path and headers, terminate TLS, rewrite requests, apply WAF rules, and balance individual requests, including HTTP/2 and gRPC streams.

Longer Answer

Use L7 (for example AWS ALB, Envoy, NGINX) for web APIs and microservices needing routing and per-request balancing; use L4 (AWS NLB, HAProxy in TCP mode) for non-HTTP protocols, extreme throughput, static IPs, or when TLS must not be terminated. Related concepts: health checks, connection draining, sticky sessions (avoid when possible), and idle timeouts that must be aligned with application keep-alive settings.

Key Points

• L4: connections, fast, protocol-agnostic.
• L7: HTTP-aware routing, TLS, WAF.
• Per-request balancing needs L7.
• Align health checks and timeouts.

Summary

Make it part of the platform: standard instrumentation (OpenTelemetry) in service templates, structured logs with trace IDs, golden-signal dashboards and SLO-based alerts per service created by default, central log, metric and trace storage with retention policies, and runbooks linked from alerts.

Longer Answer

Also manage its cost and quality: control metric cardinality and log volume (sampling, retention tiers), avoid alert fatigue by alerting on symptoms and SLO burn rather than every cause, and make telemetry easy to query during incidents (consistent labels such as service, version, environment, region). Include deploy markers so regressions are easy to correlate with changes.

Key Points

• Instrumentation built into templates.
• Logs, metrics, traces correlated.
• SLO-based alerting with runbooks.
• Control cardinality, volume and cost.

Deep dive: Observability and Operations

Summary

Make cost visible and owned: consistent tagging and allocation per team and service, dashboards and budgets with alerts, cost reviewed alongside performance in design and pull requests where relevant; then optimise: right-size, autoscale, turn off idle non-production resources, use commitments (Savings Plans, reserved instances) for steady load and spot for interruptible work, and clean up orphaned resources.

Longer Answer

Common big wins: idle or oversized databases and instances, unattached volumes and old snapshots, NAT gateway and cross-zone data transfer, log and metric ingestion, and over-provisioned Kubernetes requests. FinOps is a collaboration between engineering, finance and product: engineers make the trade-offs, finance provides visibility and forecasting. Automate guardrails (policies requiring tags, budgets per account) rather than relying on monthly surprises.

Key Points

• Tagging, allocation, budgets, alerts.
• Right-size, autoscale, schedule non-prod.
• Commitments and spot.
• Data transfer and observability are hidden costs.

Summary

Continuous integration: everyone merges small changes to the main branch frequently, and each merge is built and tested automatically. Continuous delivery: every change that passes the pipeline is releasable and can be deployed to production at any time with a push-button, low-risk process. Continuous deployment: every change that passes the pipeline is deployed to production automatically, with no manual gate.

Longer Answer

In practice the difference shows in where humans intervene: continuous delivery may keep a manual approval or release schedule for business reasons; continuous deployment removes it and relies on automated tests, progressive rollouts and fast rollback. Both require trunk-based development, comprehensive automation and feature flags to decouple deploy from release. A team that "does CI" but merges once a week from long-lived branches is not really doing CI.

Key Points

• CI: frequent merges, automated tests.
• Delivery: always releasable.
• Deployment: automatic to prod.
• Feature flags decouple deploy from release.

Summary

Mitigate first: stop the rollout and roll back to the last known good version (or forward with a known fix if rollback is unsafe), communicate status, and confirm recovery with metrics. Then investigate why the pipeline let it through and why it failed halfway, and add safeguards.

Longer Answer

Considerations: was a database migration already applied (is the old code compatible)? Are there mixed versions running? Are caches or queues holding data written by the new version? Afterwards, run a blameless postmortem: what signals could have stopped the rollout earlier (canary analysis, health checks), what test would have caught it, and how to make rollback faster and safer.

Key Points

• Stop the rollout; roll back.
• Check migrations and mixed versions.
• Communicate and verify recovery.
• Postmortem; add automated safeguards.

Deep dive: Site Reliability Engineering interview questions

Summary

Processes and signals, systemd services and logs (journalctl), file permissions and users, package management, networking tools (ip, ss, dig, curl, tcpdump), disk and filesystem management, performance tools (top, vmstat, iostat, free), text processing (grep, awk, sed, jq), SSH and certificates, and an understanding of containers' building blocks (namespaces, cgroups).

Longer Answer

Interviewers often probe with scenarios: a server is slow, a disk is full, a service will not start, a port is not reachable. A good answer shows a methodical workflow (check service status and logs, resources, network listeners, recent changes) rather than memorised commands.

Key Points

• Processes, systemd, permissions.
• Networking and performance tools.
• Text processing and jq.
• Namespaces and cgroups.

Summary

Keep desired state in Git: infrastructure as code (Terraform/OpenTofu) applied through PR-driven pipelines with reviewed plans, and Kubernetes manifests reconciled by an in-cluster agent (Argo CD or Flux). Every change is a reviewed commit, environments are folders or repositories, and drift is detected and corrected automatically.

Longer Answer

Practical design: separate repositories or folders for platform components and applications, promotion between environments by pull request (changing an image tag or version in the next environment's folder), secrets referenced from a secret manager rather than stored in Git, and policy checks on pull requests. Infrastructure GitOps is usually push-based (CI runs Terraform after merge), while Kubernetes GitOps is pull-based.

Key Points

• Git as source of truth for infra and apps.
• Reviewed plans; in-cluster reconcilers.
• Promotion by pull request.
• Secrets referenced, not stored.

Deep dive: Terraform interview questions

Summary

Agree on RTO and RPO per system with the business, choose a recovery strategy that meets them (backup and restore, pilot light, warm standby, active-active), automate the recovery steps (infrastructure as code, scripted restores, DNS failover), and test them regularly with real restores and failover exercises, measuring actual recovery times.

Longer Answer

Common gaps found in tests: backups that cannot be restored (missing keys or permissions), restores far slower than expected, dependencies forgotten (DNS, secrets, container registries, third-party services), runbooks that are out of date, and knowledge held by one person. Store backups in separate accounts or regions with immutability to survive ransomware or account compromise.

Key Points

• RTO/RPO per system.
• Strategy to match objectives.
• Automate recovery; test regularly.
• Isolated, immutable backups.

Deep dive: Site Reliability Engineering interview questions

Summary

Start by understanding the current flow and pain (value stream mapping), then automate the highest-friction steps incrementally: version control and CI with tests for every change, automated, repeatable deployments to a test environment, then to production with a rollback plan, then smaller and more frequent releases. Measure lead time and failure rate to show progress, and change culture alongside tooling.

Longer Answer

Practical sequencing: make the build reproducible; add a deployment script that does exactly what the manual checklist does; run it in staging until trusted; use it for production; then shrink batch size. Involve operations and developers together (shared on-call, shared goals), celebrate wins, and avoid a big-bang tool rollout that ignores how people work. Monthly releases often hide risk in large batches; smaller releases reduce it.

Key Points

• Map the value stream; find pain.
• Automate incrementally.
• Measure lead time and failures.
• Shrink batch size; shared ownership.

Summary

Use STAR: the situation (for example releases every two weeks with frequent rollbacks), your task, the actions (automated tests and deployments, canary releases, smaller batches, better monitoring), and measured results (deployment frequency from biweekly to daily, change failure rate from 25% to 8%, lead time from days to hours).

Longer Answer

Interviewers want specifics: what you personally did, how you got buy-in, what trade-offs you made, what did not work, and evidence. Quote metrics you can explain (DORA metrics, incident counts, pipeline duration). A good story also covers sustaining the change: documentation, ownership, and how the team kept improving after you moved on.

Key Points

• STAR structure.
• Your actions and trade-offs.
• Before/after metrics.
• How the change was sustained.

Deep dive: Behavioral & STAR interview questions