DevOps'ish 328: Samsung bricks its own fridges, Cantrill on the contagion of fear, Kubernetes flags unused PVCs, and more

A database deployment just broke production. How fast can you explain what changed? (SPONSOR)

Liquibase Change Intelligence gives teams one place to investigate database change activity, drift, policy outcomes, and risk. AI-driven analysis helps you understand what changed, what happened, and what to do next. See Change Intelligence in action →

Kubernetes v1.37: Tracking When a PersistentVolumeClaim Was Last Used (Beta) (5 minute read)

Roman Bednář of Red Hat on PersistentVolumeClaimUnusedSinceTime reaching beta and on by default. The PVC protection controller now sets an Unused condition on every claim, True with reason NoPodsUsingPVC or False with PodUsingPVC. Completed pods do not count, so a Job with restartPolicy: Never releases the claim when it finishes, and pending pods do count even when they will never schedule. The payoff is lastTransitionTime, which timestamps the moment a PVC went idle, plus a jq one-liner for everything untouched in 30 days. Kubernetes never deletes PVCs when pods disappear, so upgrade and go see what you have been paying for. SIG Storage, KEP-5541.

What I learned from joining the Blog team (8 minute read)

Kashish Verma on reviewing for the SIG Docs blog team. Posts merge twice: a draft PR with draft: true in the front matter, then a publish PR swapping that for date: YYYY-MM-DD. The trap is that the front matter says draft while the GitHub PR must not be in Draft state. Verma’s diagnosis of late posts: “Most of the time, when a blog does not go out on time, it is a front matter issue.” Netlify previews need /blog/1/01/01/{slug} appended by hand, then a check on mobile. sig-docs meets biweekly, Tuesdays at 17:30 UTC. His advice to reviewers travels well past Kubernetes: “Help the author see the problem. Do not just flag it, explain it.”

Owners mourn spoiled food after firmware update bricks Samsung smart fridges (3 minute read)

Scharon Harding on Samsung Bespoke AI fridges, mostly four-door models from 2024 on, that lost power and stopped cooling right after a SmartThings firmware push on September 22. Samsung told Ars the problem is limited to Korea and happened “due to an error during Samsung’s internal testing,” a sentence worth reading twice, because internal testing reached customer appliances. SBS Korea counts hundreds of cases; some owners heard a technician might not come until October, and Chuseok started that Thursday. Samsung would not say how many units or what prevents a repeat. No canary, no opt-out, no recovery path an owner can reach.

The contagion of fear (6 minute read)

Bryan Cantrill opens with a confession. In his first year of university, he and some classmates told the humanities lab next door that a virus had escaped and they needed to eject their floppies. Bedlam, lost work the week before finals, two letters of apology apiece. He tells it now because he has “never seen fear sown so irresponsibly by putative technologists,” pointing at ex-Anthropic employee Jacob Coxon claiming, and Anthropic’s Evan Hubinger agreeing, that the odds AI will “kill all humans” are “>10% in the next decade.” His rebuttal comes from building computers: engineering is not an act of intelligence alone; “robots will do this!” glosses over the fact that robots cannot, and whatever physical agency software has is agency we granted it. Sagan’s standard applies, and the burden sits with whoever makes the claim. It argues timelines and actuators, so it will not move anyone who thinks cognition is the only bottleneck.

What I believe about the future of software development (4 minute read)

Sixteen predictions, originally an X post, reposted to timestamp what Thorsten Ball believed in September 2026. Code review dies, unit tests might follow, the craft of writing code disappears (“Yes, there are still Italian shoe makers around. But look at your feet.”), most bugs become “you asked for the wrong thing” bugs, open source stops making sense because Linus’s Law now has artificial eyeballs, the terminal dies, and the PM/Design/Eng triad goes with it. The evidence is one anecdote: a model wrote 900 lines of Arduino C that compiled clean and ran. That is also exactly the kind of small, well-trodden program models are best at. He concedes a generation-long timeline, then asks whether you want the job that survives it.

Google Open-Sources AX a Kubernetes Style Orchestrator for Autonomous AI Agents (3 minute read)

Olimpiu Pop on Google open sourcing AX under Apache 2.0, a declarative orchestrator for agent workloads running on Agent Substrate. The framing is the useful part: agents are neither stateless microservices nor batch jobs, so each session gets an isolated actor sandbox with hard CPU and memory limits, sub-second suspend and resume, and dozens of tasks multiplexed onto shared workers while they sit idle waiting on a model. Manifests under ax.io/v1alpha1 cover Task, Workspace, Model, and Gateway. Two caveats: the community reaction is Pop’s read of Hacker News and Reddit with nobody named, and the CRD complaint he reports does not match what the repo says it does.

Jev’s Architecture Unmasked (28 minute read)

Archer Hume spent 10,000 API calls working out how TypeSafe’s Jev is built, “and why most of the grifter takes on X are completely wrong.” Jev takes shared state plus multiple-choice questions and returns probability distributions directly instead of generating text regarding its own confidence. His reconstruction: a causal transformer with a shared state prefix, isolated question branches, listwise option processing, and no autoregressive generation. Evidence includes latency profiling, token accounting, option ordering, reference card placement, and tokenizer fingerprinting against 192 public tokenizers, none of which match. One finding to act on: reversing the option order moved a classification from 0.84 to 0.96, which flips the decision on either side of a 0.9 threshold. He marks his own uncertainty carefully throughout, which is what makes it worth the time. “Black box APIs make it shockingly easy to throw a blanket over the ghost and get a rough shape of what the architecture looks like.”

git worktree gotchas (6 minute read)

Every git worktree surprise traces back to one fact: in a linked worktree, .git is a pointer file, not a directory, so anything walking up the tree hunting for a .git directory sails past the root you wanted. Four commands fix it: git rev-parse --show-toplevel, --git-common-dir, dirname on that for the main working tree, and --git-path hooks. The one that will bite is hooks. They resolve to the main working tree’s .git/hooks, so the moment you sandbox a worktree to keep an agent contained, that directory sits outside the sandbox, and hooks silently stop firing. The fix is a checked-in .githooks plus git config core.hooksPath .githooks, relative on purpose, set once to cover every worktree.

Scaling Golang CI by Replacing actions/setup-go (11 minute read)

Lukas Schwab and Peter Downs found that actions/setup-go keys its cache on OS, arch, Go version, and a go.mod hash, which says nothing about what the job does. Parallel lint and test jobs race to write different cache states to one key, and GitHub’s cache is immutable once written, so the loser restores garbage. Their cloudx-io/setup-go adds job identity, ref name, a go.sum hash, and the run ID, making an exact match impossible so every job writes fresh. Median test time fell from 131 seconds to 41, and a backtest over 4,011 commits says “86% of actions/setup-go test runs are unnecessary.” The cost is linear cache growth that needs pruning. The lesson generalizes past Go: a cache key that does not encode what the job does will eventually hand it the wrong cache, silently.

Announcing RustFS 1.0.0 GA: Production-Ready, Open Source, S3-Compatible Object Storage (3 minute read)

RustFS 1.0.0 GA under Apache 2.0, S3-compatible object storage in Rust, two years and seven months after the first commit. Erasure coding, tiering, replication, IAM, OIDC, KMS, SSE, mTLS, and OpenTelemetry, plus WebDAV, Swift, FTP, and MCP. S3 Tables puts an “Apache Iceberg REST Catalog built into the storage kernel.” Read the GA claim closely, because it covers object storage only and says nothing about the maturity of S3 Tables, replication, or KMS. The case for production readiness leans on 32,000 stars and 2.7 million self-reported deployments, with no benchmarks, no S3 compatibility matrix, no named reference deployments, and no security audit. Credit for telling you to change the default rustfsadmin credentials first.

google/ax - Apache-2.0 - Google’s open agentic orchestration runtime. The repo behind the InfoQ story above, deliberately kubectl-shaped: ax apply, ax get, and ax watch, plus ax ssh to look over a running agent’s shoulder and ax suspend and ax resume for idle ones. Three primitives: Task, Workspace, Model. It pointedly avoids CRDs, keeping state in Redis with Redis Streams because “storing millions of short-lived tasks as Kubernetes CRDs pushes etcd past its comfort zone,” which is the opposite of the complaint InfoQ’s commenters raise. Setup is heavy. Agent Substrate has to be running already, plus Go 1.27, ko, and a registry, with no local path. Google is the only model provider actually implemented, defaulting to gemini-3.8-flash. At v0.3.1, with a README warning that breaking changes are coming.

jaredpalmer/kev - Apache-2.0 - Jev-like family of decision models built on top of Qwen3.5/3.8 you can train and run on your own. Nine days old, 7,200 stars, built on the architecture in Hume’s post above. Four sizes, each a rank-16 LoRA adapter plus a small pointer head on a Qwen base, so fine-tuning runs about a dollar on an H100. The API clones TypeSafe’s System One, so their SDK works against it unchanged. Kev-27B hits 0.848 on unseen sources against Jev’s 0.857, but the gap is real on knowledge: Kev-9B scores 0.74 on MMLU where Jev gets 0.90. Training used at most 384 state tokens while the server accepts 65,536, so long documents suffer. The server is also unauthenticated by default, so never pass --host 0.0.0.0 without setting KEV_API_KEY. Unusually honest evals, including “We don’t know what Jev was trained on, so this isn’t a controlled comparison of the two architectures.”

ringlochid/oh-my-subagents - MIT, except the console - Local subagent orchestration for Codex and Claude, with persistent task state, reusable teams, and interruption recovery. Not a pile of agent markdown files, despite the name: it is a Python app you pipx install, with a per-user background service and a console on 127.0.0.1:18125, keeping state in SQLite or Postgres. It commits child assignments alongside the parent’s wait, so a closed terminal or dropped provider session doesn’t erase the record of who owned what. Capabilities are denied by default and never inherit from a parent. Eight starter workflows. Two things to know: the license is MIT except console/, which derives from n8n under the Sustainable Use License and limits commercial redistribution, and the docs invent a lot of capitalized nouns, so budget time on vocabulary. 147 stars.

blader/humanizer - MIT - Agent skill that removes signs of AI-produced writing from text. Disclosure: this one runs over this newsletter every week. Plain Markdown, so it works with any agent that supports skills. It codifies 25 patterns ranked by strength, the first five worth an edit on a single sighting: “not X but Y” constructions, the repeated dramatic closer, sayings that sound deep instead of specific, staged run-ups, and arguing with objections nobody raised. Five more only count when they cluster in one passage, which is the right call for dashes & passive voice. v3.0.0 cut 35 patterns to 25, dropping two that Wikipedia now lists as human habits. It refuses to invent a fact to fill a hole and asks instead, and it treats the content you hand it as content to edit rather than instructions.

alex/aws-ec2-vpn - No license - An OpenTofu module that boots a tiny EC2 instance which acts as both SSH Socks proxy and Wireguard VPN. Alexander Fortin’s throwaway VPN, on his own Forgejo instance. It boots AL2023 on a t4g.nano and resolves the AMI through the official SSM parameter instead of a most_recent lookup, so plans stop proposing surprise replacements. Steal that detail regardless of the rest. Cloud-init handles WireGuard, IP forwarding, and MASQUERADE, and a justfile wraps just init apply, just wireguard-conf, and just destroy. The security group opens 22/tcp and 51820/udp to the whole internet, worth narrowing if this outlives the afternoon, and the README says plainly that private keys sit in plaintext in tfvars and state. No LICENSE file, so all rights reserved by default.

Want to reach 6,800+ developers, SREs, & DevOps leaders every week?

DevOps'ish delivers a 46% open rate and 10% click rate — well above industry averages — to a global audience of practitioners, VCs, and C-suite decision-makers. Your brand gets exclusive placement (one sponsor per issue, zero competition) in a text-based newsletter that bypasses ad blockers entirely. Get in front of the people who build and run the internet!