DevOps'ish 326: Automattic's board benches Mullenweg, 100 agents hack a guy for $210, Kubernetes 1.37, and more

Kubernetes v1.37: Advancing Workload-Aware Scheduling (13 minute read)

Antoni Zawodny leads a seven-author write-up from Google and Microsoft on what actually changed this cycle. The core Workload and PodGroup APIs sit at scheduling.k8s.io/v1beta1 with gang scheduling at beta, the separate WorkloadAwarePreemption gate got folded into GenericWorkload, and v1alpha2 was replaced outright by v1alpha3 with breaking changes around disruptionMode, whose modes were renamed from PodGroup and Pod to all and single. minCount is mutable now, so controllers can resize a running gang, and only the top level PodGroup gets queued instead of every member Pod individually. The v1.36 gap where default single Pod preemption ignored a PodGroup’s disruptionMode is closed, and a new alpha CompositePodGroup API adds tree-shaped hierarchies with minGroupCount and multi-level topology constraints, which is what JobSet and LeaderWorkerSet have been waiting on. All of it is still turned off by default, so budget time for gate flipping in a test cluster. WG Workload Aware Scheduling is aiming both APIs at GA in v1.38.

Kubernetes v1.37: KubeletInUserNamespace (aka Rootless mode) Graduates to Beta (5 minute read)

Akihiro Suda of NTT walks through the beta promotion of KubeletInUserNamespace, which lets every node component, kubelet, CRI and OCI runtimes, CNI plugins, and kube-proxy, run as a non-root host user inside a Linux user namespace. Do not confuse it with user namespaces for pods (hostUsers: false, GA since v1.36), which isolates pods while node components still run as root. The gate itself is deliberately unexciting, mostly letting the kubelet swallow permission errors when setting sysctls like vm.overcommit_memory and when reading /dev/kmsg, and the namespace still has to be created outside Kubernetes by rootless Docker, Podman, nerdctl, kind, minikube, or k3s. Beta means on by default, nothing changes for existing rootful clusters, and runningInUserNamespace now shows up on the node so you can taint the ones that cannot host workloads needing real root. The payoff is blast radius: the CVEs Suda cites, including cr8escape and CVE-2026-53488 in containerd, all end at a non-root account instead of the kernel.

Kubernetes v1.37: Introducing Node Lifecycle Conditions (5 minute read)

Ryan Hallisey of NVIDIA covers five new well-known Node conditions: DrainInProgress, Drained, MaintenancePlanned, MaintenanceInProgress, and GracefulNodeShutdownInProgress. They behave like any other condition, with a machine-readable reason and a human-readable message, which finally gives operators a Kubernetes-owned place to say a node is draining instead of everyone inferring it from readiness, taints, or terminating pods. The alpha NodeLifecycleConditions gate is off by default and effectively a no-op: nothing restricts who writes the conditions, no core component reads them, and you do not need the gate on to start publishing today. Keep using kubectl cordon, kubectl drain, and taints for actual eviction behavior and treat the conditions as a status channel for dashboards and external automation. The payoff comes later, the example being DaemonSet rollouts that currently burn availability budget on nodes an admin deliberately took out of service. Pick an owner per condition now to avoid conflicting writes.

Building an AI Factory on Kubernetes (12 minute read)

Hrittik Roy maps out an eleven layer stack for running a shared GPU pool as a multi tenant platform, from bare metal provisioning with Metal3, Ironic, and Tinkerbell up through Cluster API, Node Feature Discovery, inference serving, and observability with Prometheus, OpenTelemetry, and DCGM exporters. The central argument is that utilization, not tokens per second, is the metric that matters, because accelerators are the dominant capital expense and the old device plugin model hands an entire GPU to a pod that barely uses it. Dynamic Resource Allocation went GA in v1.34 and allows richer device description, but fractional density still depends on the underlying device implementation, and operators who want hard tenant isolation end up assigning whole GPUs and eating the waste. The workload sections are concrete: vLLM, KServe, and llm-d for inference, Slinky for SLURM batch, KubeVirt for VM tenants, LiteLLM as an OpenAI compatible front door, DCGM GPU second metering and OpenCost for billing. Roy works at vCluster and the tenant cluster pattern gets the most flattering treatment here, but the layer list works as a checklist, and the closing point about NUMA alignment and collectives stalling at the slowest data path is the kind of thing people learn the hard way.

Automattic CEO Matt Mullenweg Put on ‘Leave of Absence’ (2 minute read)

Samantha Cole reports that on September 9, Mullenweg told Automattic staff in a company-wide Slack announcement that the board had voted to place him on a paid leave of absence, a vote he says he opposed. In that message he named CFO Mark Davies and board members Ann Dunwoody, Toni Schneider, and Sue Decker, said he received the resolution 50 minutes before the meeting, and said his requests for time to have independent counsel review it were denied. Davies was voted interim CEO. An Automattic spokesperson confirmed the leave and said the board has full confidence in Davies, Schneider wrote that the board asked Mullenweg to step away from the CEO role, and Davies wrote that Mullenweg stays on the board with a voice in setting direction. Every quote here comes from a named person or the company, there is no anonymous sourcing, and none of them say why. Automattic holds a large share of the people who work on WordPress, so contributors just watched leadership change hands with no stated cause, which is a rough way to run comms for anyone trying to plan around the project.

I Asked 100 Agents to Hack Me (9 minute read)

Shrivu Shankar pointed roughly 100 self-hosted agents at his own accounts for five hours and wrote down what happened. The agents ran abliterated open-weight models off Hugging Face, derived from GLM-5.3 and DeepSeek V4 Flash, served on two B300 GPUs in Modal and driven by a vanilla Codex CLI in Docker containers with internet access, a preconfigured Chrome, and scoped Railway API keys. Results: three accounts compromised through software flaws in old side projects, mostly IDOR variants and mismanaged credentials, two more via password brute forcing against variants built from public breach data, 16 social engineering attempts including a fake Substack phishing page and comments on his Hacker News posts, plus accurate aggregation of his phone number and home address from free people-search sites. No third-party zero-days, and no tier 0 accounts fell. Total cost was $210 in GPU hours, about $40 per compromised account, and he figures the same run costs under $5 within a year. The most useful note for defenders: agents shifted work onto free email and static hosting outside his control, and delayed messages kept landing after he shut the GPUs down, so a kill switch that only reaches inference is not a kill switch.

Scoop: Anthropic whistleblower gave up his equity to leave the company (2 minute read)

Anthropic researcher Jacob Coxon told Axios he quit over AI safety concerns four months in, two months short of the six month cliff, so none of his equity vested. His framing: “I no longer have anything to gain by juicing up Anthropic’s valuation.” He says he has not personally seen Anthropic trade away safety to stay ahead of rivals, but expects competitive pressure to do the damage eventually, because “if you’re under pressure to race, you have to cut corners” or skip steps in the oversight process. He also thinks threat inflation runs both ways, describing what “sometimes feels like there’s maybe excessive paranoia of OpenAI, excessive paranoia of China” used to justify moving faster. The operationally interesting part is evaluations: models noticing they are under test used to be a thought experiment and is now “just a daily fact of working with these AIs,” which makes monitoring harder at exactly the moment capability is climbing. Coxon still holds equity in his previous employer, OpenAI, which is worth knowing when weighing the incentives.

Portal by Spotify cut my Claude Code token usage by 90% (9 minute read)

Dimitri Mazmanov wired up a Claude Code plugin called shunt that routes bulk file reads and boilerplate generation away from Claude and over to a cheaper worker model, in this case Gemini 2.5 Flash running as two AiKA Modes on Portal by Spotify’s serverless runtime. The enforcement is the interesting part: PreToolUse hooks block reads past a 350 line threshold, so the routing holds even when the agent ignores the skill file telling it to delegate. A bulk-reader mode returns structured bullets instead of file contents, and a code-writer mode emits raw code straight to disk so those tokens never pass through Claude’s output at all. The headline 90% is a mean across four scenarios in one Java monorepo, comparing what Claude would have consumed reading files directly against the delegated summaries, which is a synthetic counterfactual rather than a billing comparison, so treat it as directional. To his credit he lists where it falls apart: editing breaks because worker summaries lose reliable line numbers, the worker model missed thread safety bugs Claude caught, and a 10 to 30 second round trip makes delegation a loss on small reads.

Firecracker, io_uring and validating layers of defense (4 minute read)

antiTree takes a quietly merged Firecracker patch and uses it to ask the question that actually matters: if an attacker gets control of the jailed VMM process, what stops them next? The bug was an aarch64-only TOCTOU in the jailer’s copy_cache_info(), where a write followed by an unchecked libc::chown() both followed symlinks and never verified they were operating on the file they expected. The more useful finding is that seccomp, the backstop meant to contain exactly this situation, does not hold, because Firecracker’s policy permits io_uring: io_uring_setup is allowed, and from there IORING_OP_RENAMEAT, IORING_OP_MKDIRAT, and IORING_OP_SYMLINKAT execute syscalls the filter blocks when called directly. io_uring is there because the async Block IO Engine needed it back in v1.0.0, four years ago, and that mode is still labeled developer preview. There is no working breakout in the post and he says so plainly, but the operational point stands: if you are not using async block IO, drop io_uring from the allowed set, and stop grading a seccomp policy by counting blocked syscalls when one multiplexing interface undoes the rest.

The US military just turned off ad tracking on its phones. Maybe you should too (2 minute read)

Graham Cluley reports that the Army, Air Force, Navy, and Special Operations Command have disabled mobile advertising IDs on government-issued phones and computers, which came to light through letters released by Senator Ron Wyden after months of pressure on the Pentagon. The threat model is not exotic: the IDFA on iOS and the GAID on Android persist across apps, apps hand them to third parties, brokers buy and sell the result, and pairing that identifier with app location data results in a pattern of life for a specific handset. Back in April, CENTCOM reportedly warned of “multiple threat reports concerning adversary exploitation of commercial location data,” which is exactly the targeting data you would want before launching a drone. Wyden and Representative Pat Harrigan have asked the DoD inspector general to investigate whether the services actually countered the risk. Government devices are only part of the surface area: troops in Jordan were told their personal phones would be confiscated, and the Navy told personnel last month to scrub their social media against open-source collection. Privacy people have said this for a decade, the fix is two settings screens deep, and it took CENTCOM targeting reports to move it.

Unethical Ways to Manage Technical Debt: Is ignoring it unethical? (13 minute read)

Tom Limoncelli argues that most technical debt writing is detached from how organizations actually behave, so he walks through six tactics engineers really use. He starts with the claim that the term itself is the first mistake: engineers hear “debt” as bad, MBAs hear it as capital that makes money, so he suggests calling it operational drag in front of non engineers. The list runs from ignoring it entirely, which he notes is rational given that most CTOs and engineers move on in a few years, to the electrician’s rule (touch a file, bring it up to standard) and the 20 percent per release rule, both of which keep debt in check but never fund major refactoring, and the 20 percent rule gets awkward under CI/CD where there is no clean release boundary. The strongest section argues that offering to hit an impossible deadline by taking on debt is the actual ethical failure, with AT&T Wireless as the case study: skipped Siebel CRM upgrades made a later multi release jump impossible, the company missed an FCC deadline, and selling the customer base to Cingular was cheaper than fixing the systems. This is a column, not research, and the evidence is one person’s war stories, but the naming trick and the terminology swap are cheap to try.

Introducing CUDA Rust: Two Tracks for Writing GPU Kernels (14 minute read)

NVIDIA is putting real engineering behind writing GPU kernels in Rust, split into two tracks at different levels. The SIMT track, cuda-oxide, is a custom rustc codegen backend that routes #[kernel] functions through Rust MIR, the community Pliron IR framework, and LLVM IR down to PTX; it wants Linux, compute capability 8.0 or later, CUDA 12.x, clang with libclang headers, a system LLVM, and a pinned nightly installed straight from git. The Tile track, cutile-rs, works a level up on tiles instead of scalars and JIT compiles through CUDA Tile IR, with much lighter requirements: CUDA 13.3, stable Rust 1.89 or newer, no nightly and no LLVM of your own, and the cutile crate already on crates.io so cargo add cutile is the whole setup. Both catch the classic aliasing mistake at compile time. As for what ships today, neither is production ready, cuda-oxide is early alpha, and cutile-rs is further along, already used outside NVIDIA in HuggingFace’s Grout inference engine and in mistral.rs. Coverage is incomplete, shared memory in the Tile track still requires unsafe, and interoperability with CUDA C++ and CUDA Python is roadmap rather than something you can lean on.

A Stealth Startup Thinks It Just Hacked the Memory Shortage (6 minute read)

Lauren Goode covers Kepler Computing, a San Jose outfit founded in 2018 that spent seven years in stealth and is now pitching itself as a fix for the HBM supply crunch. CEO Debo Olaosebikan and CTO Sasi Manipatruni say a 3D stacking technique plus a proprietary ferroelectric composite lets them raise HBM and SRAM density without extreme ultraviolet lithography, inside fabs that already exist, with the goal of moving data through HBM at energy costs closer to SRAM. The company has raised $468 million from GlobalFoundries, Intel Capital, AMD Ventures, Baillie Gifford, and Bill Gates through Gates Frontier, and in July the Commerce Department committed up to $245 million more. The gap between pitch and silicon is the interesting part: roughly 2,000 wafers run to date, first HBM samples promised later this year, a Singapore ramp next year, and US production not until 2028. A GlobalFoundries executive notes the composite contains iron, a contaminant nasty enough that the process has to run on dedicated or fully encapsulated equipment. Wired points at Substrate as an indication that materials science demos and millions of shipped devices are different problems.

M2: Episode 1 (or, Asahi Linux on M3) (2 minute read)

James Calligeros announced that M3 series support landed in the Asahi Linux installer, covering MacBooks and iMacs with M3, M3 Pro, and M3 Max, with the M3 Ultra Mac Studio still unsupported. Nearly everything that works on M1 and M2 machines works here: webcam, internal microphones, USB 3 up to the 10 Gb/s hardware limit, hardware accelerated video decode including AV1, WiFi, and Bluetooth. The two big gaps are full DCP support and the GPU, so do not expect performant or power efficient 3D acceleration yet, and the missing DCP work also means sleep is broken because of the firmware provided framebuffer and HDMI stays disabled on MacBooks that have the port. Installing today means Expert mode, curl -L https://alx.sh/ | EXPERT=1 sh from a macOS terminal followed by a dnf upgrade, with the team planning to drop the Expert gate in time for the Fedora Linux 45 beta barring regressions. The post credits OpenCollective and GitHub Sponsors backers directly, noting several contributors only had access to M3 hardware because of that funding, which is the unglamorous reality of years of reverse engineering getting done at all.

The Shortest IPv6 Addresses (1 minute read)

Ache went looking for the IPv6 equivalent of 1.1.1.1, meaning the shortest address you can actually type, and worked out that the winners are the ones where only the first 16 bit group is non-zero so the rest collapses into ::. Since only 2000::/3 is routable in practice, that leaves 13 variable bits, or 8,192 candidates, which is small enough to brute force. The tooling is refreshingly unglamorous: a fish loop over seq 0 8192 that builds each address with printf '%x::' and fires a single ping -6 with a 0.7 second timeout. Four responded from a residential connection: 2409:: (APNIC, an Indian state government via NKN), 2600:: (Cogent), and 2a09:: plus 2a11:: (xTom, serving dns.sb), with the xTom pair answering DNS at around 161ms. Re-running from a non-residential IP turned up three more, including 2002::, the 6to4 address for 0.0.0.0, which the author admits he cannot explain. Not load bearing for your infrastructure, but a good reminder that IPv6 notation rules have exploitable edges.

alibaba/open-code-review - Apache-2.0 - Alibaba’s internal AI code review assistant, open sourced as a Go CLI called ocr that reads Git diffs and returns line-level comments. Hybrid design: deterministic pipelines and a built-in multi-language ruleset covering NPE, thread safety, XSS, and SQL injection, wrapped around an LLM agent. Needs Git 2.41 or newer and an OpenAI or Anthropic compatible endpoint, though delegation mode hands the review to whatever coding agent you already run, with Claude Code, Codex, Cursor, and OpenCode plugins in the repo.

Want to reach 6,800+ developers, SREs, & DevOps leaders every week?

DevOps'ish delivers a 46% open rate and 10% click rate — well above industry averages — to a global audience of practitioners, VCs, and C-suite decision-makers. Your brand gets exclusive placement (one sponsor per issue, zero competition) in a text-based newsletter that bypasses ad blockers entirely. Get in front of the people who build and run the internet!