DevOps'ish 262: Docker whales, Okta and Lapsus$ galore, Apple Outage, Kubernetes storage, using mtr, Kubernetes events in Slack, and more

DevOps’ish is brought to you our friends at Honeycomb. “Stop Letting Complexity Slow You Down” Honeycomb makes it easy to understand and troubleshoot complex relationships within your distributed services. Solve problems faster. Ship reliable and performant features. SPONSORED Events Calling all front-line DevOps and SRE practitioners: Join IR Conf on April 1 (no joke), a free, half-day virtual conference for industry experts and new voices in incident response. Incident responders will come together to swap horror stories about the biggest outages, discuss best practices, and gain a better understanding of how the best are dealing with incidents. Register to: Discover trends and tooling in resiliency and incidence response Gain guidance on how to structure your teams, tooling, and processes Learn better practices from incident response experts Connect with a community of incident responders who care about this, learn from each other SLOConf - Service Level Objective Conference MAY 9-12, 2022 ...

March 27, 2022 · 5 min · Chris Short

DevOps'ish 261: Secrets with GitOps, cr8escape, B1txor20, npm sabotage (again), updated Kubernetes Hardening Guide, ArgoCD and Starboard, scripting with Go, and more

This week I wanted to spur discussion around some GitOps hurdles folks are facing. As a co-chair, it’s not only my job to make sure we’re discussing how folks are handling things. We should be providing some guidance on how to implement those GitOps Principles in a practical manner. The discussions themselves are around “Management” Clusters, Progressively Delivery, and Handling Secrets with GitOps. That last one is sticking out in my mind quite a bit: GitOps Secrets Management. I’ll preface this by saying this is my opinion. It’s is not the opinion of the CNCF GitOps Working Group or OpenGitOps. Universally, it’s a bad idea to check secrets into git. Whether they’re encrypted or not that shouldn’t really matter. They’re still secrets and, in my opinion, encrypted or not, secrets shouldn’t live in git. A shared password safe is better than git. Even better an external secret store so you could utilize the Kubernetes Secrets Store CSI Driver. To me, it comes down to a few important things. ...

March 20, 2022 · 7 min · Chris Short

DevOps'ish 260: Organizational skills a key to productivity, don't spy on employees, Elementary OS implosion, Dirty Pipe, big ole amplification attacks, GitOps galore, lavished by Linkerd, and more

We are very fortunate. I remind myself of that every day. Our household’s collective efforts have put us where we are today (being some random dude from a town called Hickory; this is far further than I’d ever imagined). I have been posting pictures from my new office that we custom built in our basement. I’m very fortunate to have a room, with a solid door and some soundproofing, to work from in these times. The only thing to bother me are the things I let into my day. It was funny, the other day, when I got a knock on the door and two of my nephews rolled in with Max and Julie. “Wow, Uncle Chris! I didn’t realize you were down here!” “Oh wow, Uncle Chris! I love your office!” Even the kids think it’s cool. Like I said, very fortunate. This weekend, I live tweeted cleaning out two bins FULL of cables. Lots of old memories brought up. All normal and some great memories. Now, I’m in a space that is just for me. Julie has a space just for herself as well. Both getting improved upon a little bit more every day. ...

March 13, 2022 · 7 min · Chris Short

DevOps'ish 259: Doctor induced roller coasters, Tessa Kriesel, NSA security guides, Kubernetes Hardening, GitOps & Flux, Get Vault Secrets into Kubernetes, and more

If you follow me on Twitter you know it has been a roller coaster week on the home front. I was getting a haircut today and a topic that I often hear lately is how terrible doctor’s are of late about informing folks of bad news. Our family went from thinking the absolute worst as laid out by doctors one afternoon. Only to have a doc twelve hours later tell us, no, everything is fine (Yay! 🥳). We’ll do this one test to prove it. We’re very confident this is a false positive. A friend of mine and I talk about this phenomenon from time to time. But, even my barber had a friend that had the news he had 4-6 years to live and the doctor said, “Yeah. You better start checking off that bucket list.” Why has this become common place? This somewhat inhuman behavior has to be being taught somehow for it to be so widespread one would think. I get it, telling people they might have to deal with the death of a loved one is hard. But, let’s rattle off all scenarios not just the death one. pat on the shoulder ...

March 5, 2022 · 8 min · Chris Short

DevOps'ish 258: Agile fails without psychological safety, Intel and Snyk acquire companies, Nvidia gets hit and punches back, GitOps with Flux, OrioleDB, and more

Rough week here on the home front. Rough week in other places too. Reach out if you’re struggling. DevOps’ish is brought to you our friends at Honeycomb. “Stop Letting Complexity Slow You Down” Honeycomb makes it easy to understand and troubleshoot complex relationships within your distributed services. Solve problems faster. Ship reliable and performant features. SPONSORED People Employees Are Sick of Being Asked to Make Moral Compromises “Moral injury is experienced as a trauma response to witnessing or participating in workplace behaviors that contradict one’s moral beliefs in high-stakes situations and that have the potential of harming others physically, psychologically, socially, or economically, and it could prompt people to leave a company.” This is an interesting take. Perfect example (potentially extreme) of this is me and Snowden. It bothered me for years. After years of therapy I can finally live with it all. Chris Lattner Formally Steps Down From Swift’s Core Team “Chris Lattner’s current day focus is serving as co-founder and CEO of Modular AI with seeking to overhaul the AI/ML infrastructure world.” ...

February 27, 2022 · 5 min · Chris Short