DevOps’ish is back.

What did the hiatus look like? Why did it need to happen at all?

Like fixing an airplane while it’s in-flight, It’s hard assessing a problem while you’re in the middle of creating it. COVID-19 is just the sort of thing that the world turned upside down enough that I needed to take a break to reassess how to do the newsletter, in general. Meanwhile, sponsorships wholly dried up (for five weeks). The incentive I created for myself to write the newsletter when things were hard was gone two weeks after my family started sheltering. The world was in chaos, my processes weren’t aligned to the new state of things, and incentives were gone. When you couple that with a massive project that I was undertaking at work (and spending all my free time researching), the newsletter had to “fall off the plate” as it were. My medical world got turned upside down, too, because of COVID-19. I was rightfully put on hold for routine treatment so the medical infrastructure could adjust also. If my doctors take a break from my treatments, I don’t stand a chance at functioning at 100%.

Why is DevOps’ish back now?

You can all thank Accurics for the return of DevOps’ish. They are enabling DevOps’ish to run for as long as our relationship lasts! Huge shoutout to the leadership team at Accurics for making this possible.

I’ve got some improved ways and expectations around news reading, too (thanks to some enhancements from Inoreader). No longer will my goal be, read all the news every day. That’s unsustainable, as I learned. Meanwhile, the cloud native ecosystem had exploded since 2016 when the newsletter started. I still aim to deliver a piece of work that I’m proud of every week. I hope you enjoy this week’s issue as much as I enjoy bringing back DevOps’ish.

Is DevOps’ish moving to Mondays?

I don’t know. Maybe. I think my days of being able to spend Saturday night producing the entire newsletter might be coming to an end as I need to slice my time more and more between work projects, meetings (which there are more of now), real-life (which is harder now), and side projects (which I’ll always have). I’d rather keep the email on Sundays, but this week’s issue is coming out on Monday because I ended up having to change mail providers in the 11th hour.


Weekly Distance DevOps Lunch and Learn: This is a fun thing to come out of the crisis. I’m going to be to be joining as a subject of discussion this week, “Q&A with Chris Short about his no good, very bad AWS bill.”

The community hat rules the company hat in open source: “For kube-state-metrics maintainer and Red Hat employee Lili Cosic, what Red Hat wants from the project is beside the point” OMG I know her!!!

The AWS bill heard around the world: I cut my fingers on some sharp edges of the cloud. Did you know that Cloudflare might not respect multipart GETs as full file GETs? That’s gonna run up a bill real quick. Thank you to AWS for bailing me out of an AWS bill that was more than my mortgage. Thank you, Corey Quinn, for helping me navigate the process (subscribe to Corey’s newsletter). I’m still working on a solution but, it’s going to be replacing Cloudflare for sure at this point. I no longer trust Cloudflare.

What makes great DevOps teams tick: “The principles of DevOps go beyond just improving software development processes. They also help foster stronger, more productive teams”


DevSecOps: Risks and Best Practices: “It is now more important than ever to understand cloud infrastructure configuration practices that are creating exposures.”

The Five Ideals of DevOps: “In The Phoenix Project, we introduced the Three Ways underpinning DevOps. In my follow-up book, The Unicorn Project, I revisit Parts Unlimited and describe my learnings through the Five Ideals.

  • The First Ideal: Locality and Simplicity
  • The Second Ideal: Focus, Flow, and Joy
  • The Third Ideal: Improvement of Daily Work
  • The Fourth Ideal: Psychological Safety
  • The Fifth Ideal: Customer Focus”

Note: DevOps’ish may earn compensation for sales from links on this post through affiliate programs.

Patterns of Distributed Systems: “Distributed systems provide a particular challenge to program. They often require us to have multiple copies of data, which need to keep synchronized. Yet we cannot rely on processing nodes working reliably, and network delays can easily lead to inconsistencies. Despite this, many organizations rely on a range of core distributed software handling data storage, messaging, system management, and compute capability. These systems face common problems which they solve with similar solutions. This article recognizes and develops these solutions as patterns, with which we can build up an understanding of how to better understand, communicate and teach distributed system design.”

How to Champion GitOps in Your Organization: “By now you’ve heard all about GitOps and are convinced that GitOps is the most efficient way for development teams to go faster without them having to become Kubernetes gurus. However, making the switch to a cloud native technical solution may be the simplest part in your Kubernetes journey. Getting buy-in from your peers and championing GitOps throughout your organization could very well be the most challenging aspect of the cloud native transition.”

Open Sourcing the etcd Security Audit: The entire report is available here.

Cloud Native Backups, Disaster Recovery and Migrations on Kubernetes: “Loosely coupled architectures enable loosely coupled teams. The benefit being that loosely-coupled teams are unblocked; they’re able to iterate as and when they can, free from the scourge of meetings, long release cycles, change control boards and more.”

Backblaze takes aim at ‘costly and complex’ AWS cloud storage: “Backblaze offers free transfer to its S3-based cloud storage – which it claims costs a fraction of AWS et al – as long as it’s more than 50TB and you commit to at least 12 months.” After my S3 and Cloudflare debacle, this is interesting. Change is desperately needed in how clouds lock our data up behind excessively expensive egress price rates.


RedMonk Top 20 Languages Over Time: June 2020: Python has overtaken Java. CSS is #7 which shows you how hard CSS is. TypeScript jumped C! R jumped Go as Go is now tied with Shell likely due to Go’s continued tumult. Rust dropped to #20 (sorry, crustaceans). I predict Rust will outrank Go in three years. PHP will also be a strong contender despite interns I’ve worked with this year having never heard of it. Is PHP the COBOL of the web?

Christian Hernandez’s very helpful helpernode: “In this live stream, Christian Hernandez and Chris Short will walk through what helpernode is, how to get the helpernode up and running, and getting an OpenShift 4 cluster stood up with helpernode’s assistance.”

Awesome Command-Line tools to boost your productivity: “The more you can speed up your terminal workflow, the more efficiently you can get work done.”

Why Should You Learn Vim in 2020: Surprisingly, the article doesn’t start with, “Because it’s not emacs.” But, it explains why everyone isn’t running VScode (even I use Vim emulation in VScode).

An introduction to crun, a fast and low-memory footprint container runtime: “Check out crun, an OCI-compliant alternative to runc for Linux container runtime.”

Go vs Rust: Writing a CLI tool: “They were both very great tools for the job. But of course, they have different priorities. On one side, we have an option which tries to keep software development simple, maintainable, and accessible. On the other hand, we have a language focused on soundness, safety, and performance.” If Rust becomes a little more user friendly (by making it an intentional goal), I’d be able to pick it up I bet, shortly thereafter. But, garbage collection is tough to pass up.

How we migrated Dropbox from Nginx to Envoy: “In this blogpost we’ll talk about the old Nginx-based traffic infrastructure, its pain points, and the benefits we gained by migrating to Envoy. We’ll compare Nginx to Envoy across many software engineering and operational dimensions. We’ll also briefly touch on the migration process, its current state, and some of the problems encountered on the way.”

The easiest and fastest way to deploy an OKD 4.5 cluster in a Libvirt/KVM host: “Long story short… We will deploy an OKD 4.5 cluster in ~30 minutes (3 controllers, 1 to 10 workers, 1 service, and 1 bootstrap node) using one single command in around 30 minutes using a tool called KubeInit.” I’ll be trying this out in the near future.

An Introduction to ZFS A Place to Start: “ZFS has become increasingly popular in recent years. ZFS on Linux (ZoL) has pushed the envelope and exposed many newcomers to the ZFS fold. iXsystems has adopted the newer codebase, now called OpenZFS, into its codebase for TrueNAS CORE. The purpose of this article is to help those of you who have heard about ZFS but have not yet had the opportunity to research it.”

University of Cambridge to decommission its homegrown email service Hermes in favour of Microsoft Exchange Online: Email is hard. Fight me.

Windows 10: HOSTS file blocking telemetry is now flagged as a risk: Running a Windows 10 rig these days for various reasons, this is quite the anti-user friendly behavior I’m beginning to expect from Microsoft.

Remote SSH Connections to WSL2: “In this article I share my learnings on remotely accessing your Windows 10 computer. My motivation was to determine efficient ways to access both the Windows environment, and the WSL2 development environment from another computer.”

bloomberg/goldpinger: “Debugging tool for Kubernetes which tests and displays connectivity between nodes in the cluster.”

DevOps’ish Tweet of the Week

Want more? Be sure to check out the notes from this week’s issue to see what didn’t make it to the newsletter.