Friday afternoon (US east coast time), some oddities around git repos being taken for ransom started to pop-up. Safe to say, I’m paranoid af when it comes to the software delivery pipeline (GitHub and GitLab are both used to manage DevOps’ish). I immediately changed passwords everywhere I have code stored (and you should too if you haven’t already). GitLab chimed in on the issue to confirm it did not appear to be a total loss of data on affected repos and provided potential fixes. GitLab also provided extensive incident details I recommend checking out. The cause? The age-old problem of exposing version control dot directory when deploying a web site. Don’t publicly expose .git or how we downloaded your website’s source code.
Find your next tech job
No more inbox spam from recruiters! Indeed Prime matches you with top tech companies and only sends tech job opportunities that match your career goals, technical skill set, location, and salary preferences. Join for free today! SPONSORED
Bay Area DevOps meetup, May 9 in Mountain View
Cloud Foundry, IBM, LogDNA share how open ecosystems, interoperability & multi-cloud are here to stay.
RSVP now to save your seat.
Try LogDNA - Start free trial SPONSORED
DevOps’ish Last Week’s Top Five
- Tweet: Hertz has sued Accenture for failing to deliver on a $32 million “digital transformation” project.
- Python Project Tooling explained
- The Difference Between Goals, Strategies, Metrics, OKRs, KPIs, and KRIs
- Istio the Easy Way (cough nomedium.dev cough)
- How to run systemd in a container
Deadlines are horrible — I have stopped apologizing for being “behind” on Kubernetes community work. I wish I could do more and commit to more things but, it’s not my full-time job. Open source work is largely volunteer-based and we shouldn’t be this stressed helping folks out. Right?
Product Marketing Manager — Ansible Automation — There are two or three open reqs on my team right now. If you’re interested let me know (if I don’t know you personally, send your LinkedIn profile too). I’ll send you a unique URL to apply if I think you might be a good fit. Not a good fit but still need a job? Indeed Prime is a sponsor this week.
Brand loyalty: Red Hat employees permanently inked with new company logo — Red Hat revealed its new logo this week. Personally, I really like it. Red Hatters have gotten the company logo tattooed on themselves for years. It’s not a cult; it’s called a great place to work. I’m sorry if folks haven’t ever had that feeling.
Call for AWS Birds-of-Feather at KubeCon Barcelona — Going to KubeCon Barcelona and use AWS? That’s probably a hard, yes. There’s a BoF forming and if you’re interested, speak up.
Sorting algorithm reference, for coding interviews and computer science classes — I’m approaching forty years old and learn new things every day. I’ll be referencing this more and more as we all move towards a world full of data and code.
You should have a personal web site — Yes, you must have a personal web site. chrisshort.net is massive and has served me very well over the years. I modeled it a little bit after Bridget Kromhout’s site which is also great.
Gail Duval Talks Mandrake Linux and /e/ Telephone — I loved Mandrake back in the day.
DevOps’ish Telegram — Join the over 230+ DevOps, Kubernetes, SRE, and other technology professionals discussing real-world problems, breaking technology events and outages, and the occasional Spotify playlist.
Accelerate: State of DevOps 2019 Survey: Nicole Forsgren, PhD is conducting the State of DevOps 2019 Survey. Your input is incredibly important. On several occasions, I have referenced the 2018 report since its release for real-world work that impacts real numbers. Nicole’s group also wrote, Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations, which I cannot recommend enough either.
A Week Later, Docker Offers Scant Details on Hub Attack — Docker did conduct a very long maintenance this week. But, it hasn’t offered much in terms of additional details regarding the Docker Hub Breach last week. There will likely be more to this story in the future.
80% of developers are not addressing Docker security — Super awesome to read after the Docker Hub Breach.
Canonical Fires Shots at Red Hat with Ubuntu Advantage Launch — ”Direct contrast to the complexity and cost of offerings from Red Hat and VMware” There is a lot to be said for a simplified billing model.
How companies adopt and apply cloud native infrastructure — Survey results reveal the path organizations face as they integrate cloud native infrastructure and harness the full power of the cloud.
What a pain in the Azzz-ure: Microsoft Azure, SharePoint, etc knocked offline by DNS blunder — I appreciate the difficult nature of DNS. But, outages like this are going to become less and less tolerated as more reliability is baked into infrastructure.
Brand New: New Logo for Red Hat — ”The new logo is quite literal: it’s a red hat, but it’s red AF and hat AF.”
Many Kubernetes Clusters — ”Zalando runs 100+ Kubernetes clusters on AWS. Each cluster runs in its own AWS account. We always create a pair of prod/non-prod clusters per ‘product community’, i.e. only half of our clusters (50+) are marked as “production” and have full 24x7 on-call support.” The reasoning behind Zalando’s Kubernetes deployment methods as detailed in the article are great. A must read.
Sherlock changelog — Like all those awesome colors in
ls? They come at a price thanks to
LS_COLORS. I’d wish I’d known this tweak years ago.
Memory Limit of POD and OOM Killer — ”Kubernetes manages the Pod memory limit with cgroup and OOM killer. We need to be careful to separate the OS OOM and the pods OOM.”
I forgot how to manage a server — ”My config management does this for me. Whether it’s Puppet, Ansible, Chef, … all of the boring parts of being a sysadmin have been hidden behind management tools. Yet here I am, trying to quickly configure a personal server, without my company-managed config management to aid me.” In a world of automation, the manual becomes foreign.
Building Operators with Ansible — I have been working hard with my co-workers and OpenShift team to make sure Ansible-based Operators for Kubernetes are a known solution. Want to run Day 2 operations inside your Kubernetes cluster? You might already have what you need in the form of an Ansible Playbook or Role. Operators are pretty dope technology. Making them with Ansible is something I’ll be talking about in two mini-theater sessions at Red Hat Summit this week.
Introducing Red Hat Quay 3 - A Registry for your Linux and Windows Containers — Normally, I would not feature a product announcement from Red Hat unless it was something I worked on. But, as I mentioned last week, I really like Quay. It is now capable of multiarch builds which is very handy as well as a few other nice features from a container registry.
Grafana Dashboards for Kubernetes Administrators — Minimum Viable Dashboards for Kubernetes via Grafana
Why Script Based Deployments to Kubernetes Don’t Scale — ”Why use Spinnaker when I can just keep doing the same thing I’ve done before?”
Synthetic Kubernetes cluster monitoring with Kuberhealthy — ”By enabling some simple synthetic checking, we stand a much better chance of catching these kinds of ephemeral and limited-scope disturbances in our infrastructure before customers or developers notice.”
Kubernetes Ingress Tutorial: Beginners Series — ”You will learn the concept of ingress resource and ingress controllers used for routing external traffic to Kubernetes deployments.”
Cilium 1.5: Scaling to 5k nodes and 100k pods, BPF-based SNAT, and Rolling Key Updates for Transparent Encryption — Cilium 1.5 now officially supports an eye-watering, “5k nodes, 100k pods and 20k services.”
gopls (pronounced: “go please”) is an implementation of the Language Server Protocol (LSP) server for Go. The LSP allows any text editor to be extended with IDE-like features.
DevOps’ish Tweet of the Week
This thread from Justin Garrison is everything.
Here's a journey of building your own Kubernetes— Justin Garrison (@rothgar) May 4, 2019
At first you think you're building a K8s that just WKS
Notes from this week’s issue can be found here.
Sponsor DevOps'ish and put your brand in front of thousands of highly skilled operators, maintainers, developers, and leaders from across the Fortune 500, Silicon Valley, and beyond.
Join the Conversation
Join the DevOps'ish group on Telegram for insight and in-depth discussions about real technical challenges facing real people. If you're into Reddit, join //devopsish. You can follow me on Twitter and LinkedIn too.