DevOps'ish 199: Women are better leaders in crisis, 97 Things Every Cloud Engineer Should Know, Merging Microservices Back Into The Monolith, Why are my tests so slow?, and more

Note: If you’re reading this, you’re winning. You beat 2020. Also, the DevOps’ish Solarwinds supply chain compromise Index has been updated. I was talking to DevOps’ish readers a bit this week. One reader, in particular, has mentioned in the past that they’d be willing to help put the newsletter together when I’m recovering from surgeries or need a day off. This morning I sat down to get a headstart on the newsletter and realized I should instead work on a checklist or HOWTO or whatever it would end up getting called. Let me introduce you to DevOps’ish’s WRITING.md file. It details how the newsletter is made and some of the guidelines around writing a newsletter, in general. It’s made me realize that I need to look into automating more of what I do. That’s one of the beautiful parts of checklists or good documentation; it tells you where to start looking into automation points. Another thing we discussed is profit sharing and how that would have to get figured out. Or rather, we need to talk about how to figure that out. Which then triggered an early afternoon of getting GitHub Sponsor up and running. Getting good, long term newsletter sponsors is difficult. Larger newsletters generally outsource it. That seems a little much for me right now. If this system proves to work well, I will flip regular sponsors over to it potentially. ...

January 3, 2021 · 7 min · Chris Short

DevOps'ish 198

Remember last week how I said things would be punchier? Well, I updated the DevOps’ish Solarwinds supply chain compromise Index. By the way, Microsoft says it was, “used by a different threat actor.” I wrote the parts of the newsletter below, which are probably helpful. ¯\_(ツ)_/¯ The past few days have been incredibly challenging mentally and physically. I’m not sure if I would do it all the same way again. Sadly, I think I’m going to have to sideline myself a little more next year. The good news is, I get to try again next year. For that, I am truly thankful. The holidays, especially religious-based ones, are challenging for me. I have a challenging relationship with religion in general. That’s for a different blog though. For those celebrating holidays the past few weeks and the weeks ahead, I wish you all the best. DevOps’ish is brought to you by Accurics People Reverse Engineering the source code of the BioNTech/Pfizer SARS-CoV-2 Vaccine “In this post, we’ll be taking a character-by-character look at the source code of the BioNTech/Pfizer SARS-CoV-2 mRNA vaccine.” Damn. That’s awesome. ...

December 27, 2020 · 5 min · Chris Short

DevOps'ish 197

This being a holiday break my intros will probably be a little bit shorter than normal. “More punchy” as my first boss at Red Hat would say. Let’s start with the thing with an ever-increasing blast radius: Solarwinds. Solarwinds Supply Chain Compromise For the record, Reuters has been all over this coverage wise. I can’t do them justice. This story has been evolving so rapidly that by the time I hit send, my coverage will be incomplete. Everything from Russian hackers to insider trading to putting a global sinkhole in place for the command and control (C2) domain. This is a truly terrifying compromise at a company I genuinely enjoyed working for in the past. No, I never touched the Orion product while I worked there. Perhaps I should have. But, over the years, I’ve been in contact with more places than I can count that use Solarwinds Orion. It really is everywhere that has a big enough footprint to justify it. Governments included. This might turn out to be bigger than the OPM hack when the dust settles. I decided to build an Index page to provide continuing coverage of the Solarwinds supply chain compromise. When in doubt, go to this page for vetted info. ...

December 20, 2020 · 8 min · Chris Short

DevOps'ish Solarwinds supply chain compromise Index

A one-stop shop for opinion, analysis, and/or coverage of the Solarwinds supply chain compromise. Coverage includes official statements and filings, accredited media coverage, industry analyisis, and noteworthy blogs, digital media, and other mediums as deemed worthwhile. Note: All links shared here have gone through the normal DevOps’ish editorial and curation process. To add content for review, issue a pull request against this file in GitHub. Official Statements Security Advisory | SolarWinds Solarwinds US SEC 8K Mitigate SolarWinds Orion Code Compromise - Emergency Directive 21-01 CISA Updates Alert and Releases Supplemental Guidance on Emergency Directive for SolarWinds Orion Compromise | CISA Microsoft Internal Solorigate Investigation Update – Microsoft Security Response Center Statement on the story from The New York Times regarding JetBrains and SolarWinds | JetBrains Blog CLASS ACTION COMPLAINT FOR VIOLATION OF THE FEDERAL SECURITIES LAWS SOLARWINDS CORPORATION, KEVIN B. THOMPSON, and J. BARTON KALSU Joint Statement by the Federal Bureau of Investigation (FBI), the Cybersecurity and Infrastructure Security Agency (CISA), the Office of the Director of National Intelligence (ODNI), and the National Security Agency (NSA) | CISA SUNSPOT Malware: A Technical Analysis | CrowdStrike Deep dive into the Solorigate second-stage activation: From SUNBURST to TEARDROP and Raindrop - Microsoft Security Raindrop: New Malware Discovered in SolarWinds Investigation | Symantec Blogs 02/17/21: Press Briefing by Press Secretary and Deputy National Security Advisor - YouTube Hearings | Intelligence Committee Solorigate Resource Center – updated February 25, 2021 – Microsoft Security Response Center Background Press Call by Senior Administration Officials on the Administration’s Response to the Microsoft and SolarWinds Intrusions | The White House Press Suspected Russian hackers spied on U.S. Treasury emails - sources | Reuters IT company SolarWinds says it may have been hit in ‘highly sophisticated’ hack | Reuters U.S. Homeland Security, thousands of businesses scramble after suspected Russian hack | Reuters U.S. Agencies Hacked in Foreign Cyber Espionage Campaign Linked to Russia - WSJ Hackers used SolarWinds’ dominance against it in sprawling spy campaign | Reuters Backdoored SolarWinds software, linked to US govt hacks, in wide use throughout the British public sector • The Register SolarWinds: Hey, only as many as 18,000 customers installed backdoored software linked to US govt hacks • The Register Investors in SolarWinds sold millions in stock before Russia breach revealed - The Washington Post Hackers used SolarWinds’ dominance against it in sprawling spy campaign | Reuters Hack Suggests New Scope, Sophistication for Cyberattacks - WSJ Exclusive: Microsoft breached in suspected Russian hack using SolarWinds - sources | Reuters Nuclear weapons agency breached amid massive cyber onslaught - POLITICO Biden hints at a tougher stance against state sponsors of cyberattacks SolarWinds hackers broke into U.S. cable firm and Arizona county, web records show | Reuters Hackers last year conducted a ‘dry run’ of SolarWinds breach Trump contradicts Pompeo in bid to downplay massive hack of U.S. government, Russia?s role - The Washington Post Trump administration says Russia behind SolarWinds hack. Trump himself begs to differ • The Register Second hacking team was targeting SolarWinds at time of big breach | Reuters Russia’s Hacking Frenzy Is a Reckoning | WIRED Russian hackers’ motive for SolarWinds cyberattack baffles US: mere espionage, or worse? | South China Morning Post Suspected Russian hackers used Microsoft vendors to breach customers | Reuters Massive data breach may have been discovered due to ‘unforced error’ by suspected Russian hackers - CNNPolitics U.S. cyber agency says SolarWinds hackers are ‘impacting’ state, local governments | Reuters SolarWinds: The more we learn, the worse it looks | ZDNet Widely Used Software Company May Be Entry Point for Huge U.S. Hacking - The New York Times DOJ Admits Microsoft Email Accounts Were Hit In SolarWinds Attacks SolarWinds hack may be much worse than originally feared - The Verge As Understanding of Russian Hacking Grows, So Does Alarm - The New York Times SolarWinds malware was sneaked out of the firm’s Orion build environment 6 months before anyone realised it was there – report • The Register Exclusive: Suspected Chinese hackers used SolarWinds bug to spy on U.S. payroll agency – sources | Reuters Suspected Russian Hackers Used U.S. Networks, Official Says - Bloomberg SolarWinds: How Russian spies hacked the Justice, State, Treasury, Energy and Commerce Departments - CBS News White House says it will hold those responsible for SolarWinds hack accountable within weeks - CNNPolitics Former SolarWinds CEO blames intern for “solarwinds123” password leak - CNNPolitics Industry Highly Evasive Attacker Leverages SolarWinds Supply Chain to Compromise Multiple Global Victims With SUNBURST Backdoor | FireEye Inc Tracing the SolarWinds exploit upstream How Russian hackers infiltrated the US government for months without being spotted | MIT Technology Review SolarWinds Hack Leaves Feds Scrambling to Determine Damage Concerns Run High as More Details of SolarWinds … Microsoft president calls SolarWinds hack an “act of recklessness” | Ars Technica Analyzing Solorigate, the compromised DLL file that started a sophisticated cyberattack, and how Microsoft Defender helps protect customers - Microsoft Security Cisco, Intel, Deloitte Among Victims of SolarWinds … NVIDIA and Intel affected by SolarWinds hack | Engadget National cyber director role in the spotlight after SolarWinds hack - FedScoop SolarWinds hack may have been much wider than first thought | Engadget DoJ says SolarWinds hackers breached its Office 365 system and read email | Ars Technica Microsoft says Russians accessed account ‘used to view source code’ in Solorigate hack | VentureBeat SolarWinds malware has “curious” ties to Russian-speaking hackers | Ars Technica SolarWinds hackers are tied to known Russian spying tools | VentureBeat Malwarebytes targeted by Nation State Actor implicated in SolarWinds breach. Evidence suggests abuse of privileged access to Microsoft Office 365 and Azure environments 30% of “SolarWinds hack” victims didn’t actually use SolarWinds Microsoft says it found 1,000-plus developers’ fingerprints on the SolarWinds attack • The Register SolarWinds Attackers Lurked for ‘Several Months’ in … SolarWinds, Microsoft, FireEye, and CrowdStrike defend conduct in major breach | VentureBeat Recovering from the SolarWinds hack could take 18 months | MIT Technology Review Chinese hackers targeted SolarWinds customers in parallel with Russian op | Ars Technica Mimecast says SolarWinds hackers breached its network and spied on customers | Ars Technica SolarWinds Experimenting With New Software Build System in Wake of Breach What We Know (and Don’t Know) So Far About the … Blogs, Newsletters, Digital Media, etc. Suspected Russian Hackers Spied on U.S. Treasury Emails - Sources | Top News | US News SolarWinds confirms 18,000 customers may have been impacted US government software provider SolarWinds confirms it was hacked - SiliconANGLE Cisco targeted in SolarWinds attack as Microsoft uncovers a second hacking group - SiliconANGLE Microsoft partnered with security firms to sinkhole SolarWinds hack C2 Researchers shared the lists of victims of Solarwinds hack SolarWinds hackers aimed at access to victims’ cloud assets SolarWinds releases updated advisory for SUPERNOVA backdoor SolarLeaks website offers source code stolen in SolarWinds hack for sale - SiliconANGLE SolarWinds Hack Lessons Learned: Finding the Next … Connecting the dots between SolarWinds and Russia-linked Turla APTSecurity Affairs Russia’s SolarWinds Attack and Software Security - Schneier on Security Injecting a Backdoor into SolarWinds Orion - Schneier on Security Behind the Scenes of the SunBurst Attack – The New Stack Sunshuttle, the fourth malware allegedly linked to SolarWinds hackSecurity Affairs SolarWinds hackers stole some of Mimecast source codeSecurity Affairs

December 20, 2020 · 6 min · Chris Short

DevOps'ish 196

Some people understand that the advancement of technology is marching at an ever quickening pace. We’re talking about exponential advancement every year. Five years ago, Kubernetes was brand new. Now it’s democratizing computing across clouds. Docker, the company behind some glue technology that made containers the new norm in software, has died twice now. The size and shape of infrastructure has changed so much in the past two years, it’s hard to remember ten years ago when Vagrant was brand new. I’ve been thinking about my typical end of year blog post this week quite a bit. Trying to accurately predict what next year will bring is difficult. When an organization offers to support something for ten years, it seems increasingly daunting to fathom what the technology landscape will look like. When I found out (the same time you did) about CentOS Stream, I appreciated it (FAQ). It feels like DevOps has touched the enterprise operating system I was using when I first started learning DevOps. But, then I realized, like most decent digital transformations, change is often met with some resistance. The best people in DevOps are the ones that can help people see the future a little clearer. Some people lean into DevOps; some people resist it. This is human nature. ...

December 13, 2020 · 6 min · Chris Short