The Perils of the Cloud

By | July 31, 2021

More and more of your data is being held in datastores hosted in the cloud. From banking to which podcasts you listen to your personal information is being kept in database run by AWS, Azure or GCP.

According to this article 38% of companies experience some sort of cloud security breach. This is hardly surprising, the security model for controlling access to cloud accounts is painful. You would never write down the password to a system in your company that controls all its sensitive information, yet environment variables like AWS_SECRET_ACCESS_KEY are placed unencrypted in .bashrc files which often live on shared drives with no access control. Your team needs access to a cloud resource? Why bother with separate accounts and only giving permissions to do what a person needs? Put the IAM credentials on Confluence, what could go wrong?

This happened to a company I worked for. I was one of the founding members and has setup our AWS system. With a team of 5 why bother with all sorts of complexity. One account with all permissions made life fast and easy, and working with a group of friends pre series A it was much easier to just send the AWS environment variables. Fast forward 9 years later and I am working at another startup when I get a call from the head of IT. Using those credentials someone had run up 70K in GPU VMs (presumably to do some crypto mining). Apologetically I had to admit that I had no idea who I had given the credentials out to as well as I was certain they had been on our companies Confluence server at some point.

I can’t login to anything these days without some sort of 2FA yet cloud resource security is mostly the equivalent of writing your password down on a post it and sticking it on your monitor.

There are of course solutions to this, don’t over privilege accounts, keep secrets in a keystore but security is the enemy of ease of use so expediency

Which brings us to SolarWinds. It was a “supply chain attack” where the attacker were able to put malicious code in a signed update. What is one of this primary things it did when compromising a company?

  1. One of the key actions SolarWinds attackers take after establishing a foothold on networks is to target the systems that issue the proof of identity used by cloud-based services, and steal the means to issue IDs.
  2. Once they have this, they can use it to create fake IDs that enable attackers to impersonate legitimate users or create malicious accounts that seem legitimate, including accounts with administrative (i.e. total) access.
  3. Because these IDs are used to give access to data and services by cloud-based services, the attackers are able to access data and email just like legitimate users, including those with total access, and they do so.

So what can you do to protect against this? Start off by assuming that even well intentioned, smart people can do stupid things to make their job easier. The NSA has a good document on this but basically

  • Don’t overpermission people
  • Use service accounts
  • No owner users should have API keys and be protected by 2FA
  • Regularly Audit
  • Monitor usage
  • Don’t grant access to groups unless you have audit controls on your group software (bonus for access on demand groups)

Lastly AWS , Azure and GCP all have best practice docs worth a read.