![](\"https:\/\/www.trendmicro.com\/content\/dam\/trendmicro\/global\/en\/devops\/22\/e\/cloud-configuration-management-guardrails\/adding-gaurdrails-devops-tn.jpg\")
<\/p>\n<\/p>\n
<\/div>\nThe Challenge<\/span><\/p>\n With a brand-new account, your initial configuration sets the tone. With existing accounts, the challenge is twofold.<\/p>\n The first is the team working with that account will already be used to operating under the existing configuration. And since they\u2019ve been doing it this way for a while and things are working, there\u2019s no motivation to change.<\/p>\n The second challenge is on the technical side. Can these guardrails be implemented without breaking anything inside the active account? What level of testing will be required? How much work is involved overall?<\/p>\n Boiling it down, this is a security feature request that needs to be prioritized. How can we approach this challenge?<\/p>\n Getting The Team Onboard<\/span><\/p>\n Everyone wants their systems to be more secure. But security is just one of the pillars of building well in the cloud. When faced with deploying a new feature that directly helps customers or deploying security guardrails that may help in the future, it\u2019s hard to argue against the customer.<\/p>\n That\u2019s completely understandable and one of the key reasons the centralized security monitoring structure is so hard to put in place in an environment that is already working.<\/p>\n The story usually proceeds like this:<\/p>\n No one likes being told they must drop their work and do something different that doesn\u2019t directly advance their goals. This is squarely on the security teams shoulders. They need to adjust their approach.<\/p>\n Until they do, let\u2019s look at this from your team\u2019s point of view. How can centralized security monitoring and audit help you meet your goals?<\/p>\n As much as auditing sounds scary, it\u2019s really just having someone double check your work. If you\u2019re able to get feedback (preferably automated) that your workloads are configured in a strong manner, isn\u2019t that a positive thing?<\/p>\n Similarly, while centralized monitoring always has challenges with context, having another team looking for security issues can add a layer of assurance that your team hasn\u2019t missed anything.<\/p>\n Additionally, centralized monitoring can have added benefits like spotting larger patterns that aren\u2019t visible with only one accounts data.<\/p>\n Evidently, there are positives for your team. They just aren\u2019t as direct or impactful as you may want\u2026which is fine as long as the cost or effort to implement isn\u2019t too high.<\/p>\n That leads to the technical implementation of these guardrails and the associated risks.<\/p>\n Digging Up Roots<\/span><\/p>\n The first step:<\/p>\n The root account is locked down, using multi-factor authentication, and not used for anything but the initial configuration of the account (AWS, Microsoft Azure, Google Cloud Platform\u2122)<\/p>\n This is probably the trickiest step to back away from. If you\u2019ve used the root account to create resources or run workloads in your account, you may have to re-launch them with a less privileged account or re-assign ownership.<\/p>\n The good news? Most cloud resources don\u2019t have ownership assigned to a user but to the account. That means any account with sufficient permissions should be able to maintain or remove those resources.<\/p>\n Backing away from root ownership is more an exercise in reducing permissions, not changing ownership. Still, there is potential for downtime here, but the risk of those elevated privileges usually justifies moving this work up as a high priority.<\/p>\n The one area that might be a \u201cgotcha\u201d is if someone is using the root account credentials on their workstation or has them embedded somewhere else like a deployment server.<\/p>\n Use the API call audit tool available in each of the big three clouds to find that access if it does exist.<\/p>\n Estimated time to resolve?<\/b> An hour.<\/p>\n Level of effort?<\/b> High due to log searches required and possible permission changes.<\/p>\n Return on investment?<\/b> Very high. Root accounts are the keys to the kingdom and should be protected at all costs.<\/p>\n API Call Auditing<\/span><\/p>\n Of course, in order to check the API call logs, those logs have to be enabled.<\/p>\n The good news is that for most accounts, those logs have been enabled by default since the account was created. That\u2019s true for Azure<\/a>, Google<\/a>, and AWS.<\/p>\n But each of the clouds does have an exception (or three) that might apply here. There was a time when API calls were either not logged by default or used a different system.<\/p>\n With Azure, \u201cClassic<\/a>\u201d resources may or may not log to the activity log. For Google, some services use the activity logs<\/a> and not the newer audit logs. In AWS, older accounts simply didn\u2019t have AWS CloudTrail enabled<\/a> and weren\u2019t logging those calls in any form.<\/p>\n For older accounts, taking a few minutes to enable this logging is a smart move.<\/p>\n The configuration is minimal and essentially boils down to providing a place to store the logs. This should not impact any production resources or result in any downtime.<\/p>\n The only downside is the possible costs associated with storing the logs. Though, again, all of the clouds have ways to easily reduce that cost over time.<\/p>\n Estimated time to resolve?<\/b> Five minutes.<\/p>\n Level of effort?<\/b> Minimal. These features are probably already one.<\/p>\n Return on investment?<\/b> High. These logs are a fantastic source of troubleshooting information for any operational issue (including security).<\/p>\n You Spent What?<\/span><\/p>\n Billing alerts are something that should be enabled on all cloud accounts by default. The CSPs won\u2019t enable them by default because what I am willing to spend on the account hosting my personal website is significantly different from what I\u2019m willing to spend on my workload supporting paying customers.<\/p>\n That means it\u2019s up to you to setup billing alerts that match your risk tolerance.<\/p>\n Again, the good news here is that this is a non-breaking change. These alerts don\u2019t stop resources in your accounts, they highlight spending that might be higher than you expect.<\/p>\n Ask any team out there, it\u2019s always better to get a notification early in the month that something is off versus a bill that is thousands and thousands of dollars higher than you expect.<\/p>\n A simple billing alert can help avoid that disaster, and alert you to any suspiciously high charges due to an attack like crypto mining. There\u2019s no reason not to apply these to your account immediately. It\u2019s five minutes that could save you thousands.<\/p>\n Estimated time to resolve?<\/b> Ten minutes.<\/p>\n Level of effort?<\/b> Moderate. You have to decide not only where to send the alerts but what to do if you receive one.<\/p>\n Return on investment?<\/b> High. It doesn\u2019t take a lot of searching to find horror stories of very large and very unexpected cloud spending bills.<\/p>\n Centralized Visibility<\/span><\/p>\n This is the step that typically meets with the most pushback. The truly interesting part of that is the reason for the pushback. This step is usually fought against because of the idea of someone looking over your shoulder.<\/p>\n The technical side of this step is relatively simple. The centralized accounts need to be already setup and then provided a role in your accounts that has read access only.<\/p>\n This means there won\u2019t be any production impact and this setup should be completely automated. The centralized teams should be able to provide a cloud-specific script that sets up the needed permissions.<\/p>\n The true issue here is the relationship between your team and the centralized services. This can be tricky waters.<\/p>\n Estimated time to resolve?<\/b> Five minutes.<\/p>\n Level of effort?<\/b> Minimal. This should be completely scripted and have zero production impact.<\/p>\n Return on investment?<\/b> Low for your team. High for the overall organization. The idea behind centralized security and audit accounts is to get a handle on the overall risk the organization faces. This is one you take for the team.<\/p>\n Organizing Access Control Permissions<\/span><\/p>\n Despite the high level of pushback in the previous step, this recommendation is by far the hardest to pull off.<\/p>\n For some reason, permissions almost always gradually drift towards \u201cadministrator\u201d levels.<\/p>\n It\u2019s often little changes here and there over time and before you know it, a resource needlessly had full administrator access to your cloud account. Therefore you need to regularly review and maintain the permissions in your cloud account.<\/p>\n Remember, the goal is to manage these permissions using a higher-level abstraction. Creating policies or roles for various tasks is a great first step.<\/p>\n There\u2019s a lot of information out there to help get you started. Here are a few examples:<\/p>\n Unfortunately, the tooling that would help you monitor which permissions are actually being used isn\u2019t nearly as mature as I\u2019d like to see. Leading the way is the AWS IAM Access Analyzer<\/a> which I\u2019m hoping other clouds will copy.<\/p>\n It should be very simple to find out which permissions assigned have never been used. Sadly, it still takes a lot of effort.<\/p>\n Estimated time to resolve?<\/b> Ongoing.<\/p>\n Level of effort?<\/b> Hard. This is a complicated and constant activity and if you remove a critical permission, the consequences could be dire.<\/p>\n Return on investment?<\/b> High. Almost all the public security breaches in the cloud stem from misconfigured permissions. This is the top security issue by far.<\/p>\n What\u2019s Next?<\/span><\/p>\n We have gone through each of the sample checklist ideas and determined the level of effort required to implement them along with a ballpark return. Check out this relevant article on how to set up guardrails to avoid cloud misconfigurations<\/a> to continue to build the foundation of great architecture.<\/p>\n\n