What’s the Problem?
It’s a beautiful day. You’re building a lambda function, and everything is going exactly as planned. The code looks great. Then you give it a test and…
"errorMessage": "An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied",
As you know, many things in IAM can cause an action to be denied. Here’s a partial list:
allowin an identity-based policy
denyin an identity-based policy
allowin a resource-based policy
denyin a resource-based policy
allowin a service control policy
denyin a service control policy
allowin a permission boundary
denyin a permission boundary
We haven’t even yet mentioned VPC endpoint policies, session policies, or granular service-specific data access controls.
Even once you know the policy type causing
access deniedyou still have to find the policy and the statement inside the policy causing the denial and potentially replace it with a least-privilege alternative.
AWS to the Rescue
When work on AccessUndenied began, AWS
access-denied messages had very little detail on the reasons for which access was being denied. In 2021, for easier permissions troubleshooting, AWS introduced changes to access-denied errors. These changes offered more detailed error messages, such as:
User: arn:aws:iam::777788889999:user/JohnDoe is not authorized to perform: codecommit:ListRepositories with an explicit deny in a service control policy
Further information can be found in the AWS IAM documentation.
These AWS error message updates have significantly improved the ability of users to troubleshoot
Access Denied messages. However, we still find significant use cases for an automated strategy for remediating
access-denied situations and facilitating access.
But Challenges Remain
AWS users trying to resolve
Access Denied scenarios still encounter many challenges:
AccessDeniedmessages still do not provide details. Among the services for which some, or even many, messages are lacking in details are: S3, IAM, STS, CloudWatch, EFS, DynamoDB, Redshift, OpenSearch, and ACM. For example:
An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied
Here’s another example:
"User: arn:aws:iam::123456789012:user/username is not authorized to perform: iam:CreateRole on resource: arn:aws:iam::123456789012:role/role-name"
- When the reason for
AccessDeniedis an explicit
denyusers can have difficulty tracking down the specific policy and statement that give the explicit
deny. Specifically, when the reason is an explicit
denyIn a service control policy (SCP), it is difficult to find and assess every single policy in the organization that applies to the account.
- When the problem is a missing
allowstatement, it can still be challenging to create the least-privilege policy that allows the desired access without granting excessive permissions.
Some of these problems are here to stay. For example, it is quite sensible that AWS does not grant unprivileged actors details on the exact content and identity of the service control policies preventing them from acting in a certain way. The need to maintain a balance between easy troubleshooting for builders and opacity to attackers requires that users do some of the work in troubleshooting more complex scenarios.
Access Undenied on AWS: An Automated Solution
Access Undenied on AWS is a free, open-source tool that runs completely locally (or in your environment). It analyzes AWS
access denied events and offers actionable remediation steps to facilitate access. You completely control its permissions and actions, and it does not send data to anyone. You can use it from the CLI on a local machine on single events, query and download batches of events for analysis, or even run it from a lambda function and have a lambda that receives an event and returns the reason that access was denied.
Full details on the output format can be found in the Access Undenied README.
Video: Conference Session
You can watch my session at fwd:CloudSec in which I discuss the challenges we ran into while building this tool, and what can be learned from them. One challenge was in constructing exact permission representations from CloudTrail events (I made extensive use of Ian McCay’s iam-dataset project, which we also contribute to). Another was using the basic SimulateCustomPolicy API to simulate more complicated IAM scenarios.
Access Undenied supports resource policies for many resources and some of the most common condition keys, but you can help us and the community make Access Undenied even more awesome by contributing to some good first issues in the repository.
If you stumble across any issues or wish to suggest features you’d like to see supported in the tool, feel free to open new issues on the GitHub repository.