Machine learning for access request approval: what your past decisions already know

Written access policies cover the obvious cases. The rest is pattern: people in a certain team, job and location tend to end up with the same groups, and owners tend to refuse the same unusual combinations. Those patterns are in your approval logs and group memberships long before anyone turns them into a policy. A few things make access a good fit for a trained model:

• Many close choices. The right answer depends on a mix of department, job, team, manager and resource, and the combinations multiply faster than anyone can write rules for them.
• Unwritten rules. For example, one regional finance team gets a reporting folder and a neighboring team doesn't. Nobody wrote that down, but every past decision reflects it.
• Rare exceptions that matter. Most requests are approved, so the few that should be refused are the ones worth a reviewer's time.
• Steady volume. Joiner-mover-leaver events, new projects and access reviews produce a stream of near-identical decisions.

Rules and role definitions capture the patterns someone has noticed. A supervised model learns from every past decision, including the ones nobody has turned into a rule, and it reports how sure it is. That confidence decides which suggestions a reviewer can confirm quickly and which need a careful look.

Both benchmarks use access records Amazon published from its own employees: the Amazon Access Samples dataset on the UCI Machine Learning Repository, and the Amazon Employee Access Challenge data. Every value is an anonymized code, with no names, folder paths or written reasons. Both tests used the StayCharted Classifier Model, which learns from the words and codes in each row, with no rules written by anyone.

Benchmark 1: predict access permissions for each group

The access permissions benchmark asked whether a model can learn who should be in each access group. It used 36,063 employees with 11 anonymized attributes and tested the 10 groups held by the most people (between 5.5% and 10.8% of employees each). The model trained on 28,793 people and was tested on 7,270 it never saw, for 72,700 decisions in total. Each group trained in about five seconds on an ordinary CPU, with no GPU.

"Given access wrongly" is oversharing: of the people who shouldn't have access, the share who would get it. "Left out" is the share of people who need access and wouldn't get it.

The team rule is a strong baseline, and the model only edges it. The model made 10% fewer mistakes (1,593 against 1,772), almost all by leaving out fewer people (17.7% against 21.1%). It did not reduce oversharing: both sit at 0.9%. For the 123 held-back people whose manager had nobody in the training data, the model left out 31.5% of those who needed access, against 39.8% for the rule. The benchmark treats that small sample as a direction, not a firm result.

Benchmark 2: machine learning for access request approval

The access request approval benchmark used 32,769 requests from 2010 and 2011, of which 94.2% were approved. The model trained on 26,103 past decisions in about two seconds and was tested on 6,666 held-out requests, 381 of which had been denied.

Accuracy says very little here, because approving everything scores 94.3% and catches nothing. Forced to answer yes or no, the model would still approve 73% of the denials. That is the plainest result in either benchmark: denials are rare and hard to predict, and no method should approve or deny on its own.

The model does much better at ranking. Its AUC (how well it puts likely denials ahead of likely approvals, where 1.0 is perfect) was 0.84, against 0.70 for the best rule and 0.60 to 0.64 for the simple rules.

Ranking is what makes a model useful in an approval process. Pick a confidence cutoff, and requests above it go to a fast-confirm lane while the rest get a careful review. Here is the request benchmark at each cutoff, with the best rule tuned to treat the same share of requests as routine:

At every volume the best rule let more denials through. At the 0.95 cutoff it was 221 against the model's 103. But 103 denials in the routine lane is still far too many to grant automatically, which is why "routine" should mean a quick confirmation by a person, not an automatic grant. StayCharted doesn't approve, deny or change permissions. It suggests, and your process decides.

The group benchmark shows the same effect. Checking the least confident 5.9% of decisions catches 1,139 of the model's 1,593 mistakes (72%). The team rule, ranked the same way, catches 1,083 of its 1,772. If every flagged answer were checked and corrected, 99.4% of decisions would be right, though that is a projection, not a measured result. For more on choosing a cutoff, see confidence scores and human review.

The benchmarks also show where a model is the wrong tool, or not enough on its own:

• Anything that widens access. An exception is a security decision. Let the model sort and pre-fill, and keep the approval with the owner.
• Groups your attributes can't explain. Two of the 10 groups couldn't be predicted from these attributes by any method tested. Even the model left out 49.0% and 62.7% of the people who needed them.
• Change. Both tests held out a random sample of past data, not future hires or a reorganization. Retrain after big changes.
• Files and folders. These tests measured group membership and application resources, not SharePoint sites or NTFS folders. Real directories with readable department names, titles and folder names may give a model more to work with, but that hasn't been measured yet.
• What a denial means. A denial is a past decision, not proof of a security incident. The figures measure agreement with your history, not breaches prevented.

Role mining looks at existing assignments and proposes roles that someone then defines and maintains. A trained classifier skips the role-design step: it predicts each person-to-group or request decision directly from attributes, and you retrain it when your history changes. The "same as most of the team" rule above is close to what a simple role gives you, and it was strong at 97.6%. If your access follows the org chart closely, a role or team mapping may be enough.

Machine learning is also unnecessary when you have a few broad groups, a handful of requests a week, or low stakes. History size matters too. In the group benchmark, accuracy rose from 95.7% with 500 people to 97.8% with all 28,793, and the share left out fell from 33.0% to 17.7%. Small histories work but miss more people. See how many examples you need.

StayCharted AI Model Trainer (AMT) trains a private model on decisions you have already made:

1. Export past decisions. One row per decision, with the attributes you have (department, job title, team, manager, location, resource or group) and the outcome (approved or denied, member or not).
2. Upload and check. AMT checks the data and runs personal-information checks and masking before training, on every plan. Your data stays in your organization's own database area, is used only for your models, and is hosted on AWS in the US.
3. Train. Use the Classifier Model when rows are mostly codes, as in both benchmarks. If rows carry readable text such as folder names or written reasons, try the AI Classifier Model, which reads meaning as well as words.
4. Read the held-out test. AMT reports accuracy overall and per category, the confusions, and the share of answers above each confidence cutoff with how many of those are right.
5. Use it. Fill CSV or Excel files, use the Google Sheets add-on, the API (Essentials and up) or the Power Automate connector. Answers below your cutoff go to the Review Queue, and reviewers' answers train the next version.

StayCharted doesn't connect to your tenant or change permissions. For SharePoint and file access specifically, see AI for SharePoint and file permissions and SharePoint permissions best practices. Plans are Free ($0), Essentials ($49) and Business ($99) per workspace per month; see pricing.

The Review Queue requires Essentials or Business. Retrain and publish a new version to use reviewer corrections in future answers.