Can a model learn which access requests to approve?

A StayCharted benchmark on 32,769 real access requests, each approved or denied by a person. 13 October 2026.

In most large organizations, access to a shared folder, an application or a group starts as a request. Someone asks, usually in ServiceNow or a similar tool. A manager or a resource owner approves or denies it. Then IT adds the person to the right group.

Most requests are routine. A few aren't. The people approving them spend the same time on both, and the routine ones teach them to click Approve without reading.

So we asked: if a model learns from the decisions people have already made, can it sort the next requests into routine and needs a person? And when it calls a request routine, how often is that a request a person would have denied?

This follows our first access test, Can a model learn who should have access?, which covered group membership.

The Amazon Employee Access Challenge dataset is a public record of real access-request decisions. Amazon released it for a Kaggle competition in 2013, and it is also published on OpenML.

It holds 32,769 requests from 2010 and 2011. Each one records:

• the resource requested;
• the requester's manager;
• their role: two levels of the organization, department, business title, role family (two descriptions) and role code;
• the decision: approved or denied, by a person.

Every value is an anonymized ID number. 94.2% of the requests were approved.

We set aside 6,666 requests (20%, chosen by a fixed hash) before training anything. They include 381 denials. Every figure below is measured on them. The model learned from the other 26,103. No held-out request was an exact repeat of a training request.

StayCharted's Classifier Model learned "Approve" or "Deny" from the 26,103 past decisions. It used the same training engine customers use in the app, and trained in about two seconds. Nobody wrote a rule.

We used the Classifier Model, which learns from the words and codes in each request, rather than the AI Classifier Model, which reads meaning. Every value here is a code, and a code has no meaning to read.

We compared it with the rules an access team might set up:

• Approve everything. What a busy approver does in practice.
• Same as past decisions for this resource.
• Same as past decisions for this resource and job title.
• Same as past decisions for this resource and team: the same manager, falling back to the department, then to the resource.

"Approve everything" is right 94.3% of the time, and approves every one of the 381 requests a person denied.

So the number that matters is wrong approvals: of the requests a person denied, the share that would have been approved anyway.

Every approach, including the model, approves most of the requests a person denied. That is the honest finding: denials are rare and hard to predict. No method should be trusted to deny or approve on its own here.

But a yes-or-no answer throws away the most useful thing the model has: how sure it is.

Each answer comes with a confidence. Here, only requests the model is very sure it would approve are treated as routine; everything else goes to a person, who decides as they did before.

At 0.95, four in five requests are routine, and nearly three in four real denials still go to a person. At 0.99, about half are routine and 86% of denials reach a person.

We gave the rules the same job. For each request, we took the approval rate of past requests like it (resource, then resource and department, then resource and manager, with smoothing for small numbers), and treated the highest as routine. At each volume, the table reports the better result of the simple and smoothed rule sorters, treating the same number of requests as routine:

At every volume the model lets through fewer denials: about half as many at 79% (103 against 221).

A standard measure of how well each one ranks requests, AUC (1.0 is perfect, 0.5 is chance), says the same thing: 0.84 for the model, 0.70 for the best rule, 0.60 to 0.64 for the simple rules. The model here is one Classifier Model with no tuning.

• These are application resources, not NTFS folders. A request for a shared folder has the same shape (who is asking, for what, and who decided), but transfer to folder requests remains untested.
• A denial isn't always a risk. A denial records an outcome, not a security incident. The dataset does not establish the reason for each denial; these figures measure disagreement with past decisions, not prevented breaches.
• The values are codes. Real requests carry folder names, titles and a written reason. That's where the AI Classifier Model, which reads meaning, can add more. We haven't measured that yet.
• One company, 2010 to 2011, a random split. Held-out requests are a sample from the same period, not next year's requests.
• We show the whole range of cutoffs, not one picked to look best. Choose yours on your own data.

Doing this with SharePoint, NTFS or your request tool: AI for SharePoint and file permissions →

1. Export past access requests with their outcome from ServiceNow or your request tool: who asked, for which folder or group, what access, and whether it was approved.
2. Upload the file to StayCharted and train. See how often it agrees, and how many denials it would have caught, on requests it never saw.
3. Ask the model about each new request through the API. Send the unsure ones, and anything sensitive, to an approver.

StayCharted doesn't approve or deny anything, or change permissions. It suggests, and your process decides. Explore file and access-request workflows →

ServiceNow is a trademark of ServiceNow, Inc. StayCharted is not affiliated with ServiceNow.

• Data: Amazon Employee Access Challenge (Kaggle, 2013), as OpenML dataset 4135 version 1 (Amazon_employee_access.arff, 2,134,522 bytes; MD5 5502ee73e563ddf5b699ffe0c633905b checked against OpenML's). 32,769 requests: Kaggle's labelled training set. Only results are published here, not the data.
• Split: a SHA-256 hash of each row (seed staycharted-request-bench-2026-10-13): 26,103 to learn from, 6,666 held out (6,285 approved, 381 denied).
• Model:
  • StayCharted Classifier Model (words), one model.
  • Each request is one row of "Field: value" lines, each value tagged with its field.
  • The engine holds back part of the training rows to choose its own settings, as in the app.
  • Run from the app's own training code (13 October 2026), outside the app's screens.
• Rules: majority of past decisions for the same key, falling back to the less specific key, then to "approve". The sorter used a smoothed approval rate (each level shrunk toward the less specific one, k = 2), fixed before the test was scored.
• Routine: the model's confidence that the answer is "Approve" is at or above the cutoff.

StayCharted AI Model Trainer learns how your team decides from your own past decisions, and sends the uncertain ones to a person.

Sources: Amazon Employee Access Challenge on OpenML · Kaggle competition.

Can AI approve access requests on its own?

This benchmark does not support that. StayCharted’s Classifier Model still approved 277 of 381 held-out denials as a binary classifier. Use scores to prioritize review within your existing authorization policy.

How much review did the model need?

At the 0.95 approval-confidence cutoff, 20.9% of requests went to a person, including 278 of 381 denials. Another 103 denials were treated as routine.

Was this tested on NTFS folders?

No. These are anonymized application-resource requests from Amazon in 2010–2011. Test separately on your own folder requests before using the approach.