A sales rep who can't open the cost screen in the ERP can still get that number by asking the company's AI assistant, "What does this product cost us?" — and get a correct answer in seconds. That looks like a strange bug, but it's actually an expected outcome. The answer depends entirely on what permissions the AI was connected with.
The Permission Boundaries Already Exist — AI Just Doesn't See Them
Every piece of software your company runs has a permission structure: roles, team visibility, folder access, personal inboxes. A sales rep not seeing cost data isn't an accident — it's a deliberate design choice, built into the ERP long before AI entered the picture.
The problem shows up when AI connects to these systems. If that connection was set up with the broadest possible read access, the AI answers based on its own permissions, not the permissions of the person asking. It searches everywhere it was given as a data source and returns whatever it finds. Whether the user could normally see that data never enters the question.
Neither the Employee Nor the AI Is at Fault
It's easy to blame the wrong side here. The sales rep asked an ordinary question — not an attempt to leak data. The AI did exactly what it was built to do within the scope it was given.
The one thing that actually determines the outcome is a decision made during setup: which account did the AI connect with, and how wide was that scope? Any integration built without asking this question risks bypassing the permission structure that already exists.
Our Rule: Don't Build a Separate Structure for AI
Our rule on this is simple: don't create a separate permission structure for AI — the same access rules that already apply to people should apply to it. If a sales rep can't see cost data, asking the AI shouldn't change that.
This separates two very different setups:
- The wrong way: connecting the AI through a service account that holds the broadest access in the company. It's fast to set up, but every user effectively gains admin-level access through the AI.
- The right way: connecting the AI under the identity of the person making the request (scoped access) and having it inherit the existing role-based access control (RBAC). The AI sees exactly what that user can see — nothing more.
To make this concrete: if an employee can't open a folder reserved for the board on the file server, the AI assistant shouldn't summarise those documents for them either. On every request, the AI should take on the identity of whoever is asking, and stay inside what that person can already access.
The second approach takes more engineering work upfront. The cost of the first approach shows up later, usually during an incident.
Where to Start in Practice
A few questions worth asking at setup time:
1. How is identity passed through? Does the AI connect using the identity of the person asking (through single sign-on), or through one shared service account? A shared account means every answer comes from the same broad set of permissions.
2. Is access restricted at the row level? If your ERP or CRM uses row-level security, the AI connection needs to inherit that same restriction. Otherwise it can reach anything allowed at the table level, regardless of who's asking.
3. Which data source is actually needed? Connecting to every system "just in case" is a common mistake. The AI should only access what a specific use case requires — not what might answer some future question.
4. Who asked what, and when? Without an audit log, there's no way to find out later which piece of information reached whom. This is the same requirement you'd already apply to a human user — it doesn't disappear because the requester is an AI.
Who Actually Decides?
This isn't a technical detail — it's a governance question. The choice made during setup is usually decided by one of three groups: IT, the relevant business unit, or management directly. Whoever makes that call needs to understand the existing permission structure — who can see what, and who can't. Otherwise the decision gets made for convenience, not for security.
If you don't know who makes this call at your company, that's a finding in itself. The Governance and Access to Company Data dimensions in our AI Readiness Assessment were built exactly to bring this question into focus.
🗓️Let's map your personalised roadmap in a free 30-minute strategy call.
Book a Free Call🧭Where does your company stand on AI today? See it across six dimensions in 4 minutes.
AI Readiness AssessmentFound this useful? Share it with your team.
