How to Vet Your MSP's AI Practices: 7 Questions Before Agents Touch Your Data

Your managed service provider probably uses AI. The question is whether they can tell you exactly how, where your data goes when they do, and what happens when something goes wrong. Most MSPs have added AI capabilities faster than they’ve built governance around them. According to a 2025 Kaseya survey, 78% of MSPs reported using or piloting AI tools for service delivery, but fewer than half had documented policies governing that usage.

If you’re evaluating a new MSP or reviewing your current one, these seven questions will separate providers who have thought through AI governance from those who are still figuring it out.

1. What AI Tools and Agents Do You Use, and for What Tasks?

Start with the basics. You need a concrete list of every AI system your MSP runs against your environment, not a vague reference to “AI-powered monitoring.” The answer should include product names, vendors, and the specific job each tool performs.

Common MSP AI use cases include automated ticket routing and triage, anomaly detection in security monitoring, predictive alerting for hardware failures, chatbot-based help desk triage, and automated remediation scripts. Each of these carries different risk profiles. An AI that categorizes help desk tickets is low-stakes. An AI agent that executes remediation scripts on your production servers is not.

A good answer names every tool, explains what each one does, and distinguishes between AI that recommends actions and AI that takes actions autonomously. If your MSP can’t produce this list within a few business days of you asking, that tells you something about how well they track their own tooling.

2. Where Does Client Data Go When It Passes Through AI Systems?

This is the question most MSPs struggle to answer precisely. When an AI tool processes your data, that data travels somewhere. It might stay within your MSP’s infrastructure, pass through a third-party API like OpenAI or Azure OpenAI, or get routed through a vendor’s cloud for inference. Each hop matters.

Ask specifically: Which third-party AI model providers process our data? Is data sent to external APIs for inference, and if so, is it encrypted in transit and at rest? Do any of these providers use client data for model training? Where does the data physically reside?

Microsoft’s Azure OpenAI Service, for instance, does not use customer data to train its models and keeps data within the customer’s selected Azure region. That’s a meaningfully different answer from “we use ChatGPT,” where data handling depends heavily on the account tier and configuration. If your MSP uses AI tools from smaller vendors, the data residency and training data policies may be less clear. Your MSP should be able to show you the relevant terms of service and data processing agreements for every AI vendor in the chain.

For businesses subject to compliance requirements like HIPAA or CMMC, data residency is not optional. Your data governance framework should extend to every system your MSP uses on your behalf, AI included.

3. What Human Oversight Exists Before AI Agents Act on Production Systems?

The distinction between “AI-assisted” and “AI-automated” is critical. AI-assisted means a human reviews the AI’s recommendation before anything happens. AI-automated means the agent acts on its own and a human reviews after the fact, if at all.

Both models have valid uses. Automated responses to well-understood scenarios (quarantining a known malware hash, for example) can improve response time. But automated remediation of ambiguous alerts, automated changes to firewall rules, or automated account lockouts based on behavioral analysis all carry real risk of false positives that disrupt your operations.

Ask your MSP to map out which AI actions require human approval and which don’t. A responsible MSP will have defined escalation thresholds: routine actions below a certain risk level can be automated, while anything above that threshold requires a human in the loop. If the answer is “our AI handles everything automatically,” push harder. As we covered in our guide on the AI agent approval gap, the speed advantage of full automation is only valuable when paired with controls that prevent the AI from causing more damage than the incident it’s responding to.

4. How Do You Handle AI Recommendations That Turn Out to Be Wrong?

AI systems produce incorrect outputs. This is not a flaw to be engineered away but a characteristic of the technology that needs to be managed. Large language models hallucinate. Anomaly detection systems generate false positives. Predictive models miss edge cases. The question is not whether your MSP’s AI will make mistakes, but what happens when it does.

A good answer covers three things. First, how the MSP detects errors: do they audit AI outputs, run automated validation, or rely on client reports? Second, what the remediation process looks like: who investigates, how quickly, and what’s the communication protocol with your team? Third, whether the MSP tracks AI error rates over time and uses that data to improve or replace underperforming tools.

If your MSP positions their AI as infallible, that’s a red flag. Every mature AI operation includes monitoring for drift, accuracy degradation, and edge case failures. You want a provider who treats AI errors as a category of incident to be managed, not a topic to be avoided.

5. What’s Your AI Acceptable Use Policy, and Does It Cover Client Data?

Your MSP should have a written policy that governs how their staff and their AI systems interact with client data. This policy should be available for you to review, not locked in an internal wiki.

The policy should address which employees can deploy or modify AI tools, what types of client data can be processed by AI systems, whether client data is ever used for internal MSP purposes (training, benchmarking, product development), and how the MSP evaluates new AI tools before deploying them against client environments.

If your own business has built an AI acceptable use policy, your MSP’s policy should be at least as specific as yours. A common gap: the MSP has an internal AI policy that covers their own business operations but says nothing about how AI interacts with client data specifically. Those are two different scopes, and both need coverage.

Ask for the document. If it doesn’t exist yet, that’s useful information. An MSP that acknowledges the gap and commits to a timeline is more trustworthy than one that claims everything is “covered by our security policies.”

6. How Are AI Tool Access Permissions Scoped?

The principle of least privilege applies to AI agents the same way it applies to human users and service accounts. An AI tool that monitors your endpoints doesn’t need access to your email. An AI agent that triages help desk tickets doesn’t need administrative credentials on your domain controllers.

Ask your MSP how they scope permissions for AI tools that access your environment. Specifically: do AI agents run under dedicated service accounts with limited permissions, or do they inherit broad access from a shared admin account? Are permissions reviewed and adjusted as AI tools are added or updated? Can the MSP show you the current permission set for each AI agent touching your systems?

This question tends to reveal how mature an MSP’s AI operations really are. Providers who have thought carefully about AI governance will have purpose-built service accounts, documented permission sets, and a review cadence. Those who adopted AI tools quickly may have granted broad access to get things working and never revisited it. A Copilot and shadow AI assessment can help identify where permissions have drifted beyond what’s needed, whether the drift originated from your MSP’s tools or your own.

7. What Happens to Your Data if You Leave?

This question matters for any MSP relationship, but AI adds a specific wrinkle. When your data has been processed by AI systems, you need to know what residue remains.

Ask: if we terminate the engagement, is our data deleted from all AI systems, model caches, and training datasets? Can you provide written confirmation of deletion? Is any of our data embedded in models or vector databases that persist beyond the engagement? What’s the timeline for full data removal?

The practical concern is that some AI architectures store processed data in ways that are difficult to fully purge. Retrieval-augmented generation (RAG) systems, for example, may store embeddings of your documents in a vector database. If your MSP uses a shared AI infrastructure across clients, your data may be co-mingled in ways that make clean separation nontrivial.

A straightforward MSP will address this in their service agreement. If the topic isn’t covered, add it before you sign. Data portability and deletion rights should be explicit, not implied.

Use These Questions in Your Next Review

Print this list. Bring it to your next quarterly business review with your MSP, or include it in your RFP if you’re evaluating new providers. The answers will tell you more about an MSP’s operational maturity than any sales presentation.

No MSP will have perfect answers to all seven questions today. AI governance is evolving alongside the technology itself. What matters is whether the MSP has a clear picture of their own AI usage, honest answers about current gaps, and a visible effort to close them. That’s the difference between a provider who uses AI responsibly and one who adopted it because the marketing team said to.

Your managed IT provider handles your infrastructure, your data, and increasingly your AI workloads. The risk profile of that relationship has changed. These seven questions will help you understand how much, and whether your provider has kept up.