Every school that evaluates AI software faces the same uncomfortable moment. The sales team is eager, the pilot looks promising, and then someone asks the privacy question. The answer should be clear. Often it is not. That is the first signal that the product may not be ready for an institution that takes student data seriously.
Privacy in education AI is not a compliance checkbox. It is a design philosophy that should show up in the product, the contract, and the support relationship. Here are the questions that matter, grouped by the stage of the conversation where each belongs.
Before the pilot: data collection and purpose
Start with the basics. What specific data does the product collect from students, and what does it actually use. The vendor should be able to describe the data pipeline in plain terms, not in legal boilerplate.
- What learner data is collected, stored, and processed by the AI system?
- Which data fields are used to generate AI outputs, and which are stored for audit or reporting only?
- Does the product use any student data to train models that serve other customers?
- Can the vendor explain why each data field is necessary for the product's core function?
Before the pilot: access and visibility
Who can see what. The answer should cover three audiences: the school staff, the vendor's support team, and any third parties. If the vendor cannot clearly separate those layers, the product may not be designed for institutional use.
- Who inside the school can access student data, and what role-based controls exist?
- Does vendor support staff ever need access to student data, and under what circumstances?
- Does the vendor share, sell, or expose any student data to third parties, including for research or model improvement?
- Are AI-generated outputs and recommendations visible to teachers, parents, and students, and can they be exported?
Before signing: storage, location, and retention
Where the data lives matters for compliance and for control. Schools in the US, EU, and other jurisdictions have specific requirements. The vendor should know them and should be able to demonstrate compliance, not just claim it.
- In what geographic region(s) is student data stored and processed?
- Does the vendor meet FERPA, COPPA, GDPR, or other relevant privacy regulations for your jurisdiction?
- What is the data retention schedule, and can the school specify a shorter retention window?
- Does the vendor provide a clear data deletion process when the contract ends?
Before signing: the exit plan
The most important privacy question is the one asked at the end of the relationship. What happens to the school's data when the contract is terminated. A vendor that cannot answer this question should not be trusted with student data during the contract.
If you cannot explain what happens to your data when you leave, you should not have trusted them with it while you stayed.
- Can the school export all student data in a standard, machine-readable format?
- What is the timeline for data deletion after contract termination?
- Does the vendor retain any data after deletion for legal, audit, or regulatory reasons, and for how long?
- Will the vendor provide a signed deletion certificate confirming compliance?
A practical evaluation approach
Privacy evaluation should not be a separate phase tacked onto the end of the sales process. It should be woven into the pilot. Run a small pilot with synthetic or limited data, then evaluate the vendor's responsiveness to privacy questions before expanding the scope.
This is the approach Nivorius uses when designing privacy-first AI products for education. Every feature, every data pipeline, and every integration point is evaluated against a simple test: would I be comfortable explaining this to a parent. That test catches more gaps than any compliance checklist.
Part of the Nivorius research and consulting team, focused on practical applications of AI in education and enterprise contexts.
