What breaks in an AI-built education app?

What breaks in an AI-built education app?

Roles and children's data. An education app has teachers, students, parents and administrators, each allowed to see different things, and AI-generated code tends to treat them all as logged-in users. Schools will ask about student privacy and, if young children use the app directly, children's privacy law applies.

What is really going on

Education data is a specific kind of sensitive. It relates to children, it is held on behalf of institutions and it tends to outlive the school year. Parents, teachers and administrators have different rights over it, and the law treats data about minors more strictly than data about adults.

In the United States, schools that receive federal funding are subject to FERPA, and they usually require software vendors who handle student records to act under their direction and protect the data accordingly. Separately, a service that collects personal information directly from children under 13 is subject to COPPA. Other countries have their own rules.

On the engineering side, it comes down to who can see which student's data, whether the roster is right, and what you collect, keep and share. Those are the questions a school will ask before it lets you near a classroom.

Where education apps go wrong, most common first

1. Roles are collapsed into "logged-in user"

   How to tell A student can open another student's work or a teacher view by changing an id. A parent sees a class rather than their child. Teachers see classes they do not teach.

   The fix Model school, class, teacher, student and parent relationships explicitly, and enforce each read and write against those relationships on the server or in database policies.

2. More student data is collected than the product needs

   How to tell Sign-up asks for birth dates, addresses or photos that no feature uses. Analytics tools receive student identifiers.

   The fix Collect only what a feature needs, keep student identifiers out of analytics and advertising tools and set a retention period.

3. Children use the product directly and nobody has checked the rules

   How to tell Children under 13 can create their own accounts without a parent or school authorising it. There is no age gate or consent flow.

   The fix Find out whether COPPA applies to you. Where it does, use a compliant consent route, or have the school create and authorise accounts, and get legal advice on which.

4. Roster import and account lifecycle are fragile

   How to tell Imports duplicate students, merge two children into one account or leave graduated students active. Teachers share logins because adding a colleague is hard.

   The fix Match students on a stable identifier, handle duplicates deliberately, deactivate accounts at year end and give each teacher their own account.

5. AI features send student work to a model provider

   How to tell Essays, questions and chat transcripts go to an AI model. The provider's terms on retention and training use have not been read, and the school was not told.

   The fix Check retention and training terms, minimise identifying details, tell schools which providers you use and let them opt out.

6. No way to export or delete a student's data

   How to tell A school or parent asks you to delete a child's records and the answer involves manual database work across several tables and backups.

   The fix Build a tested deletion and export procedure covering all stores, including logs and backups, before the first school asks.

How we diagnose it

1. Map people and relationships. Write down every role and who may see whose data, then turn it into a set of test cases.
2. Attack across roles. As a student, a parent and a teacher from another school, try to read and change records you should not reach, through the app and the API.
3. List what you collect and where it goes. Trace each field about a student to its storage, logs, analytics and vendors, and mark the ones the product does not need.
4. Check the age and consent path. Establish who creates accounts for young children, who consents and what you record, then compare it with what the applicable rules ask.
5. Read the code behind it. We read authorization, roster handling and data export and deletion, and rank findings by harm to students.

What fixing it changes

• Explicit role and relationship checks on every read and write.
• Data collection reduced to what the product uses, with retention limits.
• A legal answer on the rules that apply, and an account creation flow that fits it.
• Reliable roster import and year-end account clean-up.
• Tested export and deletion, and a clear statement of which AI vendors see student work.

When you do not need us

We can test and read the software, but we do not give legal advice or certify compliance with FERPA, COPPA or any other law. If you sell only to adults, such as corporate training, much of this does not apply and the B2B page is the better starting point.

Your next step

Straight answers