There's a piece of startup advice so embedded in tech culture, it barely earns debate anymore: “move fast and break things” (and, quiet part out loud: “ask forgiveness later”). It's produced some remarkable companies, within the safe space of merely breaking beta release code that does little more than raise a few clients’ eyebrows. But the mantra has also produced a long list of tech vendors who show up to a university procurement conversation and can't understand why no one’s buying, figuratively nor literally.
As the founder of skillyAI, I’ve been on both sides of this divide, and sat in all the seats: teaching at Carnegie Mellon, leading product and design teams at various startups, consulting at BCG, and now building skilly to power the people who power humanity. There’s no shortage of lessons learned along that path, but when we talk about the dangers of “breaking things”, the lesson I’ve cemented is this: privacy isn't a bolt-on feature to check the compliance box. It's a fundamental design requirement which has to be present at the first line of code; otherwise, the product will never fulfill the needs of the institution it's meant to serve.
April 2026 made the case better than I can. Instructure, the company behind Canvas (the learning management system used by ~41% of U.S. higher ed institutions), was hit twice in ten days by the extortion group ShinyHunters. The attackers claimed to have pulled 3.65 terabytes of data, including names, emails, student ID numbers, and private messages, across nearly 9,000 schools and universities worldwide. It wasn't one campus' IT team that failed. It was a single vendor sitting underneath thousands of institutions, which meant one bad week became a sector-wide emergency.
That's the risk profile universities are underwriting every time they sign an AI vendor contract, and they’re painfully aware of it. In Ellucian's latest survey of higher ed professionals, data security and privacy is the single largest barrier to AI adoption, cited by 61% of individuals and 56% of institutions, even as 66% of institutions report they're already using AI in some form, up from 49% a year earlier. Adoption is accelerating and skepticism about vendors is rising at the same time. That combination is exactly why the vendors who treat privacy as a first-class design constraint, not an afterthought, are the ones who will still be in the room a year from now.
FERPA Isn't Red Tape. It's the Terms of Trust.
To an outside observer, FERPA can look like bureaucratic friction… just one more hurdle to an AI rollout that could otherwise move at Silicon Valley speed. That reading misses what FERPA actually protects: the basic terms under which a student agrees to trust an institution with their data in the first place.
A university isn't just another business evaluating a vendor's feature set. It's legally and ethically bound to protect the people it serves, many of whom hand over sensitive academic, financial, and personal information as a condition of attending. An AI company that treats this obligation as an obstacle to build around, rather than a pillar to build on, has misunderstood who its buyer is and what its buyer is responsible for.
What Privacy-by-Design Actually Requires
In practice, privacy-by-design means asking a different set of questions from the very start of a product's development:
- What data does this feature genuinely need, versus what would simply be convenient to collect?
- Where does that data live, who can see it, and for how long?
- Can this system explain, to a provost, a general counsel, or a worried parent, exactly what it's doing with student information and why?
Retrofitting these answers after a product already exists is far harder than designing for them on day one, and universities can tell the difference. A CIO reviewing an AI vendor's data architecture doesn't need long to figure out whether privacy was a founding principle or something stapled on to win a contract. It's part of why skilly was built with a private data layer from day one: an institution's knowledge stays out of public model training by default, not as a setting a customer has to remember to toggle on (or worse, re-toggle after a new update lands that conveniently unlocked privacy settings… a classic “break things” move).
The Tech That Ultimately Wins, Moves at the Institution's Pace
A lot of AI companies talking to higher ed have the incentives backward. The instinct is to treat university caution as a hurdle to clear with faster sales cycles or more aggressive pilots. But the vendors who build durable relationships with universities will be the ones willing to slow down and build trust the way institutions require it: transparently, with clear data governance, and with a real answer to "what happens to our students' data" that doesn't require a lawyer to parse.
That's not a compromise on ambition. In a sector where a single privacy failure can end a vendor relationship, and a reputation, overnight, it's the only strategy that scales sustainably. IBM puts the average cost of a US data breach at north of $10 million; even education's comparatively lower average, in the $3.8-4.4 million range, isn't a number any procurement office wants attached to a new AI contract.
The Imperative Is Only Getting Stronger
As AI tools become more embedded in how students search, ask, and get answers on campus, the volume and sensitivity of the data flowing through those systems will only grow. The vendors who treated privacy as a first-class consideration early will be the ones universities trust to handle that growth. The ones who didn't will spend the next several years trying to retrofit trust they should have built from the start.
"Move fast and break things" was never really a strategy fit for institutions built to last generations. The partners who bring lasting value to higher ed will be the ones who know how to scout new paths without burning the forest behind them.