AI Is an Accelerator, Not a Replacement: Three Product Decisions From an Edtech Platform Serving 2M Students
Authored by: Jono Ellis
My hot take as a CPO in edtech: AI hasn’t changed my job, and it hasn’t changed what an edtech product needs to do either. It’s a tool that helps us ship code faster and produce content at scale, but the fundamentals of the job are the same as they’ve always been: solve a real problem, do the user research, listen to users, obsess over the experience, and ship something useful.
AI is an accelerator, not a replacement. The product leaders who forget that end up shipping impressive demos that don’t move outcomes, and every quarter I see teams reach for an AI feature when the real problem is a product decision they haven’t made yet.
Three decisions we’ve made at Cognito, where we now support over two million students, have reminded me why the fundamentals still hold, and why treating AI as an accelerator rather than a substitute is the difference between shipping something useful and shipping noise.
1. Listen to users: The AI feature we let students turn off
The clearest example is our AI marking tool. We built it in-house, trained on thousands of our own handwritten exam questions. A student types an exam-style answer, the tool marks it against how an examiner would, and tells them what they missed.
Then we measured the impact. Students on Cognito improve by around two grades on average, but when we checked whether the AI marking was driving that lift, it wasn’t. The lift held whether students used the feature or not, and some told us directly they preferred marking their own answers because they engaged more when the reasoning was on them, not on the AI. So we built a toggle to switch it off, and some schools have asked us to disable it entirely across their students.
That’s not a strategy-deck decision – it came from listening to users and taking them seriously enough to build against our own assumption. The tool ended up doing the accelerator job well: speeding up feedback for students who want it, without pretending to be the reason they’re learning.
2. Solve a real problem: Freemium calibration
The second one is less glamorous but more constant. We’re a bootstrapped freemium product, so we have to think carefully about how much of the platform to give away. Give away too little and users don’t stick around long enough to see what’s good – give away too much and there’s no reason to ever pay.
Over the last year we ran twenty variants of the free-tier split (yes, twenty). Video lessons and study notes stayed free, while flashcards, quizzes, and exam-style questions became free with weekly limits and unlimited on Cognito Pro. Some variantes increased conversion but hurt long-term retention. Others delighted free users but gave them little reason to upgrade. We eventually landed on a balance where the free product remained genuinely useful while Pro removed the limits for students who wanted to practise more. Ultimately it’s a balancing act between whether the free tier delivered enough value to be worth using in its own right, and whether it converted to Pro at a rate that funded the rest of the platform. Optimising one at the expense of the other is a good way to break the funnel while thinking you’re improving it.
The AI temptation on this one is to solve the freemium question with an AI feature, whether that’s auto-personalising the free experience or adapting the tier logic. We didn’t go that way because the real problem was calibration, and the real research was watching how students used the tiers we had.
3. Obsess over the experience: Don’t automate the productive struggle
The third decision is the one I think about most, and it’s the hardest to hold the line on. There’s a whole category of AI features in learning products that quietly remove the thing that makes the learning happen: auto-generated flashcards, LLM-summarised notes, one-click essay drafts. Each strips out the productive effort that creates the memory, which is what thirty years of cognitive science say you need.
We built the platform around active recall, dual coding, and worked examples for that reason. Some of the most-requested features on our roadmap would undermine what we’ve built, because they’d feel helpful in the moment while costing the student the learning, so we turn most of them down. Obsessing over the experience of a learning product means protecting the friction that makes the learning stick, even when users would rather you smoothed it away.
The job hasn’t changed
None of this is particularly groundbreaking, but when there’s a lot of hype it can be easy to lose sight of the fundamentals of good product leadership. Listen to users, solve real problems, obsess over the experience, ship something useful. What AI has changed is the stakes on getting them right, because it’s easier than ever to ship a feature that looks impressive in a demo but doesn’t move the outcome.
If you’re leading product in an AI-heavy category and wondering whether the job has changed, my honest read is that it hasn’t; what’s changed is the discipline required to keep doing it well. Treat AI as a good accelerator and you’ll build things that matter; treat it as a replacement and you’ll ship features nobody uses.
Author Bio: Jono Ellis, Chief Product Officer, Cognito (cognito.org)
Jono Ellis is Chief Product Officer at Cognito, a study platform used by over two million students preparing for GCSE, A-Level, IB and AP exams. He writes about product decisions in edtech, growing bootstrapped freemium products, and the parts of AI that move outcomes for learners.