A Cohort Can Look Healthy While Three Students Are Already Lost

A healthy-looking cohort and a healthy cohort are not the same thing.

If you run cohort-based cybersecurity training, you know the picture I mean. Submissions are landing. The completion line on the dashboard is climbing. The room is quiet, heads down, everyone working. By every signal you have access to, things are going well.

None of that tells you whether a handful of students stopped understanding two topics ago and have been coasting on pattern-matching ever since.

That is the uncomfortable part. The dashboard measures the average, and the average is exactly where the people in trouble hide.

The average is a hiding place

Here is the mechanics of it. A cohort of, say, 24 trainees produces a completion rate. Twenty are genuinely tracking the material. Three or four are not, but they are still submitting. They copy a working command from a peer. They adjust a script until the output looks right without understanding why it does. They pass the checkpoint.

On your metrics, those students are indistinguishable from the ones who actually understood. The line goes up. The cohort looks healthy.

A correct submission built on a flawed process is a future problem wearing a good grade. In a certification programme it surfaces later, usually at the worst time, when the material stops being isolated exercises and starts building on itself. The student who was pattern-matching in week two hits week five and has nothing underneath them.

Why lab-heavy programmes make this worse

This is not a generic edtech observation. It is specific to what we do.

A lab session is the one place where the gap between looking competent and being competent is widest, and it is also the place where you can see the least. You can read a written answer and tell whether someone understood. You cannot read a terminal session the same way. What you get is the end state: the flag captured, the box owned, the exercise marked complete. You do not get the twenty minutes of confusion, the borrowed command, the moment the student stopped reasoning and started guessing.

The most important things happening in a training programme are invisible to the people running it. In a hands-on cybersecurity course, that invisibility lives inside the lab, and the LMS was never built to look there. It records submissions. It does not record process.

Quiet is not the signal you think it is

We tend to read a quiet room as a working room. In my experience running cohorts across very different environments- intensive programmes and standard ones, groups of twelve and groups of over fifty, highly selective intakes and beginner courses for people with no technical background - quiet means different things depending on who is quiet.

A strong student is quiet because they are absorbed. A struggling student is often quiet because they have stopped asking. They have decided that falling behind is embarrassing, or that the instructor is busy, or that they will catch up on their own tonight. They rarely do.

Mixed-skill cohorts make this a structural problem, not a motivation problem. When a single pace serves a group with a real spread of ability, the fast students are under-stretched and the slow students quietly drop off the back. The pace is set for the middle. The people at the edges are the ones you can least afford to lose track of, and they are the ones the average erases.

This is a design question, not an effort question

The instinct is to say the instructor should catch it. Walk the room, watch the screens, notice who is stuck.

A good instructor does exactly that. But look at the ratios this actually depends on. When one instructor is responsible for fourteen or fifteen trainees in a live lab, there is no version of walking the room that gives real coverage. By the time you reach the third screen, the first has moved on. The instructor is not failing. The model is asking a person to be in fifteen places at once and see inside every terminal while they do it.

So when a cohort looks healthy and a few students are already lost, it is not because someone was careless. It is because the tools we run these programmes with report submissions and treat everything before the submission as a black box. The visibility gap is built in.

That is the part I keep coming back to. The metric we trust most - completion - is measuring the one thing that hides the problem, and the moment where the problem is actually visible, the live lab, is the moment we can see the least.

The open question

I do not think this is solved by trying harder or watching more screens. I think it is a question about what our programmes are built to see.

Adaptly exists because better tools didn't. We’re at early stage and looking for the right training providers to build this with - a small number of founding design partners who recognise this problem in their own cohorts and want to work on it seriously. Not a customer. A partner. If, after a conversation, it doesn’t seem like the right fit, we’ll be the first to say so.

But before any of that, the honest question is one I’d ask any programme director I respect: how do you tell the difference between a cohort that’s quiet because it’s working and a cohort that’s quiet because a few people have checked out?