research – David's Blog

Don’t chase the score

In general, the growing emphasis on ‘measurement’ of different educational outcomes is a good thing. This is especially the case with wellbeing in schools. As the data-collection tools for wellbeing have become more sophisticated and prioritised, first by schools and now governments, it has attracted more attention, energy, and resources. As a result, we have higher quality data, better materials, and refined practices.

Because wellbeing researchers and schools are trying to harness the scientific method, they rely heavily on the process of quantification. Quantification allows us to take inherently ‘uncountable’ and intangible human experience and turn it into numerical data. For example, the feeling of trust that a student has for their teacher can be quantified into a number using a rating scale. This is helpful because it allows for statistical analysis and more meaningful discussions that are less hindered by subjective language.

But quantification is not the same as measurement.

Measurement is counting the quantity of a unit of material via an agreed standard.

Quantification is turning something that cannot be counted or measured directly into a number via subjective opinion.

So when I use a scale to measure my weight and I’ve gone up from 76kg last month to 78kg this month, that’s because I am heavier. We can measure weight directly.

But when a student’s self-reported ‘trust score’ goes up from ‘3’ last month to ‘4’ this month that does not mean that they trust me more. It might mean that. But it might instead be because it’s the student’s birthday today or because their football team won on the weekend or because they now trust their other teachers less and so, relatively, they feel more trusting of me. Or maybe it’s because scoring something as complex and nuanced as trust on a numerical scale of 1 to 5 is a very crude method. We don’t actually know. And that’s because we cannot measure trust directly.

And all of this is fine. As long as we don’t try to chase the score or place too much value on the score or data.

Ultimately, we must be focussed on optimising the wellbeing of our students, not the wellbeing data. Those two things are not the same.

Read the fine print

If you work in education and haven’t been living under a rock for the past ten years, chances are you’re familiar with Dr Carol Dweck’s work on mindset. For decades, Dweck has been studying the effects that our beliefs about ability have on learning behaviours and our future success.

If you believe that ability is mostly the result of practice and hard work, you tend to work harder, practice more, accept more feedback and tackle more challenging problems. And guess what happens…you get better at whatever you are working on. Dweck calls this a growth mindset.

If you believe that ability is mostly the result of predetermined genetic factors or inherent ‘talent’, you don’t practice as diligently, are resistant to feedback and tackle less challenging problems. (After all, there’s no point practicing if ability is genetic.) And guess what happens…you don’t get better at whatever ability it is you think is ‘talent’ based. She calls this a fixed mindset.

Despite some vocal critics of Dweck’s work, there are significant benefits associated with nurturing a growth mindset in children. But like all psychological theories, we need to be careful not to skim the headlines of research and, consequently, develop blunt, broad-spectrum, low-resolution approaches.

Here are just a few of the situations in which Dweck herself, a staunch proponent of growth mindset, has explained that a fixed mindset is healthier and beneficial:

When faced with certain acute mental or physical health conditions, those who believe they will be able to work their own way through it or ‘get over it’ may be less likely to seek professional or medical help and therefore increase the risk of harm.
When faced with issues associated with sexual orientation, those who accept that this is who they are and this is who they’re meant to be seem to respond more effectively and adjust more healthily than people who think they should be resisting or trying to change something about themselves.
When faced with the realisation of aging, graceful acceptance of the inevitability of physical change is often associated with more healthy adaptation of behaviour. In Dweck’s words, we are less likely to “run around nipping and tucking”.

As educators, we should be consuming high-quality research findings. But when we do, it’s important to read the headlines and the ‘fine print’.

Why so early?

As human and wellbeing science continues to mature, it forces us to ask questions about schooling that are a little bit uncomfortable. One such question is: “Given what we now know about delayed circadian sleep rhythms in adolescents, why do secondary schools still start lessons so early in the morning?

The underlying biochemical processes that drive an altered, later sleep cycle in teenagers have been well established. Whilst different people have different chronotypes which affect our propensity to want to sleep at certain times, the average adolescent doesn’t begin to feel the effects of sleep-inducing hormones until about 10:45pm. Even, if they fall asleep by 11:00pm, many teenagers require 9-10 hours of sleep, which means they shouldn’t be waking until 8 or 9:00am. It would seem ideal then, that school should begin at 10am to facilitate sleep, health and performance.

This is exactly what a recent UK study found. A shift in start time from an 8:30am to 10:00am not only saw grades significantly improve, but rates of absence due to illness halved. It’s hard to imagine what long-term impact these two outcomes might have across a population.

Many other studies have shown similar outcomes. Whilst it is likely that changing technology and social media are contributing to the widespread sleep deprivation we see in senior schoolers, part of the problem is within our control.

So why don’t schools respond to the research and begin lessons later? Because it’s inconvenient. Think of everything that needs to be changed: bus times, sport training, meetings, lunch times, etc…not to mention the impact on parents…it’s a hassle.

Those are all real concerns. And maybe they justify a maintenance of the status quo. Or maybe we need to think more creatively about this problem.

Data as art

In a recent conversation discussing some of the limitations of wellbeing data, a trusted colleague mentioned to me that he views empirical data as a form of art. It might feel odd to think of scientific data as art but it is also a beautiful concept.

The Collins Dictionary defines ‘art’ as consisting of “paintings, sculpture, and other pictures or objects which are created for people to look at and admire or think deeply about.” Data, particularly from the human sciences, is absolutely intended to create a ‘picture’ for us to think deeply about. Like art, data is not an actual snapshot of reality but rather a creative representation of reality. In fact, often the most effective data – data that moves us and affects us, is data that is represented graphically, typically crafted with much thought given to the colour, contrast, form, and dimension.

This is not true of all data. Some data is highly objective and clean – we could call this realism. Some data is quite crude and bold – impressionist. Other data attempts to quantify the inherently subjective human experience – expressionist. And, of course, like Rothko’s painting, some data is distinctively abstract.

All data, however, share the same purpose – to tell a story. These stories help put language to experience, to challenge our view of the world, to help create a sense of coherence and meaning.

When we view data from the human sciences as art, we are able to see it for what it really is; not fact or truth, but a way of harnessing human creativity and ingenuity to transcend our own small, individual lives. Like art, data allows us to view the world differently, with greater integrity. It has the power to open our eyes and capture our hearts.

This is why data matters.