Randomised Controlled Trials in English Schools

[EducationState]

Currently, the Department for Education is funding a number of experiments (known as randomised controlled trials (RCTs)) in English schools www.gov.uk/government/news/new-randomised-controlled-trials-will-drive-forward-evidence-based-research

Apart from questions regarding whether the £140 million being spent on RCTs is money well-spent, there are other issues concerning the use of RCTs, which parents may not be aware of, which the DfE have perhaps not made as clear as they might, and which by writing this thread, in some small way it is hoped to address.

RCTs are normally conducted in medical research and have proven highly effective, but come with stringent ethical controls, and similar controls will more than likely be in place for those being carried out in English schools.

However, just as in medical research, as such trials are experimental, there is the possibility that an education experiment's instrument/intervention (e.g. new teaching method) will have a negative rather than positive or neutral impact. In medical research, negative impact can be fatal (e.g. the side-effects of a new drug treatment). In education, negative impact is highly unlikely to be fatal, but may, for example, set a child's learning back some time.

This thread, therefore, is meant as both a reminder and a warning. It is a reminder that parents should expect to be notified before their child participates in such an experimental trial and also parents should expect to be given the choice of whether they allow their child to participate. If in the unlikely situation that parents are not notified nor given the choice then this lack of consultation ought to be taken up with the school.

The warning to parents on the other hand is that being experimented on even in educational contexts (e.g. the classroom) could have a negative effect on your child, and this is clearly something that needs bearing in mind.

[noblegiraffe]

That's fantastic news, I'm really pleased that there will be a decent evidence base for what is passed off as best practice in schools.

At the moment kids are experimented on anyway with endless new initiatives. At least now we might get some idea of whether these initiatives are working.

[thecatfromjapan]

I hope it does have the effect you see, noblegiraffe.

I find the constant, unregulated experimentation totally annoying too, and I'm just a parent. I fondly hope that my children's teachers will buffer them from the worst of the wild changes.

Anyway, let's just hope it all goes as Mr Goldacre hopes.

Was he an odd choice to write this report? Is he a proper expert in cross-disciplinary research? I only know him as a quasi-celebrity science journalist. Not that that necessarily means this is a bad idea. Stamps were "invented" by a non-expert in anything, weren't they? They were a good idea. And I do like a lot of what he writes.

[thecatfromjapan]

By the way, what if you want to opt your child out of one of these trials? Would the LEA have to place your child in another school; could you choose that school; would they pay for transport if it is a long way away?

[ReallyTired]

I don't think its realistic to ask parents' permission before doing an educational experiment on their child. Teachers need a certain level of automony and should not be required to ask parents for permission every time they make a decision. Rather than asking permission, prehaps experiments should need ethical approval.

[scaevola]

The way I read e release is that they are not imposing a teaching method as part f the RCT. Rather that they are matching cohorts of schools by intake, then monitoring the teaching and other interventions (that the schools are selecting anyhow) then comparing outcomes. There would then be a collaborative effort between the participating schools and researchers to establish, on the basis of evidence, which methods and interventions lead to the best outcomes. And they can then be encouraged for general use on the basis of evidence (rather than just imposing something that someone somewhere thinks is a good idea).

[FidgetPie]

I think RCTs in education are a great idea. Having a strong evidence base for what is more or less effective can only be a good thing.

It has benefited our medical health so why not try it on other aspects of development. As Noblegiraffe says - at least then people will be able and expected to prove / justify new initiatives or explain why they are doing things a certain way. Very happy for DD to be involved (and surely no different to the present where one teacher, school or LA takes one approach while another does it differently).

I think Ben Goldacre was / is originally a doctor, which might be why he was chosen.

[thecatfromjapan]

I know he's a trained medic. but there are a lot of other people who are extremely qualified in this area - Professors of medical ethics; specialists in the development of randomised trials; professional educationalists; professionals involved in the development of public policy; etc. and they weren't asked.

As I said, I read BG's paper and I liked it. I was particularly interested in his suggestions about teachers being at the forefront of developing these trials and in his idea that career progression be linked to undertaking research.

I suppose I am just worried that it is all just a bit of "puff" for some current experiments that are underway at the moment and will just disappear. I think what noblegiraffe said in her first post is pretty much where I stand on this: the massive, sweeping changes that have been inflicted - they really need to stop.

I guess I worry that they have chosen BG for headlines, and precisely because he lacks professional clout in this field there will be no comeback when it all quietly disappears.

I really did think BG's suggestions were very interesting.

[ReallyTired]

Education is strongly linked to child pychology. Child pychology is linked to medicine. If RCT trials have dramatically improved medicine then they may well improve education. I don't think we should dismiss Goldburg's suggestion just because he is a doctor.

Finding out what inteventions work is a job for a skill statistican rather than a teacher. Clearly teachers need to work closely with top statisticans to use each other's skill sets to find out what works.

Our year 6 children are all part of big experiment this week when they sit the grammar paper. Some of our youngest children (year 1) are going to be doing phonics test in a few weeks time. All these changes have been done with no randomised trials.

[noblegiraffe]

Ben Goldacre will have been picked (if indeed he was picked, rather, I suspect that he simply browbeat the government into taking notice of his suggestions and this is all his idea) because he has been banging on about this stuff for years.

Here are his many articles on the appallingly designed Durham Fish Oil trials
www.badscience.net/category/fish-oil/

And his many articles on Brain Gym
www.badscience.net/category/brain-gym/

Remember, this rubbish has been going on in schools with real children.

He's a big fan of evidence-based policy in politics too, not just education.
www.badscience.net/category/evidence-based-policy/

[elastamum]

Not sure why Ben Goldacre is considered the best person to front this, but moving to evidence based intervention can only be a good thing. But the use of the term RCT is a bit misleading. The first study described sounds like more of a longditudanal study than a RCT.

Anyway, just to balance a previous posters comment that RCT's can result in fatal side effects. It is fairly rare that this occurs and well designed RCT's are usually preceeded by a pilot small scale study and have to go through stringent regulatory and ethics approval - unlike the sort of experimentation we frequently see in education

[ReallyTired]

I suppose that Ben Goldacre has been picked for his enthusiam and passion for the use RCT in all different areas of life.

I feel its important that teachers are at the forefront of research, but someone who has a degree in early years may need help with complex post graduate statistics. Everyone has different talents and a good RCT needs a range of people with different talents.

Many years ago I was part of a research team of hundreds of scientists and mathematicans world wide who did research into the affects of depleted uranium on soldiers. No one had the knowledge to do the research on their own.

Maybe we can have better analysis of existing interventions as well as experiments.

Good research is collabative. Teachers don't have a strong research tradition and prehaps help from other professions to make teaching a research based profession.

[Trill]

How else do you suppose we go about finding out what works and what does not?

[noblegiraffe]

I don't think he was an odd choice to front this campaign, but then I have seen him talk. He is an articulate and enthusiastic campaigner for evidence-based decision making and a consummate dispeller of pseudo-science and bullshit claims. He has written before about the poor evidence base for certain school initiatives (also see his interesting discussion on the evidence about whether exams are getting easier) so he hasn't solely concentrated on health in the past.

He is also an incredibly engaging speaker and so may actually (as opposed to some dry academic) be able to convince teachers to take on extra work and conduct research in their own classrooms.

I don't think it really matters that he doesn't work in education because this isn't about education really, it's about improving the evidence base for initiatives in education.

[tiredaftertwo]

I think concerns about experimentation are unfounded.

Education practice and policy changes all the time, often with a shaky evidence base (yes, Mr Gove, I'm looking at you, stop fiddling with public exams in the back row, and listen to me when I'm talking).

The changes are often not controlled, outcomes may not be defined beforehand or carefully monitored, and so on. One of the many advantages of this way is that negative impacts will be likely to be picked up sooner and if serious the trial can be abandoned (as happens in medical research).

It is good news IMO - if teachers can be persuaded to get on board.

[RunnerHasbeen]

So, between experimenting and gradually implementing changes proven to work and rushing in changes that a random politician thinks is best as it represents his schooling - the RCTs are the thing you are worried about? Really?

[thecatfromjapan]

Are all these slightly aggressive questions directed to me?

Why?

If you read my posts, it's clear that I find the idea interesting, and that I have actually read Ben Goldman's paper (before this thread was started; because I was genuinely interested in the idea; because I generally like Ben Goldman's work) - I don't think many of you have, and yet I'm supposed to be a straw man against which you are constructing some hallucinatory argument? Odd.

If you'd read his paper, you would know that, yes, Reallytired, Ben Goldman envisions these trials to be team efforts in the main, with teachers mainly involved in shaping the studies, putting them into practice, and identifying areas of possible research. Though obviously, if you are a statistician and a teacher, you'll be off to a head start in any number-crunching. But, I see your response there as a genuine, interested response. And it was nice to read it.

I'm depressed, frankly, by the snippiness, and yes, the stupidity of some of the responses to my questions. There has been a tendency to read anything posted on mn increasingly in the most cynical, aggressive and mean-spirited fashion. It doesn't enable the spread and facilitating of ideas and creativity - it actively inhibits it. That is what I mean by stupidity. You are not stupid women. I recognise you from other threads. So why are you acting here in a fashion to put a full stop to thought, rather than promoting a conversation?

I can;t help but feel my only "crime" was to ask why Ben Goldman was chosen. I think that's a valid question. I very, very sincerely am interested in this idea. Everyone on this thread has pointed out that the current situation really shouldn't prevail. I do wonder if this might be the equivalent in education of Labour giving the Bank of England its independence. It's fascinating. So it is worth considering exactly how seriously the government intend to take this.

I'll repeat: there is no-one on this thread arguing that this is a bad idea. To pretend otherwise is, frankly, the equivalent of picking a rather drunken fight at your mate's birthday party. It's not a good look.

[thecatfromjapan]

Seriously, when the fuck did mn change from a place you could have collaborative, constructive conversations - based on an assumption that the majority of you understand and accept the rules of civilised communication - to a multi-topical argument - with the rules of the bunfight?

Sorry if I sound pissed off - I am. This is not AIBU.

[Trill]

thecatfromjapan are you sure you posted that on the right thread? Who do you think is having a go at you? And who is Ben Goldman?

[noblegiraffe]

Cat, I think the slightly aggressive questions are directed at the OP, not you!

[StarlightMcKenzie]

Kids are experimented on all the time in schools, by latest daft government initiatives, to HT's 'turnaround' styles to whims of the class teachers.

RCT at least regulate this and build up a bank of knowledge about effectiveness and I would hope, give the teaching participants a reminder/education/insight into EBP.

[StarlightMcKenzie]

FWIW, I think BG is a patronising arse.

And imo teachers prefer to think of teaching as an art rather than a science so are (in a general sense) unlikely to get on board.

[tiredaftertwo]

Umm, I think the questions were brisk and precise, not aggressive. Thecat, I thought your first post was clear, and made sense. The OP did, however, issue a clear warning and a call for parental consent to be required and people are discussing the implications of that. That's fair enough, surely?

I have seen the behaviour you describe on other threads - and share your frustration - but I do not see it here, and I think there have been some good descriptions of the way complex multidisciplinary research can work.

[thecatfromjapan]

Sorry.

I'm probably just a bit defensive. I've been away for a bit and ... I don't know. It all seems a lot rougher than I remember.

Starlight - that's a very interesting break from the "BG is great" consensus I usually hear - I have a few rather forceful scientist friends who think he is the leader of the righteous. Refreshing, actually, to read you saying that.

Trills - Ben Goldacre is the author of this report we're discussing. He's a science-based journalist and a best-selling author. He writes for the Guardian.

Now, here is what I want to know: who is Education State?

[UnderwaterBasketWeaving]

Ben Goldacre is very well qualified to promote RCT in educational research.

He has been campaigning for many years to change the publishing policies of pharmacological research, as at the moment drug companies are not required to publish studies that show their drug to be dangerous or ineffective. Instead the studies are just halted and swept under the carpet. Meaning the ONLY published work is in support of the drug companies.

In terms of educational research, this is just formalising what teachers do anyway. They try different teaching techniques with different groups at different times as a matter of course. What works for a very able group, just won't work for a more challenged group, so they try different tactics to find something that works. Also, teachers may change techniques to keep the classroom vibrant, to keep challenging students rather than letting them fall into routine that might become tired and unstimulating.

Until now, educational research has been carried out as "Action Research". Educational research is like the even-more-rubbish version of social science research.

Introducing this kind of rigour, to assess the effect of things that teachers do normally, is a good thing (unless you happen to be the teacher trying to write up research on top of a full timetable!).

[UnderwaterBasketWeaving]

ReallyTired "Rather than asking permission, prehaps experiments should need ethical approval."

Honestly, teachers wouldn't be trying to study poor practice, they'd only be trying things they truly believed to be positive in influence, and if it became clear that things weren't working they'd stop.

I imagine ethical approval would be required initially, but the biggest issues would be in ensuring that SEN/G&T students would be sufficiently supported (although anything good enough for study by RCT should be easily differentiated and to the benefit of all learners, otherwise what's the point?!)