Can teachers really be researchers?

As the debate over evidence-based teaching continues there appears to be two separate strands emerging:

Strand 1: Teaching should be evidence-based (or evidence-informed).

I certainly have no issue with this, although the view of ‘what works’ is perhaps a secondary debate.

Strand 2: Teachers should also be researchers.

At a superficial level this appears like a pretty good idea – imagine the amount of evidence teachers could gather if they were all carrying out their own research studies within their own schools?

To be honest, it’s more than likely that strand 2 would lead to complete chaos. Let’s face it; there is enough bad educational research out there already, without research naïve teachers adding to it.

I view educational research through the lens of a psychologist and hold a very similar view to other psychologist-teachers (e.g. @turnfordblog) that (in terms of science) if psychology is in the Dark Ages then Education is in the Stone Age when it comes to research. Thus, I tend to make references to psychology when discussing education and, as a result, I take a positivist view of the research process.

So what’s the problem?

This list is by no means exhaustive, but it does represent at least some of the issues that need to be discussed before strand 2 can be fully recognised.

1. Sample Size:

Small sample sizes are common is school-based educational research. The majority of schools in their entirety don’t have enough participants to ensure an acceptable sample size, so carrying out a study using a few classes can only give us some small indication of the effect of any independent variable. The other issue is that we can’t force our pupils to take part in a study and even when they do volunteer, they must still be given the right to withdraw themselves (or the data) from the study.

2. Replication:

The replication debate is huge in psychological research at the moment. It has been found that many of our long-standing assumptions about human behaviour are based on studies that simply cannot be replicated. Replication studies are rarely published, null hypothesis studies end up in a dusty cupboard somewhere or in a folder marked ‘failed studies’. One study doesn’t make a theory so implementing interventions based on the results of single teacher conducting a study in a single school with a small sample tells us very little about anything.

3. Generalizabilty:

Does a study conducted in a middle-class school with a low number of students receiving free school meals, below average number of ethnic minority pupils and low number of special need pupils tell us anything about those pupils in deprived, inner-city or ethically-mixed schools? If research is to be useful then it needs to inform us about learning, not just about learning is a particular school (although this data can be useful at a more local level).

4. Bias:

Accept it or not, we all want to be ‘right’. Bias is a major problem in psychology and there is no reason to believe it won’t also be an issue with the teacher who is trying to support a hypothesis (or perhaps prove a point). Bias is usually unconscious but can often also be deliberate.

5. Lack of research training:

I’m a teacher and a Chartered Psychologist. I’m also conducting research as part of a part-time PhD at the University of York. Even though I have a psychology degree, a Masters in Education, and have attended more research methods workshops and seminars than I care to recall, the process of research still often baffles me. Those teachers whose degrees have not included a substantial amount of science or social science research training are far from equipped to carry out serious research.

6. Analysing results – the use and abuse of statistics:

How would a teacher know if the results they have obtained are significant? Most undergraduates are baffled by statistics, as are many post-graduates and post-docs. A non-British academic recently told me that us Brits don’t do stats very well, mainly because the support isn’t there at undergraduate and post-graduate level. Statistical analysis is confusing and often very time consuming and even the best statistical software packages can seem like a long-forgotten language. What about p values? Should we trust effect sizes? Is it alright for us to shave off some of our outliers to get an acceptable level of significance (it’s not by the way – it’s called p-hacking and it very much a no no)? What about Type 1 and Type 2 errors?

What can be done?

I can only offer a few suggestions:

1. Don’t go it alone.

Partner up with other schools to get bigger sample sizes. Partner up with universities or research centres that can give advise on how to carry out research or involve you in some of their own research.

2. Replicate replicate, replicate.

Get other teachers in other schools to carry out the same study.

3. Publish/Blog results (even null hypotheses) – and accept advice/criticism.

Make others aware of what you are doing and take on board the advice offered. Let’s face it; sometime it’s hard to get past the egos that dominate the Internet. We all have something to say and we don’t like being criticised for it but if we’re serious about using research to inform our teaching we really need to get over it.

4. Get some training.

CPD is a major issue and most isn’t worth the time and effort (or money) involved. Introductory workshops in research methods are often cheap (and sometimes free) and there are plenty of resources available online (try OpenLearn from the Open University). Linking such CPD to a recognised research qualification would be a great incentive.

It’s perhaps time to move away from the debate about the acceptability of teacher-researchers and try to work out how it can be practically done. Even though there is a great opportunity here for educational research, there is an equally realistic possibility that it could end in disaster and confusion.

Advertisements

4 thoughts on “Can teachers really be researchers?

  1. Andrew Sabisky

    Some points to the contrary:

    1: Small studies can be aggregated with other small, similar studies in a meta-analysis to yield clearer results. So long as results from one study are not over-interpreted, this problem need not be fatal. Teachers are certainly in no worse position here than educational researchers – in fact, arguably they are in a better one.

    2: True: I agree, but again, teachers are not in a worse position than the researchers of today.

    3: Arguably teachers – rightly – may care less about generalizability than researchers. The rest of us may want widely applicable solutions, but the teacher’s most pressing problem is “will this work in my school?”. Local research – even small-scale – is well placed to answer this question

    4: Again, teachers are no worse off than academic psychologists.

    5: Fair point, but a lot of the complication in psychological research comes from an inability to randomize coupled with samples that are unrepresentative of the population to begin with. Over time various complex statistical methods have been used to correct for these issues, with varying – often highly overstated – degrees of success. Teacher-researchers are well placed to avoid both these issues that crippled much current research from the start.

    6: I could not agree more, but again, teachers are no worse off here than present-day researchers, especially psychologists. Again, teachers (especially in secondary schools) may be in a better position because they have Real Maths People on board who have a maths degree and teach statistics. Psychological research nearly always lacks a Real Maths Person.

    All your suggested solutions are excellent. I liked the blog very much but though a few counterpoints may be worthwhile.

    Reply
    1. Marc Smith Post author

      Thanks Andrew, I’ll try and respond point by point:

      1. Absolutely, however, all these studies would have to be available in sufficient detail in order to carry out the meta-analysis. Academic psychologists have access to research papers published in peer-reviewed journals, requiring them to reach a certain standard and (more importantly) have the ability and/or reputation to have their papers published in such journals. The problem then becomes one of access and quality.

      2. Yes, I would agree – education would face the same ‘replication crisis’ as psychology. Teachers carrying out research would need to know what research has already been undertaken and have that information in enough detail in order to attempt a replication.

      3. Again, yes. However, are we looking for specific interventions or something more general? Will it work in my school? Will it work for this specific class? If it works now, will it work with the new cohort of year 7’s in September?

      4. Academic psychology has been dealing with bias for decades and researchers should be aware of the issues. The naive teacher-researcher would be less aware of the problems encountered over bias – which is why high quality research training is vital.

      5. Agreed.

      6. I think you are perhaps overestimating the ability of many school maths departments to deal with the complexities of advanced statistics. Not all maths teachers have maths degrees and those who do don’t necessarily have the knowledge and skills to deal with research statistics.

      These are not barriers to school-based research by teacher-researchers, simply a highlighting of the problems we face – all of which can be overcome. My argument is not ‘Should we?’ but ‘How should we?’

      Reply
  2. Dick van der Wateren

    Reblogged this on Blogcollectief Onderzoek Onderwijs and commented:
    My answer to the question in the title is a reluctant yes. Some teachers certainly can be researchers, but most, as the author of this post, Marc Smith, points out, lack the necessary training. I am less reluctant to answer positively the question whether teachers should be researchers. Teachers should definitely do research in their classroom, although not necessarily the kind of randomised double blind field trials that academic researchers might prefer. In my opinion, teachers should try to develop a scientific approach to their own teaching. This means that we view teaching as a continuous experiment of trial and error of our teaching practice. John Hattie’s 8 Mindframes may offer a framework for this process (http://youtu.be/6xpcXobZF1k). What works and does not work is essentially judged on the basis of our observations, requiring some sort of standardisation. It would even be better if we could collaborate with academic researchers and compare data from different classes and schools, as suggested in this post, but I see no reason for us to wait for that to happen.

    Pedro De Bruyckere alerted me to this interesting post at ‘Psychology in Education’, suggesting teachers to “… welcome researchers who are looking for cooperation into your class and schools.”

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s