//
you're reading...
Leadership Issues, Teaching and Learning

Homework: What does the Hattie research actually say?

The Highly Influential Work by John Hattie

This is an excellent book.  It is an attempt to distil the key messages from the vast array of studies that have been undertaken across the world into all the different factors that lead to educational achievement.  As you would hope and expect, the book contains details of the statistical methodology underpinning a meta-analysis and the whole notion of ‘effect size’ that drives the thinking in the book.  There is a discussion about what is measurable and how effect size can be interpreted in different ways. The key outcomes are interesting, suggesting a number of key factors that are likely to make the greatest impact in classrooms and more widely in the lives of learners.

My main interest here is to explore what Hattie says about homework.  This stems from a difficulty I have when I hear or read, fairly often, that ‘research shows that homework makes no difference’. It is cited as a hard fact in articles such as this one by Tim Lott in the Guardian: Why do we torment kids with homework?   Even though Tim is talking about his 6 year old, and cites research that refers to ‘younger kids’, too often the sweeping generalisation is applied to all homework for all students.  It bugs me and I think it is wrong.

Research cited for younger children

I have written about my views on homework under the heading ‘Homework Matters: Great Teachers set Great Homework’ . I’ve said that all my instincts as a teacher (and a parent) tell me that homework is a vital element in the learning process; reinforcing the interaction between teacher and student; between home and school and paving the way to students being independent autonomous learners.  Am I biased? Yes.  Is this based on hunches and personal experience? Of course.  Is it backed up by research……?  Well that is the question.

So, what does Hattie say about homework?

Helpfully he uses Homework studies as an example of the overall process of meta-analyses, so there is plenty of material. In a key example, he describes a study of five meta-analyses that capture 161 separate studies involving over 100,000 students as having an effect size d= 0.29.  What does this mean?  This is the best typical effect size across all the studies, suggesting:

  • improving the rate of learning by 15% – or advancing children’s learning by about a year
  • 65% of effects were positive
  • 35% of effects were negative
  • average achievement exceeded 62% of the levels of students not given homework.

However, there are other approaches such as  the ‘common language effect’ (CLE) that compares effects from different  distributions. For homework a d= 0.29 effect translates into a 21% chance that homework will make a positive difference.  Or, from two classes, 21 times out of a 100, using homework will be more effective.   Hattie then says that terms such as ‘small, medium and large’ need to be used with caution in respect of effect size.  He is ambitious and won’t accept comparison with 0.0 as a sign of a good strategy.   He cites Cohen as suggesting with reason that 0.2 is small, 0.4 is medium and 0.6 is large and later argues himself that  we need a hinge-point where d > 0.4 is needed for an effect to be above average and d > 0.6 to be considered excellent.

OK.  So what is this all saying. Homework, taken as an aggregated whole, shows an effect size of d= 0.29 that is between small and medium?  Oh.. but wait… here comes an important detail.  Turn the page:  The studies show that the effect size at Primary Age is d = 0.15 and for Secondary students it is d = 0.64!  Well, now we are starting to make some sense.  On this basis, homework for secondary students has an ‘excellent’ effect.  I am left thinking that, with a difference so marked, surely it is pure nonsense to aggregate these measures in the first place?

Hattie goes on to report that other factors make a difference to the results:  eg when what is measured is very precise (eg improving addition or phonics), a bigger effect is seen compared to when the outcome is more ephemeral. So, we need to be clear:  what is measured has an impact on the scale of the effect.  This means that we have to throw in all kinds of caveats about the validity of the process.  There will be some forms of homework more likely to show an effect than others;  it is not really sensible to lump all work that might be done in between lessons into the catch-all ‘homework’ and then to talk about an absolute measure of impact.  Hattie is at pains to point out that there will be great variations across the different studies that simply average out to the effect size on his barometers.  Again, in truth, each study really needs to be looked at in detail.  What kind of homework? What measure of attainment?  What type of students?  And so on…. so many variables that aggregating them together is more or less made meaningless?  Well, I’d say so.

Nevertheless, d= 0.64!  That matches my predisposed bias so I should be happy.  q.e.d.  Case closed.  I’m right and all the nay-sayers are wrong. Maybe, but the detail, as always, is worth looking at.  Hattie suggests that the reason for the difference between the  d=0.15 at primary level at d=0.64 at secondary is that younger students can’t under take unsupported study as well, they can’t filter out irrelevant information or avoid environmental distractions – and if they struggle, the overall effect can be negative.

At secondary level he suggests there is no evidence that prescribing homework develops time management skills and that the highest effects in secondary are associated with rote learning, practice or rehearsal of subject matter; more task-orientated homework has higher effects that deep learning and problem solving.  Overall, the more complex, open-ended and unstructured tasks are, the lower the effect sizes.  Short, frequent homework closely monitored by teachers has more impact that their converse forms and effects are higher for higher ability students than lower ability students, higher for older rather than younger students.  Finally, the evidence is that teacher involvement in homework is key to its success.

So, what Hattie actually says about homework is complex.  There is no meaningful sense in which it could be stated that “the research says X about homework” in a simple soundbite.  There are some lessons to learn:

The more specific and precise the task is, the more likely it is to make an impact for all learners.  Homework that is more open, more complex is more appropriate for able and older students.
Teacher monitoring and involvement is key – so putting students in a position where their learning is too complex, extended or unstructured to be done unsupervised is not healthy.  This is more likely for young children, hence the very low effect size for primary age students.

All of this makes sense to me and none of it challenges my predisposition to be a massive advocate for homework.  The key is to think about the micro- level issues, not to lose all of that in a ridiculous averaging process.  Even at primary level, students are not all the same.  Older, more able students in Year 5/6 may well benefit from homework where kids in Year 2 may not.  Let’s not lose the trees for the wood!  Also, what Hattie shows is that educational inputs, processes and outcomes are all highly subjective human interactions.  Expecting these things to be reduced sensibly into scientifically absolute measured truths is absurd.  Ultimately, education is about values and attitudes and we need to see all research in that context.

 

PS. If you are reading this from Sweden, Tack för läsning. Låt mig veta era tankar om denna fråga.

Discussion

40 thoughts on “Homework: What does the Hattie research actually say?

  1. Hattie et al. have undoubtedly produced work of outstanding value. Too easily; however, his evidence is simplified & distorted. The use of technology in his research is an apt example. The effect size would intimate that technology has no positive impact upon learning. But the research doesn’t include iPads for example, which I believe are wholly different to more traditional technology. Indeed, how we use iPads is predominantly for feedback & collaborative learning – two positive effect sizes – feedback being the daddy! People have dismissed using new technology with Hattie’s ‘evidence’ – but that is an incomplete story. ‘Some’ technology I believe has a negligible impact on learning – often due to insufficient pedagogy, whereas some work brilliantly well, motivating students, enriching feedback etc.

    Hattie’s evidence is a great starting point for all educators when evaluating impact or devising improvements, but people need to know the data and recognise how their different contextual factors impact the data. Otherwise we have the ‘Tim Lott effect’ i.e. people grandstanding their opinions as incontrovertible facts!

    Posted by Alex Quigley | October 21, 2012, 5:07 pm
  2. Thank you for a this good posting! I used to teach in Finland where homework was never graded. During my teacher education the reason for homework was made very clear: it is an opportunity for the student to revisit what s/he learned at school, and thus make an additional memory of it (and in the best scenario also move it to the long term memory, i.e. provide “automatic” transfer). Secret of successful homework: keep it short & simple.

    Posted by Nina | October 21, 2012, 5:21 pm
  3. Tom – Hope you won’t be offended, but I can’t really agree with this one. An effect size of 0.29 is well below the hinge point of 0.4 – Hattie explains that we have a responsibility to focus on effect sizes above 0.4 and there are SO MANY things which will make more of a different. Good feedback has an effect of over 1.0 – THIS is what we (and students) should be spending their time doing if we really want to make a difference. All an effect size of 0.29 shows is that it’s not actively doing any harm. However, when we take this (http://learningspy.co.uk/2011/09/12/should-we-stop-doing-good-things/) spending undue time on homework really could be preventing students making better progress. Maybe the solution is set action of feedback AS homework?

    Does this make sense?

    Posted by David Didau (@LearningSpy) | October 21, 2012, 6:22 pm
    • Hi David. I think my point is that Hattie’s work should be divided up – Secondary homework as an effect size of 0.64. The 0.29 is a very crude average that loses meaning. Even within Hattie’s framework 0.64 is a strong effect worth looking at. In practice, I’m thinking of all the stuff my students do for homework… I don’t know how we’d run lessons without it. Tom

      Posted by headguruteacher | October 21, 2012, 6:56 pm
  4. Hi Tom, The article is thorough and well presented. Homework has a bad name because there is so much “busy work” being given that does not impact on learning. Homework that is integral to ongoing learning, with clear feedback, either direct and individualised, or group based within discussion, might give the higher figures from Hattie. Working within a commonly articulated purpose focuses learners on the task, they can see the point.
    Chris Chivers

    Posted by Chris Chivers | October 22, 2012, 7:55 am
  5. Really enjoyed this post, partly because it supports what I think! In our department we’re puttting more of our effort into giving feedback in lesson time and making sure students having time to improve, rather than setting big homework tasks for teachers to write long comments on (that are rarely read!). So far this year we’ve focussed on learning key words, single exam questions, simple ‘find out’ tasks about science in the news. These short tasks can be marked in class and feedback given. Students missing HW can be dealt with simply and those who complete their HW each week are clearly benefitting from the reinforcement. I no longer risk setting a piece of HW that is essential for understanding the following lesson. More trouble than it’s worth!

    Posted by Ellie Russell | October 23, 2012, 12:52 pm
  6. Reblogged this on languagesupportuk and commented:
    Just interesting reading

    Posted by Liz | October 29, 2012, 9:43 am
  7. You also have to consider what is defined as homework. In some studies it is taken to be work done at home. This is not the same as homework as we understand it.
    If a student does their own revision, it probably has a positive effect. We need to separate this from homework set by a teacher and completed or not by students. I don’t believe that Hattie makes this distinction.
    I spent a summer reading lots of research about it and came to the conclusion that it is unhelpful at primary but that the jury is still out for secondary.
    I also suspect that it is the feedback effect that creates positive effects for homework and not the setting of homework itself.

    Posted by Rob Anthony | November 3, 2012, 1:11 pm
  8. Head Guru Teacher
    Thanks for the courtesy of reading my work, and not tripping over headlines of those who not bother. I knew when writing the section on homework that it could be misinterpreted (as has class size, streaming, finances, etc.) and tried my best to make the picture clear. Yes, homework is one influence where there is a critical moderator – there is a major difference between its effects in primary and secondary.

    I like your summary but can I add one more critical comment …

    Visible Learning is a literature review, therefore it says what HAS happened not what COULD happen. The message about the low effect in primary school means that there is a high probability that many homework practices may not be working. The key is that this highlights the importance for schools to now evaluate the effectiveness of its primary homework practice. And if it turns out to be like most other practices (low effect), there there is an invitation (indeed an imperative) to try an alternative set of practices re. homework – and evaluate their impact on learning, involvement in learning, and increasing the students (and parents) understanding about the language of learning.

    I do NOT recommend abandoning homework, and I do provide some direction for effective homework policies (as you note), but most of all the invitation is to “Know thy impact”.

    Posted by John Hattie | November 6, 2012, 10:16 am
    • John
      Thanks so much for your comment. I must say I am thrilled that you have added to the discussion in person. I agree with what you have said entirely. Your work is making a serious impact because people are hungry for research evidence – however, the detail in your book is rarely reported; too easy to go for the average simplification. I am very happy to hear that you do not recommend abandoning homework! It is all about effectiveness – and, yes, impact. Great!

      Anyway, thanks very much indeed. Tom Sherrington

      Posted by headguruteacher | November 6, 2012, 10:36 am
      • Since John has engaged directly with this, as usual, excellent blog post by Tom, perhaps he could answer the broader question of how his data is being used within education policy and reform? I am having a mini-debate about it with former education colleagues especially on how to measure the actual benefits of informal/non-formal learning – this is because the Wellcome Trust recently produced a series of extensively-researched reports on informal learning within science and having read them I am still not clear what the proof of benefit actually is! I am inclined towards the Sutton Trust/EEF toolkit being developed at Durham which gives a ‘precise’ table of effect size and relative cost of the intervention – out of school activities don’t do that well on this basis. Of course some might say (if you are a parent who is asked to help your child with it, even more so) that homework is a type of out of school activity/informal learning …

        Posted by behrfacts | February 2, 2013, 3:35 pm
      • You wanted a comment from Sweden… I wholly support the approach you have in this article and am glad for Johns comment. In Sweden, simple bylines from Visible learning has all to many times been presented as truths in education (not at least from the employers) such as “class size do not matter” etc. I have advocated Johns research quite heavily in Sweden since I think we in this country to long have not been interested in evidence when it comes to teaching.

        But in doing so I always stressed that it should be used as a base for discussions, and that behind every sampled effect is a plethora of information that can enrich a pedagogical discussion among teachers. I have done this very much in the same way as you have done above. It is also important and crucial for real school development that every teacher evaluates the methods he or she uses and if the desired learning is achieved as John points out.

        Used in the way you describe above, Visible learning is a tremendously valuable collection of information on education. It is at this moment translated into Swedish, which will make it more accessible to the common teacher, and I hope it will be used in schools and in teacher education to broaden and inspire pedagogical discussions and not as a book of finite answers.

        Posted by Per Kornhall | January 9, 2014, 4:05 pm
      • Dear Per, Thanks a lot for the response. It is interesting that we are all dealing with same questions. I love John Hattie’s book but it is frustrating to see it reduced to a list of good and bad strategies as if the effect sizes are absolute measures. Used intelligently, the ideas are very useful – even just the fact that teachers are talking more about evaluating impact and effect as a consequence is a step forward.

        Posted by headguruteacher | January 9, 2014, 4:27 pm
  9. I think in another post, you have made the point that the quality of homework is likely to be an arbiter in progress. It would be worth a survey of activities given for homework to ascertain their worth in promoting learning, or whether they are given just because there’s a need to set homework.

    Activity at home, linked to the learning in class, cf flipped classrooms, would appear to have a stronger base than a photocopied worksheet.

    It’s another example of #marginalgains. Look at the dynamics.

    Posted by Chris Chivers | March 4, 2013, 7:57 am
  10. Thanks for the analysis, it really helped when trying to understand what the research actually says.

    Show My Homework / @ShowMyHomework

    Posted by Show My Homework | July 16, 2014, 3:13 pm

Trackbacks/Pingbacks

  1. Pingback: Homework: What does the Hattie research actually say? | … Not the Principal's Office!!! - October 21, 2012

  2. Pingback: Homework « @impactEduk - November 10, 2012

  3. Pingback: On homework « Verbal Tea - December 4, 2012

  4. Pingback: Leading 21st C learning : getting my bearings for the journey ahead… « headguruteacher - December 9, 2012

  5. Pingback: Leading 21st C learning: getting my bearings for the journey ahead… « Wilmslow High School's 'Lookout for Learning' - December 11, 2012

  6. Pingback: Great Lessons 5: Journeys « headguruteacher - February 8, 2013

  7. Pingback: What’s the Difference? Putting on my Thinking Hattie | i-Biology | Reflections - March 8, 2013

  8. Pingback: The Data Delusion: On average, it’s a bit more complicated. | headguruteacher - March 17, 2013

  9. Pingback: Primary school homework – who is it meant to benefit? | Educating Brentwood - March 18, 2013

  10. Pingback: Homework Matters: Great teachers set great homework | headguruteacher - April 28, 2013

  11. Pingback: A Year in the Life of Headguruteacher | headguruteacher - May 10, 2013

  12. Pingback: Four Successful Homework Strategies - EDUTRONIC | Christopher Waugh - June 1, 2013

  13. Pingback: How much should homework count? | i-Biology | Reflections - June 10, 2013

  14. Pingback: The Great Homework Debate Part II – What is the effect size of homework on student learning? | paddington teaching and learning - June 12, 2013

  15. Pingback: Winning Ugly: The Secrets of ‘Gritty’ Teaching and Learning | huntingenglish - July 14, 2013

  16. Pingback: Winning Ugly: The Secrets of ‘Gritty’ Teaching and Learning — HuntingEnglish - July 30, 2013

  17. Pingback: Ofsted 2013: Appropriate homework — From Good to Outstanding - September 3, 2013

  18. Pingback: The Goldfish Bowl » Getting on top of homework - September 22, 2013

  19. Pingback: All the small things | mrbunkeredu - January 21, 2014

  20. Pingback: A Blog Blog: The things that seem to matter | headguruteacher - January 27, 2014

  21. Pingback: Engagement, Teaching and Bias | Moments, Snippets, Spirals - February 1, 2014

  22. Pingback: Always Evidence-Based Practice. Or Not? | Moments, Snippets, Spirals - February 20, 2014

  23. Pingback: Homework and Retention | cavmaths - July 10, 2014

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Archives

Blog Stats

  • 836,270 hits

Twitter Updates

Follow

Get every new post delivered to your Inbox.

Join 21,521 other followers

%d bloggers like this: