Interviewed by the employee magazine educ.alla Utbildningsförvaltningen – Göteborg
Visible Learning has had a major impact in Sweden and among your results the study shows a very small effect of smaller groups or classes and of homework assignments in certain age groups. Can you see a change in attitude around these issues among school professionals and connecting groups, or are there still many who maintain that these factors have a large significance for learning? Is there a difference between groups (parents, students, teachers, politicians)?
The key issue with influences that have very small effects (and many that have effects much smaller than many would expect or hope for) is to ask “Why are they so small”. The reason, I would argue, that smaller class sizes make little difference is because teachers have not changed their teaching methods as they move from larger to smaller. The reason why homework in elementary schools is low is that the homework is often an “extra” (or a project) and not an opportunity to reinforce the learning that has begun in the classroom.
It is certainly the case that many do not want to believe evidence as their own “experiences” tell them different. Research starts from the premise of attempting to falsify your pet theories and many parents, teachers and politicians work from the premise of attempting to find beliefs and evidence to support their prior beliefs. This confirmation bias means they spend millions of dollars on the wrong issues, and thus do major damage to the learning of our children.
Your study of meta-analyses is very comprehensive, but I understand you keep adding to it. How does the database look today? Have you discovered new factors that have significant impact on learning and by that concern the teaching profession?
Yes I am now close to 1200 meta-analyses (up from the 800 in VL). What is remarkable since I published the first study (in 1989 based on 134) is that the “story” underlying the data has hardly changed. Some of the more interesting (new) effects include Bullying (-.24), parental employment (.03), sleep (.07), single sex compared to coed schools (.08). Philosophy in schools (.43), Service learning (.58), conceptual change programs (1.15), and collective self-efficacy (1.57).
As always, when it comes to research, there are some objections to your results. What would you yourself say is the most controversial area of your research?
Meta-analysis is a well-rehearsed and well known method. While some bloggers have troubles with effect-sizes, they are an established part of the statistical scene. Other bloggers claim that even I say that “half the data in VL” is wrong – how absurd and I have never made such a claim! Yes, I made a mistake calculating on statistic (the CLE) in early editions of VL but it is hardly important given the attention to the effect-sizes (although I regret that error) and the very minor part played by the CLE (which almost every reader and critic has ignored anyway). I would have wanted more moderators in the literature other than the more obvious and often less interesting (like achievement levels compared to gender, age) – but this is a reflection on the research not methods. Also, too many studies are based on surface rather than deep learning, but again this is reflection of what so often occurs in our classrooms. The data from the meta-analyses is hardly controversial but the more important part is the “explanation” – I have yet to find any alternative story than the one I proposed in the various VL books.
I read that you don’t see your research results as a manual or answers to questions but that it is about that teachers should equip themselves with a different approach. They should apprehend that the role as a teacher includes evaluation of the effect they have on their students. What are your best tips on how teachers can get such feedback from their students?
Yes, there are multiple ways for a teacher to be excellent and we should stop prescribing a narrow set of ways to teach. The fundamental claim in VL is “Know thy impact”: teachers and school leaders who continually seek to find evidence on their impact are the most successful. This begs the question as to “what is impact,” “what is the magnitude of this impact,” and “how many students are gaining at least a year’s growth for a year’s input.” These questions are best debated across teachers and schools and are at the heart of maximizing impact. Note, the power of teachers’ collectively working together (see Q1 above).
Can you comment on that people see your results as a list of things possible to generalize in the classroom?
I do, however, argue that there is a “practice of teaching” – there are ways we can think about teaching that work more successfully than others – and many which we should abandon (e.g., learning styles, beliefs about tracking or not, fiddling with curricula and not expectations).
Literacy training is in focus in Sweden, especially after the last PISA-evaluation. What do you think of the correlation between time reading and reading comprehension? Are there any studies of the effects of reading on screen for reading comprehension?
Our recent meta-synthesis shows that learning to read by about age 8 is critical in our Western society, and thus there needs to be even more emphasis on language (oral and written) especially in the 0-3 age group – as after 8 it is very hard for students to catch up. It is not merely “time on task” but exposure to language, the deliberate teaching of reading, successful diagnosis of the needs of students to read and comprehend, and the continuous monitoring of the success of teachers to provide at least a year’s reading progress for a year’s input – Pfost et al. 2014 (link opens in a new window).
Another question, linked with literacy and poor reading results, is when you should start giving grades. A recent government commission of inquiry, carried out by the Swedish neuroscientist professor Martin Ingvar, proposes giving grades to pupils in year 4. Today we start doing that in year 6. Do you believe that earlier grades will help Swedish children to gain better results in for example reading?
What an unusual debate. Grades can be beneficial to students WHEN they are accompanied with information that the student understands about “where to next”. Without this kind of feedback, grades are of little use at any age. I would be asking about the nature and quality of feedback at every age, asking about how and what students understand when they are given feedback (such as grades), and teach students and teachers to optimize the “where to next” feedback. Debates about grades alone are not helpful.
Another public inquiry report proposes reintroducing ratings of orderliness. Do you have any comments on that?
Sorry, what is orderliness?
In Sweden there is a discussion claiming that there is a negative attitude to learning in certain groups of boys, this is seen as an explanation for the fact that boys generally get lower grades. How do you think it is possible to change this? Is it important to focus on just boys in this respect?
The effect-size of the differences between boys and girls is very small – so why would I worry. Yes we socialize boys and girls differently, and yes boys are less likely to pre-plan (e.g., in writing). When we teach them to pre-plan the differences reduce a lot. I would ask about pre-planning for ALL students not segregating boys from girls as this implies that boys are more alike to other boys, girls are more alike to other girls – whereas this is just not the case.
Concerning principals and other organizers of work in upper secondary school: What do you think is the most important prerequisite for a teacher to be able to do a good job?
By continuing asking and seeking evidence of their impact, and school leaders should be having these discussion about impact across teachers in the school – and asking the questions in Q4.
How should an upper secondary school teacher think to organize teaching in the best way possible?
No differently – they should also be concerned about the nature of impact, ensuring that students are aware of what success in a series of lessons look like near the start of the lessons, and having transparent (to the students) and defensible decisions about the best proportion of surface and deep learning (in their teaching and in their assessments).
Finally, the relationship between teacher and pupil has great importance, but how about the impact of all the other relations in school? Do you have any comments on for example Michael Fullan’s studies on school culture?
School culture is as critical as teacher-student and student-student relationships. You need a culture of high trust for teachers and students to discuss their impact, what is working, and as critically what is not working. All learning feeds on error, mistakes and misunderstanding. High trust is necessary for this learning to progress. Also most of us are wary of progressing if there is unfairness, fear of failure, and criticism for “not knowing”. Absolutely we need high trust to explore, to make mistakes, to learn, and to have fun in learning.
- Pfost, M., Hattie, J., Dörfler, T. & Artelt, C. (2014). Individual differences in reading development: A review of 25 years of empirical research on Matthew effects in reading. Review of Educational Research, 84, 203-244.
The interview originally appeared in the employee magazine educ.alla Utbildningsforvaltningen – Goteborg