Expert Q&A: Exploring Australia’s PIRLS results

Australia’s results in the 2021 Progress in International Reading Literacy Study (PIRLS) were released last week by the Australian Council for Educational Research (ACER). PIRLS has measured trends in year 4 students’ reading literacy achievement every 5 years since 2001. In our latest expert Q&A we speak with PIRLS National Project Manager for Australia and ACER Senior Research Fellow Kylie Hillman about some of the interesting findings to come from this cycle.

How did Australian year 4 students perform on PIRLS 2021 compared to their international counterparts? Also, what is the relationship between achievement and student behaviours and attitudes?

Compared to other year 4 students who participated in PIRLS, Australian students have performed quite well. Australia’s average score was higher than that of 28 other countries, similar to the average score of students in 8 countries, and lower than that of 6 countries – Singapore, Hong Kong SAR, the Russian Federation, England, Poland and Finland.

The average reading score for Australian students has also remained stable between this cycle of PIRLS and the previous one in 2016, which was not the case for most countries that assessed year 4 students in both cycles of PIRLS – 21 of those countries recorded a significant decline in average reading scores. I think we can agree that this is a positive finding for our students, given the context of education and learning during 2021.

Over the past cycles of PIRLS, we have found a pattern in which students who report that they enjoy reading, and students who are confident in reading, tend to perform at a higher level in the reading assessment than students who say that they do not like reading and students who are not confident in their reading. I don’t think that many educators would be surprised by that, and it is certainly a trend that is present in many countries and many cycles of PIRLS.

But those 2 attitudes – enjoyment of reading and self-confidence in reading – don’t always overlap as much as you might expect. We found that some of our more confident readers tell us that they don’t like reading – about 15% of very confident students said that they did not like reading – and some students who are not confident in their reading skills still say that they enjoy reading very much.

In PIRLS 2021, principals were asked to report the number of weeks in the current academic year during which normal operations had been affected by COVID-19. Was there a correlation between disruptions and student performance across the jurisdictions? How did this compare internationally?

This is the first time a disruption of this kind has occurred on a global scale during a cycle of PIRLS, and it had an impact on the study in a number of ways. Many countries had to delay when they assessed students, some countries had to change their plans for how they would conduct the assessment, and some countries ended up testing their students when they had moved into year 5, instead of towards the end of year 4. We are also releasing the results later than originally scheduled because of COVID delays.

Ninety-two per cent of participating Australian students were in schools in which principals reported that normal instruction was affected by COVID-19-related restrictions, while the lowest proportions of disruption were recorded in Russia (39%) and Chinese Taipei (23%). Of course, these are national-level results and, certainly within Australia, we know that school closures and periods of remote learning differed depending on where students were.

According to the principals’ responses, 100% of Victorian students were in schools that had 8 or more weeks of interruptions to normal school operations in 2021, while in the ACT and New South Wales the figures are 83% of students and 78%, respectively. Interruptions in the other jurisdictions are somewhat shorter, with most students being in schools in which normal operations were interrupted by 2 weeks or less. In Tasmania, close to 60% of students were in schools with no interruptions at all during 2021.

We can’t draw a direct line between remote learning or lockdowns and student performance, though, because the study isn’t designed to test for causal relationships between something like a pandemic and students’ reading levels. What we can say is that, for perhaps the first time, students in a wide variety of countries shared a common experience – having to undertake at least some of their learning outside their normal classroom environments. So, the pandemic certainly forms part of the background or context of the PIRLS results for many countries.

I understand teachers were asked to indicate how they felt about their profession – how regularly they felt enthusiastic about their job or felt pride in the work they did. How did our teachers’ responses compare to previous cycles of PIRLS?

We have noted a statistically significant decrease in the proportion of students being taught by a teacher who is very satisfied with their profession (according to their responses to a number of questions) – from 58% in 2016 to 49% in 2021. There has also been an increase in the proportion of students with a teacher who is less than satisfied with their profession – in 2016, we had 2% of students being taught by a teacher who was less than satisfied with their job, now we have 10% of students with a teacher with low job satisfaction.

As the questions and scale changed between PIRLS 2011 and PIRLS 2016, we can’t really tell if this is something relatively new or something that has been happening over the past 10 years. Either way, it is a concern, and I will be very interested to see what the results of the next Teaching and Learning International Study (TALIS) will tell us about how our primary and lower secondary teachers are faring (the main data collection period is July and August 2024).

It has been a challenging few years for many educators, not just because of COVID. There are many factors at play in the satisfaction of teachers. For example, we know from the PIRLS 2021 results that close to two-thirds of year 4 students were in classrooms that faced some limitations to teaching due to students not being ready to learn, including students who might be uninterested, disruptive or needing extra support in reading. Six per cent of students were in classroom that, according to their teachers, were limited a lot by those factors.

In PIRLS 2021, students were asked to indicate how much time they spent using a computer or tablet to find and read information for schoolwork on a normal school day. What have we learnt about the use by Australian students of digital devices in their education?

This question is a new one for PIRLS – we have to add new questions sometimes as the educational environment changes, and the prominence of digital devices in the lives of young people has most certainly changed since I was at school! We had a single desktop computer for the entire primary school. According to teachers surveyed in PIRLS 2021, most Australian year 4 students (88%) had access to digital devices (desktop computers, laptops or tablets) during their reading instruction.

The majority of Australian year 4 students reported that they spent 30 minutes or less per school day using digital devices to find and read information. In terms of reading performance, students who were using digital devices – either for more than 30 minutes or 30 minutes or less – tended to score higher on the reading assessment than students who reported spending no time using these devices to read and find information for schoolwork.

We also compared responses to this question for different groups of students and found some interesting differences. Male students, for example, seem to report higher usage of digital devices for schoolwork than female students – 30% of male students reported spending more than 30 minutes a day on devices for their schoolwork compared to 24% of female students.

We also looked at this by the number of books students said they had at home, which can be an indicator of socioeconomic differences. The proportion of students with few books at home who said that they spent no time on digital devices for schoolwork was significantly higher than the proportion of students from homes with many books who weren’t using digital devices for schoolwork – 20% compared to 14%. This raises the question of access and resourcing, and we will need to interrogate the data in greater depth to see if we can identify what lies behind these differences.

Results from PIRLS 2021 show Australian students who are regularly assigned (once a week or more) longer fiction books with chapters scored a statistically significant 14 points higher, on average, than students whose teachers assigned longer books less than once a week. Do you think this is an important finding?

I certainly think it is an interesting finding, particularly when you consider that we don’t find significant differences in reading scores when comparing how often students are assigned other texts, like short stories or poems or any types of informational text.

When students are reading and interacting with longer narratives texts – chapter books, as they often call them – it’s not just the number of pages or words: there is often more you can discuss, more opportunities for students to make connections to characters, to develop their opinions. You are often asking more of students in terms of character development over the course of the story, there might be one or more subplots for them to keep track of.

Eighty-six per cent of year 4 students are assigned longer fiction books with chapters at least weekly, according to their teachers. That means 14% of students aren’t being assigned those longer, more involved texts regularly. Most teachers would have some say in how often they assign these types of books, so this might be one of those findings uncovered by PIRLS that can make a difference to classroom practice.

But assigning longer books relies on students having solid foundational skills in reading. PIRLS focuses on students in Year 4 because they are usually at a key transition point in their schooling, during which they move from learning how to read, to reading in order to learn. Individualised support for the 1 in 5 Australian students whose reading achievement is below the proficient standard – particularly the 6% who did not reach the Low benchmark – is essential to prevent them falling behind in other learning areas, as more of their schooling draws on their reading skills.

References

Hillman, K., O’Grady, E., Rodrigues, S., Schmid, M., & Thomson, S. (2023). Australia’s results from PIRLS 2021. Australian Council for Educational Research.

Kylie Hillman will be presenting on PIRLS and the relationships between home activities, attitudes and reading performance at ACER’s Research Conference 2023. It will take place on Sunday 3rd and Monday 4th September at the International Convention Centre Sydney (ICC Sydney). To find out more about all the conference speakers and Masterclass events, and to take advantage of early bird registration discounts visit www.researchconference.com.au.