How much device use achieves the best outcomes?
The evidence for computer and device use in education is at best, mixed.
Moderate use of digital devices seems to have some positive impacts on student outcomes, including obtaining digital skills.
Improvements only occurred in certain areas of learning.
Students who frequently used computers and devices at school had significantly lower educational outcomes.
Careful consideration is needed to use digital technology to its best effect.
The use of digital technologies within education has much potential, and there is no question that some use can support learning. But to date, the impact of computer and device use for learning on educational outcomes has been described as at best, mixed. Some studies show improvements while some show declines in performance.
For examples of effective use of digital technologies in learning see 'Using digital technologies to enhance learning'.
A large review on the impacts of educational technology assessed the findings of 234 meta-analyses, involving 2.12 million students. Authors described the impacts of educational technology on learning outcomes pre-covid as ‘at best average, more likely well below,’ but noted that digital technologies had above average impacts for students with special learning needs.
Large-scale population studies comparing standardised assessments are more consistently finding trends that some technology use can support learning, but there is a limit to the amount of technology use that can achieve this.
The 2015 OECD report on the use of digital technologies in school, 'Students, computers and learning,' agrees that digital fluency is important for equal opportunities for employment and study on leaving school.
The report analysed data from 64 countries when reviewing this topic. They found that students who used computers moderately, defined as up to half an hour a day approximately), have some improvements in educational outcomes compared to students who rarely use computers (this included high schools). They noted improvements only occurred in certain areas of learning.
Students who frequently used computers in school (defined as more than half an hour per day) had significantly lower educational outcomes. Results were replicated in 2018 PISA data. Scores also improved when teachers used digital technologies to teach, or with students, but declined when students used devices independently.
Student use of digital devices to search the internet was linked to improved learning outcomes, but all other tasks (including learning apps, simulations, online homework, e-textbooks and others) were linked to reduced outcomes.
Testing was done both on paper, and digitally. Singapore, with only moderate use of technology, came out top for digital skills.
Students' digital skills have been found to have declined in Australian research, in a comparison between digital literacy scores in 2011 with 2014, despite increased device use. Authors noted the declines were concerning and warrant serious attention. They noted that children learn very different skills on tablets and smartphones to the basic technology skills required for the workplace, and warned against assuming that children who use mobile devices were more widely competent with technology.
Achievement in technology also declined in New Zealand Year 4 and 8 students between 2016 and 2021, despite increased use of digital devices to learn, and changes to the technology learning area of the New Zealand Curriculum.
The OECD report noted that 'ensuring every child achieves a baseline proficiency in reading and mathematics seems to do more for creating equal opportunities than can be achieved by expanding or subsidising access to high tech devices or services.'
A recent report by the Reboot Foundation (2019) has also analysed the connection between educational technology and learning, using PISA and the 2017 NAEP (National Assessment of Educational Progress) data, and their results have replicated the 2015 OECD results. They noted that "the results regarding tablet use in fourth-grade classes (equivalent of NZ Year 5) were particularly worrisome, and the data showed a clear negative relationship with testing outcomes. Fourth-grade students who reported using tablets in 'all or almost all' classes scored 14 points lower on the reading exam than students who reported 'never' using classroom tablets. This difference in scores is the equivalent of a full grade level, or a year’s worth of learning."
A meta-analysis by Higgins, Xiao & Katsipataki (2012) found that studies linking the use of technology with student performance show small positive changes, but noted that this is a correlation. Further analysis showed that the schools with higher ICT use in the studies had students who performed better, and the researchers concluded that the technology itself was unlikely to be the cause of the differences in pupil performance.
When the researchers analysed scientific studies with experimental design (that is, research that uses a control group and compares it against the intervention group), they found that technology-based interventions tend to produce slightly lower levels of improvement when compared with other researched interventions. They note that careful thought is needed to use technology to its best effect.
From their research analysis, they recommend that digital technology can be useful 'as a short but focused intervention to improve learning, particularly when there is regular and frequent use (about three times a week) over the course of about a term (5 - 10 weeks). Sustained use over a longer period is usually less effective at improving this kind of boost to attainment.'
While these research findings may have controlled for variables such as socio-economic status and prior performance, they all have limitations, and further research is needed to better understand how to use technology in education more effectively.
References not included in hyperlinks:
Higgins, S., Xiao, Z., & Katsipataki, M.(2012). Impact of Digital Technology on Learning: A meta-analysis about the impact of digital technology on academic achievement [PDF file].