Effects of the reading medium on the text comprehension of university students

Design

The present study had a transversal experimental design.

Participants

One-hundred and twenty-eight students (43.7% women) participated in the study. They were freshmen from the university careers of Engineering (52.3%) and Health Sciences (47.7%) from the University of Mendoza (Age = 18.6 years, SD = 2.61 years). We defined the presence of developmental, psychological, neurological, learning or reading disorders as exclusion criteria. The convenience sampling method was applied.

Instruments

Procedure

Before starting the study, students gave their informed consent to participate. The anonymous and voluntary nature of their participation was explained, as well as the possibility of suspending the activity at any time without negative consequences. The tasks were administered in the faculty classrooms. Participants were assigned to three groups according to the reading medium: 1) reading on PC, 2) reading on paper, 3) reading on smartphone. Reading on PC screens was done on LED monitors, while reading on smartphones was done on the students’ devices. The vocabulary test was conducted on a computer in all cases, as well as the reading survey (which was administered through a Google form).

This study complied with the ethical guidelines 5344/99 of the National Council for Scientific and Technical Research, as well as the 1975 Helsinki Declaration and its subsequent amendments, and was approved by a CONICET committee.

Statistical Analysis

The associations between the study variables were examined through Spearman correlations (considering that reading habits had an ordinal level of measurement). The effects of the reading medium, chosen field of study, and vocabulary were analyzed using a 2 × 2 factorial ANOVA, including age as a covariate. The Shapiro-Wilk test was used to check the normality assumption, and type I error was reduced by applying the Bonferroni adjustment for multiple comparisons. The effect size was reported using the partial eta-squared coefficient.

Descriptive statistics and associations between variables

Performance in text comprehension was relatively low among engineering students (M = 51.4%, SD = 17.9%) and very low among Health Sciences students (M = 36.2%, SD = 14.7%) (see the next section for a comparison). Their vocabulary scores, however, were similar (Engineering: M = 13.9, SD = 4.49; Health: M = 14, SD = 4.78) (T(130) = -0.115, p = .909). Regarding their reading habits and screen use, most members of both groups reported watching television and playing video games for less than an hour per day (Engineering: 83.71% and 73.77% respectively; Health: 75% and 82.14%, respectively), while internet use was more frequent (more than three hours per day, Engineering: 42.6%, Health: 58.9%). As for recreational and study reading, most do it for less than an hour per day (Engineering: 55.74% and 90.16%, respectively; Health: 44.64% and 85.71%, respectively). The most chosen medium for recreational reading was paper among Engineering freshmen (39.3%) and the smartphone among Health Sciences students (44.6%); while for study reading, they preferred the computer in Engineering (47.5%) and paper in Health Sciences (51.8%). This last difference was statistically significant ( ${\chi }^2$(2) = 7.54, p = .023). It is noteworthy that 21.4% of students indicated the smartphone as their preferred study medium (18% in Engineering, 25% in Health). The correlation matrix is described in table 1. The only significant predictor of text comprehension was vocabulary (rho = .198, p = .023).

Effects of Reading Medium and Chosen Field of Study on Text Comprehension

Text comprehension scores were analyzed based on the students’ reading medium and their chosen field of study using an ANCOVA, including the subjects’ vocabulary as a covariate. Main effects were found for the reading medium (F(2.123) = 3.13, p = .048, ${\eta }_p^2$ = .026), the chosen field of study (F(1.123) = 40.14, p < .001, ${\eta }_p^2$ = .239), and vocabulary (F(1.123) = 4.41, p = .038, ${\eta }_p^2$ = .021), with no significant interactions observed. Engineering freshmen comprehended the text better than Health Sciences students, regardless of the reading medium. Post hoc comparisons indicated that comprehension of the text read on a smartphone was lower than that of the text read on a computer screen (p = .048) (see table 2).

Effects of Preferred Reading Medium on Text Comprehension

The previous ANCOVA was repeated, adding as factors: the preferred medium for study reading and the preferred medium for recreational reading, in separate models. No main effects or interactions were found for any of these variables (F < 1.395, p > .218). Two additional analyses were conducted, considering whether the medium in which they read in our study matched their preferred medium for studying or recreational reading. No main effects or interactions with these variables were observed either (F < 1.078, p > .373).

Effects of Vocabulary Level on Text Comprehension

To examine in more detail the effects of students’ vocabulary level and its possible interaction with the reading medium, a new variable was created to classify them according to their performance on the vocabulary K-BIT test. The sample was divided into two groups with vocabulary scores above the median (“high group”, n = 49) or below (“low group”, n = 58), while scores equal to the median (n = 20) were excluded from the analysis. The previous ANOVAs were repeated, but this time the vocabulary level was not included as a covariate but as an additional factor. Main effects of the reading medium (F(2.98) = 3.603, p = .031, ${\eta }_p^2$ = .068) and the field of study (F(1,98) = 38.04, p < .001, ${\eta }_p^2$ = .280) were again observed, but there was also a medium × field × vocabulary interaction (F(2.98) = 3.553, p = .032, ${\eta }_p^2$ = .068). Post hoc comparisons indicated that comprehension was worse when reading on a smartphone compared to a PC screen (p = 0.004) and paper (p = .043) for engineering students with low vocabulary levels. Additionally, comprehension in this group was also lower than that of engineering students with high vocabulary levels who read on a smartphone (p = .002) (see figure 1).

Figure 1Reading comprehension by reading medium, field of study and vocabulary score Note. Estimated marginal means and their corresponding 95% confidence intervals are shown. Comprehension: Percentage of correct responses in the comprehension task. Vocabulary: Vocabulary score group.

Effects of vocabulary, reading habits and medium

The contribution of vocabulary to the comprehension of expository text is considered in the most widely disseminated theoretical model, the Simple View of Reading (Hoover and Tunmer, 2018), and its most recent version, the Active View of Reading (Duke and Cartwright, 2021), which identifies it as a pivotal skill between the processes of decoding and accessing the meaning of written text. Additionally, it has been found that expository texts, in particular, present greater demands in terms of specialized lexicon and general world knowledge (Mar et al., 2021), skills closely linked to the vocabulary test. In this line, we find convergent evidence of the importance of vocabulary for comprehension in other studies conducted in the local adolescent (Abusamra et al., 2020) and university (Tabullo et al., 2020) populations.

The effects of the reading medium should be considered within the framework of the accumulated evidence over the past decades. While two meta-analyses agree on indicating a relatively small advantage in the comprehension of expository texts (but not narrative texts) read on paper compared to digital media (Delgado et al., 2018; Clinton Clinton-Lisell, 2019), others found no significant differences at a general level (Fontaine et al., 2021; Li and Yan, 2024). However, the authors did observe better comprehension on paper for texts longer than a thousand words (Li and Yan, 2024) or of a more technical nature (Fontaine et al., 2021). These effects have been linked to less attentive and more superficial reading favored by screens, or to interference effects related to the demands of navigating digital texts (Clinton, 2019; Delgado et al., 2018). These conclusions are supported by neuroimaging studies that found a higher metabolic cost at the prefrontal cortex level for reading on screens (Lee et al., 2024), and indications of less deep semantic processing for digital reading, inferred from its effects on the N400 potential (Froud et al., 2024). Additionally, an eye-tracking study indicated a more strategic rereading pattern focused on relevant content in subjects who read on paper (compared to tablets), and better performance in recalling a scientific text (Jian, 2021). While the mentioned reviews mainly focused on reading on computer or laptop screens, or did not discriminate by device type, another recent meta-analysis that examined handheld devices (tablet, e-book) (Salmerón et al., 2024) found similar effects, but of lesser magnitude than on larger screens. The authors interpreted that this format might be offering a reading experience more similar to that of a book, thus reducing the performance gap. It is worth noting that a recent study, also conducted in Argentina, found no differences in the comprehension of the same expository text when comparing its reading on screen and on paper (Tabullo and Puliafito-Hamann, 2024). It is also important to note that the differences between digital and paper reading can be reduced or exacerbated by contextual factors, such as time pressure (Ackerman and Lauterman, 2012) or the goals and instructions of the task (Sidi et al., 2017; Lauterman and Ackerman, 2014) and the level of supervision of the activity (Fountaine et al., 2021). On the other hand, none of these works considered reading on smartphones.

Various studies suggest that extensive use of smartphones (characterized by quick interactions, for entertainment purposes, and oriented towards immediate gratification) is associated with a decrease in concentration, reflective thinking, and cognitive effort in daily life. This phenomenon would also affect reading on these devices. A large-scale national study found significantly worse performance in the comprehension of an expository text when comparing reading on smartphones and computer/laptop screens (Cotton et al., 2023). Another neuroimaging study, which compared reading on smartphones and on paper, found evidence of greater cognitive load at the level of prefrontal activity and respiratory frequency in subjects who read on smartphones, as well as greater difficulty in comprehension (Honma et al., 2022). Unexpectedly, we could not observe this difference with respect to the group of students who read on paper in the general analysis, but this effect did manifest when considering the subjects’ verbal abilities. It is worth noting that no effects were found for the preferred reading medium for study or recreation, nor for the match between this preference and the study medium. On the contrary, a previous study that analyzed reading the same text on a computer found better performance for students who usually study on those screens compared to those who preferred paper (Tabullo and Puliafito-Hamann, 2024).

The effect of the reading medium was moderated by the subjects’ verbal ability and also by the chosen major. Among engineering freshmen, subjects with lower vocabulary performance exhibited poorer comprehension when reading on smartphones (compared to paper and PC) and compared to those with better vocabulary reading on the same device. This result suggests that the potentially disruptive effects of the smartphone are amplified for those subjects whose more limited vocabulary constitutes an additional difficulty when approaching the text (as it does not facilitate lexical-semantic access). Subjects with better vocabulary, on the other hand, could compensate for the additional difficulty of the device. A previous study conducted on university students also found an interaction between verbal abilities and reading medium, although in this case, no differences were seen in low-scoring subjects; rather, it was those with higher vocabulary who benefited the most from the digital format (Piovano et al., 2018). Crucially, the medium in this case was the ebook, whose size and mode of use make it more similar to a book and which lacks the potential sources of distraction inherent to the smartphone (Fontaine et al., 2021). Interestingly, Health Science students did not exhibit any of these effects. Given their systematically lower comprehension performance (see the following section), it could be concluded that a floor effect obscured the possible differences related to smartphone use.

Finally, we did not find effects of reading habits on comprehension, as has occurred in previous studies (Acheson et al., 2008; Tabullo et al., 2020); which should probably be interpreted as a limitation of self-report measures to adequately describe the reading experiences of the subjects. In contrast, when more objective measures of text exposure are applied, such as the Author Recognition Test, robust and consistent effects are observed throughout development (for a meta-analysis, see Mol and Bus, 2011).

Differences between freshmen from different Majors

The effect of the chosen university major on text comprehension was surprising, especially because it far exceeded the effects of vocabulary and reading medium in magnitude. The closest precedent we could find in the literature was the work of Amavizca-Montaño and Álvarez-Flores (2022), who compared the comprehension of advanced Mexican university students from different majors. While the authors observed relatively low performance in general terms, they also found a small advantage in the critical level of comprehension for Health Science students (and also for Engineering students, although to a lesser extent) compared to other majors. This effect was attributed to greater exposure to scientific and social literature in their curricula. In contrast, our study differs not only in the effect (since the advantage was observed for those who chose Engineering) but also in the sample, as it was conducted with incoming students who could not yet have been influenced by the literature of their majors. The explanation for the effect should therefore be sought in their prior educational trajectory, which was not considered in this work. In relation to this, a recent study conducted in five Latin American countries found that the socioeconomic level of schools was one of the main predictors of secondary students’ performance in comprehension tests similar to the PISA assessment (Flores-Mendoza et al., 2021). In this study, the socioeconomic level of the school was defined based on the characteristics of the student community, the material and pedagogical resources of the school, its infrastructure, and sanitary conditions. This variable was a more robust predictor than the socioeconomic level measured at the household level or the management of the school (public or private), and therefore constitutes a possible candidate to explain the differences observed among the freshmen to our majors. Another interesting predictor of comprehension (with an effect independent of socioeconomic level) was the fluid intelligence of the students, measured with the Raven’s Progressive Matrices test. Similarly, a study conducted on 10-year-old children also found that fluid intelligence is a significant predictor of reading comprehension (Vernucci et al., 2021). We can hypothesize then that the effect of the chosen major on text comprehension could be explained at least partially by a better average performance in fluid intelligence among the group of engineering freshmen. This hypothesis is supported by previous results from a local study, which indicate superior performance in the Raven test, as well as in calculation and analogical reasoning tests, for students of natural sciences majors (Exact and Engineering) compared to social sciences (Psychology and Sociology); which can be detected as early as the first year and increases throughout the majors (González et al., 2008). On the other hand, the effect of the major cannot be attributed to differences in verbal abilities, because: 1) we controlled for the effect of vocabulary, and yet the effect of the major remained significant, and 2) vocabulary scores were not significantly different between the groups of Engineering and Health Science freshmen.

Study limitations

As limitations of the present study, we must first point out the relatively small sample size, which affects the generalizability of our results. Since it has been noted that the level of supervision of the activity can obscure potential differences between reading media (Fontaine et al., 2021), future studies could include a control condition in which students perform the activity in private (e.g., at home), thus increasing exposure to potential sources of interference such as internet browsing or social media use (when reading is done on screens). Although we controlled the contribution of verbal abilities to comprehension, general cognitive domain variables, such as fluid intelligence or executive functions, were not considered. Including these measures could explain the unexpected advantage observed among Engineering freshmen. On the other hand, the use of self-report measures may have prevented us from adequately describing the actual reading habits of the students, so future studies would benefit from applying more precise and less subjective measures, such as the Author Recognition Test or the use of reading diaries. Finally, although individual preferences regarding the reading medium were considered, other potentially relevant variables, such as attitude or motivation towards reading, as well as the interest aroused by the text and emotional responses during the task, were not included.