Analysis of the Cognitive Load Produced by the Use of Subtitles in Multimedia Educational Material

Benjamín Maraza-Quispe, Olga Melina Alejandro-Oviedo, Walter Cornelio Fernandez-Gambarini, Luis Ernesto Cuadros-Paz, Walter Choquehuanca-Quispe, and Eduardo Rodriguez-Zayra

Analysis of the Cognitive Load Produced by the Use of Subtitles in Multimedia Educational Material and Its Relationship with Learning

Abstract—The research aims to develop an analysis of the cognitive load produced by the use of subtitles in multimedia educational material and its relationship with learning. The methodology used for the development of the research presents a qualitative approach of experimental type, the population selected for experimentation is made up of 100 students of the fifth year of secondary school of Regular Basic Education, of which a sample of 45 students was selected through a simple random sampling, which have been divided into 3 groups of 15 students each,  four  learning sessions are developed using subtitled multimedia videos and without subtitle and a Pre test and Post test are applied in order to analyze the cognitive load involved in the use of subtitled videos in the learning sessions, the scale standardized by Pass and Van Merrienboer is used. The results show that the application of subtitles in the videos mean a potential proposal in the teaching-learning processes, due to the improvement in learning since 30% of students improve their performance in relation to the initial Pretest, in addition to promoting inclusion through support for people with hearing disabilities, they also represent a constant support material. In conclusion, the application of subtitles in multimedia material is evaluated by the student and the developer, because not all videos are productive to apply them if there are videos where subtitles of an intralinguistic, bilingual and other nature are required; except for situations where the learning condition warrants it.

Index Terms—Analysis, cognitive load, subtitles, videos, multimedia, learning.

1. Introduction

Technology has opened doors to different branches of support such as multimedia material, since the 80’s it has been seen as a great support in education, because as educational material it has revolutionized education due to tutorials, documentaries or informative videos that have been the protagonists of non-face-to-face education, however these have always been accompanied by different contents that are part of multimedia,  using visual material (Videos), auditory material (Audios) and textual material (Subtitles), the latter

Manuscript received March 30, 2022; revised April 22, 2022.

The authors are with Facultad de Ciencias de la educación de la Universidad Nacional de San Agustín de Arequipa, Arequipa-Peru, Peru (bmaraza@unsa.edu.pe).

being the most questioned, since subtitling supposes a greater cognitive demand because according to the Theory Proposed by Sweller [1] in the Theory of Cognitive Load, it affirms that the human being is capable only of processing only 3 of the 7 things that are around him,  because the process of these would suppose a cognitive overload in the working memory of the knowledge agent, however these also suppose an improvement to the learning because they function as educational material of support to the student. In this context, this research focuses on analyzing and comparing learning based on the Theory of Cognitive Load [1] and the Theory of Multimedia Learning [2].

From the perspective of cognitive load theory, there can be many reasons for lack of learning. The level of experience can serve as both a facilitator and a barrier, as can the investment effect of the experience [3]. Cognitive load theory is basically a group of principles that guide efficient learning [4] and is rooted in cognitive perspectives. This theory assumes that meaningful learning is possible if the foreign load is removed in addition to benefiting from the German load. According to [4], there are several ways to use cognitive load sources. Based on the assumptions of [5], some of the guidelines have a specific focus on multimedia learning environments. For example, in their ninth (9th) and tenth (10th) directives, [4] they define how to deal with redundancy. According to the principle of redundancy [5], it is assumed that on-screen text does not contribute to the learning process when accompanied by graphics and narratives. Text becomes redundant, so designers need to be aware of it and avoid cognitive overload. In their study,[5] they tried to observe the learning processes of non-native English learners in multimedia environments and found that providing audio and video together contributed to the learning process, while adding subtitles to video made no difference. In a normal learning scenario, a video narrated with subtitles is not an adequate teaching material due to redundancy, but the case may not be valid in a learning environment with a second language. The use of subtitles is accepted as a common way to support non-native learners and reach more people [6].

Subtitling means the translation of a text transmitted orally in an original language from a multimedia product (commonly dialogues in films), it is usually overwritten to the image of the product, however in later studies it states that subtitling is a communication prepared for the language that uses written language and that works as an additional and synchronous channel semiotically, which is part of a transitory and polysemiotic text [7].

According to [8] types of subtitles are presented: Intralinguistic: Those that occur after the transfer of information from one standard language to another, that is, a translation; Reverse Intralinguistic Subtitles: They are those that are in a language other than audio, being considered the strangest because they are not found in the cinema or in the arts and Bilingual Subtitles are those that the soundtrack is in one language and subtitling is applied in two different dialects, they are usually used in film festivals. The ability to combine applications and knowledge is what characterizes the realization of multimedia [9], so that since the twentieth century with the appearance of communication tools as powerful as television and computers people were able to expand their knowledge, product of new multimedia applications.

Cognitive load is related to the amount of cognitive resources a person requires to perform a given task [10], from another perspective, mental workload is described in terms of an interaction between task requirements and human capabilities or resources [10]. Likewise, Sweller [5] states that cognitive load is the workload imposed by a particular task on the human cognitive system. In short, mental load is closely related to the interaction that occurs between the characteristics of the learner and the complexity of the task. Among these characteristics can be taken into account its format, its complexity, the use of multimedia, time pressure and the pace of instruction, on the other hand the age must be taken into account the level of expertise and spatial ability of the apprentice.

[pic 1]

Fig. 1. Structure of Human Cognitive Architecture (ACH).

Source: Retrieved from [11]

Fig. 1 shows how the architecture of working memory works and its functions, because it is the information from the outside that enters the sensory memory, only as stimuli, it is not processed, the stimuli remain in the memory for 1 to 3 seconds and then they are forgotten, these if necessary pass to the working memory in which they are processed to build them as relevant and processed information,  in 30 seconds it is a matter of reviewing keep it constant in the long-term memory, it is for this reason that the greater the stimuli from the outside the wasted information is greater and details of the information are left aside, because the working memory only consists of 7 stimuli in process, the others are discarded and from these a perception of the outside and what is happening in the environment is obtained.

There are three types of cognitive load: intrinsic, extrinsic and pertinent, the first arises from the interaction between the nature of the object or complexity and the experience of the learner; it is not related to instructional design, so it is impossible to modify it. As for the extrinsic, it is the one that has more information than necessary and can become distracting and hinder the task, this type of cognitive load is unnecessary as detailed by Sweller [5] and finally the Germanic or pertinent load, this is the product of a good instructional design, with it the resources of the working memory are used for the beneficial construction of schemes, through processes of abstraction and elaboration [12].

Sensory Memory: they constitute sensory channels such as visual and auditory, which are the main key to our ability to find different stimuli in the environment, which are stored for a short period of time, this does not assign meanings to any of these stimuli. Therefore; as these channels are separated, the information is processed independently and the Working Memory: Sweller [5] defines it as short-term memory (name with which it was previously known), is characterized by its limited storage capacity, this memory is responsible for the work and processing consciously around two or three elements at the same time, paying attention to it for 15 to 30 seconds [13]

As for the measurement of cognitive load, the elements that are presented: Mental load: It is originated by the interaction between the characteristics of the task and the characteristics of the subject. Different tasks can be more complex because the steps to follow are taken into account or how much precision these require, on the other hand several subjects may have different skills and different levels of expertise; Mental effort: This aspect refers to all the resources necessary or that are destined to solve a specific task, this reflects the instantaneous cognitive load, this is usually measured when the participants are in realization of the same or immediately after its completion; Performance: According to [14], performance is an aspect of cognitive load, this is understood as the quantifiable achievements of the learner, as would be his score after a success of questions. This can be measured while the task is being carried out; Methods of measuring cognitive load: According to [11] in his article Theory of Cognitive Load, multimedia design and learning: a state of the art, he discerns two different methods for the measurement of cognitive load, an analysis based on estimating the amount of mental load that a person would require to perform a task, basing these estimates on critical judgments in the domain of the task and on mathematical models of analysis,  however, there is also the empirical method that measures mental effort through subjective methods or instruments, such as the Likert test, which records the introspective report of mental effort, on the other hand there are also assessment scales or tests such as the one proposed by [14] which I would consider more accurate.

...