[article | discuss (0) | ]
Ali Farhan Munify AbuSeileek
King Saud University, Kingdom of Saudi Arabic
This study explored whether hypermedia annotation presentation in different locations of the text facilitates EFL vocabulary acquisition and reading comprehension. It also examined participants' preferences for the location of hypermedia-annotated text (at the end of the text, in the margin, at the bottom of the screen, or in a pop-up window). Data were collected from 80 intermediate adult EFL learners who read annotated texts and received support in different conditions of hypermedia annotations. A survey of prior knowledge was carried out, followed by vocabulary and reading comprehension tests, and a questionnaire. Results indicated that students who had access to hypermedia annotations outperformed their peers who used traditional glosses listed at the end of the text. There was also a positive impact for texts with hypermedia annotations located in the margin of the text. Learners also preferred hypermedia annotation presentation in the margin. Finally, no relationship was found between participants' preferences and their level of achievement.
Annotations, Hypermedia, Glosses, Location of Glosses, Learner Preferences
Hypermedia annotation is a technique used in language learning and teaching to enhance reading comprehension and vocabulary acquisition. It is based on the use of technology-enhanced language-learning facilities to gloss words and provide their meanings in different modes (textual, visual, and auditory), forms (pictures, videos, sound, and text), or locations in the text (at the side, bottom or top margin, in a pop-up window, and the end of the text). While surfing hypermedia-annotated texts, the reader can enjoy interacting with different types of information which are presented digitally on the computer screen. Hypermedia annotations is one of the advantages of computer-based texts and consulting them for different types of multimedia information is no longer seen as an interruption to the process of understanding the text because information is available as the reader wants it at the simple click of a mouse. However, the way the hypermedia annotations are presented or processed in different locations of the text may affect the reader's understanding of the text, either positively or negatively.
An annotation--or a gloss--is defined as adding comments or notes about difficult words, phrases, or ideas in order to provide their definitions or meanings in a particular context. Lomicka (1998, p. 41) outlines that "glosses are most often supplied for 'unfamiliar' words, which may help to limit continual dictionary consultation that may hinder and interrupt the L2 reading comprehension process." Roby (1999, p. 96) also stresses that a gloss
supplies "what is perceived to be deficient in a reader's procedural or declarative knowledge" about a certain word or expression. The gloss contains different types of information such as synonyms, encyclopedic comments, grammatical notes (Hullen, 1989), translations, cultural, historical and geographical references, and guiding questions (Lomicka, 1998). Researchers study annotations under different categories. For example, Roby (1999) sorted annotations into the following subdivisions: gloss authorship, gloss presentation, gloss function, gloss focus, gloss language, and gloss form.
Although the technique of glosses was generated by medieval learners when they read a foreign text (Hullen, 1989), Lomicka (1998) notes that the use of glossing as a teaching aid to improve an L2's reading comprehension was largely unexamined by researchers until late in the last century. Stewart and Cross (1991) point out that the purpose of glossing is to produce independent readers since this technique has a focusing effect. Moreover, Jacobs (1994) reports that learners can benefit from glosses because they are effective techniques that aid L2 vocabulary acquisition. Jacobs, Dufon, and Hong (1994) list the following four reasons for the widespread presence of vocabulary glosses in reading texts: enhancing comprehension, increasing vocabulary learning, catering to students' preferences, and providing greater use of authentic texts. Glosses also remain a common and acceptable aid for EFL language textbooks (Davis, 1989).
Hypermedia annotations have several advantages over traditional glosses. Glosses in computer-based annotated texts can be approached both globally and linearly (Martinez-Lage, 1997), while glosses in paper-based texts can only be approached linearly. Though traditional glosses usually contain textual information, computer-based glosses can take different forms such as video, sound, and pictures (Wolff, 1987; Garza, 1991; Chun and Plass, 1996; Lomicka, 1998; Ercetin, 2003; Ariew and Ercetin, 2004). More than one type of multimedia annotation may be used together in computer-assisted annotations. Jones and Plass (2002), for example, used a combination of written and pictorial annotations, and Al-Seghayer (2001, p. 202) developed "printed text definition coupled with still pictures, and printed text definition coupled with video clips" for enhancing vocabulary acquisition. Unlike traditional glosses, hypermedia annotations in a digitized electronic text can be linked to the defined word and can appear in a pop-up window at any location on the screen when the target word is clicked.
This study explores reading comprehension and vocabulary acquisition in second or foreign language in hypermedia environments under two major headings: (a) hypermedia annotations and language learning and (b) method of gloss presentation.
Hypermedia Annotations and Language Learning
There is controversy regarding the effectiveness of using different types of annotations on foreign and second language learning. On the one hand, many studies have found that hypermedia annotations are useful for several aspects of language learning (e.g., see Davis, 1989; Hulstijn, Hollander, & Greidanus, 1996; Chun & Plass, 1997; Martinez-Lage, 1997; Roby, 1999; Bell & LeBlanc, 2000; Gettys, Imhof, & Kautz, 2001; Melissa, 2004; Juffs, Eskenazi, Callan, Wilson, Heilman, Collins-Thompson, & Pelletreau, 2006; Pelletreau, 2006). Several studies have also found that the computer is very useful for facilitating reading comprehension. For instance, Blohm (1982) reports that students who used computer-based annotations recalled reading texts significantly better than those who had no access to glosses. Lomicka (1998) also draws the conclusion that computer-based reading with access to glosses may promote a deeper level of text comprehension than no glossing.
Other studies which compare computer-based glosses to paper glosses have found a
significant effect for multimedia use in learning. Roby (1991) reports that students who used a computer dictionary looked up significantly more words than those who used a paper dictionary. The students in the computer condition were also more satisfied with the semantic support available to them than those in the paper condition. Aust, Kelley, and Roby (1993) also report that students looked up significantly more words in electronic conditions than in traditional conditions and that the students consulted references in electronic conditions more often than the students in conventional conditions.
However, other studies have shown that glosses do not have an effect on vocabulary acquisition and reading comprehension. Jacobs et al. (1994, p. 26) report that "although those learners with glosses outperformed their peers who did not have glosses on a vocabulary instrument administered shortly after reading the passage, that difference disappeared when the vocabulary instrument was readministered four weeks later." Black, Wright, Black, and Norman (1992) confirm these findings and indicate that readers do not benefit from computer-based glosses for long-term retention. Johnson (1982) affirms that glossing disrupts the global reading of texts. Sakar and Ercetin (2005) also report a negative relationship between annotation use and reading comprehension, multimedia annotations such as pronunciations, audio recordings, and videos affecting reading comprehension negatively. Other researchers find no relationship between reading comprehension and glosses (Dillon & Gabbard, 1998; Guillory, 1998).
Method of Gloss Presentation
Gloss presentation--how and where the information is displayed--is very important for the EFL reader because it affects the cognitive aspects of text processing such as comprehension and reading speed. Roby (1999) proposes using boldface type to indicate that certain words have been glossed. Liu and Reed (1995), Chun and Plass (1996), and Brett (1998) demonstrate a major effect for highlighted words because they are learned or retained better than unmarked language items. Martinez-Lage (1997) reports that highlighted words allow students to read without disturbing the process of reading and immediately provide readers with their meaning. Other studies have outlined the use of the invisible hypermedia annotations. Davis (1989) concludes that invisible and unobtrusive annotations allow users to consult meanings whenever they desire. Al-Seghayer (2001) also finds that computer-based glosses do not interrupt the reading process because the glosses are invisible until readers click on the glossed word.
Several studies have investigated students' clicking behavior. For instance, De Ridder (1999, 2000) reports that the clicking behavior does not affect text comprehension negatively, nor does it slow down the reading process. On the other hand, some studies have noted that more intensive clicking on hyperlinks may hinder reading comprehension. De Ridder (2002) stresses that the constant interruption of the reading process in the marked condition may prevent students from building up a coherent representation of the text.
Many studies have also stressed the need for conducting more research on hypermedia annotation presentation. Chun and Plass (1997) suggest conducting further research to investigate the effect of multimedia materials on improving reading comprehension. De Ridder (2002) reports that the issue of how glosses are presented is still under investigation and that the method of gloss presentation may affect the cognitive aspects of text processing. Stark (1990) states that pop-up windows should not cover up the portion of the text in which a glossed word is located so that users can view the gloss and the glossed word's context together. Roby (1999) proposes carrying out research based on dedicating a portion of the
screen as a gloss space because cluttered screens hinder rather than help readers. He suggests that all glosses could appear in the lower right corner of the screen in a box. In addition, Frenckner (1990) stresses the importance of conducting studies which focus on the method of gloss presentation such as typefaces and margins because the method of presentation affects comprehension and reading speed.
Jacobs et al. (1994) investigated students' opinions about the presentation of glosses in different locations in the text: in the margins, at the bottom of the page, or at the end of the text. Participants expressed a preference for glosses in the margin. However, to date there do not seem to be studies focusing, as their main objective, on comparing the efficacy that different locations of hypermedia annotations have on vocabulary acquisition and reading comprehension. The present study was designed to fill this gap.
The purpose of the present study was to investigate the effect of hypermedia annotation presentation in different locations of the text on vocabulary acquisition and reading comprehension for EFL learners and to measure their attitude towards the location of the annotations. The study addressed the following questions:
1. Do learners who have access to hypermedia annotations while reading an EFL passage perform significantly better than those with traditional glosses listed at the end of the text on measures of vocabulary acquisition and reading comprehension?
2. Does the location of the gloss (at the end of the text, in the margin, at the bottom of the screen, or in a pop-up window) affect learners' performance on a vocabulary test and reading comprehension test?
3. Do students prefer reading passages with hypermedia annotations or passages with traditional glosses listed at the end of the text?
4. Where do students prefer the glosses to be located: at the end of the text, in the margin, at the bottom of the screen, or in a pop-up window?
5. Is there a significant correlation between students' level of achievement and their attitude towards the location of the gloss?
Participants and Design
Ninety-eight students from four sections of English language courses took part in this study. The data from 18 students were excluded from the analysis because these students did not complete at least one of the tests involved in the project, leaving a total of 80 students whose results are reported here. All of the students were Saudi students enrolled in a second-semester intermediate English class in the Department of English Language and Literature at King Saud University in Saudi Arabia. They were enrolled in the BA English program which aims at enhancing cross-cultural understanding and values. The students' ages ranged from 19-21 with a mean of 19.8 years. Only three students had been to the United States or the United Kingdom, with a medium length of stay of about 2 months. The rest had no language experience in the United States or the United Kingdom. In other words, the students in the study
had limited English proficiency. All students had used a computer prior to the study with an average of 6 years of experience. All of the students were males due to cultural values that support segregation of males and females in classes. Their scores in the secondary general English exam were over 90%. Their mean estimated English GPA was 3.2 out of 5 (good) in the first semester.
For the purposes of this study, a score range between 68 and 75/100 on the reading and vocabulary sections of the department's English Placement Test was considered intermediate level and is roughly equal to level 5 in the IELTS, FCE, and CCSE; level 2 in Cambridge exams; AP according to ARELS exams; and OP according to Oxford exams. The 80 participants were placed in the intermediate level based on their scores and randomly assigned to one of the four groups: control group, first treatment group, second treatment group, and third treatment group. There were no significant differences (F (3,76) = 1.342, p = .236) in the mean scores of the participants in these groups on the test: (a) control: traditional glosses listed at the end of the text (n = 20, M = 71.10, SD = 2.24), (b) the first treatment group: computer-based glosses in the margin which appeared on the same line as the glossed word (n = 20, M = 71.10, SD = 2.24), (c) the second treatment group: computer-based glosses at the bottom of the screen (n = 20, M = 69.95, SD = 2.23), and (d) the third treatment group: computer-based glosses in pop-up windows (n = 20, M = 70.87, SD = 2.44). In order to investigate the effect of the location of the glosses, the scores of the control group versus those in the treatment groups on a series of tests over a period of several weeks were analyzed.
All participants in the four groups and the pilot study used the same material and the same glosses. To ensure the independence of each group over several weeks and to minimize the possibility for students sharing what they were doing in each group, students were blinded about the material used in this study. That is, they had access to the material only during the lessons. In addition, they were told that this study would be conducted just for academic purposes and that their results in the course would not in any way be affected by their answers on the tests used in the study. Students were also asked not to write their names on the tests; numbers were used instead. It was occasionally difficult to follow all students in the four groups because some students were sometimes absent. During the last three weeks of the study, the instructor was sometimes forced to postpone a lesson in order to include the results of students who were absent in a given class.
Seven texts comparable in length (about 1,000 words each) were read by 27 students in the pilot study and three raters. Two of the raters were EFL teachers, while the other was a native speaker of English. The texts were extracted from the following stories in the AMRA textbook: "The house on Mango Street" by Sandra Cisneros, "The far and the near" by Thomas Wolfe, "The poet" by Hermann Hesse, "Through the tunnel" by Doris Lessing, "The people before" by Maurice Shadbolt, "The lottery ticket" by Anton Chekov, and "Apparition" by Guy de Maupassant. Within this pilot study, the texts were evaluated for interest and difficulty level and were selected based on the following criteria:
1. The book was selected by a team of seven experts for intermediate level students in the reading course in AMRA series in the Ministry of Education. Six of them were EFL specialists (two linguists, two methodologists, one curriculum designer, and one supervisor of English language) and one native speaker who was a specialist in applied linguistic studies.
2. The texts contained some words that the participants would not know. Five hundred and forty words (6% of the total words) were marked as difficult by at least two of the raters, while only 270 of these words were glossed because they were not known by all students in the pilot study.
3. The texts were interesting for the participants in the study.
4. The length was about 1,000 words because learners at the intermediate level are not trained to read long texts, and this length allowed students to read a text carefully within 50 minutes.
Instructional Software: Overview of Organization and Content
In order to gloss target words, a program was designed for this study. The program was written in a Visual Basic Editor and then published on CD-ROM. Visual Basic was chosen for programming because it easily integrates hypermedia annotations in texts. The computer screen presented the title of the text and the text itself, while the left-hand frame was reserved for marginal glosses and the bottom frame for bottom glosses. In addition, the program allowed hypermedia annotations to appear in a pop-up window.
The target words used in the program were linked to two types of information in the gloss. The first contained the synonym of the glossed word, consisting mainly of one word. The other type provided the definition of the word, usually a phrase of 2-6 words. Glosses were designed in a way that allowed users to see only one gloss at a time. When they clicked on the target word, the gloss for that target word was presented. When they wanted to check on the meaning of another word, the first gloss disappeared. This strategy was adopted to enable the participants who had limited English proficiency to concentrate more on checking the meaning of a single glossed item.
The program was used for glossing words as follows:
1. Marginal glosses: hypermedia annotations in the side margin
As users click on the glossed word, the meaning appears beside the glossed word in the margin. Marginal glosses are the closest to the glossed word because they appear on the same line as the glossed item (see Fig. 1 below).
2. Bottom glosses: hypermedia annotations in the bottom margin
As users click on a glossed word, the meaning appears at the bottom of the same screen of the computer with no need to scroll down the window. Although bottom glosses appear in the same window, their distance from the glossed word is much more distant than that of the marginal gloss (see Fig. 1).
3. Pop-up window glosses: hypermedia annotations in a pop-up window
The English definition appears in a pop-up window when users click on the defined word. The window does not cover up the portion of the text in which a glossed word is found (see Fig. 1).
4. List of words at the end of text: traditional glosses listed at the end of the text
Another version of the texts was created in a scrolling window for use by the students in the control group. Students see underlined words and have to scroll down to the end of the
text to see their meaning. This version contained the same glossed words and glosses as the ones used by the treatment groups.
The students took a test each week for a total of nine tests over the course of the project. Each test consisted of two sections: a vocabulary recognition test and a reading comprehension test. The vocabulary recognition test contained 25 items. Each item had four distracters, along with the correct answer, in order to minimize potential effect of guessing. Each correct response received two points, with a maximum test score of 50. The test questions were chosen based on the glossed items and an earlier pilot study.
The reading comprehension test followed a recall protocol format, a technique used successfully in related studies (e.g., Lee, 1986; Chun & Plass, 1996), in which participants were required to summarize in English the reading passage they had just read. The free recall protocol was scored on the basis of 50 propositions that represented the idea units of each of the reading passages, and each proposition was 1 point. To establish interrater reliability, 10% of recall passages were randomly selected and separately scored by each of the raters. Differences were discussed until consensus was reached. The recall passages were then scored by the same three raters, and the interrater reliability was found to be .97.
The tests were extensively field tested in the pilot study. That is, students in the pilot study who were enrolled in a program of intensive language courses studied the same material used in this study over 9 weeks and they took nine tests in vocabulary recognition and reading comprehension over those 9 weeks as a regular part of their course. At the end of the pilot study, unsuitable vocabulary items and propositions in the reading texts were eliminated, leaving 25 vocabulary items and 50 propositions. The individual tests were then aggregated into a single posttest. The internal consistency of Cronbach's alpha reliability for this test instrument was .86 (.83 for the vocabulary test and .89 for the reading comprehension test).
The final instrument used in this study was a two-item questionnaire designed to investigate students' opinions about glossing. The first item asked them to indicate their preference for computer-based glosses or traditional glosses listed at the end of the text. The second item asked about their preference regarding the location of the gloss: (a) at the end of the text, (b) in the margin, (c) at the bottom of the screen, or (d) in a pop-up window.
The study was conducted in a computer laboratory in the Department of English Language and Literature at King Saud University. The lab had 40 multimedia computers. An experienced instructor in English with advanced computer expertise monitored the tests so that students would not give positive answers to questions in order to please their instructor. He explained to the participants that their scores on these tasks would in no way affect their grade in the course.
The study was conducted during a 9-week period. Every week students studied a text which was followed by a vocabulary/reading comprehension test. One of the seven texts, 'The people before' by Maurice Shadbolt, consisted of three sections and was long compared to other passages, so students completed it over a 3-week period. Each session consisted of reading the text, taking a vocabulary/reading comprehension test, and completing a survey of prior knowledge in which students were asked whether they thought they already knew the words before reading the text or from reading the text itself.
Students received a brief introduction prior to each stage of the project. Before the project, the participants had been provided with an introduction to the reading text and the program. The course instructor explained to the participants what they would do in the experiment on the screen. The computer program guided the participants through the steps in sequence: reading the text, vocabulary/reading comprehension test, and prior information. The students used a simple point-and-click mouse function for answer selection, submission, and text navigation.
In the process of reading the text, each of the participants in the treatment groups could further choose to consult the hypermedia-annotated words, while those in the control group were able to consult the glosses listed at the end of the text. After reading the passage, each participant immediately took the vocabulary/ reading comprehension test. After taking a break after the final test, participants in the treatment groups completed the questionnaire.
Means and standard deviations of students' overall scores on the vocabulary acquisition and reading comprehension tests were computed. A t test was used to analyze the test scores of students in the three treatment groups versus those in the control group. A one-way ANOVA was used to analyze the students' test scores with respect to any differences attributable to the location of the gloss (at the end of the text, in the margin, at the bottom of the screen, or in a pop-up window). Scheffé post hoc tests were conducted to compare any differences among the different groups. To identify students' preferences for type and location of glosses, means, standard deviations, and frequencies of responses to the questionnaire items were calculated. Finally, Pearson correlation coefficients were calculated to explore relationships between students' preferences for the presentation of the annotated words in different locations and their level of achievement on the test.
Question number one asked whether learners who have access to computer-based glosses would perform significantly better than those with glosses at the end of the text on measures of vocabulary acquisition and reading comprehension. Table 1 lists the means and standard deviations of the student's scores on the posttest comparison of those means (t values).
*p < .005
**p < .001
According to the figures in Table 1, the overall mean of students' scores in the three treatment conditions was significantly higher than that of the students in the control group (t = 4.97, p < .05). The mean score of the students in the three treatment conditions was also significantly higher than that of the students in the control group on the vocabulary portion of the posttest (t = 3.11, p < .05). The results of comprehension followed a similar pattern. The mean score of the students in the three treatment groups was significantly higher than that of the students in the control group (t = 5.67, p < .05). In summary, the t-test analysis showed that the presence of hypermedia annotations (margin, bottom, and pop-up window) led to significantly greater vocabulary acquisition scores and reading comprehension scores compared to the availability of traditional glosses at the end of the text.
Question number two focused on the effect of the location of the glosses (at the end of the text, in the margin, at the bottom of the screen, or in a pop-up window) on learners' performance. Table 2 presents the results of a one-way ANOVA of students' posttest scores for the four locations of the glosses.
*p < .001.
The data in Table 2 show that there was a significant effect of the location of the glosses on the students' scores on the posttest (F(3,76) = 36.51, p < .001). Post hoc comparison (Scheffé) showed that the 'margin' treatment group significantly outperformed the other groups at the p < .05 level. The 'bottom-of-screen' group also had significantly higher scores than each of the control group and the 'pop-up-window' group (p < .05). These findings result in the following order of the effect of the location of glosses: margin > bottom of screen > pop-up window/end of text.
The data in Table 2 show a similar significant effect for the location of the glosses on the students' scores on the reading comprehension posttest (F(3,76) = 16.03, p < .001). Post hoc comparison (Scheffé) showed that the 'margin' and 'bottom-of-window' groups significantly outperformed the 'pop-up-window' and 'end-of-text' groups and that the 'margin' group outperformed the 'pop-up-window' group (p < .05). These findings result essentially in the following order of effect: margin/bottom of screen > pop-up window/end of text.
Questions three and four asked about students' preference for reading passages with traditional glosses versus hypermedia annotations and the location of glosses. Analysis of the
students' responses on the questionnaire indicated that 95% of the students preferred hypermedia annotations. Regarding the location of glosses, 78.8% of the students preferred the glosses to be located in the margin, 15% at the bottom of the screen, 5% in a pop-up window, and 1.3% at the end of the text (see Table 3).
Question five focused on the relationship between students' level of achievement and their attitude towards the location of glosses. In order to see whether students' attitude was meaningful in terms of their achievement in the treatment conditions (end of text, margin, bottom of screen, and pop-up window), the means of the students' overall achievement was compared with the means of their preferences indicated on the questionnaire. Table 4 shows the correlation between students' test scores and their preferences.
The data in Table 4 shows that Pearson correlation coefficients between the students' scores on the vocabulary and comprehension tests did not show any significant differences between students' achievement and their attitude towards gloss locations in the text.
Based on the findings of this study, it can be affirmed that learners who had access to hypermedia annotations while reading an EFL passage performed significantly better than those who had access to glosses at the end of the text on measures of reading comprehension and vocabulary acquisition. The immediate availability of glosses for hypermedia-annotated words enhanced students' reading comprehension and vocabulary acquisition. This finding has been supported by other research done to identify the effect of hypermedia annotations on students' performance. For example, Davis (1989) found that annotated words were useful to several aspects of language learning. Hulstijn et al. (1996) also supported the use of glossed words in L2 reading. Lyman-Hager, Davis, Burnett, and Chennault (1993) reported that students who had access to hypermedia-annotated words retained vocabulary better than those who used the traditional gloss. However, the results of the current study differ from those obtained by Sakar and Ercetin (2005) who found a negative relationship between annotation use and reading comprehension. The present study reports the opposite: participants performed better on the vocabulary and comprehension tests.
This study also investigated the effect of gloss location in the text (at the end of the text, in the margin, at the bottom of the screen, or in a pop-up window) on learners' performance on the vocabulary test and comprehension test. The results of the study reveal that texts with hypermedia annotations in the margin were the most highly preferred, followed by texts with annotated words at the bottom of the screen, at the end of the text, and finally in a pop-up window. As a further step, the study examined the correlation between students' level of achievement and their attitude towards the location of glosses. There was no correlation between students' attitude and their level of performance.
The first part of this finding, higher performance for the students in the 'margin' group and lower performance for those in the 'end-of-text' group, was expected for the following reasons. First, most participants (78.8%) indicated a preference for marginal glosses over those in other positions. Second, students in the 'margin' condition had faster and easier access to marginal glosses than traditional glosses listed at the end of the text (see Jacobs et al., 1994). Third, as students in the 'margin' condition clicked on a word, they saw the meaning of the word somewhere near the same line in the text. However, students in the 'end-of-text' group had to scroll down to the end of the text to see the gloss, an action which may have caused them to lose track of the context of the annotated word and interfered with their efforts to develop an understanding of the text.
Students in the 'bottom-of-screen' condition had lower scores than those in the 'margin' condition likely for similar reasons. The bottom-of-screen glosses were presented in locations that were often far removed from the annotated word. For example, sometimes the target word was on the first line of the text or in the first paragraph while the gloss appeared at the end of the screen. This finding confirms Jacobs et al.'s (1994, p. 26) statement that "participants ... indicated a near unanimous preference for marginal glosses over those placed in locations more distant from the text." Furthermore, the split in the locations of the annotated word and its gloss may produce a split in students' being able to follow the types of information in both locations. Chun and Plass (1997) warned against the negative effect of attention split between the types of information presented in glosses.
The second part of the findings that the 'pop-up-window' group had the lowest performance was unexpected. Roby (1999, p. 98) warned that "pop-up windows [should be] positioned so that they do not cover up the portion of the text in which a glossed word is found." Although Roby's warning was considered when the present study was designed, and no glosses covered any part of the text so that users could read the gloss and see the glossed word's context together, the students in the 'pop-up-window' condition still had the lowest performance. This result may be attributed to several reasons. First, the way the pop-up window appeared may have directed users' attention more to the pop-up window than the annotated words in the text. Second, the pop-up window occupied much more space than the textual glosses in the margin. The presence of the pop-up window may have distracted users and interfered with their processing of the information in the text.
The third reason may be related to the level of linguistic competence of the students in this study, intermediate-level EFL learners. Students with limited English proficiency may need to concentrate more on a text when they read than advanced-level learners. A glance outside the line such as that occasioned by the use of a pop-up window may mislead readers and cause them to lose track of where they are in a text. This finding is similar to the one reported by Ariew and Ercetin (2004) who used pop-up window annotations and found them to have a negative impact on reading comprehension for intermediate-level learners. They suggested that practitioners should be cautious about using hypermedia annotations, especially with lower proficiency learners.
Fourth, since the project lasted several weeks and all participants in the four groups used the same material and the same glosses, it could be that the low results of the 'pop-up window' is attributable to the possibility for students' sharing what they were doing in each group. However, this possibility is just a speculation because all participants in the four groups were equally aware of the tests. Furthermore, many procedures were put into place to minimize this possibility. For example, students had access to the material only during the tests and they were also told that this study would be conducted only for academic purposes and that their results in the course would not in any way be affected by their answers on the tests used in this study. All these speculations could be tested in other studies to investigate their effects. A similar study could be conducted to compare the effect of hypermedia annotations presented in different locations of a text on groups of learners of different levels of linguistic competence. A future study could also include the effect of the distance of the location of the gloss using hypermedia annotations in which the gloss is presented directly after the glossed item.
The results of this study should be interpreted cautiously because they are based on a small number of participants and limited texts used over a limited period of time. Therefore, similar studies are needed to determine the effect of gloss presentation in different locations of the text on students' performance. Future research is also needed to address issues regarding gloss presentation in different locations in the text under different conditions (textual, visual, and audio). A future study might be devised whereby students are asked to work on longer texts in several sessions over a longer period of time. Another point to be considered in future studies is based on the suggestion of one of the participants in this study who proposed including glosses that directly follow the glossed word. This suggestion opens more perspectives for future studies such as developing more questionnaire and interview formats which contain open-ended questions that take students' points of view into consideration.
In summary, the findings of the present study bring us a step closer to understanding the most suitable ways in the presentation of the hypermedia annotation in different locations in a text. The technique of hypermedia annotation makes the process of providing information while reading a text more accessible globally and linearly. It helps readers compensate for their lack of information and becomes especially important for those with limited levels of linguistic competence. The findings of the study revealed that there was a main effect for texts with hypermedia annotations over a traditional list of words at the end of the text and a positive impact for hypermedia annotations in the margin, the location that was also most preferred by the participants. Nevertheless, no relationship was found between participant's preferences and their levels of achievement in the posttest.
Al-Seghayer, K. (2001). The effect of multimedia annotation modes on L2 vocabulary acquisition: A comparative study. Language Learning & Technology, 5 (1), 202-232. Retrieved October 22, 2007, from http://llt.msu.edu/vol5num1/alseghayer/default.html
Ariew, R., & Ercetin, G. (2004). Exploring the potential of hypermedia annotations for second language reading. Computer Assisted Language Learning, 17 (2), 237-259.
Aust, R., Kelley, M., & Roby, W. (1993). The use of hyper-reference and conventional dictionaries. Educational Technology Research & Development, 41 (1), 63-73.
Black, A., Wright, P., Black, D., & Norman, K. (1992). Consulting on-line dictionary information while reading. Hypermedia, 4 (3), 145-169.
Bell, F., & LeBlanc, L. (2000). The language of glosses in L2 reading on computer: Learners' preferences. Hispania, 83 (2), 274-285.
Blohm, P. (1982). Computer-aided glossing and facilitated learning in prose recall. In J. Niles & L. Harris (Eds.), New inquiries in reading research and instruction: Thirty-first yearbook of the National Reading Conference (pp. 24-28). Rochester, NY: National Reading Conference.
Brett, P. (1998). Using multimedia: A descriptive investigation of incidental language learning. Computer Assisted Language Learning, 11 (2), 179-200.
Chun, D., & Plass, J. (1996). Effects of multimedia annotations on vocabulary acquisition. Modern Language Journal, 80 (2), 183-198.
Chun, D., & Plass, J. (1997). Research on text comprehension in multimedia environments. Language Learning & Technology, 1 (1), 60-81. Retrieved October 22, 2007, from http://llt.msu.edu/vol1num1/chun_plass/default.html
Davis, J. (1989). Facilitating effects of marginal glosses on foreign language reading. Modern Language Journal, 73 (1), 41-48.
De Ridder, I. (1999). Are we still reading or just following links? In K. Cameron (Ed.), CALL and the learning community (pp. 195-116). Exeter, UK: ELM Bank Publications.
De Ridder, I. (2000). Are we conditioned to follow links? Highlights in CALL-materials and their impact on the reading process. Computer Assisted Language Learning, 13 (2), 183-195.
De Ridder, I. (2002). Visible or invisible links: Does the highlighting of hyperlinks affect incidental vocabulary learning, text comprehension, and the reading process? Language Learning & Technology, 6 (1), 123-146. Retrieved, October 22, 2007, from http://llt.msu.edu/vol6num1/deridder/default.html
Dillon, A., & Gabbard, R. (1998). Hypermedia as an educational technology: A review of the quantitative research literature on learner comprehension, control, and style. Review of Educational Research, 68 (3), 322-349.
Ercetin, G. (2003). Exploring ESL learners' use of hypermedia reading glosses. CALICO Journal, 20 (2), 261-83.
Frenckner, K. (1990). Legibility of continuous text on computer screens--A guide to the literature. Stockholm: Royal Institute of Technology.
Garza, T. (1991). Evaluating the use of captioned video materials in advanced foreign language learning. Foreign Language Annals, 24 (3), 239-258.
Gettys, S., Imhof, L., & Kautz, J. (2001). Computer-assisted reading: The effect of glossing format on comprehension and vocabulary retention. Foreign Language Annals, 34 (2), 91-106.
Guillory, H. (1998). The effect of keyword captions to authentic French video on learner comprehension. CALICO Journal, 15 (1-3), 89-108.
Hullen, W. (1989). In the beginning was the gloss. In G. James (Ed.), Lexicographers and their works (pp. 100-116). Exeter, UK: University of Exeter.
Hulstijn, J., Hollander, M., & Greidanus, T. (1996). Incidental vocabulary learning by advanced foreign language students: The influence of marginal glosses, dictionary use and reoccurrence of unknown words. The Modern Language Journal, 80 (3), 327-339.
Jacobs, G. (1994). What lurks in the margin: Use of vocabulary glosses as a strategy in second language reading. Issues in Applied Linguistics, 4 (1), 115-137.
Jacobs G., Dufon P., & Hong, F. (1994). L1 and L2 vocabulary glosses in L2 reading passages: Their effectiveness for increasing comprehension and vocabulary knowledge. Journal of Research in Reading, 17 (1), 19-28.
Jones, L., & Plass, J. (2002). Supporting listening comprehension and vocabulary acquisition in French with multimedia annotations. The Modern Language Journal, 86 (4), 546-561.
Johnson, P. (1982). Effects on reading comprehension of background knowledge. TESOL Quarterly, 19 (4), 503-516.
Juffs, A., Eskenazi, M., Callan, J., Wilson, L., Heilman, M., Collins-Thompson, K., & Pelletreau, T. (2006). Promoting robust learning of vocabulary through computer assisted language learning. Paper presented at the meeting of the American Association of Applied Linguistics, Montreal, Canada.
Lee, J. (1986). On the use of the recall task to measure L2 reading comprehension. Studies in Second Language Acquisition, 8 (2), 201-212.
Liu, M., & Reed, W. M. (1995). The effect of hypermedia-assisted instruction on second-language learning. Journal of Educational Computing Research, 12 (2), 159-175.
Lomicka, L. (1998). To gloss or not to gloss: An investigation of reading comprehension online. Language Learning & Technology, 1 (2), 41-50. Retrieved October 22, 2007, from http://llt.msu.edu/vol1num2/article2/default.html
Lyman-Hager, M., Davis, J., Burnett, J., & Chennault, R. (1993). Une Vie de Boy: Interactive reading in French. In F. Borchardt & E. Johnson (Eds.), Proceedings of CALICO 1993 Annual Symposium on Assessment (pp. 93-97). Durham, NC: Duke University.
Martinez-Lage, A. (1997). Hypermedia technology for teaching reading. In M. Bush & R. Terry (Eds.), Technology enhanced language learning (pp. 121-163). Lincolnwood, IL: National Textbook Company.
Melissa, B. (2004). L2 glossing: To CALL or not to CALL. Hispania, 87 (3), 541-552.
Pelletreau, T. (2006). Computer-assisted vocabulary acquisition in the ESL classroom. Unpublished doctoral dissertation. University of Pittsburgh, Pittsburgh.
Roby, W. (1991). Glosses and dictionaries in paper and computer formats as adjunct aids to the reading of Spanish texts by university students. Unpublished doctoral dissertation. University of Kansas, Lawrence.
Sakar, A., & Ercetin, G. (2005). Effectiveness of hypermedia annotations for foreign language reading. Journal of Computer Assisted Learning, 21 (1), 28-38.
Stark, H. (1990). What do readers do to pop-ups, and pop-ups do to readers? In R. McAleese & C. Green (Eds.), Hypertext: State of the art (pp. 2-9). Oxford, UK: Intellect Limited.
Stewart, R., & Cross, T. (1991). The effect of marginal glosses on reading comprehension and retention. Journal of Reading, 35 (1), 4-12.
Wolff, D. (1987). Some assumptions about second language text comprehension. Studies in Second Language Acquisition, 9 (3), 307-326.
Dr. Ali Farhan Munify AbuSeileek is a Jordanian assistant professor working in the Department of English Language in the College of Arts at King Saud University. He holds a Ph.D. in TEFL/CALL. He has published papers and designed several CALL programs for EFL learners. His major research interest is CALL and its application in EFL teaching and testing, machine translation, and CALL program development.
Dr. Ali Farhan Munify AbuSeileek
Department of English Language and Literature
College of Arts
King Saud University
P.O. Box 2456
Kingdom of Saudi Arabia
Phone: (office) + 0096614675437; (work) + 0096614675429; Mobile: + 00966551464049
P.O. Box 7456
Phone: 00962 5 3654041