[article | discuss (0) | ]
Defense Language Institute
This article is a revised version of a keynote address given by the author at CALICO '98, the fifteenth annual CALICO symposium, in San Diego, California in July 1998. CALICO wishes to thank Dr. Clifford for his keynote address at CALICO '98 and his permission to publish it here. The opinions expressed in this article are those of the author and do not necessarily represent those of the Defense Language Institute Foreign Language Center.
INTRODUCTION Much has transpired since I last addressed a CALICO conference, and the progress we have made in this field is truly astounding. However, the advances we have made in technology have not been matched by equivalent progress in the sophistication of our courseware. On the contrary, many of the current computer delivered language programs seem to be based on one or more of the following misperceptions:
1. language is a simple phenomenon;
2. language acquisition is a simple task;
3. presentation of information is all that is required for language learning;
4. access to information eliminates the need for presentations;
5. every learning activity is appropriate for all learners;
6. automating poor teaching practices will improve the instructional process;
7. poorly designed “shovelware” is better than well designed, but costly, learning activities.
I doubt that many of us would consciously agree with these assumptions, but they have apparently become default options in the design and development of most Computer Assisted Language Learning (CALL) programs. As a result, a disparity exists between our announced instructional goals and the true capabilities of our courseware. It is interesting to note
this disparity has neither dampened our enthusiasm for the potential benefits of applying technology to language problems nor restricted the claims of many developers. For instance, one advertisement appearing in several software catalogues promises to deliver (for less than $100!) a “universal translator” that will operate on your personal computer. The fine print includes a few caveats, but the advertised capabilities greatly exceed those reported in the professional literature on machine translation.
While enthusiasm is often beneficial, unrealistic claims can lead to overstatements of our capabilities—overstatements and claims that damage the field's credibility and reduce support from students, administrators, and the public in general. The detrimental impact of unfulfilled expectations in translation programs can have implications for instructional programs as well. From the perspective of a consumer, learners might reason, “Since computers can't translate language, they must not understand language, and I don't want to learn from a teacher who doesn't understand the subject matter.” From a development perspective, our inability to parse students' input with confidence is a major hindrance in the design of student feedback routines.
SEVEN AXIOMS FOR CALL DEVELOPMENT
In the midst of the excitement about CALL, it is appropriate that we occasionally remind ourselves of the significance of the challenges we still face. The following seven axioms are presented in the hope that they will lead to improved CALL programming and more realistic statements of expected learning outcomes.
Axiom 1: Language is not a simple phenomenon.
Even with massive computing capabilities, the challenge of machine translation is daunting. Baker, et al. (1994) describe this challenge as coping with ambiguity. In a test of disambiguation methods, they reported that after applying one level of disambiguation to narrative text, the average number of possible “parses” per sentence was 27.0. Even after applying six levels of disambiguation, the average number of possible parses per sentence was still 1.5. In an article on a machine translation project at the University of Helsinki, which had received financial support from international businesses and the European Union, Bollag (1997, p. A22) declared, “It is this ambiguity, inherent to all human speech, that has humbled ... [those] working in computational linguistics. Their ultimate goal—teaching computers to translate automatically from one language to another— seems more elusive than ever, and they have had to settle for less ambitious objectives.”
The enthusiastic overstatement of our computers' ability to deal with language communications is not a new phenomenon. With great confidence the newsletter article “Computer-Aided Translations” (1984, p. 2) boldly announced
Automatic translation of the spoken voice is on the way [original emphasis]. Your words are translated automatically, on the fly, by phone circuitry you pay for by the message unit. The year? About 1994, according to industry estimates.
This prediction was obviously made by the communications industry, not by experts in machine translation.
For most of us, the complexity of language often goes as unnoticed as the air we breathe; we are only reminded of its importance when communication efforts fail. These communication failures may be minor or important depending on the significance of the communication task involved. We only smile when we encounter a sign such as the one purportedly discovered in a Tokyo hotel with the message, “We invite you to take advantage of the chambermaid.” Much more serious would be interpreting a crime witness's statement that the accused “thought nothing of it” as “did not think about it at all” (Hewitt & Lee, 1996, p. 29). Rather than being a simple phenomenon, language is the most complex of human activities. The fact that we discover new concepts and cultural perspectives as we learn another language that are accessible only through that language supports the conclusion that language not only describes but also defines our reality.
Axiom 2: Language acquisition is not a simple task.
There is obviously much more to learn about language acquisition than we currently know. However, we do understand enough to know that successful language learning programs satisfy a set of explicit principles. The two lists below of curricular and instructional requirements for successful second language acquisition offers a reference against which we can assess the adequacy of CALL courseware.
The major curricular requirements for successful language acquisition relate to the scope, sequence, and length of the teaching program and include
• a curriculum that presents the language in communicative settings;
• a corpus of language that continually expands in task, discourse, and cultural complexity as learners progress in their abilities;
• a course design that simultaneously increases the number of topical domains introduced, while re-entering familiar topics with greater depth and precision;
• an instructional sequence that extends over a sufficient length of time for learners to acquire the cognitive understanding, psychomotor skills, and cultural awareness needed to accurately communicate facts, concepts, and feelings with members of another society.
The major teaching and learning requirements for effective language acquisition relate to the effective combination of subject matter, learner, and pedagogical variables. These requirements include
• interactive communication practice that requires learners to comprehend and produce the target language;
• assistance with input comprehension and timely, formative feedback to learners about their language production;
• individualized feedback to learners that recognizes and applies differential performance and accuracy expectations depending on each language component's degree of difficulty and individual learners' aptitude, motivation, and prior experience;
• presentation of new language content and concepts at a pace that is appropriate for learners and for the difficulty of the material to be mastered;
• modes of instruction that recognize individual learners' learning styles and their desire for expansion, enhancement, and enrichment of the initial target language input;
• learning activities that build from controlled communication exchanges to unpredictable, real-world language use.
Axiom 3: Presentation of information is not all that is required for language learning.
In projects in which the objective of the courseware is to replace a human delivering a lecture, the development task is certainly simplified, but humans do not efficiently acquire second language skills from lectures alone. The January 1998 issue of Syllabus, which carried the subtitle Video and Presentation Technologies, is illustrative of the current state of computer delivered instruction. The emphasis throughout the issue is on oneway teaching or presentations with only limited learner participation.
An important corollary to this axiom is that it is not the level of the language presented that determines the level of instruction for learning
exercises; rather, it is the level of response expected from learners that determines the level of instruction. For instance, presenting a video or audio clip of a news broadcast could be an appropriate communicative stimulus for a learning activity at the American Council on Teaching of Foreign Languages (ACTFL) Advanced proficiency level, but if learners' responses are restricted to a lower level activity such as recognizing the main topic of the broadcast, then the effective instructional level of the exercise is reduced to the ACTFL Intermediate or even Novice proficiency level.
At a recent symposium on Assessing and Advancing Technology Options for Language Learning at the University of Hawaii, a call was issued for the evaluation of multimedia CALL based on hypotheses about ideal conditions for second language acquisition (see Chapelle, 1998). In attempts to develop software under these conditions, the two greatest challenges facing courseware developers have been to provide interactive communicative practice and to generate formative feedback based on learners' communication efforts. In programs in which communicative practice and feedback have not been considered by CALL developers, their absence is the programs' most serious deficiency.
It is clear that the learning requirements for second language acquisition exceed those of strictly cognitive disciplines. It is precisely the higher demands of language instruction that led me to conclude when working with public school teachers in Minnesota in 1976 that computers will not replace teachers, but teachers who use computers will replace teachers who do not.
Axiom 4: Access to information does not eliminate the need for adequate presentation.
Just as presentations do not eliminate the need for interaction and feedback, access to information does not eliminate the need for effective organization and presentation of that information. Few, if any, language teachers would tell their students that they should skip class to sign onto the Internet. However, teachers' enthusiasm for this new source of authentic language materials, coupled with students' often naive ideas about language learning, can lead to unwarranted conclusions. A well intentioned idea proposed to me recently provides an example of such errant reasoning. Reduced to its essential elements, the argument was
since: “the best way” to learn a foreign language is to live in the country, and the Internet has made all countries accessible to anyone with a computer
therefore: we could save a lot of money by giving students access to the Internet and closing the Defense Language Institute.
The Internet has been compared to both a worldwide electronic landfill and a worldwide library. I prefer the second comparison, but nonetheless significant differences exist between the Internet and a library. The Internet does not have a quality review process that selects its holdings. It is because of the lack of such a review process that the best search engines on the Internet are not as productive for research and individual learning as the on-line card catalogue in a library. The capabilities of the Internet will surely increase, but just as libraries did not eliminate the need for teachers, neither will the Internet. Even for situations in which teaching would only require the presentation of information, the functions performed by the teacher in organizing and sequencing information is a valuable service that greatly enhances the efficiency of the learning process.
A more productive option is to use the Internet as a delivery mode for teacher prepared instruction. But this approach is not without its own set of challenges. Slatin and Sharir's (1996, p. 20) report on a course taught at the University of Texas using the Internet is generally positive, but it also recognizes “the principal limitations of the World Wide Web as an environment for learning and teaching can be summed up in two phrases: limited bandwidth and limited interactivity.” They go on to explain that “The bandwidth issue is not, strictly speaking, inherent to the Web, and it will be addressed (and deferred) by continuing changes in the global telecommunications infrastructure. But the limitation on interactivity is integral to the Web, and a far more serious problem for art and education.”
Axiom 5: Not every learning activity is appropriate for all learners.
While the validity of this axiom is apparent even to inexperienced teachers, it would seem that the design of most courseware is determined more by the author's own teaching preferences or by the capabilities of the technology than by learners' needs. CALL's shortcomings in this area may be related to the historical development of the field. At first, the profession's expectations for automated learning activities were limited to computerdelivered versions of teacher-developed worksheets. Now that developers have moved to true courseware development efforts, our expectations should be elevated to match the expectations of professional instructional materials. At a minimum, the selection of courseware learning activities should be based on a thoughtful consideration of course objectives, learner characteristics, and the instructional steps needed to bring learners from their current level of abilities to a mastery of the course objectives. If the author chooses to address a specific type of learner rather than providing a range of alternative learning activities, then that decision should be clearly communicated in the program's descriptive materials.
A simple example of product differentiation for different learners can
be found in the toy industry's practice of including labels for suggested age ranges on its packaging materials. Most people readily understand these guides and find them useful. Every CALL program developer has an audience in mind, but that information is seldom communicated to potential users. Creating such a system for CALL will require some thought because the categorization of descriptors must be understandable to those inside and outside the field. Given the inherent ambiguity of language, this problem may be more difficult than one might imagine. The toy industry's age-based rating system is relatively self-explanatory, but you may have heard about the college student who bragged to his friends that he had assembled a picture puzzle in less than an hour. He was especially proud of his accomplishment because the puzzle had come in a box that said “5 to 6 years.” With CALL programs, the age of learners is not the major factor to be considered in categorizing instruction. CALL programs should provide suitability guidelines for learners with different proficiency levels, performance profiles, language backgrounds, and learning styles.
Axiom 6: Automating poor teaching practices will not improve the instructional process.
Learning activities that provide a good match between learner characteristics and lesson objectives are difficult to create, but, once created, they retain their instructional effectiveness when transferred to a computer delivery format. The same is not true for mediocre teaching. In fact, computerizing poor teaching invariably further reduces the effectiveness of these learning activities. Developing excellent CALL courseware requires expertise in the language, instructional technology, and language pedagogy. This combination of skills may occasionally exist within a single person, but, generally, instructionally effective courseware is the result of a team effort that brings together language, teaching, and programming skills.
Axiom 7: Poorly designed “shovelware” is not better than well designed, but costly, learning activities.
The pressure on CALL faculty to produce visible products is multifaceted and pervasive. Administrators may need to show a return on investment for new computer laboratories, students may be demanding courses that are “up-to-date,” and tenure review procedures may foster a philosophy of “more is better.” Regardless of the underlying motivational factors, the creation of automatic page turners for large volumes of text, audio, or video material invariably violates more than one of the axioms listed here.
Recognizing the need to establish quality expectations for computer projects in the humanities, the Modern Language Association (MLA) sponsored the creation of “Guidelines for Evaluating Computer-Related Work in the Modern Languages.” The preface to these guidelines (1997, p. 50) states, “faculty members who pursue computer-related work as part of their formal assignments should be prepared to make explicit the results, theoretical basis, and intellectual rigor of their work, as well as its relevance to the discipline.”
The application of the MLA guidelines would certainly improve the quality of CALL products within the academic community, but we need more than adherence to these guidelines. Just as every CALL courseware product should be accompanied by a list of minimum hardware and software requirements, it should also include information about course objectives, the students for whom the program was designed, the kinds of learning activities included, and the nature of feedback provided to the learner. In this regard, the National Foreign Language Resource Center at the University of Hawaii is developing an evaluation schema that could lead to improved clarity of CALL capabilities and objectives (see http:// nts.lll.hawaii.edu/flmedia/evaluation/general/gencriteria.htm).
The enthusiasm of those working in the CALL field is both a great asset and a potential liability. Without this enthusiasm, we would not have made the progress that is so evident in the presentations and displays at this conference. However, the challenges associated with teaching human communication will not be solved by enthusiasm alone. In the face of increasing pressures to do everything “better, faster, and cheaper,” we must not let our enthusiasm mislead either ourselves or our students. To this end, it will be necessary for us to describe with greater clarity and precision the capabilities of our programs. The following sample questions should be addressed as minimal parameters in the design or descriptions of CALL programs.
1. CHARACTERISTICS OF THE TARGET AUDIENCE For which age group(s) is the program intended? Should users of the program have already had any particular language experiences or mastered any specific language skills?
2. COURSE AND LESSON OBJECTIVES Is the program oriented toward the development of achievement, performance, or proficiency skills? If the focus is on the develop-
ment of supporting skills such as pronunciation, vocabulary, or grammar, what are the skills to be taught? If the focus is on achievement, what is the content of the lesson(s)? If the objective is to teach performance skills such as gisting, transcribing, or translating, what skills are to be learned under which topical domains at what level of difficulty? If the objectives are proficiency-based, what is the proficiency level of the communication tasks presented to learners?
3. STUDENT LEARNING ACTIVITIES Do learners only have to recognize correct answers in a multiple choice format? Are learners expected to produce correct forms or short answers? Are learners expected to communicate concepts using rehearsed or formulaic language? Are learners expected to communicate using language in new, unrehearsed settings?
4. FEEDBACK TO LEARNERS Is answer judging limited to right or wrong? Must learners correct their own work by comparing their responses to a model response? Does the program parse learners' language input and highlight irregularities? Is each irregularity identified by the program marked with a hot link or branching option for help or remedial activities? Does the program provide a natural response to learners' communication attempts and then coach them on how to improve the accuracy of their communication?
5. LENGTH OF THE PROGRAM What is the time it takes the average learner to complete the lesson? How long does it take the average learner to master each lesson's objectives? Of the time spent, how much is devoted to receiving instruction, to practicing language, and to accomplishing real-world tasks in the target language?
If we address these key instructional variables, we and our students will be better able to assess the value of CALL courseware. We will be able to differentiate automated worksheets from lessons and lessons from courses. We will be able to apply CALL programs to those situations where there is the greatest need and the best chance of success. Most important, we will have created our own “truth in advertising” standards.
Baker, K. L., Franz, A. M., Jordan, P. W., Mitamura, T., & Nyberg, E. H. (1994). Coping with ambiguity in a large-scale machine translation system. Computational Linguistics, 1, 90-94.
Bollag, B. (1997, May 16). Advances may allow computers to become translators someday. The Chronicle of Higher Education, pp. A22-A23.
Chapelle, C. A. (1998, March). Multimedia CALL: Lessons to be learned from research on Instructed SLA. Paper presented at the conference on Assessing and Advancing Technology Options for Language Learning, the University of Hawaii, National Foreign Language Resource Center.
Computer-aided translations. (1994, August). Science and Technology for the Executive, pp. 1-2.
Guidelines for evaluating computer-related work in the modern languages. (1997). ADFL Bulletin, 28 (3), 50-51.
Hewitt, W. E. & Lee, R. J. (1996). Behind the language barrier, or `You say you were eating an orange?' State Court Journal, 20 (1), 23-31.
Slatin, J. M., & Sharir, Y. (1996). Multimedia in cyberspace: Teaching with virtual reality. Syllabus, 10 (3), 16-20.
Ray Clifford (Ph.D., University of Minnesota) is Provost of the Defense Language Institute Foreign Language Division. He publishes regularly in leading journals and has given over 100 papers and professional workshops. He has participated in several national foreign language task forces and is past president of the American Council on the Teaching of Foreign Languages.
Dr. Ray Clifford, Provost
Defense Language Institute Foreign Language Division
Presidio of Monterey, CA 93944-5006