My Lifelong Journey: DaRT: A CALL System to Help Students Practice and Develop Reasoning Skills in Choosing English Articles

Rika Yoshii
Alastair Milne
California State University, San Marcos

Abstract:
DaRT is a Computer Assisted Language Learning (CALL) system for helping English as a Second/Foreign Language students master English articles. The system uses a diagrammatic reasoning tool described in previous studies (Yoshii, 1997, 1998) to present communicative contexts for exercises in choosing appropriate articles. These diagrams make explicit the "intent-assumption" factors involved in article choice for many situations and permit students and teachers to discuss these factors without relying on English language explanations which depend on concepts foreign to most Asian students. DaRT can generate numerous exercise sentences, provide help based on student interactions, and choose the next exercise based on student performance. DaRT functions, therefore, as a tutorial to help students acquire the skills necessary for analyzing and applying the "intent-assumption" factors in choosing English articles. This article describes the major stages of the project, provides details of the system components, and concludes with the results of an evaluation of the system by Japanese students.

KEYWORDS

English as a Second/Foreign Language, Articles, Diagrams, Reasoning Tool, Pascal, LISP

INTRODUCTION

Many researchers and teachers declare that English articles (THE and A(N) and zero) are one of the most difficult aspects of English for English

121

as a Foreign/Second Language (ESL/ESL) students to learn, regardless of the student's native language or experience with English (e.g., Covitt, 1976; Little, 1980; Dulay, Burt, & Krashen, 1982; Rinnert & Hansen, 1986; Master, 1990).¹ Yet article misuse in formal situations can cause serious misunderstandings and stigmatize the speaker or author. This problem is particularly acute for Asian students since the concept of articles does not exist in Asian languages such as Japanese, Korean, or Chinese. Observing these problems led us first to find out what the Asian students' difficulties with articles actually are, then to design a reasoning tool to help them work out article choices, and finally to develop a Computer-Assisted Language Learning (CALL) system, entitled DaRT (Diagrammatic Reasoning Tool), to help them use the tool to develop their reasoning skills.

WHY ARTICLES ARE DIFFICULT

Review of the Literature

Master (1990) and Rinnert and Hansen (1986) stated that because articles are unstressed when spoken, are often omitted in certain styles, and the zero article is invisible in any case, it is difficult for students to acquire the system from simple exposure to the language or to realize the importance of using articles appropriately. The researchers also stated that very few ESL/EFL textbooks present systematic approaches or coherent grammars for teaching the article system. Covitt (1976), too, stated that the article system is not explained well in most textbooks and pointed out that teaching the system requires many contextualized examples. Our own survey of 16 randomly selected recent textbooks revealed no changes; all of the 16 textbooks devoted only a few pages to the article system, relied on grammatical terms such as "definite," "specific," and "identified," and gave very few examples. Sugamoto (1976) noted that students often choose articles based on the "viewpoint" of their native language. Little (1980) found in an oral survey that nearly all the students enrolled in the Intensive English Program at the University of California, Riverside, said they had received little or inadequate instruction and claimed that their teachers (both nonnative and native speakers) obviously had not really understood the article system very well.

Interviews and Questionnaires

The ESL instructors we interviewed at the University of California, Irvine (UCI) and California State University, San Marcos (CSUSM) reported that they do not have enough time to teach and practice the article system,

122

with its heavy dependence on context, in class. They complained that they must let the students graduate from their programs knowing that they will need many more realistic examples to be able to understand the system. A tool which would present many examples of many different situations would clearly help augment classroom instruction.

These teachers also complained that, when correcting article usage in student papers, they are obliged to rely on their own immediate assumptions of what students intended to say when they used articles without any knowledge of implicit context on which to rely. The teachers said they risk miscorrecting article usage and altering the meaning of the message that students intended to convey. Miscorrection of usage can only heighten student confusion and reinforce the impression that the teachers themselves do not understand the article system. The UCI instructors we interviewed mentioned that for these reasons the students refute the teachers' corrections with the result that the teachers feel that at least 25% of students' errors will never be eliminated. Here again, a tool which would help students and teachers discuss article choices unambiguously would facilitate student-teacher communication.

We also visited two ESL classes at UCI to interview the students about their experiences in learning the English article system. These students, the majority of whom were Asian, were regular students fulfilling their ESL requirements at the university. Our findings include the following:

1. The students stated that their teachers often explain their corrections as "it just sounds better that way" or "sometimes that would be OK" without listing any factors involved in their decisions.

2. All the students stated that it is easy to memorize the rules but difficult to decide which rule(s) to use and to explain their choice of a rule for each situation. They feel that rules with expressions such as "definite," "specific one," and "when identified" used to describe article-related concepts are too foreign to them, too ambiguous to be useful, and too vague to be applied to real-life situations.

3. Almost all the Japanese and Chinese students stated that since they do not have articles in their languages, they usually do not pay close attention to the articles in English and, thus, cannot accurately reproduce them from sentences they hear.

The results of the UCI interviews are echoed in the responses to a survey we conducted with 14 Japanese students studying at CSUSM and at surrounding community colleges. Eight of these students had been in the

123

US for more than a year and were taking regular university courses. The other six had been in the country less than six months and were enrolled in an ESL program. It should be pointed out here that the Japanese educational system starts teaching English, at the latest, in the seventh grade. All of these students had had at least six years of English instruction. The survey items and students' answers are listed below:

Item 1

You find the article system to be difficult (0 [no] -10 [yes] scale)

The mean of students' answers was 7.0, with 10 students choosing 7 or above. The mean among the newer students was 6.3. We speculate that this lower figure indicates an underestimation of the true difficulty of the system caused in part by a general lack of knowledge.

Item 2

Circle all that apply

a) It is difficult to memorize the rules circled by 4 students (2 older, 2 newer) b) It is difficult to apply the rules circled by 12 students (7 older, 5 newer) c) I don't understand why we really need the articles circled by 4 students (1 older, 3 newer) d) It takes time and thinking to select articles circled by 11 students (5 older, 6 newer) e) I have little or no trouble with articles not circled by any students

Item 3

For A versus THE: when/why is it difficult? Describe or explain. Student's verbatim answers follow. Please note that some of the responses are difficult to interpret.

How specific does it need to be to use THE is not clear. (older student)

example: "[] teacher who teaches computer classes also teaches English." (older student)

example: "[] dog which lives in my neighborhood" where "dog" was never mentioned before. (newer student)

When repeating the same noun; the difference between personal and public items. (newer student)

example: "a system of politics" vs. "the system of politics" what's the difference? (older student)

When to use THE is not clear at all (newer student)

124

Some nouns always take THE but others do not. Why? (older student)

It appears that the students do not yet understand even the simplest differences between A and THE. It is important to notice that the students did not provide enough context for their examples to make it possible to determine the correct articles. This situation could be due to their lack of understanding of what information is needed to make the choice and lead us to believe that they indeed memorized the rules but did not understand them.

Item 4

When you listen to a native speaker of English, do you pay attention to articles? (0 [no] -10 [yes] scale)

The mean of students' responses was 2.4 with 11 students choosing 2 or less. It is interesting to observe that although the newer students gave a lower rating on how difficult articles are; the mean of their responses was 1.6. Whether due to a lack of understanding of the article system or to the habits ingrained by their native language, the students virtually ignore the articles they hear in everyday conversations around them.

Based on the remarks from both the UCI and CSUSM students, we concluded that the reasons for article choices need to be made more explicit while, at the same time, avoiding explanations that assume that students already know what an article system is. We believed that by doing so, the articles themselves will be put into prominence.

PROPOSED SOLUTION

Following the review of the literature, the questionnaire, and the interview results above, we decided to develop a reasoning tool that would

1. make article-selecting factors explicit and accessible;

2. avoid using terms such as "definite," "specific," and "identified;" and

3. allow teachers and students to discuss article choices unambiguously.

To reach the first objective, we started with the speaker-listener dyad described by Brown (1973) and used by Celce-Murcia and Larsen-Freeman (1983).

125

To reach the second and third objectives, we decided to use schematic diagrams to present article-choice factors when providing the context for choosing correct articles. The article-choice factors in the speaker-listener dyad are a combination of the speaker's intent when referring to a referent and the speaker's assumptions about whether the listener will be able to identify that referent. The dyad therefore explains the concepts that terms such as "definite" and "specific" are meant to describe. We extended the dyad to cover plural nouns, generics, and nonspecifics and included as many speaker's intentions as we could identify from various textbooks and examples of actual English usage. Additional cases of speaker intent were based on analyses found in Burton- Roberts (1976), Master (1987, 1988b), and Celce-Murcia and Larsen- Freeman (1983). These "intent-assumption" factors provided a point of departure from which we could move on to cover other article-choice factors.

We then developed the DaRT system to allow the student to practice article selection using these diagrams. DaRT is designed to give students exposure to many different situations, individualized attention, and individually adapted sequences of exercises. Our goal was to enable students to develop skills for analyzing article-selection factors and thereby decrease their feeling that articles are vague or foreign to them.

DIAGRAM OVERVIEW

In describing the diagrams, we will refer to the speaker as Speaker and to the listener as Listener. The basic framework of the diagram consists of Speaker's head and Listener's head, nodes in a schematic display, and Speaker's sentence. Figure 1 shows the basic framework of the diagram.

Figure 1

Basic Speaker/Listener Framework

126

The diagram contains a larger Speaker's head with his/her view of a smaller Listener's head. Speaker's view of Listener is emphasized by placing Listener's head within Speaker's head. Within the heads, a graph of nodes with arrows connects the nodes to the noun phrase in Speaker's sentence displayed in a box below Speaker's head. (Figures 4 -9 below illustrate complete diagrams.)

In Speaker's head, the noun phrase's referent is indicated by an arrow from the phrase to the node denoting that referent. This referent always has the number 89 written on it (except in the case of a unique referent, such as "the earth" or "the sun"). The number 89 is a randomly selected number whose only significance is that it is consistently used to represent Speaker's referent and to distinguish this referent from other referents in Listener's head. If there is a possibility that Listener may have other referents in mind, those nodes appear with numbers other than 89. Intent-link icons are placed on the arrow from the noun phrase to the referent to indicate in what way the noun phrase refers to the node. Figure 2 lists the intent-link icons.

Figure 2

Intent-Link Icons

Notes:

IS-REP link is for speaking about a representative member of a class. ISALL-OF link is for speaking about all members of a class. IN-link is for speaking about the category of the referent group. IS-ANY-OF link is for speaking about any member of a class. IS-IN link is for stating a category.

All intent-link icons, except the IS-and IN-links are used for generic and nonspecific cases.

The way in which Speaker believes Listener can interpret the noun phrase is indicated in Listener's head. If Speaker assumes that Listener knows the referent, this same referent node also appears in Listener's head. If Speaker believes that Listener could confuse the referent with other referents, those other referents are shown as other nodes in Listener's head with an arrow from each back to Speaker's phrase and labeled by a CAN-BE-link icon (see Figure 3).

127

Figure 3

CAN-BE Link Icon

Where appropriate to the situation, Listener's head is divided into the current focus and long-term memory. The part of Listener's head that a specific Speaker intent does not consider can be grayed out, including the entire Listener's head when Speaker need not consider Listener's assumptions at all.

Figures 4 through 9 show examples of the diagrams for various situations.

Figure 4

Diagram for Introduction Situation

In Figure 4, the noun phrase "a student" refers to the referent node "student-89" in Speaker's head. Student-89 does not occur in Listener's head. Therefore the appropriate article is A.

128

Figure 5

Diagram for Ask-Awareness Situation

In Figure 5, the noun phrase "a big dog" refers to the referent node "dog89" in Speaker. The node "?dog-89?" in Listener's head depicts Speaker's wondering whether Listener is aware of dog-89. Thus, the correct article is A.

Figure 6

Diagram for In-Current-Focus Situation

In Figure 6, the noun phrase "the car" refers to the referent node "car-89" in Speaker's head. This node is in Listener's current focus, and there are

129

no other nodes in Listener's current focus that could be confused with "car-89." The Speaker is not considering Listener's long-term memory. Therefore, the article is THE.

Figure 7

Diagram for Entailment Situation

In Figure 7, the noun phrase "the engine" refers to the referent node "engine-89" which is entailed by "car-89." Since the node "car-89" is in Listener's current focus, "engine-89" is automatically in Listener's current focus. Therefore, the article is THE.

Figure 8

Diagram for Categorization Situation

130

In Figure 8, the noun phrase "students in my class" refers to the referent group node "students-89." The arrow from this node to the class node "students" indicates that the group "students-89" belongs to the class "students in my class." Since the intent here is the categorization of the group, the article is zero.

Figure 9

Diagram for Case of Generic Article A

In Figure 9, the noun phrase "a lion" refers to the class node lions with the link labeled as IS-REP. Therefore, "a lion" is being used as a representative of all members of the class. For all generic and nonspecific cases, a single class node is shared by Speaker and Listener. The article is A.

DaRT EXERCISES

The aim of the CALL exercises on articles is for students to pretend to be the speaker and choose the correct article for the situation depicted by the diagram. The exercises are not quiz problems, giving only reports of right or wrong. DaRT is a tutor which gives help in interactive sessions to train students to analyze the factors depicted in the diagrams. The important features of DaRT's exercises are listed below:

1. Pedagogical control is based on student interactions The system looks at the history of student interactions to decide what "intent-assumption" situation to present next. The

131

basic philosophy behind the pedagogical control is to allow students to demonstrate understanding of a given "intent-assumption" situation by repeating an exercise for that situation with different sentences.

2. Feedback is based on student interactions When wrong choices are made, a hint text is given based on which incorrect choice was made and the number of times that choice was made. The actual number of times available for each choice is a consequence of the pedagogical design of the exercise. These hint texts are designed to guide students to discover the correct answer for themselves. The system does not give the correct answer unless students exhaust all available hints for a given wrong answer. Also, if students make the correct choice, the system confirms their choice and states the reason why it is correct. This kind of feedback makes sure that even if students choose the correct answer by chance, they are still exposed to the reason their choice is correct.

3. Numerous sentences are generated by the program The system does not have a fixed set of preestablished sentences; it creates new sentences by recombining words in its dictionary. Students are not likely to encounter the same sentence for a long time and can keep on using the system for as long as they wish.

4. Recording the Student Interactions To help teachers review how students are doing and to aid in formative evaluation of the software, DaRT records students' interactions into a new log file each time they use the system. The log contains information on which exercises students tried, which answers they selected, and how many trials they attempted before they got the correct answer.

Outline of a DaRT Exercise

The list below shows how the program works.

1. The system draws a diagram for an "intent-assumption" situation and creates the sentence whose meaning is described by the diagram. The sentence contains a blank space where the article is to be filled in.

132

2. The system draws a palette of article icons from which students select what they believe to be the correct article.

3. The system directs students to choose the article that they think will complete the sentence.

4. If students click on the correct icon, the system puts the article in the sentence, confirms that the answer is correct, and displays the reason that the article is correct.

5. If the icon clicked on by students is not correct, the system displays a hint text and directs students to try again. The first hint text for a wrong answer points out Speaker's intent as shown by the diagram and the sentence. ("Trivial" errors such as A vs. AN or singular vs. plural are not taken into account.) Subsequent hints for that wrong answer, if selected again, point out the part(s) of the diagram that students do not seem to be considering.

6. If the limit of trials for any one incorrect choice is reached, the system gives the correct answer, fills in the blank in the sentence with the article, and displays the reason the article is the correct choice. Students can then choose either to move on to the next exercise or to quit the program.

All texts remain displayed on the screen until students either try the exercise again or go on to the next exercise. Figure 10 shows a sequence in which the student ultimately selects the correct article after getting hints for selecting wrong articles.

Figure 10

Getting the Correct Answer

133

134

Figure 11 shows a sequence in which the student reaches the limit of selecting incorrect answers and the program eventually displays the correct answer.

Figure 11

Reaching the Limit on Incorrect Answers

135

136

SYSTEM OVERVIEW

The DaRT system is composed of two main parts: a processing mechanism written in Macintosh Common LISP and a graphical user interface written in THINK Pascal—a dialect of object-oriented Pascal. We decided to use two different programming languages for two major reasons: (a) our original prototype with hard coded exercises was written entirely in Pascal, and we wanted to take advantage of the user interface parts; and (b) LISP, a language favored for Artificial Intelligence programming, has powerful capabilities for generating sentences and intelligently analyzing students' answers. The two parts run simultaneously and become alternately active. The active part signals the waiting part when that part is to become active again and passes along relevant information. The design and development of the system follows current software engineering principles, including the use of object libraries and separation of data from control, in order to achieve demonstrable correctness and a high degree of maintainability and extensibility.

SYSTEM COMPONENTS

The Pedagogical Controller

This component determines the overall sequence of events. It chooses an "intent-assumption" situation for an exercise and invokes each of the components that follow. It also determines the subsequent step for students based on their history of interaction. Under the current pedagogical design, if the correct choice is made on the first try, the next exercise will either use a different "intent-assumption" situation or a more challenging version of the same situation, if available. However, if the correct choice requires more than one trial or is in fact never entered, the next exercise will continue to work on the same "intent-assumption" situation with a different sentence.

The Sentence Generator

This component generates new sentences. Given an "intent-assumption" situation, it randomly obtains one of a list of sentence templates already provided for the situation and randomly picks a noun for the noun phrase (see Figure 12).

137

Figure 12

Sample Sentence Templates for INTRODUCTION situation

The sentence generator then uses the noun to look up an entry in DaRT's dictionary, which contains sets of fillers (words and phrases prechosen to match the meaning of the noun) for all possible template components (see Figure 13).

Figure 13 Sample Dictionary Entry for "student" and "students"

138

Each template component is filled in by randomly choosing a filler from among these sets (see Figure 14).

Figure 14

Filling a Template

The Diagram Generator

This component produces internal representations of diagrams. Given an "intent-assumption" situation and a noun phrase, it picks a diagram template for the situation and augments it with information to match the noun phrase. These representations are used by the user interface to create the actual diagrams to be displayed. Each internal representation includes the following information:

• the properties of the referent node;

• the type of the intent link;

• whether the listener knows the referent or not;

• whether there are other nodes the listener is believed to know, and whether there should be CAN-BE links from any of them;

• when appropriate, what part of the listener's head should be grayed out; and

• labels for the nodes, matching the noun phrase.

Text preceded by double semicolons (;;) annotates the actual templates in LISP (see Figure 15).

139

Figure 15

Sample Internal Representation of Diagram

140

The Answer Analyzer

This component receives the answer selected by students, the correct answer, and (for incorrect answers) the number of times this incorrect answer was entered. It uses a set of rules to determine which hint or confirmation text to give (see Figure 16).

Figure 16

Sample Answer Analysis Rules

Some rules deduce the part of the diagram or sentence that students do not seem to be considering. It creates texts for hints and the confirmation of the correct answer by putting together generic text templates chosen according to whether the analyzer is pointing out a lack of attention to diagram components, to the speaker's intent, or both. The use of generic display texts, made specific to the situation at the time of display, obviates the need to pregenerate or prestore display texts for all possible situations and sentences.

The User Interface

The User Interface is responsible for all displays to students and for interpreting all student actions back to the answer analyzer. This process entails

141

• prominently displaying the speaker's sentence;

• constructing the diagram display for that sentence;

• drawing the intent link and any assumption links between the sentence and the diagram;

• displaying the hint texts, directions, and confirmation of the correct answer given by the Pedagogical Controller;

• creating the icon selection palette from which students choose articles;

• enabling the palette for selection of those phrases from which selection is to be made;

• disabling the palette once selection has been made; and

• timing the interaction with students.

Display of the Speaker's Sentence and Links to the Diagram

The user interface receives Speaker's sentence for each exercise from the Pedagogical Controller. The sentence is received in four parts: the segment before the article (if any), the article itself, the noun phrase, and the segment following the noun phrase (if any). This composite string permits the interface to build the full sentence for display while knowing the functions and screen locations of all its parts and without having to depend on a natural language parsing of its own. This process is necessary for such functions as replacing the article with an underscore, correctly placing all links between the noun phrase and the diagram, and filling in and highlighting the article. For most of the exercise, the article is replaced by an underline in the display, which is set to be longer than the longest article to avoid giving a hint about which article is correct. When the article is filled in, the user interface resizes the sentence and rechecks the positioning of the links. The visual positioning of the sentence, with the diagram centered over it, is the basis for positioning all other parts of the display.

Construction of the Diagram Display and Creation of Display Texts

The diagram information that the user interface receives, as described in the Diagram Generator section above, includes the essential parts that the diagram must show for the situation but not the actual graphical data or screen positioning. In this way, the interface is left free to assemble the specified diagram by whatever methods are practical and appropriate to it and to refine those methods during development while still meeting the Controller's requirements. Furthermore, improvements to the screen design and the aesthetics of the diagrams themselves involve changes only to the interface. The Macintosh concept of "resources" has been a powerful

142

tool for designing and creating data for diagram construction.

The user interface additionally receives text from the Controller to match the noun phrase in the sentence. This text is needed for two essential purposes: to create the various labels for all the nodes the interface will draw in the diagram and to complete the display texts, which, when provided by the Controller, still contain references to generic sentence components in a format the user interface can easily find and replace.

Displaying, Enabling, and Disabling the Article-Choice Icon Palette

A palette of icons, which represent the four possible entries for the omitted article and on which students click to make an article choice, is drawn at the left. As soon as the directions to make a selection have been given to the student, the icon palette is enabled to receive mouse clicks. As soon as a choice has been made, the palette is again disabled. This disabling prevents students from accidentally selecting more than one choice and avoids any resulting confusion.

Timing Control

The user interface is responsible for all aspects of time control in student interactions. In this version, no time constraint is placed on students. They may spend as much time as desired reading the hint text or confirmation text and relating the text to the situation described on the screen. The system never moves ahead to the next phase of an exercise or to a new exercise until students direct it to do so.

Extensibility of the System

The system's components are designed to allow future incorporation of additional "intent-assumption" situations into the diagrams. All data for sentence-and diagram-dependent features are kept separate from DaRT's general processing mechanisms. For most changes, only data files need to be updated. No change to the structure of the program is required. For example, adding more sentences only requires the addition of new templates to the data file storing the templates and new words to the dictionary. Handling more "intent-assumption" situations only requires the addition of the internal representations of their diagrams to the data file and intent texts to the file of response texts.

143

PILOT STUDY

We conducted a pilot study to assess student reactions to DaRT's instructional design and the user interface features. The results of this assessment will be used to revise and refine the system.

Eleven students volunteered to participate in the pilot study, five of whom had also participated in the survey described earlier. Of the eleven students, six were regular college or university students who had been in the U.S. for longer than a year, while the others were enrolled only in the ESL program at CSUSM. We had no previous connection with any of the students and felt confident we would receive unbiased reactions to the system. At least four days before they used the system, the students were given a training document describing the diagram notation and summarizing its relation to article selection. The document was written in Japanese since we were not sure of the students' proficiency in English. Before the students used the system, they were allowed to ask questions about the document so they could start with a clear understanding of the diagram notation. They were then asked to work through the exercises for at least an hour. Most students spent approximately one hour on the exercises, but one student came on two different days and spent a total of two hours on the exercises. After they finished the exercises, they completed a rather lengthy evaluation questionnaire. (See the DaRT Project Questionnaire in the appendix to this article.) Some students also made additional oral comments. Data from each student were identified with individual codes for correlation purposes and to avoid any personal influence on data interpretation by the authors.

All students rated the usefulness of the DaRT system at 4 or 5 on a five- point scale (5 being the most useful). All but one student said a combination of traditional lectures and DaRT would be most effective. This student indicated that the traditional approach alone would be most effective despite the fact that she rated the usefulness of the system as a 5. She also indicated that she did not like computer technology in general.

When asked what they had learned by using DaRT, three students said they learned virtually everything they knew about when to use A versus THE. All they had understood previously was that A should be used first and THE thereafter. Other students reported learning

1. the importance of speaker assumptions about the listener (5 students);

2. situations of re-enacting a discovery, as in "remember, we found a car that was too big for me?" (5 students);

3. Generic-THE as in "the automobile changed our lives" (1 student); and

144

4. The differences in meaning between two such statements as "A lion is dangerous" versus "Lions are dangerous" (2 students).

It is possible, of course, that the students learned more than they described in their responses. We recognize that it is not reasonable to expect students to be able to recall every detail of what they learn.

The students felt that the visual presentation of the context and the opportunity to repeat many different exercises as much as they wanted were the major reasons for DaRT's utility. At least three students said these features will continue to help them retain what they learned. The time they estimated for being able to answer an exercise question correctly varied from 15 minutes to several hours. As described below, all students correctly answered all exercises on the first attempt by the time they approached the end of their sessions with the system.

In answering which exercises were most difficult for them, four students said they were all equally easy. Five students said generic and nonspecific situations, in which a class node is shared by the speaker and the listener, were the most difficult. Two students found the indefinite plural situations with the IN link between the referent node and its class to be the most difficult. In answering which diagram notations needed improvement, three mentioned the graying out of Listener's head (when assumptions about the listener are not involved), two mentioned highlighting a node to show that it is prominent for the listener, and two mentioned the IN link for categorization of a group. Of the intent-link icons, six students said IS-ANY-OF and IS-IN icons were confusing and that they could not tell the difference between the meanings these icons represented.

In reviewing the logs of their interactions, we found that most of the errors were made during the first half of their sessions and that almost all answers were correctly chosen on the first try in the second half. Almost all the cases in which the students could not answer a question on the first try were for nonspecific A, generic A, generic zero, categorization of a group, and "prominent for the listener" situations. We found only five cases in which students chose an incorrect answer two consecutive times in an exercise. These incorrect answers were for nonspecific A, generic A, and "prominent for the listener" situations. Therefore, the students' own assessments of what was confusing or difficult matched the logs of their interactions.

Concerning the best language for the training document and the on- screen texts, there was little agreement among the students. Those who wanted an English training document with Japanese texts on the screen stated that English sounds more natural for explaining concepts which occur only in English but that when explanations appear in English on the screen, it takes them time to read and understand the text. However, all

145

except one student rated the hints and explanations to be useful (4 or 5 on the five-point scale).

Students' major complaints focused on the link icons for generic and nonspecific cases. They felt that they spent more time trying to understand the meaning of the icons than in actually applying the concepts. Other concerns included the tendency to ignore the sentences and look only at the diagrams. Two students said that the ease of analyzing the diagrams made them fear that they may not be able to readily apply what they learned to real-life situations. They suggested the importance of follow-up role-playing exercises in classrooms.

For additional features to be added to the system, six students wanted balloon help on the intent-link icons to review their meanings, five students wanted balloon help on other parts of the diagrams, and seven students wanted a summary of how well they did. They also requested that the prompt and other displayed texts be darker and in a larger font.

We are pleased with how quickly the students learned the "intent-assumption" factors which had previously escaped them despite their many years of English instruction. We are also pleased with the evidence of the effectiveness of the diagrams in presenting these factors and of the DaRT system's ability to help the students quickly get used to these factors in a variety of situations.

CONCLUSION AND FUTURE WORK

The DaRT system with its diagrammatic reasoning tool addresses a long recognized and pervasive problem for ESL/EFL students whose first language does not have articles. It

1. provides students and their teachers with a tool to discuss article usage without relying on English terms or concepts that are foreign to most students;

2. makes visually explicit the "intent-assumption" factors in choosing English articles;

3. gives students a large corpus of sentences and situations in which they can practice article selection and accumulate appropriate experience in the language;

4. delivers continual instruction on the "intent-assumption" factors as hints and explanations; and

5. allows students to proceed at their own pace in an individualized sequences of exercises.

Based on the student evaluation results, we plan to include help balloons to explain diagram components, design better IS-IN and IS-ANYOF intent-link icons, find a better way to depict categorization of groups,

146

improve the display of node prominence, and use darker and bigger fonts for hint and explanation texts. Since the students could not agree on the language for the training document and texts on the computer screen, we believe we should produce versions in English and the students' native languages. To address student concerns about tending to pay too little attention to the exercise sentences, we plan to display the sentence first and then add the diagram later when students click on a "Give me context" button. We also recognize the DaRT alone cannot guide students to complete mastery of the article system; follow-up role-playing exercises are essential.

We also recognize a current limitation of DaRT's pedagogical controller in selecting a sequence of exercises. We plan to extend it to remember student performance for a sequence of exercises, instead of the preceding exercise only, and to use this information to select the next exercise.

Our next step is to conduct an experiment with ESL classes at CSUSM to assess the effectiveness of the DaRT system in helping students use English articles correctly in increasingly natural contexts. In extending the diagrams, we hope to cover the instructional THE cases and the use of articles with noncount nouns. The latter causes special problems for Japanese and Chinese students, whose native concepts of countability differ widely from those of English (Sugamoto, 1976).

147

APPENDIX

DaRT Project Questionnaire

BACKGROUND

1. Last 4 digits of your ID Number:

2. How long did you use the system? (total hours)

How much time did you spend on the article selection exercises?

3. User perception (1 = strongly disagree; 5 = strongly agree)

(1 2 3 4 5) a. I enjoy using computer technology.

(1 2 3 4 5) b. I enjoy learning about the English article system.

(1 2 3 4 5) c. I believe computer technology is useful for learning English.

4. PEDAGOGY

4a. Which diagrams/situations were most difficult for you?

(Rank order the following, 1 = most difficult.)

when the 89 node in the listener's head had ? ? around it.

when the arrow in the speaker's head went to the IN-link icon.

when the 89 node was in the listener's head, and no other node in the listener's head had CAN-BE links.

when the 89 node was in the listener's head, but there were other nodes in the listener's head with CAN-BE links.

when the 89 node was not in the listener's head. when the speaker and the listener shared a class node.

Other (specify)

148

4b. For the one you marked as most difficult in Question 4a, please explain in what ways it was difficult or confusing.

5. Diagram suggestions:

5a. How effectively did each icon remind you of the relationship it stands for? (1 = did not help me remember; 5 = reminded me right away)

IS (1 23 4 5)