August 11, 2000 (Preliminary Report)

9.  Results and Discussion
Tables 2 and 3 summarize the numbers of errors made by each of the 20 subjects in both Control and Experiment Groups for Tests 1 through 7. These two tables are converted into a 3-D graphics as in Figure 1-1 for the Control Group and Figure 1-2 for the Experiment Group respectively. These two graphics clearly show that errors in the Experiment Group are consistently fewer than in the Control Group, except for Test 1.

Figure 2 compares the mean numbers of errors between the two groups for Tests 1 through 7. Before interpreting these results, Fisher's F test was conducted for Test 1 to see whether the two groups, the Control and Experiment Groups, can be said to belong to the same mother population. The test yielded an F value of 1.5424 (df=19, 19), indicating that the difference between the observed sample variances are not statistically significant at p = 0.05 and that the two sets of samples are most likely to have been derived from the same mother population.

Having thus confirmed the statistical homogeneity of the two sample populations, the t-test  was then conducted for all the seven assignments. As shown in Table 1, the observed differences in the mean error scores between the two groups are found statistically significant at p < 0.05 for Tests 2 through 7 consistently, confirming out initial hypothesis that the use of the BLC Concordancer was actually quite effective in reducing the number of surface errors -- as far as the two target areas are concerned.

It is also noteworthy that the "full and explicit error correction" given to the Control Group did not make any notable contribution to reducing the number of errors in their writing throughout the seven assignments (Mean=3.43, STD=0.23). This is in confirmation of the finding of previous research that instructor's feedback, however detailed and accurate it may be, does not make in itself any significant influence on the overall quality of students' writing.

Figures 3-1 and 3-2 show that, of the two grammatical categories that are most problematic for average Japanese EBP writers, errors in prepositions seem to be more easily avoided by using the BLC Concordancer than errors in articles. The reason for this may be that, while use of articles is largely dependant upon the semantics, rather than the syntax, of a given sentence, the choice of a particular preposition from among other possible alternatives in a given context can be determined more or less locally and syntactically -- often within a few words to both the right and left of the prepositional slot in question. And, as such, users can simply submit a query to the BLC Concordancer, containing either the main verb (or VP) or noun (or NP) as a search string to find out the preposition that goes with it.

The following are some of the typical errors found in the assignments submitted by the subjects in the Control Group.

   1.   your participation fee of  (=> for) the party is still outstanding  [T3, CG-S3]
   2.   my visit in (=> to) Toronto  [T4, CG-S12]
   3.   information of (=> on|about) Canadian business  [T4, CG-S6]
   4.   invitation of  (=> to) career guidance seminar  [T5, CG-S14]
   5.   kindly explained about (=> x) the meanings of . . . for  (=> in) detail  [T7, CG-S18]
   6.   please contact with (=> x) me if you . . .  [T7, CG-S1]

Had they used the BLC Concordancer in writing and reviewing their assignments, they would have been able not only to avoid making these simple errors, but also learn, for instance, that the combination "fee + for" is usually followed by an NP-EVENT as in "fee for this service" and the combination "fee + of" is followed by a MONEY-AMOUNT as in "fee of $40" and so on and so forth. [5]

This is basically what the subjects in the Experiment Group have done in doing the assignments under the condition that they make full use of the BLC Concordancer. As we have already seen, the positive effect of this data-driven, self-learning tool is clearly manifested in the much fewer average numbers of errors they made in their writing from Tests 2 through 7 than those of their counterparts in the Control Group.

[5] See Appendix 5 for a sample printout of BLC Concordancer's KWIC data for a search string "fee (for|of)".
[Data Download: MS Excel Zip File (17 KB)]  [Back]
Table 2  Numbers of Errors in Control Group (without BLC Concordancer)
Table 3  Numbers of Errors in Experiment Group (with BLC Concordancer)