Comments on McNamara?s

?How can one measure the extent

of a persons bilingual proficiency??

Leon A. James

Center For Comparative Psycholinguistics,

The University of Illinois

Urbana

I greatly enjoyed reading Dr. McNamara?s paper, and I am glad to have this opportunity to make a few general comments on it. I would like to begin with his proposal that the investigator interested in measures of bilingual proficiency ?need only measure those skills and those aspects of skills with which he is directly concerned? (p.70). Dr. McNamara proposes in his figure 1 (p.70) 16 cells in each language representing the various skills, which are potentially involved in language proficiency. A relevant question that ought to be raised in this connection is whether it is possible, in practice, to measure the various skills tabulated in this scheme independently. That is, assuming for example, that one were interested in measuring a person?s skill in understanding the syntax of oral speech - which is one of the 16 cells in the figure-can one use a test which would measure this skill independently of skills represented in adjacent cells, such as semantic and phonemic decoding? Even a superficial analysis of this problem suggests that such cannot be the case. Phonemic decoding is essential in the meaningful perception of an utterance and semantic disambiguation must precede or accompany syntactic analysis. Examination of the ?direct? measures used by Dr. McNamara and presented in the appendix leads to the same conclusion. Thus, criterion variable, 9, which is a listening comprehension skill, logically involves all four of the cells under the ?decoding - listening? heading. Similarly, criterion variable 1, which is reading comprehension, involves all four cells under the heading ?decoding - reading.? Other criterion variables such as reading vocabulary, syntactic interferences, number of words, and so on, also involve two or more cells in the schematic figure. Given this situation, one must interpret Dr. McNamara?s admonition that we must limit ourselves to those measures with which we are directly concerned in terms of the traditional four-way classification of listening, reading, speaking, and writing. However, we must consider the fact that even these more global skills are not independent of each other. One can hardly imagine (except in abnormal situations involving various forms of aphasic dysfunction) someone proceeding in writing skills without a prior or parallel achievement in reading sills, nor is it likely that one could develop speaking skills without any kind of listening skills. In view of these interdependencies, it would appear that some global measure of language proficiency is necessary and that it is difficult to imagine that an investigator could limit him to the measurement of sub-skills that in fact do not develop or exist independently.

My second point, relates to the distinction made by Dr. McNamara between ?direct? and ?indirect? measures Of language proficiency. In principle, such a distinction would seem to be potentially useful if by ?indirect? measures one would mean to refer to tests that assumed to be mediated by language competence but that, on the surface, involve some apparently unrelated skill such response latency in motor behavior, which in fact is involved in Lambert?s autornaticity measure of bilingual dominance. Similarly, a language background questionnaire and self-evaluation ratings of bilingual proficiency can be conceived of as indirect indices. But in what way can one say that fluency tests, word completion tests, semantic richness tests, and tests of reading speed are ?indirect? measures of language proficiency? Surely all of these measure skills directly involved in the four basic aspects of language use and it is a matter of complete arbitrariness to call them ?indirect? as opposed to ?direct?. This issue is not purely one of terminology since in Dr. McNamara? s analysis one is treated as criterion for validation while the other is treated as tests to be validated. Hence the important issue of validation is completely vitiated by this arbitrariness. The most that could be done in this case, given the absence of a global validating measure of language proficiency, is to present a table of intercorrelations of the 29 measures and the problem of their validation remains unsolved.

I come now to Dr. McNamara? s method of indexing bilingual proficiency. Since this estimate is based on a difference score between the various language proficiency tests referred to above, all the difficulties just mentioned will also apply to the measure-of bilingual proficiency. But there is an added difficulty here, which ought now to be taken up. The rationale for using the difference score is that the investigator wishes to eliminate and control out that part of the variance, which is attributable to monolingual skills. For example, if we wish to estimate the bilingual?s relative ease in reading speed in his two languages, we are told to subtract his reading speed in the second language from his reading speed in his first language and, in some cases, the difference score must be divided by the reading speed in the first language to obtain a percentage difference of imbalance. The assumption underlying this procedure is that a common type of competence mediates the two measures and that the same factors influencing one measure will also influence the other measure in a similar manner. How valid is such an assumption?

Let us take the case of bilingual proficiency in fluency as measured by the ratio - difference score of ratings of word frequency. In this test, as it is described in the paper by Dr. Cooper (p. 125), the subject rates the frequency of words chosen from particular domains in his two languages. The ratio - difference score, when computed, allows him to make such statements as ?the respondent?s English dominance over Spanish for words in the home domain is 10% while her English dominance for words in the school domain is 29%,? and so on. The assumption implicit in this conclusion is that the aptitude underlying the two indices, namely verbal fluency, is the same in the two languages since the respondent acts as her own control. My point is that this assumption is not necessarily true and that despite the fact that the aptitude involved in verbal fluency is constant, the actual performance of fluency may be affected by factors extrinsic to aptitude for fluency - factors, which are not necessarily comparable in the two languages. The evidence, which I can suggest for this assertion, is only informal but nevertheless compelling. It relates to a psychological aspect of bilingualism which, to my knowledge, has not been extensively investigated although it has often been described subjectively in what I might call ?folk bilingualism? following Hoeningswald?s proposal for a ?folk linguistics.? The phenomenon manifests itself as a décalage between the psychological states involved in bilinguals using their two languages. In extreme instances, such as might be the case with co-ordinate bi-cultural bilinguals, one might even speak of a schizophrenic type de-duplication of the bilingual?s personality, such that when he speaks language one he is not the same person as when he speaks language two. This difference in personality may exhibit itself in different ways of perceiving the environment, different interpersonal perceptions and behaviors, different role expectations and role manifestations, all of which, by the way, are objectively measurable. I believe that these manifestations are present to a greater or lesser extent in all bilinguals. The consequences of this kind of a psychological décalage on linguistic performance are potentially very great. The difference score involved in the measurement of bilingual proficiency is completely confounded by these effects and to my mind invalidates its use. This problem involved in the measurement of bilingual proficiency is connected with the failure on the part of Dr. McNamara and others in this field, to heed Chomsky?s warning that performance measures in general and in principle are inadequate estimates of competence. In the present instance, the performance measures involved in the measurement of bilingual proficiency are potentially affected by non-linguistic extraneous factors that vary differentially in the two languages and that were not taken into account.

The solution to these problems is not an easy matter, but it seems to me that it does not lie in direct comparison of measures in the form of simple difference scores. There is a difficult dilemma here. On the one hand, I am suggesting the use of a global monolingual measure to be standardized in relation to the performance of monolinguals and with due attention to the difficulties of estimating competence from performance. On the other hand, I would not want to have neglected the purely bilingual nature of knowing two languages, which comes about by the unique fact of a co-location of two different linguistic and cultural systems in the brain of a single individual, the investigation of which is pursued in studies on interferences and on translation. It may be that these two aspects, that is, monolingual competence in each of two languages and bilingual interferences, may have to be studies separately.

Before I attempt to summarize my remarks I would like to make note of the absence among the various measures proposed and used by Dr. McNamara of tests that are relevant to the individual?s bilinguality per se. I don?t know if this is the best way to refer to this type of a test, but what I have in mind includes the potential linguistic and psycholinguistic advantages of being bilingual. I am intentionally putting it in terms of the positively evaluative term of ?advantages? to counteract the implied negative evaluation in the universally used term of ?interferences?. It may help to remind ourselves that the notion of ?linguistic interference? in the speech of bilinguals stems from the psychological concept of transfer and despite the fact that it is well know that transfer effects may be either negative or positive, its negative aspects in linguistic interferences have been generally emphasized in experimental reports to the exclusion of an interest in any positive effects. An examination of the literature on the problem of the effects of bilingualism on education and intellectual development, such as the excellent review in Dr. McNamara?s book, reveals an exclusive interest on the part of researchers in this area in the negative effects of bilingualism. The supporters of bilingualism are apparently quite happy in demonstrating that there is no evidence for negative effects of bilingualism. The Peal and Lambert study, which has demonstrated the superiority of a bilingual group of children in the Montreal setting, is a notable exception to this trend, and these authors have permitted themselves to at least examine the possibility that early bilinguality may contribute to greater cognitive flexibility in the child. Lado, in his book on language testing, devotes one or two paragraphs to the problem of measuring linguistic and psycholinguistic sills that are facilitated by the learning of a second language. Among these he mentions an increased awareness of the arbitrariness of the sound-meaning relation, an increased flexibility in the manipulation of alternative ways of expressing analogous ideas and a keener realization of the polysymous and homophonous nature of words. To these we may add the proposal of a weakening of linguistic chauvinism and a broadening of linguistic intuitions relating to language particulars and language universals. I have no specific proposals for procedures to measure any of these skills, but recognition of their existence and importance would undoubtedly lead other to implement the development of such measures.

And now, I would like to briefly summarize my remarks. My first point was that the subdivision of language skills beyond the four basic ones of listening, reading, speaking and writing, while theoretically possible, cannot in fact be achieved by tests, and therefore the suggestion that the investigator limit himself to those aspects that he is interested in, as opposed to paying attention to more global skills, is to my mind a weak proposal. My second point was that the distinction between ?direct? and ?indirect? measures, unless it is based on a clear theoretical criterion, is in fact arbitrary, and this arbitrariness vitiates the attempt to validate some measures by means of others. Thirdly, I argued that the difference score as proposed for a measure of bilingual proficiency is inadequate in principle due to the fact that correlated non-linguistic factors may differentially affect scores on proficiency tests in the two languages, hence rendering their direct comparison questionable. These considerations follow from the principle that performance measures cannot be taken as simple and direct indices of competence:

Finally, I pointed out the apparent exclusive interest shown in bilingual interference on the negative transfer side to the detriment of measures dealing with the potentially positive linguistic effects of bilingualism.

I would not want to end my comments with an enumeration of my criticisms of Dr. McNamara?s paper. It is a strange fact that a commentary, more often than not, turns out to be a negative piece of work. In this instance, my commentary thus far, fails to reflect the great interest with which I have read and reread Dr. McNamara?s contribution and the personally useful function, which it fulfills as a summary of the measures that investigators have used in this area. At several places in his paper, Dr. McNamara furthermore shows the same perspicacity and lucid evaluation of certain central issues in bilingualism that he has previously demonstrated in his excellent book on bilingualism and education, as well as in his editorship of the recent issue of the Journal of Social Issues dealing with bilingualism and to which he has contributed several articles. We surely owe him an expression of gratitude for taking upon himself this useful task.