PROBLEMS IN
ASSESSING LANGUAGE PROFICIENCY
Dr. Leon James
(c)1999
Chapter 4 of a book now out of print: Foreign
Language Learning (Rowley, Mass.: Newbury House, 1970).
PROBLEMS IN ASSESSING LANGUAGE PROFICIENCY
A criticism of current practices in FL education that recurs throughout the earlier chapters is that the traditional methods used for assessing language proficiency attained by students are inadequate. Grade-point averages obtained in FL courses and standardized discreet-point tests (both written and oral) are not sufficient indicators of the ability to use the FL for communicative purposes. One could even argue that these assessment procedures are not only insufficient, bur irrelevant. Furthermore, one important reason why research on the comparative effectiveness of different teaching methods has never yielded definitive results is that the methods and procedures employed for assessing what the student has acquired have been inadequate. There is a critical need in the language testing field for tests of communicative competence. Some possible approaches to this problem were briefly outlined in the last section of Chapter 3 under the heading of "Criterion evaluation." (See p. 106). This chapter considers the problem of assessment in greater detail. The first section provides the rationale for a "functional" approach and gives a few illustrations on how one would go about developing tests of language skills. The last two sections have a more "molecular" orientation and are concerned chiefly with the assessment of transfer effects between the individual's two languages. This part of the book is more "experimentally" oriented than the rest, and here the FL teacher can gain a
glimpse of the academic researcher in action. It is involved, esoteric, and sometimes abstruse, and may be of only peripheral interest to the practitioner. Nevertheless, I encourage the reader to plough..: through these sections, not only because he will gain a better under-., standing of the psycholinguist's mind (hopefully!), but also because they raise interesting and important issues on the nature of bilingualism: Are there different "types" of bilinguals? To what extent does the native language affect the acquisition of the second language? Under what conditions may we expect positive and negative transfer? And many others.
A FUNCTIONAL APPROACH TO THE ASSESSMENT OF LANGUAGE SKILLS
The problem of assessing language skills is a practical concern. Developments in this area have proceeded under the impetus of necessity rather than under the guidance of a coherent theoretical under- standing of the nature of language and communication. There is nothing opprobrious in such a strategy, given the various social needs which the field of language testing has had to meet, and no doubt the activities associated with this endeavor will-and should-continue in the near future. On the other hand there is no wisdom or advantage in ignoring the potential contribution of theoretical developments in the fields concerned with the study of language and in this section I shall attempt to outline what I consider to be the implications of recent theoretical developments for the practical problems which are of concern to the language testing field.
4.1 COMMUNICATIVE COMPETENCE
Because of their practical orientation, the contributors to the language testing field are more aware than their theoretically oriented colleagues that language is primarily a communicative skill. Or so it would seem. And yet a cursory examination of the "state of the art" in language testing indicates that the progress of the work to date has been chiefly oriented towards the assessment of linguistic com-
PROBLEMS IN ASSESSING PROFICIENCY 151
petence rather than communicative competence. Since much of the rationale of the proposal to be outlined relates to the distinction to be made between these two types of competencies, it is necessary to examine it in some detail. The distinction is recognized intuitively at certain embarrassing moments in our social interactions when we don't know what to say or how to say something, when we fail to communicate "a point," when we "put our foot in our mouth," when we misunderstand "the words" of someone, or, at less awkward moments, when we infer intent and other psychological states on the basis of someone's utterances, when we identify a person's geographic origin and his social and educational background on the basis of his speech, and so on. The study of linguistic competence has traditionally excluded these wider considerations involved in the use of language as indicated by the emergence of "paralinguistic," "exolinguistic," "sociolinguistic," and "psycholinguistic" hybrid fields whose purpose was to cover various aspects of the communicative act. Linguists have acknowledged the fact that the study of language use must necessarily encompass these wider competencies, sometimes referring to the difference as linguistic competence versus linguistic performance, but they have generally maintained that the study of linguistic competence per se need not concern itself with social-psychological factors. The development of language tests as measures of linguistic skills derives its justification from this traditional position taken by linguists. However, the adequacy of such a position must now be challenged as a result of the evidence that has accumulated in these hybrid disciplines as well as from recent work in linguistics itself, especially, but not exclusively, that aspect of the field that has concerned itself with the semantic component of language. Let us consider some specific issues involved in this argument. It has been established (see for example the review by ENin-Tripp, 1967) that speakers of a language have a command of various "codes" that can be defined as a set of restriction rules that deter- mine the choice of phonological, syntactic, and lexical items in sentences. These restriction rules form a code by virtue of the fact that they are related to the same set of variables such that, given the presence of a particular variable, it will then require a specifiable set of rules determining the choice of these items in speech. For ex-
ample, the choice of address form in English (using the title "Mr." followed by last name), the choice of a particular phonological variant (the "-ing" ending of the progressive form of verbs), and the choice of certain lexical equivalents ("Good Morning" vs. "Hiq are all determined by the same social variable which relates to the status relation between the speaker and listener. (Contrast the utterance "Good morning Mr. Gibbs! How is your work coming along" with "Hi, John! How's your work comin' along?"-both of which are possible for the same speaker, but he would not use both under the same situation: they belong to different codes.) The choice of "Tu" and "Vous" in French (and in German along analogous lines) is determined by specifiable social factors between two interlocutors. In Thai (and several other languages) it is impossible to carry out a conversation without explicitly encoding certain relations that hold between speaker and listener (status, age, degree of friendship, family ties, etc.). These selection rules, and others of this type, are as necessary a part of the linguistic competence of a speaker as those with which we are more familiar in syntax such as accord in gender, number, and tense, and it would seem to be entirely arbitrary to exclude them from a description that deals with linguistic competence. There have been a number of recent proposals (e.g., Katz and Fodor, 1963; Osgood, 1968) that attempt to describe the meaning relations among words in terms of a set of semantic features (e.g., "animate," "male," "human," etc.). Certain linguistic facts can be explained by assigning particular semantic features to words. For example, if one assumes that MAN is coded for an "animate" and a "human" feature (among others), DOG for "animate" and "non- human," TABLE for "nonanimate" and "nonhuman," SPEAK for "human" and RUN for "animate," then one can explain why the following sentences are permissible or nonpermissible, as the case may be:
The man did not wish to speak. "The dog did not wish to speak. "The table did not wish to speak. The man ran away. The dog ran away. "The table ran away.
PROBLEMS IN ASSESSING PROFICIENCY 153
The nature and identity of the semantic features that must be postulated is still a matter of dispute. One practical criterion for the choice of a feature is whether it helps explain a linguistic fact. Thus to account for the fact that the expression "to beg proudly" is anomalous (as opposed to "to beg humbly" which is apposite), Osgood (1968) postulates a "supraordinate-subordinate" feature for interpersonal verbs (to beg" is coded for "subordinate" and "proudly" for "supraordinate"-hence the anomaly of the combination). I have pointed out elsewhere (Jakobovits, 1968c) that OLD MAN is coded for "passivity" and "weakness" which accounts for the fact that the following two utterances are paraphrases of each other:
(a) Don't worry about him. He can't interfere. He's just an old man .
(b) Don't worry about him. He can't interfere. He's just a
passive old man.
weak old man.
The addition of "passive" or "weak" in (b) is redundant, showing that OLD MAN is coded for a "passivity" and a "weakness" feature. Note further that (c) is anomalous:
(c) "Don't worry about him. He can't interfere. He's just
an active old man.
a strong old man.
It is possible under certain circumstances to ask native speakers directly whether a particular word is coded one way or another for any particular feature. The Cross-cultural Atlas of Affective Meanings represents such an attempt in which a group of native speakers in some 20 language/culture communities were asked to code 600 words in terms of three generalized affective features: Evaluation, Potency, and Activity (see Osgood, 1964; Jakobovits, 1966; 1969). The relevance of this Atlas for the present discussion lies in the fact that the affective features of words that represent translation equivalents are sometimes universal (e.g., MOTHER is code as "+Evaluation," DEATH is coded as "-Evaluation," AIRPLANE is coded as "+Activity," ROCK is coded as "+Potency," etc.) and sometimes
cross-culturally variable (e.g., PRAYER is coded "+Evaluation" in the United States but "-Evaluation" in Yugoslavia). The "+Evaluation" and "-Activity" coding of SUNDAY in American English accounts for the interpretability of the sentence "On this island, my friend, every day is Sunday." In Delhi Hindi, SUNDAY (meaning: the first day of the week, the Day of the Lord) is coded as "O Evaluation" and "O Activity" (i.e., "Zero") and the sentence in question would not be interpretable. Note, however, that in order to account for these linguistic facts it is necessary to take into account specific cultural differences between Delhi Hindi and American English cultures. Again it would be a matter of pure arbitrariness to declare that these cultural factors are to be excluded from an account of linguistic competence. Finally I would like to mention one other specific issue which argues against a restricted view of linguistic competence. One would certainly insist that any serious account of linguistic competence would include the derivation of the meaning of a (well formed) sentence. Central to such an account is the fact that speakers of a language are capable of recognizing whether two sentences have the same meaning, that is, that they are paraphrases of each other. That the following two sentences are paraphrases of each other, namely
(a) Kiki chewed up the bathroom rug.
(b) The rug in the bathroom was chewed up by Kiki.
can be adequately demonstrated by showing the transformations which relate the two sentences to each other. But what would it take to demonstrate that the next two sentences are paraphrases of each other?
(c) You're on, Rex.
(d) You're next, Rex.
The relation between these two sentences is in terms of functional equivalence (of intent) within the context in which either of them may be uttered. It would do no good to argue that the relation involved is arbitrary, nonlinguistic as might be the case between two equivalent secret passwords, since it can be shown that functional equivalence in meaning can be rendered by novel paraphrases (e.g.,
PROBLEMS IN ASSESSING PROFICIENCY 155
the telephone is out of order, the telephone isn't functioning, the telephone doesn't work, the telephone is dead, etc.). In order to account for a speaker's ability to recognize the equivalence of these sentences, one will have to consider factors other than those usually included in a restricted sense of linguistic competence (for example, intention of the speaker, see below).
4.2 THREE LEVELS OF MEANING
In order to be able to account for the minimum range of linguistic phenomena involved in communicative competence, it will be necessary to incorporate in the analysis of an utterance three levels of meaning. I shall refer to these as linguistic, implicit, and implicative.
4.2.1 Linguistic Meaning
I shall retain this term to refer to the traditional concerns of linguists as outlined in such recent treatises as Chomsky (1965) and Katz (1966). Briefly, this includes a dictionary of lexical meanings and their projection rules, syntactic relations as defined by a derivative transformational theory, and honological actualization rules.
4.2.2 Implicit Meaning
I shall use this term to refer to the elliptically derived conceptual event which an utterance represents. Let me use the following ex- ample to clarify what I mean. Consider the meaning of "agreed to" in each of the following sentences:
(a) The University agreed to pay his moving expenses.
(b) After the President invoked the Taft-Hartley Act, the strikers agreed
to go back to work.
(c) Joan agreed to marry him.
(d) They agreed to settle the matter out of court.
Note that in each of the four instances "agreed to" has some different particular implications which one can infer on the basis of the situation to which the sentence as a whole refers. These particular implications are a function of the situational context that the sentence in which the verb is used refers to and therefore are potentially infinite. The dictionary meaning of the verb applies certain definite restrictions on the kind of particular implications it may have in a sentence and thus, although these implications are potentially in- finite, they are not randomly variable. There are many other in- stances in which this type of nonrandom but infinite variability principle operates. The concept of "triangularity" can be particularized as a visual form (among others) with infinite, but nonrandom, variability: each particular triangle will have its own unique properties different from any other particular triangle. What I am arguing is that the effective information that is being transmitted in communication via sentences-that which the speaker intends the listener to understand-is the particular meaning, not the general, for this is what represents the conceptual event ("idea") that he attempts to communicate to the listener. In order to recover the particular meaning of a word intended by the speaker, the listener must engage in an inferential process which makes use of his knowledge of the general (dictionary) meaning of words as well as his knowledge of the overall situation to which the sentence as a whole refers. This inferential process yields what I have called "the elliptically derived conceptual event" which an utterance represents and which I have used as the definition for implicit meaning. Note that the effective carrier of the message is the implicit meaning of the sentence, not the linguistic one. Hence, we see once more that linguistic competence must include factors outside a narrow view of linguistic manipulations and that the assessment of such competence by means of language tests must therefore also include these larger issues.
4.2.3 Implicative Meaning
We have seen that words have a relation to a generalized abstract meaning (linguistic meaning) as well as to a particularized meaning
PROBLEMS IN ASSESSING PROFICIENCY 157
that relates to a specific conceptual event (implicit meaning). Utterances as a whole have, in addition, a relation to several aspects of the speaker: his intention, his psychological state, his definition of the interaction and certain "claims" (both intentional and unintentional) he makes about the status of his utterance, etc. In some cases these implications are necessary to recover the intended meaning of the utterance (e.g., "Do you have a match?" is not a question to be answered verbally but a request for fire to light a cigarette). In other cases, the implications of an utterance are by-products not essentially related to conveying intended meaning (e.g., phonological clues that reveal geographic origin). I shall refer to both types of implications as the implicative meaning of an utterance.
4.3 A FUNCTIONAL APPROACH
Within the framework that I have outlined the problem of assessing language skills becomes the problem of describing the specific manner in which an individual functions at the three levels of meaning identified above. The relations that hold between the three levels, both for a speaker and a listener, is a matter to be determined empirically for each individual, although it is to be expected that such empirical studies will show that some of these relations will be universal for all language users. The development of specific tests for assessing these relations should be guided by an overall model of the functional elements in communicative competence. Because of the practical concerns involved in language testing, one need not post- pone the task of developing language tests until such time as adequate theoretical models become available. In fact, it might even be the case that such empirical evidence that the language tester accumulates would constitute useful information for building such theoretical models. I would argue, however, that the development of language tests on a piecemeal basis and without considerations de- rived from a model that encompasses the full range of phenomena involved in communicative competence, can only yield artificial laboratory exercises that have little significant relationship to the use of language in real life situations. I believe it is presently possible to outline the major elements to be included in a model of language use
even if it may be insufficient in details, and I have described elsewhere (Jakobovits, 1968c) a classification scheme based on an analysis of the three levels of meaning discussed above. I will not discuss the full classification scheme here but present below an over- view of it:
1. A functional analysis of meaning
1.1 linguistic meaning
1.2 implicit meaning
1.2.1 aspectual
1.2.1.1 extended duration
1.2.1.2 limited duration
1.2.1.3 inception
1.2.1.4 iteration
1.2.1.5 gradualness
1.2.1.6 chronicity
1.2.1.7 termination
1,2.1.8 collectiveness
1.2.1.9 inaction
1.2.1.10 directionality
1.2.1.11 intensity
1.2.1.12 nonchalance
1.2.2 affective
1.2.3 aesthetic
1.2.4 contextual
1.2.5 situational
1.3 implicative meaning
1.3.1 sociolinguistic 1.3.1.1 geographic origin
1.3.1.2 education and SES
1.3.1.3 professional
1.3.2 emotional
1.3.3 intention
1.3.3.1 mands
1.3.3.1.1 request
1.3.3.1.2. command
1.3.3.1.3. prayer
1.3.3.1.4. question
PROBLEMS IN ASSESSING PROFICIENCY 159
1.3.3.1.5. advice
1.3.3.1.6. warning
1.3.3.1.7. permission
1.3.3.1.8. call
1.3.3.2 pseudo mands
1.3.3.2.1. superstitious
1.3.3.2.2. magical
1.3.3.3 intraverbals
1.3.3.3.1. grammaticalness
1.3.3.3.2. word associations
1.3.3.3.3. conventions
1.3.3.3.4. expressions
1.3.3.3.5. recitations
1.3.3.4 private tacts
1.3.3.4.1. feelings
1.3.3.4.2. opinions
1.3.3.4.3. rationalizations
1.3.3.5 public tacts
1.3.3.5.1. instructing
1.3.3.5.2. describing
1.3.3.5.3. entertaining
1.3.3.6 autistic utterances
1.3.3.6.1. verbal fantasy
1.3.3.6.2. self-tacting
1.3.3.6.3. self-manding
1.3.3.7 autoclitics
1.3.3.7.1. importance
1.3.3.7.1.1. stress pattern
1.3.3.7.1.2. repetition
1.3.3.7.1.3 adverbial intensification
1.3.3.7.1.4. syntactic inversion
1.3.3.7.2. existential status
1.3.3.7.2.1 assertion
1.3.3.7.2.2. negation
1.3.3.7.2.3. obligation
1.3.3.7.2.4. possibility
1.3.3.7.2.5. probability
1.3.3.7.2.6. certainty
1.3.3.7,2.7. hear-say
1.3.3.7.2.8. permission
1.3.3.7.2.9. imperative
1.3.3.7.2.10. irony
1.3.3.7.3. relational status
1.3.3.7.3.1. explicit hinges
1.3.3.7.3.2. implicit hinges
1.3.3.7.3.3. zero hinges
2. Elements of style
2.1 internal consistency
2.2 appropriateness
2.3 effectiveness
2.4 aesthetic value
2.5 affective value
3. Selection factors in the act of composing
3.1 intention
3.2 audience
3.3 situation
3.4 style
3.4.1 intraverbal influences
3.4.2 formal constraints
3.4.3 blendings
4. The structure of the message
4.1 syntactic
4.2 conceptual
4.4 A PROGRAM OF TEST DEVELOPMENT
It is not possible within the context of this section to specify in detail a program of test development that will take into account the full range of linguistic phenomena covered by the classification scheme sketched out above. Furthermore, the development of such a testing
PROBLEMS IN ASSESSING PROFICIENCY 161
program would require the collaboration of a team of workers with specialized skills which I do not possess. Nevertheless, I can perhaps offer some tentative suggestions with respect to some methodological approaches that may be used in connection with many of the linguistic phenomena involved in the classification scheme.
4.4.1 Judgments of Acceptability
This technique consists of asking a subject to make an evaluation of the acceptability or appropriateness of an expression, or alternately, presenting him with two similar expressions and asking him to choose the more appropriate of the two. The technique is well suited for assessing sensitivity to aspectual qualities of words. Aspectual quality (1.2. 1) is part of the implicit meaning of a word and is based on inferences one can draw about characteristics of the specific conceptual event implied by a word. These characteristics are not specified directly by the dictionary meaning of the lexical items involved but are instead implied by them. Some examples of aspectual qualities in English are the following: extended duration (as exemplified by the following contrasts: "to stare" versus "to notice," "fearful" versus "frightened," "state of normalcy" versus "state of crisis"), limited duration ("to strike" versus "to beat," "to gulp" versus "to sip"), inception ("to doze off" versus "to doze," "obsolescent" versus "obsolete," "adolescence" versus "manhood"), and iteration ("to nag" versus "to criticize," "to sob" versus "to yelp," "wave" versus "line"). It is possible to construct phrases that contain modifying elements which are either consistent or inconsistent with the aspectual quality of the modified element. In each of the following examples the second phrase contains an inconsistent modifier; given a choice, the subject should choose the first alternative if he is sensitive to the implicit meaning of the expression:
He was frightened momentarily/He was fearful momentarily
He took a long sip/He took a long gulp
He had been dozing for some time when the phone rang/He had
been dozing off for some time when the phone rang.
He criticized me only once/He nagged me only once
4.4.2 Semantic Differential Technique The usual method requires a subject to rate a word on a set of seven-point bipolar adjectival scales (see Osgood, Suci, and Tannenbaum, 1957). The selection of scalar opposites varies with the
purpose of the test. Scales for the assessment of three dominant and universal affective features have been identified empirically for English and some 20 other languages (see Osgood, 1964;Jakobovits,1966). These consist of Evaluation (for American English: nice- awful, good-bad, sweet-sour), Potency (big-little, powerful-powerless, strong-weak), and Activity (fast-slow, alive-dead, noisy-quiet). For other affective features, different scales would be used. The technique is general enough to allow for changes in the stimuli that are rated (e.g., they may consist of phrases and sentences) as well as the nature of the scales used for the ratings (e.g., instead of the seven-step scale one may use an "either-or" two-step scale). Sensitivity to the aspectual quality of intensity (1.2.1.113, for example, can be revealed by asking a subject to say which of two choices is more or less intense: "icy" versus "cold," "filthy" versus "dirty," "ravenous" versus "hungry," "to shatter" versus "to break," etc.
4.4.3 The Questionnaire Method
This approach is convenient for assessing a subject's sensitivity to implicative aspects of the meaning of utterances: socio-linguistic (1.3.1,), emotional (1.3.2.), and intentional(l .3.3.), Since nonverbal factors (e.g., gestural, visual, situational) provide particularly important clues to implicative meaning, the problem of assessing an individual's skill in this area is likely to be quite involved. Without implying that these nonverbal factors should be left out of the testing program let me suggest that utterances, even in the absence of such clues, do allow for the derivation of quite a bit of information about the speaker-and the skill involved in this can be assessed more readily within the confines of the usual testing situation. Phonological variables present in tape recordings offer clues to socio-linguistic characteristics of the speaker and his emotional state. Syntactic and contextual variables offer clues to intentions of the speaker. Here are
PROBLEMS IN ASSESSING PROFICIENCY 163
some examples in connection with intentions (1.3.3.). "Pass the salt," "Hands up," "Take your coat off. It's hot in here," "Now listen to this," are utterances which reveal the speaker's interest in inducing some action in the listener. These utterances are classified as mands (1.3.3.1.) and the four given here are instances of a request, a command, an advice, and a call, respectively. The speaker continually uses devices in utterances that have the function of commenting on certain aspects of the message. These are called autoclitics (1.3.3.7.). For example, stress pattern and repetition indicate that the speaker wants to lay emphasis on his utterance ("You should not infer from this that..." "I like it very, very much"). Comments on the existential status (1.3.3.7.2.) of assertions include, among others, obligation ("I had to give in"), possibility ("Couldn't it happen?"), probability (He is likely to forget"), hear- say ("The two accident victims are reported to have died"), irony ("Aren't we having fun!"), etc.
4.4.4 Acting out Situations
The questionnaire approach just described assesses decoding (passive skills). Reversing the task by asking a subject how he would say something under specific conditions becomes an encoding task that assesses corresponding active skills. He now assumes the role of the speaker by acting out the communicative act called for by the instructions. These may be simple or complex. An example of a simple situation would be to ask the person to transform an utterance in such a way as to express certain intentions. For instance, the sentence "He will be going to the meeting," which is an assertion, is to be transformed to indicate negation (He will not be going to the meeting), obligation (He has to go to the meeting), permission (He is allowed to go to the meeting), certainty (He most definitely will be going to the meeting), irony (Wouldn't he just love to go to the meetind), and so on. More complex instructions might require the subject to construct public tacts (1.3.3.5.) appropriate to various situations (reporting a past event, describing an ongoing activity, telling a joke, etc.). Encoding and decoding skills are not necessarily related. Thus, one might successfully identify the geographic origin
of a speaker without being able to imitate him; or, one may re- cognize that a speaker is angry without being able to talk like an angry man upon demand. (The latter case exemplifies a further distinction to be made between volitional control of encoding skills and the ability to carry out the same skills under natural conditions but without volitional control.)
4.4.5 Paraphrasing
"Rewording" is a traditional test for assessing understanding. But rewording often changes the tone or style of utterances, which means in fact that the alternative construction is not semantically equivalent: it may have different implicit and implicative meanings. I use the term semantic modulation to refer to the possibility of alternative phonological strings that are related in linguistic meaning but are different in their implicit and implicative aspects. Consider some examples in semantic modulation:
He spoke loudly/He spoke with a thunderous voice
The light was too bright/It was a blinding light
It was uncomfortably hot and humid/The weather was stifling
He worked extremely hard/He worked his fingers to the bone
The second sentences of each pair in these examples are recognized as paraphrases of their respective first sentences (they have equivalent linguistic meaning) but they are marked in their implicit meaning by the aspectual quality of intensification (1.2.1.1 i.). The ability to manipulate the implicit meaning of an utterance without changing its linguistic meaning is an important skill in "creative writing." Similarly, the ability to manipulate implicative meaning is a mark of the effective writer and speaker (contrast "That's not the way to do it" with "If I were you, I would do it differently").
4.4.6 Intraverbal Skills
Intraverbals (1.3.3.3.) refer to aspects of verbal behavior that are not marked for intent. The most obvious instance of intraverbal in-
PROBLEMS IN ASSESSING PROFICIENCY 165
fluences is grammaticalness. Another instance is the associative relation that holds between words as exhibited in the traditional word association test. Conventional phrases (greetings, acknowledgement of thanks, condolences, etc.), expressions (proverbs, idioms, platitudes), and recitations (lines of poetry, prayers, etc.) all involve intraverbal skills that should be included in any testing program that attempts to assess communicative competence.
4.5 SUMMARY
I have pointed out that the traditional approach to the assessment of language skills derives its rationale from a restricted definition of linguistic competence. I have argued that such a restricted view is totally inadequate to account for the most basic phenomena in language use and, hence, language tests based on this rationale are nothing but artificial laboratory exercises that have little in common with the skills involved in the everyday use of language. To study adequately the latter process, one must recognize the basic functions of communication via language and the total information available in the communicative act. I have described in an outline form a classification scheme for identifying the functional elements to be taken into account in the study of communicative competence and have suggested some methodological approaches that one might adopt in the development of a testing program designed to assess skills involved in the use of language.
4.6 COMPOUND-COORDINATE BILINGUALISM
Since Ervin and Osgood's (1954) discussion of the problem, compound-coordinate bilingualism has received the attention of several investigators (see, for example, Lambert, Havelka, and Crosby, 1958; Jakobovits and Lambert, 1961;Preston, 1965; Kolers, 1963; and others: see the review by Macnamara, 1967). The distinction between the two types of bilinguals offered by Ervin and Osgood (1954) was a unidimensional one, in terms of the semantic relation between the bilingual's two languages. The compound bilingual was supposed to have a single meaning system hooked up to two
different input (decoding) and output (encoding) channels corresponding to the bilingual's two languages. This type of bilingualism was assumed to come about as a result of learning the second language in the same environmental setting as the first, and using the first language as the indirect channel of acquisition. In this kind of a psycholinguistic system every word in the second language was a mere replica of a word in the first language with a one-to-one correspondence in meaning between the two translation equivalents. This is why the "compound" bilingual system is sometimes referred to as a "fused" system (Weinreich, 1953).
On the other side of this semantic continuum is located the "separate" system of the coordinate bilingual who is assumed to possess two independent meaning systems corresponding to his two languages. This situation is achieved, according to this view, as a result of direct language acquisition in a linguistic-cultural community different from one's own. There is no one-to-one relation between the two meaning systems and, in fact, translation equivalents are mere approximations, their closeness depending on the similarities in the two cultures involved. A direct test of the semantic distance between a few translation equivalent words for French- English bilinguals was carried out by Lambert, Havelka, and Crosby (1958). These authors divided their sample of bilinguals into three groups. One was composed of uni-cultural compounds who learned French in school by the indirect method through the medium of English. The second group was composed of uni-cultural coordinates who learned French by the direct method and often used it in a setting different from that where they habitually used English; the two settings, although different, were nevertheless both North American. The third group was composed of bicultural coordinates; these were often native speakers of French (in Quebec) who lived in the bicultural setting of Canada. Having selected a set of common words in English (House, Drink, Poor, Me) and their French translation equivalents, the investigators then obtained semantic differential ratings for each word using the appropriate scale in English and French. The results showed that the semantic distances between translation equivalent terms were largest for the bicultural coordinates and smallest for the uni-cultural compounds (a statistically significant difference), with the uni-cultural coordinates falling near
PROBLEMS IN ASSESSING PROFICIENCY 167
the uni-cultural compounds and not significantly different from them. These results, which represent partial confirmation of the predictions of Ervin and Osgood (1954), are further strengthened by a subsequent study (Jakobovits and Lambert, 1961) which demonstrated that reducing the meaning of a word (through semantic satiation brought about by verbal repetition, see Lambert and Jakobovits, 1960) in one language also brings about a corresponding reduction in meaning of its translated equivalent in the second language for compound bilinguals but not for coordinate bilinguals-as indeed the theory would predict.
The theory of compound-coordinate bilingualism as described above may be quite useful in understanding certain aspects of the semantic functioning of bilinguals. However, it may be advantageous to extend the compound-coordinate distinction to other areas of psycholinguistic functioning. The remainder of this section will explore some possible ways by which such an extension may be achieved.
It would be useful to recognize at the outset that bilingualism involves two different aspects of the question of interrelatedness of two distinct language/culture systems. One such aspect (R3 deals with the relation between the bilingual's two language/culture systems, and may be represented as follows:
RA=f(XL1?XLz)
where X stands for the speaker and L for the language spoken. In this diagram the first language is represented by a rectangle marked by
horizontal lines (on the right), while the second language is represented by a rectangle marked by vertical lines (on the left). The overlapping area of intersection is marked by both horizontal and vertical lines and represents interference. In this example L, is larger than L, to indicate dominance. The second aspect of the question of interrelatedness deals with the comparison between each of the two languages of a bilingual to that of an idealized monolingual member of the two respective language/culture communities (RB & Rc). This relationship may be represented as follows:
bilingualism, where X stands for an idealized monolingual. It is possible to devise tests which would measure the extent of interrelatedness that exists for each of these two aspects within various areas of psycholinguistic functioning (e.g., semantics, grammar, decoding, etc.) and several of such tests will be described below. At this point, however, it would be useful to restate the definition of compound-coordinate bilingualism keeping in mind the extended perspective presented above.
The ideal coordinate bilingual represents a state of complete functional independence of two language/culture systems in all their aspects: semantic, lexical, syntactic, phonological, morphemic, cultural, attitudinal, etc. In his case, RA (see diagram) has a minimum value while R, and Re have maximum values. In its extreme form, coordinate bilingualism represents linguistic schizophrenia: one individual with two separate and theoretically unrelated cultural-linguistic systems co-occur within the same body. In compound the two linguistic systems are so fused and inter-dependent that the second language represents a mere alternative
PROBLEMS IN ASSESSING PROFICIENCY 169
channel for the overt manifestation of the same underlying system represented by the first language. In this state R, and R, have maxi- mum values, while the other, R,, has a minimum value. In its extreme form compound bilingualism represents a translation machine: for every coded bit in L, there must be a corresponding bit in L, with the relationship between the two exactly specified. Let us now examine some implications which follow from these considerations.
4.6.1 Degree of Bilingualism
Under this topic we should include both the question of bilingual interference as well as bilingual proficiency. Bilingual interference can be defined, following Mackey (1965), as the use of elements in one language while speaking another. The consequence of such intrusions is an increase in the value of R, and a decrease in the value of R,: the bilingual's speech in L, (XLZ) Will differ from the idealized version (XLZ ) in direct proportion to the extent of interference from L, (RA); furthermore, the bilingual's first language may show uncharacteristic departures from the standard (XL3-a problem he did not encounter as a monolingual. One would expect that the most common form of bilingual interference would be in the direction of L,: that is, high values in RA would lead to low values in Rg leaving Re relatively unimpaired. However, there is evidence (see Mackey, 1965) that in certain areas of psycholinguistic functioning, particularly in semantics, the value of Re may decrease as a result of backlash interference (see below).
Bilingual proficiency has sometimes been defined (e.g., Macnamara, 1967) in terms of the relative competence of the bilingual in his two languages: where the two competencies are equal, bilingual proficiency is said to be maximal (sometimes referred to as a state of "balanceness," see Lambert, Havelka, and Gardner, 1959). Where competence in one language surpasses that in the other, a state of bilingual "dominance" is inferred. I have pointed out elsewhere (Jakobovits, 1967) the difficulty with such a definition. When bilingual proficiency is estimated, as has often been the case (see Macnamara, 1967), by means of indirect measures which may be
contaminated by nonlinguistic factors, the relative or difference score would not be acceptable without prior determination of interaction effects. When bilingual proficiency is established by means of direct competence tests, then the difference score obscures the distinction between RB and RC, two elements which are theoretically independent. For these reasons it would be preferable in discussing the problem of degree of bilingualism, to keep separate these two r_elations expressed as XLZ-f gL2 (RB) on the one hand and XL1¨ XL1 (RC) On the other. The first of these must be measured by proficiency tests in the second language standardized on monolingual speakers. The second relation is of interest to the problem of what might be called "backlash interference"-the intrusions into and modifications of one's mother tongue due to the learning of a foreign language.
One may raise the question of the relation between the two aspects of bilingualism discussed above, namely degree and compound-coordinatedness. Let us recall the elements in our system:
Compound-Coordinate
Degree of Bilingualism (D) Bilingualism (CP-CR)
(i) Proficiency in L2 (PL2) (i) RB=f(XL2¨ XL2)
(ii) Proficiency in L, (PL1) (ii) RC=f(XL1¨ XL1)
(iii) RA=f(XL1¨ XL2)
These relations have already been explicated above. Now let us consider the following two hypothetical cases:
Case Y:
PL2 is high; PL1 is unimpaired
RB is maximal; RC is maximal; RA is minimal
Case Z:
PL2 is low; PL1 is unimpaired
RB is minimal; RC is maximal; RA is high
As can be seen, case Y represents a bicultural balanced coordinate bilingual: proficiency in the second language is high while functioning in the mother tongue is unimpaired; furthermore, this bilingual functions as a native in both languages (RB and RC) without showing cross-linguistic interference (RA). This case then is the linguistic schizophrenia referred to earlier. Case Z is weak in the second language (PL2) while he retains the characteristics of his language/culture
PROBLEMS IN ASSESSING PROFICIENCY 171
group (PLl and RC) he shows considerable interference in the second language (RA) which renders his speech in that language uncharacteristic (RB). In other words, case Z represents a uni-cultural dominant compound bilingual. The reader may explore for himself other possible combinations using this schema. By manipulating the various elements in a systematic fashion, one at a time, some interesting cases arise, but these will not be taken up here. Instead, let us consider the extension of the model by the addition of two other dimensions. One dimension relates to the various areas of psycholinguistic functioning as might be indexed by such tests as semantic distance, word association structure, adequacy of translation, language switching, reading comprehension, pronunciation accuracy, and some additional ones which will be described below. The other dimension to be considered is that of development over time. Figure 1 is a schematic representation of a four dimensional model of bilingual description using the elements discussed above. The first dimension, that of de- gree of bilingualism (PL), is given by the relative size of the two rectangles: the left hand rectangle with the vertical lines represents the second language (L2) and the right hand rectangle with the horizontal lines represents the mother tongue (L1). Proficiency in L2 and L1 is indicated by the size of their respective rectangles. The second dimension, that of compound coordinate bilingualism, is indicated as follows:RA (i.e" XLI¨ XL2 or interference)is given by the amount of overlap between the two rectangles; RB (i.e., XL2¨ XL2)and RC (XL1¨ XLl)is given by the density of the vertical and X horizontal lines within the rectangles: high density represents a high degree of relationship. The third dimension, that of psycholinguistic area of functioning is represented on the vertical axis (a. vs. b. vs. c3, and development in time, the fourth dimension, lies along the horizontal axis (1 vs. 2 vs. 3). Let a stand for grammatical intrusions, let b stand for pronunciation accuracy, and c for a measure of word association structure (see below for details). Let I stand for the beginning stages of second language acquisition, let 2 be several months later spent in a bicultural setting, and let 3be at the completion of the language acquisition process. Looking at part a.l of the figure, we notice a dominant uni-cultural compound (L, on the right is dense and larger than L2 on the left, which it envelops) whose
FIGURE 1. A FOUR-DIMENSIONAL MODEL OF BILINGUAL DESCRIPTION.
grammatical performance in L2 is poor (note its small size). His pronunciation in L2 (b.l) is practically unintelligible as the phonemic system of L1 completely dominates it. A test of word association structure (c.l) shows relatively better performance (note larger size of L2). A few months later, and with the opportunity of bicultural experience such as that offered by extensive travel in the foreign country, our language learner has considerably improved in gram-
PROBLEMS IN ASSESSING PROFICIENCY 173
matical performance (note larger size of L2 at a.2), although his pronunciation still shows heavy contamination (b.2). His greatest progress is exhibited on the word association test where his performance increasingly resembles the cultural norm (note increase in size of L2 at c.2 and greater density of vertical lines). At this point an interesting phenomenon occurs, "backlash interference." As a result of becoming more bilingual the individual experiences a typical departure from his mother tongue as compared to idealized monolingual speakers of that language. At the end of the successful acquisition process, the grammatical performance of this bilingual (a.3) shows the coordinate pattern (large, dense rectangles with no overlap). His pronunciation (b.3) is good but not native, and his word associa- tion structure (c.3) shows a curious pattern: his performance is excel- lent (large rectangles) and native-like (dense lines), but the two linguistic systems interact (note overlap): the bilingual has developed what might be called a "cross-cultural semantic system"-a kind of "coloring" or "flavoring" of semantic factors which comes about through cross-cultural contact (see Jakobovits, 1966).
Once again the reader may want to explore further the various combinations that are possible in this kind of a schema. The ad- vantage of what may appear to be a gratuitously complex representation is that it facilitates systematic elaboration of possible stages of bilingualism and may thereby increase our understanding of the processes involved.
4.6.2 Translation and Language Switching
In their analysis of compound coordinate bilingualism, Ervin and Osgood (1954) are led to the hypothesis that extensive translation on the part of a coordinate bilingual will transform him into a compound system. They argue that continued interaction between the distinct meaning systems of the two languages washes out the differences between them. However, the opposite effect is also logically possible: continued interaction can act as discrimination training through contrast and forced differentiation and, in fact, experienced translators seem to be attuned to stylistic and connotative differences between translation equivalents to a degree not apparent
in less experienced bilinguals. The Ervin and Osgood model furthermore neglects the problem of a language switching mechanism which must be postulated to account for the fact that bilinguals, and especially compounds, are able to function in one language without extensive interference from the other language. Figure 2 represents an elaboration of the Ervin and Osgood model by including hypothetical language switches on the decoding and encoding sides as well as filter mechanisms (separators) whose function it is to channel signs into their respective languages at the decoding and encoding stages. Consider first the compound bilingual. If the input switch is "on," incoming signs will be shunted into either LA or LB, depending on the "language set." In other words the switch has directional properties. The signs are then fed into the common meaning system where they are decoded. Now, if the output switch is also "on," the
FIGURE 2, A MODEL OF COMPOUND (TOP) AND COORDINATE (BOTTOM) BILINGUALISM.
PROBLEMS IN ASSESSING PROFICIENCY 175
encoding response will "exit" through LA or LB, again depending on the direction of the switch. One must also consider the case where the bilingual (compound or coordinate) is set to decode from either language (e.g., in a bilingual environment). In that case the switch is turned "off" and the incoming signals are filtered through the "sign separator" mechanism and fed into the meaning system through LA or LB. For the coordinate bilingual, there is no need for a "response separator" mechanism since the meaning systems in the two languages are distinct and feed into separate output systems. For the compound, however, such a filter on the encoding side must be postulated to allow for the possibility that the output switch be "off" (e.g., where the bilingual is set to encode in either language). On the other hand, it is necessary to postulate a "translation switch" for the coordinate to allow for his capacity to attend to LA in the decoding stage and to render its equivalent in the encoding stage. Now let us see how the model depicted in Figure 2 can be used to make predictions about the behavior of bilinguals. The following are some outcomes of the assumptions that have been made:
1. For both compounds and coordinates decoding will be less efficient when the input switch is "off" (e.g., when he has to attend to signs in both languages at the same time-see Measures 17 and 18 below). The reason for this expectation is that the route through the sign separator is less direct and the filtering process should take measurable time.
2. The requirements to encode in either language (see Measures 16 and 19 below) should be more disruptive for the compound than the coordinate, again because of the time requirements of the filter separator.
3. Instructions to translate into a specified language should be a more efficient task for the compound than for the coordinate bilingual. The greater efficiency of this task for the compound can be seen in Figure 2: If the output switch is "on" and directed, say, toward La, the meaning system feeds directly into the output sys- tem. For the coordinate bilingual, there must first be an "equivalence search" between the two meaning systems through the translation switch, and this should take measurable time. It should be noted that if the translation time is very short, such as might be the case for word-to-word translation of common lexical items, the coordinate's
disadvantage relative to the compound would be difficult to measure. This might account for the failure of some investigators (see Macnamara's 1967 review) to find latency differences under these conditions. Where the translation task is less automatic (unfamiliar words, phrases, sentences) the present prediction should be verifiable. The reader can explore further for himself several other predictions that Figure 2 suggests. Macnamara (1967) has discussed the kind of experimental data that would be required to verify the validity of the switch model-the number of switches it is necessary to postulate and whether switches take measurable time for operating. Some translation and switching tests are described below.
4.6.3 Degree of Interrelatedness
Under this topic we shall discuss some of the factors that lead to compound-coordinate bilingualism. We should consider three types of factors: language acquisition context and usage, attitudinal and motivational variables, and cross-cultural distinctiveness. Within each of these areas, it is necessary to distinguish among the various aspects of psycholinguistic functioning. In the semantic area the evidence on connotative distance between translation equivalent words resulting from separate acquisition contexts and usage has already been discussed above. The effects of some attitudinal variables on semantic change has been discussed by Lambert (1967) and his coworkers (see, for example, Lambert, Gardner, Barik, and Turnstall, 1963). In the latter report it was shown that semantic Interdependence is positively related to higher achievement in an intensive language acquisition program. The measure involved was the semantic distance (as indexed by the semantic differential) between the meaning of some words in English (the mother tongue) at the beginning and at the completion of the course, as compared to the meaning of the translation equivalent words in French. Thus all three aspects of interrelatedness were presumably involved, although, unfortunately, they were not all directly measured. Let us examine how this type of change would affect the three aspects of interrelatedness discussed earlier in this paper. The value of RC (i.e, XL1¨ XL1) may be presumed to have decreased from its previous maximal value: there is
PROBLEMS IN ASSESSING PROFICIENCY 177
a decrease in coordinateness and the meaning of words in the mother tongue of this bilingual is no longer representative of monolinguals of his culture. RA (i.e, XL1¨ XL2) in the semantic area decidedly increased: as Lambert and his associates put it, the students "permit the two languages to interact, and this is to their advantage in the learning of the second language" (p. 368). This would lead to greater compoundness. However, change in the value of RB (i.e" XL2 ¨ XL2,) is in doubt: the students' semantic systems may have progressed X towards the native pattern of L2-in which case the value of RB increased promoting coordinateness, or, also possible, their semantic system may have moved away towards a compromise position which is not characteristic of the culture and is colored by the mother tongue, as Lambert et al. put it: "...these students develop generalized super linguistic concepts which incorporate the unique semantic features of the concepts in each of their two languages" (p. 368). Thus it would appear that contrary to what one might expect from an emphasis on "direct" teaching methods, an increase of compound- ness in the semantic area of connotation during certain stages of the acquisition process may facilitate progress in a FL.
Figure 3 represents another hypothesized relation between attitudinal factors and compound-coordinateness. The two psychological variables considered here are ethnocentrism and language learning orientation. The former trait is measured by a questionnaire intended to ascertain the extent of the student's rigidity of belief in the superiority of his own culture, while the latter trait is ascertained by his answers to questions about his purposes for learning a foreign language: instrumental reasons (e.g., usefulness in work or travel, meeting the Ph.D. language requirements) versus integrative reasons (e.g., getting to know the people better; see Lambert et al., 1963, for greater detail). Figure 3 says that high ethnocentrism coupled with an instrumental orientation are psychological factors which promote compound bilingualism. The reason for this is presumably that a highly ethnocentric individual would not tolerate a decrease in the value of Re (i.e, XL1¨ XLl) as this would estrange him from his own culture in whose superiority he so firmly believes. The over all implications of the relationship represented in Figure 3 are rather intriguing, since it indicates that the opposite number of the highly ethnocentric, instrumentally oriented student, namely the self-
FIGURE 3. HYPOTHESIZED RELATION BETWEEN COMPOUND-COORDINATE BILINGUALISM AND ATTITUDINAL STRUCTURE.
deprecating, integratively oriented "culture grabber" characteristic of young, enthusiastic language learners eager for "foreign culture," would also promote compoundness: this comes about, presumably, because this kind of an individual freely allows a decrease in RC (i.e, XL1¨ XL1)as he eagerly "leaves behind" the norms of his culture. X ~X Figure 3 also shows that coordinateness will be promoted by moderate levels of ethnocentrism and bi-directional tendencies in orientation: in other words the psychologically conflictual individual. Such a person would tend to maintain RC (i.e, XL1¨ XLI) at a high level, while moving at the same time towards high values for RB (i.e, XL2¨ XL2). This type of an individual would be expected to experience the "threes of anomie" as his linguistic schizophrenia develops into the full-fledged coordinate pattern.
PROBLEMS IN ASSESSING PROFICIENCY 179
The third type of factor to be discussed in this section, as mentioned above, is cross-cultural distinctiveness. A general hypothesis may be formulated as follows: cultural distinctiveness promotes coordinate bilingualism while cultural similarities inhibit it. It would be difficult for an Acadian in Halifax, Nova Scotia, or a Cajun from Louisiana, to be a coordinate bilingual. Similarly, one cannot expect a Japanese immigrant to San Francisco to be a compound bilingual, even after twenty years and half-a-dozen American monolingual grandchildren. Although it is true that certain semantic characteristics related to affective structure tend to be universal (see Osgood, 1962; Jakobovits, 1966), other characteristics tend to be uniquely peculiar to individual cultures, and these must be considered in evaluating the factors that determine compound-coordinate bilingualism.
4.6.4 Some Measures of Compound-Coordinate Bilingualism and Their Interrelatedness
In this final section we will consider a number of tests which may be used to index the various aspects of compound-coordinate bilingualism as discussed here. The following is a brief description of some 27 measures of compound-coordinate bilingualism. The first nine are in the semantic area; the tests numbered 10 to 14 are in the lexical area; those numbered 15 to 19 are translation and language switching tests; 20, 21, and 22 are grammatical tests that require
transformation and paraphrasing; tests 23-25 deal with a judge's subjective ratings of various bilingual skills; and the last two (26 and 27) are proficiency tests in the two languages (discrete-point type).
4.6.4.1 Semantic Distance Test
The geometric distance formula D is used with the semantic differ- ential to index connotative differences between words (described in Osgood, Suci, and Tannenbaum, 1957). When translation equivalent concepts are used for the same individual, as in Lambert, Havelka, and Crosby (1958),it becomes one index of RA, (i.e" XLI¨ XL2) where a large D would indicate coordinateness. When a bilingual's ratings are compared with a monolingual (or with a norm profile for
a monolingual group), it becomes an index of RB or RC, depending on whether one deals with L2 or L1 (large D values indicate compoundness).
4.6.4.2 Semantic Cross-satiation Test
This is an indirect test of semantic distance and is complicated by the unavoidable occurrence of language switching due to the nature of the test. The bilingual repeats a word in L, (or LP) and a test is then made for the semantic satiation which is presumed to transfer to L2 (or L1). Jakobovits and Lambert (1961) have used this test with clearcut results: cross-linguistic satiation occurs with compound systems but not with coordinates.
4.6.4.3 Associative Interference Task
This test incorporates a transfer of learning design in which bilinguals first learn to memorize a list of commonly occurring words in one language, then receive further training on an interpolated list of words which is made up of the translation equivalent words in the other language of the previously learned list. The theoretical predictions based on transfer of learning theory are clear: because of the identity in meaning of the two lists for compounds, positive transfer is expected; for the coordinates, either less positive transfer or actual negative transfer should be exhibited. This test was successfully used by Lambert, Havelka, and Crosby (1958).
4.6.4.4 Semantic Congruity Test
The procedures for this test were developed by Osgood (1966)in connection with the investigation of semantic feature analysis. The subject is given a set of verb-adverb combinations and asked to make a judgment as to their acceptability. Take for example the following items involving the combination of the verb to attack and three adverbs.
Apposite Permissible Anomalous
To attack violently X
To attack stupidly X
To attack meekly X
PROBLEMS IN ASSESSING PROFICIENCY 181
The acceptability judgments appear on the right. Osgood assumes that the acceptability rating is determined by the interaction between the semantic features for which the verb and adverb are characteristically coded. This test, when used with bilinguals, would be a measure of both RB and RC. By careful selection of the items it should be possible to test quite subtle differences in meaning. Because of this, the test is likely to be more sensitive and efficient in detecting small differences than the previous ones described.
4.6.4.5 Metaphor Sensitivity Test
The principle involved here is similar to that in the previous test but it is considered separately here because the nature of the competence that underlies the judgment process is believed to develop differently in the two situations. More specifically, metaphor sensitivity is believed to originate "deeper" in the semantic structure of a language and represents a type of competence that cannot be learned adequately from dictionaries. This type of "Sprachgefuhl" would be particularly sensitive to coordinate achievement in the second language (RB).
4.6.4.6 Semantic Range Test
Mackey and Savard (1967) have described an index of "coverage" which assigns a numerical value to words on the basis of a combination of characteristics which include power of definition, inclusion, extension, and combination-all measures relating to some aspect of the semantic range of words. It is proposed that these or equivalent indices be adapted for the use of individual speakers in order to permit comparisons of individual scores to the population norm. These would then become indices of RB.
4.6.4.7 Word Association Test
Words in both languages are given as stimuli and the subject is free to respond in either language by giving as many words as he can think of within a certain period of time, usually 60 seconds (see Nbble, 1952). The following indices can be derived from this procedure: (i) number of responses to the stimuli in L1; (ii) number of responses to the stimuli in L2;(iii) the difference between (i) and (ii); (iv) number
of responses in the language of L1 for stimuli in L1; (v) number of responses in the language of L2 for stimuli in L2; (vi) number of responses in L2 for stimuli of L1; (vii) number of responses in L1 for stimuli of L2; (viii) number of translation responses for stimuli in L2 number of translation responses for stimuli in L2;(ix) proportion of responses in L2 for Stimuli in L1 [i.e., [(vi)/(I)];(x) proportion of responses in L1 for stimuli in L2 [i.e., [(vii)/(ii)];(xi) number of "bilingual" responses for L1, where applicable (i.e., cognates or homographs for related languages) (xii) number of "bilingual" responses for L2, again where applicable.
4.6.4.8 Paired Associate Task
This test is a measure of learning ability with meaningful materials. The bilingual's comparative performance in the two languages may be estimated, and the general level of his performance compared to the norm within the population. Various procedures for paired associate learning are possible. A convenient procedure which does not require a memory drum is that used in the Modern Language Aptitude Test (Carroll and Sapon, 1958).
4.6.4.9 Semantic Structure of Affective Meaning
This rather complex procedure consists of the following steps: (a) semantic differential ratings of concepts in both languages are obtained from the bilingual;(b) these ratings are then analyzed by means of Tucker's three mode factor analysis (see Levin, 1965) and compared to group factor analyses using various factorial similarity indices (see for example Kuusinen, 1967). These measures should reveal significant departures of semantic structure from the cultural norm (RB and RC).
4.6.4.10 Word Association Structure
There are several methods which have been used to arrive at indices of similarity in word association structure (see Deese, 1965). Kolers (1963), Rosenzweig(l959), Lambert and Moore (1966), Jenkins and Russell (1960), and Moran (1966) have made use of some of these indices in comparisons of word association structure across languages.
PROBLEMS IN ASSESSING PROFICIENCY 183
4.6.4.11 Semantic Cluster Analysis
This measure is an index of vocabulary structure based on factor analysis and cluster analysis techniques described in Osgood, Suci, and Tannenbaum (1957) under the heading "concept factor analysis." Visual representation of vocabulary structure is possible and facilitates comparisons (see for example, Osgood and Luria, 1954 and Jakobovits and Osgood, 1967).
4.6.4.12 Spew Hierarchy
Relying on certain complex arguments presented by Underwood and Schulz(1960) and summarized in their "Spew Hypothesis," Osgood, Archer, and Miron(1962) developed elicitation procedures which yield association data that when treated by means of the information statistic H, reveal a response hierarchy that is stable and haracteristic of a language/culture group. Comparisons of such "spew hierarchies" (to which word frequency distributions are related in a crude way) should reveal the degree of the bilingual's coordinateness.
4.6.4.13 Word Categorization Task
There is considerable evidence (Mandler, 1966; Jenkins and Russell, 1952; Cofer, 1965) that the lexical items in the vocabulary of an individual are structured in his memory along patterns that are organized in clusters which can be revealed under conditions of free recall and spontaneous emissions. There would be considerable interest in determining the stages of development of such vocabulary clusters in learners of a second language and the extent to which compound-coordinateness affects the establishment of such clusters.
4.6.4.14 Word Stereotypy
This measure is related to the H index used in the determination of the spew hierarchy described above (4.6.4. 12). It gives an indication of the "richness" of a concept and its stereotypy, that is the extent to which subjects under conditions of a restricted word association task will tend to give the same high frequency responses. Again, the interest here, as in the last few measures just described, lies in the
determination of the degree of correspondence between the bilingual's vocabulary structure and that of the population norm.
4.6.4.15 Translation Test
Facility in translation as indexed by latency and accuracy measures may be related to compound-coordinate bilingualism in so far as it involves language switching and semantic correspondence, as discussed earlier in this section. It would appear that a translation test involving sentences or meaningful discourse should be more revealing than the translation of single words since the latter is likely to be either too simple and automatic-when the word in isolation has a single meaning, or too difficult and conflictual-when the word has multiple meanings. One should also keep in mind the possibility that translation may be a separate special skill only partially related to the usual skills of decoding and encoding required in everyday discourse. If this were the case, translation should not be used as a measure of language proficiency.
4.6.4.16 Language Alternation Test: Encoding
The purpose of this test is to force the bilingual to encode in both languages in rapid succession. The task has two levels of difficulty: one consists of naming objects shown to the bilingual in each language, alternating with each new stimulus; the other level consists of persuading him (a difficult task with some!) to tell a story while alternating languages for "chunks" of discourse as he goes along. The size of these chunks (i.e, words, phrases, sentences) at reasonable fluency is likely to be the significant variable.
4.6.4.17 Language Alternation Test: Decoding A
Here the bilingual's ability to understand written and oral material is being tested. The text is made up by alternating languages using phrase chunks. Here is an example of a sentence in such a text using English and French as the two languages: "The government of Henry the Seventh, de son fils, et de ses petits enfants was, on the whole, plus arbitraire, than that of the Plantagenets."
PROBLEMS IN ASSESSING PROFICIENCY 185
4.6.4.18 Language Alternation Test: Decoding B
This differs from the previous test in that the alternating chunks are smaller. Here is an English-French example: "A complex appareil used in deux ou plus parts de la world suggests un rapport between eux in very measure to leur complexity." Understanding of both kinds of texts can be evaluated by requiring the subject to answer questions based on the text. Latency measures may also prove to be useful.
4.6.4.19 Pronunciation Switch Task
This test is added for the special case where the two languages possess homographs either because of linguistic closeness (as in English and French) or because of linguistic contact (e.g., English origin words in the linguistically unrelated languages of India that use a common script). A list of homographs is prepared and the bilingual is instructed to read it as fast as he can by alternating languages with each word. Both speed and accuracy are recorded. Here is an ex- ample of such a list in English and French: FORMER, OR, CHAIR, SEIZE, LAME, POUR, TAPER, COMMENT, HABIT, MAIN, SON, COIN, FOUR, CASE, ON, CENT, RIDE, CHAT, DRESSER, BUT, PIN, SORT, DOT, LOIN, BOND.
4.6.4.20 Capsule Sentences
The peculiar (but nevertheless, regular) transformations that characterize newspaper headlines result in sentences whose meaning often eludes non-native speakers of a language whose proficiency otherwise is quite adequate. This test is designed to measure the bilingual's knowledge of these peculiar transformations by requiring him to recover the original sentence from the "capsule" sentence. For example: given the headline "Tells Parties for Army Aids by U.S. Firm," the subject may recover "He reveals that parties are given by a U.S. firm on behalf of army personnel." The recovered sentence should contain not only the factual information but also the sub rosa implications ("reveals" for "tells").
4.6.4.21 Sentence Transformation Test
The purpose of this test is to measure the bilingual's capacity to transform sentences from one mode to another (e.g., from active to passive; from negative question to active declarative, etc.; see Miller,1962). Error and speed are the two relevant measures.
4.6.4.22 Cloze Procedure
In this test a text is mutilated by deleting every fifth word and the incomplete text is then given to the subject for completion. The choice of words in the bilingual's attempt to restore the text is compared to the original or to the performance of native monolinguals. Accuracy can be scored in terms of exact type or in terms of semantic correspondence to the original. The Cloze procedure was first described by Taylor (1953).
4.6.4.23 Subjective Evaluation of Competence
A competent bilingual judge, in a face-to-face interview with the subject, evaluates the bilingual's competence in five areas: phenol- ogy, syntax, vocabulary, oral comprehension, and fluency. The judge has available a rating sheet which he fills out while he is conversing with the subject. Here is an example of such a rating sheet:
Accent
foreign:-----:-----native
Grammar
inaccurate:-----:----accurate
Vocabulary
inadequate:------:---adequate
Fluency
uneven -- : --- even
Oral Comprehension
incomplete : ------: --- complete
PROBLEMS IN ASSESSING PROFICIENCY 187
The six-point scale is defined, from left to right, as follows: extremely, quite, more left than right, more right than left, quite, extremely. A pretest of the reliability of this scale using two sophisticated judges rating French-English bilinguals yielded interjudge correlations in the high 80's and low 90's for the five scales.
4.6.4.24 Subjective Evaluation of Compound-Coordinateness.
During the interview the judge engages the bilingual in conversation about his language acquisition history and usage, asking sufficiently specific questions to enable him to rate the individual on four measures of compound-coordinate bilingualism: compound acquisition, coordinate acquisition, compound usage, and coordinate usage. In each case a rating is given which may vary from 0 to 100. In a pretest using French-English bilinguals, interjudge reliability was found to be in the high 60's and low 70's for the four measures. Compound acquisition and usage correlated positively with each other (in the 70's) and negatively (in the 60's) with coordinate acquisition and usage (the latter also correlating positively with each other, in the 70's).
4.6.4.25 Subjective Evaluation of Attitude
Two measures are included under this heading: an orientation rating (discussed earlier in this Chapter) and a rating of the attitude of the subject toward bilinguality: how advantageous or important it was to be bilingual, whether he'd require his children to achieve a high degree of bilingualism, and so on. Both of these ratings may vary from 0 to 100.
4.6.4.26 and 27 Proficiency Tests in Both Languages
In many languages special proficiency tests have been developed and are widely known and used. Where these are not available, achievement scores in language courses or some other substitute tests would have to be used. In all cases, the question of validity will have to be carefully considered. These problems cannot be dealt with here and the reader is referred to the discussions in Lade (1961)and Mackey (1965), and the discussion in Chapter 3 and at the beginning of this Chapter.
4.7 TRANSFER EFFECTS FROM ONE LANGUAGE TO THE OTHER
Transfer is perhaps the single most important concept in the theory and practice of education. In its most general form the principle of transfer refers to the hypothesis that the learning of task A will affect the subsequent learning of task B and it is this expectation that justifies educational training in schools as a form of preparation for the subsequent demands that society will impose upon the individual. In fact, some educators (e.g., Ferguson, 1956) consider transfer as the general case and learning as the specific case where task B is highly similar to task A and learning of task B thus constitutes further practice on task A.
The general paradigm that applies to all transfer studies can be summarized as follows:
Experimental Group: Learn A Learn B
Control Group 1: Learn B
Control Group 2: Learn X Learn B
In the experimental group, the transfer effects that are presumed to operate are of two types: specific and nonspecific. The former are attributed to the habits acquired in learning task A which are utilized in the learning of task B. The latter are attributed to general factors involved in the learning of any task such as warm-up effects and the development of learning sets. By comparing the performance of the experimental group to that of Control Group l, the combined effects of specific and nonspecific transfer can be assessed, whereas by com- paring their performance to that of Control Group 2, the specific transfer effects can be assessed without the contamination of the nonspecific factors. The interest in transfer in second language learning has been most explicit in attempts at specifying interference effects through contrastive analyses between the second language (L2) and the first (L1). The most extensive analysis of the potential mechanisms of interference between two languages is that of the classic work of Weinreich (1953). Today contrastive analysis is a widely accepted technique among second language teachers and is commonly used to guide classroom and laboratory activities in terms of the type of materials to be introduced at specific times and the nature of the
PROBLEMS IN ASSESSING PROFICIENCY 189
practice exercises to which the learner is to be exposed. In addition the operation of transfer is implicitly assumed both within the language learning situation and in the applicability of the course instruction to skills the individual will be called upon in using the language outside the school. The expectation of transfer within the language instruction situation relates to the notion of cumulative gains to be obtained so that language acquisition is conceived of as a step-by-step building process of successively more complex stages and whatever is learned in the early stages is expected to be retained and transferred to later stages. There is the further expectation that knowledge of the common elements which constitute the structure of the language will transfer to the various skills; for instance, a grammatical relation learned orally is supposed to facilitate the acquisition of the same relation in reading or writing.
The external applicability of the classroom training similarly rests on assumptions of transfer effects so that a student drilled in pattern practice is expected to be able to transfer these skills to the real life situation of an ordinary conversation with a native speaker. Reading of literature in the second language and indirect exposure to the foreign culture through the content of sentences, pictures, and films are similarly justified in that such knowledge is expected to facilitate later understanding of "field-like" situations. Obviously then, a major portion of what constitutes second language teaching today rests its justification on the explicit or implicit recognition of the operation of transfer effects.
4.7.1 The Extrapolation of Knowledge
The literature on transfer (when the term is considered in its broadest sense) is possibly more extensive than that on any other topic in psychology and education. It can be subdivided into two categories: on the one hand, one finds a mass of experimental reports on highly specific aspects of transfer effects under laboratory learning conditions; on the other hand, one finds a substantial body of writings which deals with the operation of transfer in real life settings such as the school and industrial training programs. In a pattern now familiar in the social sciences whereby scientific exactitude is in-
versely proportional to realistic concerns, we find that whatever specific knowledge there is on transfer concerns tasks and settings that are largely irrelevant to crucially important educational concerns, and the amount of specific and reliable knowledge on the latter is practically nonexistent. Careful reviews of the vast literature pertaining to transfer are invariably pessimistic. Concerning the work on paired-associate learning Betts (1966) concludes that "the data presented here and the equations derived from these data are much too tentative to be offered to educators as practical solutions to their problems" (p. 135). Concerning the work on interference theory in rote learning one of the most authoritative experts on the subject recently con- cluded that "thus far, there is little empirical evidence in support of [the] assumption [that] forgetting outside the laboratory is a function of the same variables and represents the same processes as are observed in formal studies of interserial interference" (Postman,1961, p. 166). In an extensive, formal, and detailed report on restricted aspects of learning Russian under controlled laboratory conditions, Crothers and Suppes (1967) conclude that "...in these second language learning experiments the proportion of correct responses usually seemed to depend more on the detailed structure of the item than on the presentation-order variables that were examined" (p. 311). This pessimistic conclusion is particularly note-worthy because the work of Crothers and of Suppes is often cited as examples par excellence by proponents of the formalized experimental approach to problems in learning and teaching.
I am citing these conclusions, which are representative and describe accurately the "state of the art," because I want to draw attention to a tendency that has become quite widespread in the social sciences generally and in education, including second language teaching, specifically. This tendency consists of an attempt to justify teaching practices of a particular kind by appealing to experimental findings in the laboratory that deal with materials of an entirely different order of complexity than the learning task in question. An example of this, which by now ought to be notorious, is the description of language learning and language behavior that is based on conditioning principles developed through laboratory studies with rats and rote learning studies with human subjects. A more recent
PROBLEMS IN ASSESSING PROFICIENCY 191
instance, one which is very much topical today, concerns the justification of pattern drill practice in second language teaching by appealing to laboratory experiments on motor skills and to principles of generalization and transfer derived from stimulus conditioning and verbal rote learning studies. While I think it is both justifiable and desirable to apply knowledge which one might gain from laboratory experimentation to educational and social concerns, I do not think that this extrapolation carries greater scientific merit than the extrapolation that comes from knowledge acquired outside the laboratory through observation, experience, and insight. The extrapolation process in both instances is not a rigorous matter and cannot be scientifically justified. Chomsky's evaluation of this tendency has been quite harsh: "...there has been a natural but unfortunate tendency to "extrapolate,' from the thimbleful of knowledge that has been attained in careful experimental work and rigorous data processing, to issues of much wider significance and of great social concern. This is a serious matter. The experts have the responsibility of making clear the actual limits of their understanding and of the results they have so far achieved, and a careful analysis of these limits will demonstrate, I believe, that in virtually every domain of the social and behavioral sciences the results achieved to date will not support such 'extrapolation' "(1968, Preface).
I suspect that these remarks will be welcomed by those practitioners of the teaching arts who have always preferred to follow their flair and who have maintained a distaste for the analytic and molecular formulations of the experimentalists. However, I would like to dispel immediately any sense of security that a smug subjectivity may provide. It would be a mistake to assume that because extrapolation from the laboratory is not a scientific process, therefore we must give up all attempts at developing systematic evaluation criteria whereby teaching methods can be made more effective. It is possible to be systematic without at the same time being rigid about the manner in which we make additions to our knowledge. The analysis of transfer effects in second language learning which I sketch in this section is molecular and has, I believe, a systematic character which could potentially organize the knowledge that language teachers already have from their experience in the classroom in such a way that they could make more effective use of such knowledge by
rendering it more explicit. Hopefully, it might have the additional quality of enabling them to increase the usefulness and signficance of the knowledge they may gain through additional and continuing experience in teaching.
4.7.2 Some General Principles of Transfer
In this attempt to summarize some principles of transfer I will rely on the experimental work that is extant without, however, the pretense that the statements are scientifically rigorous. One should more properly consider them as fairly specific descriptions of current knowledge about transfer effects, knowledge which was arrived at by various procedures, which include interpretation of experimental data, observational data gathered through the experience of educators, and selections from what is common knowledge in the culture (sometimes called "common sense").
A general formulation of the transfer problem must deal with five basic elements: task A, training or practice on task A, training or practice on task B, and the relation between task A and task B. Since our specific interest here lies in language learning, let us refer to the five elements as follows: PL1 (proficiency on task A), PL2 (proficiency on task B), TL1, (training in L2), TL2 (training in L2) and RL1-L2 (the relation between L, and L2). The transfer effects to be R expected in second language learning can then be expressed by the following formula:
PL2 = f(PL1, tL1, tL2, RLI-L2)
This formula says that attained proficiency in L2, will be some joint function of attained proficiency in L1, training in L1, training in L2, and the relationship between L1, and L2. Consider now some of the possible relations that may hold in this formulation (see Ferguson, 1956) and how they are reflected by this formula. In the case where tL2 is constant and RL1-L2 is fixed we have:
PL2 = f(PL1)
PROBLEMS IN ASSESSING PROFICIENCY 193
where PL1= f(tL1). This says that given fixed amounts of training, proficiency attained in L2 is a function of native language proficiency, which no doubt reflects a general language ability. It is known that verbal ability and verbal intelligence are significant factors in second language learning although the shared common variance is fairly small. Furthermore, certain special abilities whose development may very well depend on TL1 factors in early life, such as memory for coding phonetic materials and sensitivity to grammatical relations (see Carroll, 1966), seem to share substantial common variance with PL2
In situations where tL1 is very large, has attained a stable value(e.g., an adult learning a second language), the general formula reduces to:
PL2=f(tL2)
where RL1-L2 is assumed fixed. This expresses the usually recognized function in second language learning where achievement is said to be a function of training in that language.
Where tL1 is very large, PL1 stable, and tL2 fixed, the general formula reduces to:
PL2 = f(RL1-L2)
which can be recognized as expressing the common knowledge that "exotic" languages are more difficult to learn than the more familiar ones.
An interesting feature which emerges by rearranging the elements in the general formula is expressed by the equation:
PL1=f(PL2, RL1-L2)
This equation says that when RLI-L2 has a particular value (e.g., with closely related languages) the learning of a second language may come to have interference effects on the mother tongue. This refers to what I have called elsewhere "backlash interference" as distinct from the more familiar linguistic interference which is expressed by the equation:
PL2 = f(PL1,RLI-L2)
One of the factors which is known to be important in the operation of transfer concerns the common elements between the two tasks; in the present discussion this refers to RLI-LZ. I have pointed out earlier that this relation has three different aspects, as follows:
1. RA = f(XL1 - XL2)
2. RB = f(XL2 - XL2)
3. RC = f(XL1 - XL1)
where X refers to a particular speaker (either of L1 or of L2) and X refers to an idealized monolingual of L, or L2. Equation 1 (RA) refers to the relation between the bilingual's two languages; equation 2 (RB) refers to the relation between the bilingual's second language and that of an idealized monolingual speaker of L2, equation 3 (RC) of an idealized monolingual speaker of L1. These three relations are theoretically independent and their simultaneous value within the various linguistic domains defines the bilingual's position on the compound-coordinate continuum.
4.7.3 Transfer Effects in Second Language Learning
In attempting to assess the transfer effects in second language learning it will be necessary to consider the various relations separately since the same situation may have differential effects on each of them. Consider for instance the values to be expected for each of these relations on the basis of Osgood's three laws of interlist similarity (as reviewed by Jung, 1968). The three principles can be stated as follows:
1. Two tasks in which the stimuli are the same but the responses different (A-B, A-D) will yield negative transfer which increases the more the responses get dissimilar1 .
1Following established practice in the literature on paired-associate learning, the first letter in expressions like A-B or A-D refers to antecedent stimulus factors while the second letter
PROBLEMS IN ASSESSING PROFICIENCY 195
2. If the responses are the same, but the stimuli are different (A-B, C-B), positive transfer is expected which increases the more the stimuli get similar.
3. When both stimuli and responses are different in the two tasks (A-B, C-D), negative transfer is expected which increases the more the stimuli get similar (see Figure 4).
FIGURE 4. TRANSFER EXPECTATIONS IN SECOND LANGUAGE ACQUISITION BASED ON OSGOOD'S TRANSFER SURFACE. (OSGOOD. 1953. FIGURE 172. p. 532.)
refers to consequent response factors. The comma in expressions like "A-B, A-D" separates the first task from the second. Identity of letters is intended to indicate that the same factors are believed to hold in the two tasks. Thus, the expression "A-B, A-D" refers to two tasks in which the stimulus conditions are the same but the response conditions different. Please note that "stimulus" and "response" as used here do not carry the usual rigorous denotations and connotations which they have in the literature on "S-R theory."
In second language learning "stimulus similarity" may be taken to refer to the environmental conditions that are antecedents to linguistic utterances: these include the physical, external environment (to be referred to as EL1 Or EL2) as well as the mental, internal environment (to be referred to as EL1, or EL2). "Reponse similarity" may be taken to refer to the structural relations between the linguistic systems of the two languages (to be referred simply as L1 and L2).
We are now ready to express the transfer expectations in second language learning which Osgood's three laws would suggest (see Figure 1):
1. In a compound setting with the indirect method of teaching (where EL1 and EL2 are highly similar, and so are EL1 and EL2), we would expect negative transfer which increases the more L1 and L2 get dissimilar.
2. With closely related languages (where L1 and L, are similar) we would expect positive transfer which increases the more EL1 and EL2 (and EL1 and EL2) get similar, i.e, in a compound setting with the indirect method.
3. When EL1 and EL2 (and EL1 and EL2) are relatively different (as with the direct method in a coordinate setting) and L1 and L2 are different (as with unrelated languages) negative transfer is expected which increases as EL1 and EL2 get similar.
These principles can be restated from the point of view of the interest in maximizing positive transfer and minimizing negative transfer.
Hypothesis 1. With unrelated languages a coordinate setting (symbolized as CR) will yield less negative transfer than a compound setting (symbolized as CP).
Hypothesis 2. With related languages a CP setting will yield more positive transfer than a CR setting.
Note that the second hypothesis here derived seems to be in conflict with the widespread inclination among most proponents of the audiolingual method to maintain as much as possible a direct learning method and to avoid as much as possible any use of L1 in the learning situation. There is some evidence, however (Lambert,
PROBLEMS IN ASSESSING PROFICIENCY 197
Gardner, Barik, and Turnstall, 1963), which supports the expectation that individuals studying a second language that is closely related to their first language (in this case French being studied by Americans) do better in the course if they show evidence of allowing a CP interaction in their meaning systems. In our terms, EL1 and EL2, were not kept separate even though there was an effort on the part of the instructors to maintain overt environmental separation (EL1 and EL2)by offering to the American students enrolled in the intensive summer course the distinctive French-Canadian atmosphere in bilingual Montreal and not permitting, under threat of expulsion, any use of English. Despite these efforts on the part of the instructors the students who acted internally as CP bilinguals achieved higher grades. To be sure, this evidence is only suggestive with respect to the transfer expectation expressed as Hypothesis 2 and, because it seems to conflict with our common sense expectations, we should be wary of it. Nevertheless, now that we are alerted to its possibility, we should watch out for possible confirming evidence whenever possible. Jung (1968) describes an elaboration contributed by Martin of Osgood's three laws of similarity. Consider the three components of specific transfer in two successively learned paired-associate tasks. They are response learning, the establishment of forward associations, and the establishment of backward associations. The findings are apparently consistent with the following statements:
1. With the A-B, C-B paradigm (different stimuli, same response) there is positive transfer from the first task (A-B) to the second task (C-B) for response learning.
2. With the A-B, A-D paradigm (same stimuli, different responses), there is negative transfer for the establishment of forward associations (A-B connections in the first task conflict with the A-D associations in the second task).
3. With the A-B, C-B paradigm (different stimuli, same responses), there is negative transfer for the establishment of backward associations (B-A connections established during learning of A-B associations in the first task) and conflict with B-C connections that tend to be established during C-B learning in the second task.
4. Low degree of first learning and low degree of response meaningfulness facilitate response learning and minimize the establishment of backward associations. Hence these two factors in combination will result in greater positive transfer in the A-B, C-B paradigm (since they augment the positive transfer mentioned in statement 1 and decrease the negative transfer referred in statement 3). With the A-B, A-D paradigm, these two factors produce less negative transfer since they minimize the effect expressed in statement 2.
Now let us examine what kind of expectations these principles would lead us to in the second language learning situation to see if this exercise will help us organize our knowledge.
4.7.3.1 Coordinate Training with Related Languages
This corresponds to the A-B, C-B paradigm where EL1 and EL2 (and eL1 and eL2) are made as different as possible and response learning (L1 and L2) are relatively similar. According to Hypothesis 2 derived earlier from Osgood's three laws of similarity (see also Figure 4), the overall effect to be expected is zero transfer (as opposed to the positive transfer effect with CP training). With Martin's elaboration of Osgood's three laws we can specify in greater detail the effects in several areas (see Table 1).
4.7.3.1.1 Encoding Skills.
Positive transfer is expected for response learning of the second language due to the similarities in structure to the first language; however, CR training may reduce this effect because of the emphasis on separate contexts and the discouragement of the use of the learner's first language. The net effect is therefore in doubt.
4.7.3.1.2 Decoding Skills.
Because the background context is made different through CR training while the structures of the languages remain similar, negative transfer is predicted due to the expected tendency of the learner to relate L2 responses to L1
environment. Symbolically, this inference pertains to the relation L2-EL2 or L2 -El2 which refers to back-
PROBLEMS IN ASSESSING PROFICIENCY 199
PROBLEMS IN ASSESSING PROFICIENCY 200
PROBLEMS IN ASSESSING PROFICIENCY 201
ward associations (where EL2 - L2 Or eL2 - L2 corresponds to the forward associations involved in 4.7.3.1.1 above). Since the L2 - eL2 relation refers to the decoding process (understanding),we would therefore expect less efficient acquisition of the "passive" skills.
4.7.3.1.3 Decoding Skills (early childhood bilingualism).
Because a young child's experience with the mother culture is more limited, the negative transfer expected in 4.7.3.1.2 should be minimized. According to the transfer principles discussed earlier, lower degrees of first learning should facilitate backward associations (L2 - EL2); hence, the expectation will favor early childhood bilingualism in decoding skills. (The same situation would hold for immigrants in a country where their L1 usage becomes unstable.)
4.7.3.1.4 Backlash Interference.
Departures from nativeness involving one's mother tongue as a result of contact with L2 are expected to be less serious if the two cultures, as well as the two languages, are similar, as is often the case. Since we are dealing with related languages, the negative transfer in backward associations (i.e9 in the decoding skills represented by L2- EL2 and L1 - EL1) will be a source of difficulty as stated in 4.7.3.1.3 above. Thus with a CR setting, RC=f(XL1-XL1)will be maximized. As will be recalled from the earlier discussion, this refers to the possibility of backlash interference (when it is minimized) which has the effect of reducing the nativeness of the developing bilingual in his mother tongue. It follows that a CP setting with the indirect method when the languages are related should minimize Re.
4.7.3.1.5 Backlash Interference (early childhood bilingualism).
Less interference is expected with lower degrees of first learning (as discussed earlier), i.e., when experience with the mother culture is still limited as in early childhood bilingualism.
4.7.3.1.6 Interlingual Interference.
As in 4.7.3.1.1, positive transfer is expected due to the similarities between L1 and L2 leading to a reduction in response interference [RA = f (XLI - XL2)].
4.7.3.1.7 Degree of Nativeness.
Degree of nativeness in L2 [i.e., RB=f(XL2-XL2)] would be expected to be high with CR training, although when the two cultures, as well as the two languages, are closely related, this factor of setting is not likely to be as important as in the case when the cultures are quite different. Note that similarities in culture and language are expected to facilitate nativeness in L2, but CR training may not capitalize on this effect; hence, in relative terms, this is not the most efficient training condition.
A similar analysis can be undertaken for the three additional cases where CR training is given for unrelated languages and CP training for either related or unrelated languages. The results of the analysis for the four cases are summarized in Table 1.Seven areas of transfer are examined. As can be seen, the most efficient condition for second language acquisition is that of CP training with highly related languages; CR training with highly related languages is not as efficient, a conclusion which is in agreement with the statement in Hypothesis 2 considered earlier. The least efficient condition is that of CP training with unrelated languages; CR training with unrelated languages is more efficient. An interesting effect which the table points up is the fact that expected facilitation effects may not materialize due to training conditions (e.g., 4.7.3.1.1 and 4.7.3.1.7 for the CR, Related condition); similarly, expected interference effects may not materialize thanks to training conditions (e.g., 4.7.3.1.1 and 4.7.3.L~ for the CR, Unrelated condition). In some instances, the training condition converts an expected interference effect into facilitation (e.g., 4.7.3.1.4 and 4.7.3.1.7 for the CR, Unrelated condition).
The effects summarized in Table I should be viewed in relative rather than absolute terms. For instance, a question mark indicates that relative to some other condition, the expectations of facilitation are not as good (e.g., 4.7.3.1.7 for the CR, Related condition relative to the CP, Related condition). Furthermore, the processes and conditions to which the table refers are abstract conceptualizations of aspects of language behavior and conditions of learning. In real life situations these neat abstractions become "fuzzy." Thus what has been talked about as "CR or CP training" becomes in fact degrees of
PROBLEMS IN ASSESSING PROFICIENCY 203
compoundness and coordinateness, and what is referred to as "response learning" or "decoding skill" turns out to be in practice a conglomeration of a multitude of skills and innumerable facets of knowledge each of which can be focused upon for observation and analysis. These practical considerations do not invalidate the abstract analysis, but they indicate that one must always be clear about what particular condition or language behavior process is in question. In a subsequent section I shall discuss the kinds of techniques and instruments that are available for assessing particular language behavior functions.
4.7.4 Areas of Transfer in Language Learning
Up to now I have been considering transfer effects in language behavior without specifying the specific skills involved. A systematic analysis must concern itself with specific skills within specified domains (e.g., syntactic competence in speaking, or vocabulary knowledge in reading) and, in addition, with the degree of proficiency attained in each of these skills at the time the analysis is relevant. A model of CP-CR bilingualism which I have described earlier specified changes in second language acquisition along four different dimensions: attained proficiency in L1 and L2, a three-fold relation between L1 and L2 (RA, RB, and RC), psycholinguistic area of functioning, and development in time. The general transfer equation described above [namely, PL2=f(PL1,tL1, RL1-L2)] and its various derivatives did not specify the nature of the functional relation. Theoretically the nature of this function can be zero, positive, or negative, and can be linear or nonlinear. An example of a nonlinear transfer effect is provided by the case where degree of practice affects amount and type of transfer. Thus phonological interference effects due to dissimilarity between L1 and L2 (e.g., 4.7.3.1.6 in the table above) are known to be particularly strong during the early stages of second language learning and diminish with time and practice. On the other hand, backlash interference (e.g., 4.7.3.1.4 in the table above) is expected to be strongest at later stages of L2 learning and to be minimal at the beginning.
Obviously, the transfer expectations expressed in the table above must be further qualified with respect to the nature of the transfer function that holds for each specified skill, within the specified domain, for specified time and amount of practice factors.
Let us turn to an analysis of the mechanism of transfer within the various language domains. Weinreich (1953) defines interference as "those instances of deviation from the norms of either language which occur in the speech of bilinguals as a result of their familiarity with more than one language" (p. I). The terms "bilingualism" and "language," Weinreich suggests, ought to be defined in a technical sense so that they would refer to the presence of two linguistic systems whether these be recognized as separate languages or not. From a theoretical point of view the mechanism of transfer would seem to be the same whether the contrasting forms and patterns are taken from different languages, different varieties of the same dialect, or even different codes from the same dialect; the amount of interference will depend on the number of "mutually exclusive forms and patterns," according to Weinreich who, at the same time, ex- presses pessimism with respect to the possibility of specifying (at this stage of our knowledge) the "total impact of one language on another." I don't think this pessimism is justified since it is common knowledge that within standard learning situations existing in our High Schools and Colleges some languages are learned more easily than others and it would certainly be possible to make overall statements about transfer relations between L, and Lx, Ly, Lz, etc., on the basis of such learnability data. In fact, Cleveland, Mangone, and Adams (1960) present a table (p. 250-251) which specifies the time requirements for attaining specified levels of proficiency in various languages for individuals with certain predetermined levels of aptitude enrolled in a highly intensive language course. Their table shows that Americans with average aptitude,can attain proficiency in speaking Italian, French, Spanish, Rumanian, or German (among others) in two-thirds of the time it takes them to achieve the same proficiency in Russian, Creek, or Finnish (among others),and in only half the time it takes them to learn the same functional proficiency in Chinese, Japanese, or Vietnamese (among others). At any rate Weinreich's suggestion of estimating the overall impact of one language on another by describing the various forms of interference
PROBLEMS IN ASSESSING PROFICIENCY 205
and tabulating their frequency would certainly be useful for teaching purposes as an aid in anticipating areas of difficulty. I would suggest that in addition we should be aware of the areas of overlap and similarity since, as pointed out in the discussion above, these similarities can be turned into facilitation effects through positive transfer by manipulating the teaching conditions in an appropriate fashion.
A transfer analysis between two languages is likely to be very involved. To indicate some appreciation of the complexity of such a task, here in summary form is the analysis which appears in Mackey (1965). He discusses the various relations between L, and L, along four levels: phonetics, grammar, lexicology, and stylistic usage.
1. Phonetics
1.1. Articulation
1.1.1 Vowel phonemes
1.1.2. Consonant phonemes
1.2.Catenation
1.2.1. Permissible combinations
1.2.2. Combinations causing sound changes
1.2.3. Linking and separating words
1.3.Rhythm
1.4.Intonation
2. Grammar
2.1 Differences in system
3. Lexicology
3.1. Form
3.2. Word meaning
4. Stylistic usage
Now let us consider a few specific examples of transfer effects to be expected between say English and French. The absence of a vowel phoneme (1.1.1.1. above) should be less interfering than its presence in a slightly different form (1.1.1.23. The former corresponds to an A-B, A-D paradigm which, if B and D are sufficiently distinct should produce close to a zero transfer effect, while the latter corresponds to an A-B, A-Pr paradigm which produces a distinct negative transfer effect. Thus an English speaker learning French should acquire with- out difficulty the vowels [o] t8t, [e] th6, and [oe] boeuf, which do not exist in English. If the vowels are very similar, as in the French [aj] ail and the English [al] eye, we would expect facilitation through positive transfer. Similar transfer effects can be illustrated with an example from the grammatical domain. Presence or absence of cases (2.1.1) as between English and French on the one hand, and Latin and German on the other, would be less of an interference than instances where both languages have common types of inflectional categories which vary in function (2.1.2). For example, the English and French verb systems (2.1.2.1) overlap in tense system but dif- ferentiate as to their functions. Thus, English I hear he's coming (hear = present tense) is rendered in French J'ai appris qu'il viendra Cai appris = perfect tense). Similarly, English I hope he comes (comes = present) is rendered in French J'espere qui'l viendra (viendra = future). At the lexical level the transfer effects of cognates are well known both in terms of their facilitative effect (as in the huge savings involved in learning French vocabulary for an English speaking person) as well as their negative effects (as with the notorious pitfalls involving the faux amis). The transfer problems involved in word meaning and stylistic usage seem even more complex than those involving the phonological and grammatical aspects, undoubtedly because our methods of analysis are so much less systematic at the moment in the former instance. However, I
PROBLEMS IN ASSESSING PROFICIENCY 207
believe that systematization of our intuitive knowledge about meaning is possible at this time (see my attempt in Jakobovits, 1968c).
4.7.5 Operation of Transfer Effects: A Surface Feature?
The implications for transfer effects in language learning of the current return of interest in the universal laws that underlie the structure of all natural languages and the related interest in nativistic theories of language deserve our attention. There are certain fundamental differences which must be noted between the acquisition of a second language beyond adolescence and first language acquisition during infancy. In the first place the notion of second or foreign language aptitude has no parallel in first language acquisition. The latter appears to be a maturational process whose development is universal and regular throughout the human species. Exposure to adult language and communication constitutes both the necessary and sufficient condition for the attainment of native proficiency. In the second place it is a highly probable fact that unless the first language is acquired by early adolescence, the capacity to acquire any human language is lost forever. No such effect is true of second language learning. Thirdly, the acquisition of speaking fluency in a second language under intensive training conditions by a well motivated adult with high aptitude is achieved in the order of between 400 to 600 hours and in slightly less than two months (see Carroll, 1963). The time factor involved in first language acquisition during infancy is of an order of magnitude that is nine times larger when one considers the time of onset of speech at 18 months and mastery of complex syntactic combinations at 36 months of age.
These three basic differences between first and second language acquisition strongly suggest that the processes involved in the two learning situations are distinct in some fundamental way. Conditions of teaching and transfer effects appear to be quite important in second language acquisition; they have little or no effect in first language acquisition (see Chapter 1). First language acquisition can aptly be described as a discovery procedure guided by innate universals, where the paradigms of concept formation and hypothesis test-
ing appear much more relevant than the conditioning paradigms that have been applied to the second language learning process (e.g., in Rivers, 1964). I think these observations are significantly related to the fact that sophisticated linguistic descriptions of the competence of native speakers concern themselves to a considerable extent with the underlying structure of language, whereas contrastive analyses between two languages have dealt with relatively surface features of the two languages. The latter fact may of course be the result of the relative recency of transformational descriptions of grammar but I am suggesting that,in view of the distinctions just raised between the processes involved in first and second language acquisition, similarities and differences of surface features may be more relevant for the operation of transfer effects in second language learning than deep structure relations, especially when one believes that at some level of depth all natural languages are describable in terms of one universal system. Perhaps it would be more useful to talk about descriptions at different relative depths rather than surface versus deep structures,but until we know more about this problem I see no theoretical justification for giving a greater weight to transformational descriptions of grammar over phrase structure descriptions in guiding our teaching approach to second language learning.
4.7.6 Attitudinal Factors in Transfer Effects
In my analysis of the variables to be considered in the operation of transfer effects I have thus far limited myself to the structural factors that emerge from a contrastive analysis between the two languages and to the manipulation of environmental conditions of acquisition. But as Weinreich (1953) has pointed out it is necessary to consider nonstructural factors that relate to the individual's contribution to the realization of transfer effects in his speech. I shall refer to these as "attitudinal" factors. I would like to consider three of these, the first two being those discussed by Weinreich.
The first of these concerns the individual's tendency to resist transfer effects which would cause serious confusion in his attempt at communicating in L2. Because the instances involved in these particular interferences are defined by the structural demands of L2,
PROBLEMS IN ASSESSING PROFICIENCY 209
Weinreich considers them "structural resistance" factors. For example, in the case of phone substitution such as [r] for [R] where there is only one trill phoneme, the stimulus situation which is the occasion for interference relates to a structural difference between the two languages, namely different pronunciations of equivalent phonemes. The resistance factor to this interference is the danger of confusion with another phoneme in L2. Another example, this time from the lexical field, is the outright transfer of words (such as the use of Telephon in German rather than Fernsprecher). The resistance factor for this type of transfer is the danger of potential homonymy as well as the uncongeniality of the word form in L2.
The second type of attitudinal factor pertains to a nonstructural variable that may affect the occurrence of transfer such as resistance factors related to cultural and social-psychological considerations, especially loyalty to the recipient language and intolerance on the part of the community for linguistic importations. Thus French-English bilingualism in French Canada has for many years been marked by high tolerance for English importations both at the gram- matical and lexical levels. Even Public road signs exhibit a strong English influence (mainly American English)(e.g., CEDEZ which is a direct translation of the English YIELD). In recent years the government of Quebec has embarked upon a vigorous program intended to counteract this drift (e.g., the CEDEZ signs are slowly being replaced by signs of PRIORITE A GAUCHE and PRIORITE A DROITE, literally "priority to the left" and "priority to the right") and there is increasing evidence of spreading social resistance to English importations in the speech of bilinguals in that province, a trend that is especially strong among the educated and the intellectuals.
The third attitudinal factor, also nonstructural, relates to the individual's learning strategy. One approach is characterized by the learner's attempt to make use of his knowledge of L1 as a reference structure in terms of which the elements and relations in L2 are being assimilated. The other approach is characterized by the learner's attempt to keep the two languages as separate as possible. Certain conditions of teaching may facilitate one or the other of these two approaches (e.g., direct vs. indirect methods). The structural relations between L1 and L2 may also influence the individual towards one or the other of the two strategies. Ultimately, however, the individual
himself has the greatest contribution to make and the teaching method itself cannot totally control the manner in which the individual chooses to approach the learning problem. The expectation is that the assimilatory strategy will yield greater transfer effects than the "separative" strategy. The resultant facilitative or inhibiting effects will depend on the conditions of training and the relation between the two languages, as discussed previously. Where transfer effects lead to interference the individual with an assimilatory strategy will encounter greater difficulties than the individual with a separative strategy; on the other hand, where the conditions are such that transfer effects are facilitative the individual with the assimilatory strategy should have an advantage. I do not know to what extent learning strategies of this type are stable and relatively in- variant, nor how they can be manipulated by the teaching situation. It is clear, however, that the training conditions and the choice of the second language to be taught (when such choice exists) ought to take into account such individual differences in learning strategy.
4.7.7 Transfer of Grammatical Rules
Earlier I indicated my belief that extrapolation of findings from laboratory experiments to real life learning situations is not a rigorous and scientific undertaking. The analysis of transfer effects presented in the previous discussion and the specific transfer expectations outlined in Table 1 must be viewed with the proper understanding of the limitations of extrapolation. Predictions and generalizations arrived at through this process that run contrary to the expectations of the experienced language teacher must be viewed with some suspicion; those that concur with his expectations are thereby rein- forced and strengthened still further; those that are unexpected or not previously considered by the teacher deserve his careful consideration and it is these that ultimately justify systematic analyses of this type since they bring into focus generalizations and observations that otherwise would remain unnoticed. One problematic issue with which I have not dealt in any substantive fashion but which is no doubt in the mind of the reader concerns the question of what is it that we are really talking about when we refer to stimulus condi-
PROBLEMS IN ASSESSING PROFICIENCY 211
tions, response conditions, associations, and the like in the context of language learning. It is quite clear that we do not mean by these terms what the strict S-R behaviorists mean since by no stretch of the imagination can one seriously claim that the antecedent conditions of an utterance are rigorously specifiable "stimuli" or that an utterance can be considered as made up of elementary "responses" that are identifiable in terms of separate and unitary motor or neurophysiological elements,and even much less that the two sets of factors can be functionally related to each other in a one-to-one fashion. Any such claims simply cannot be serious. It seems to me that the only senses which the terms "stimulus," "response" and "association" may properly have in the present context are the following: a stimulus is some event which is hypothesized to be an antecedent to another event and related to it in some cause-effect fashion; this second event is a response; association refers to the belief that the stimulus event is in fact a contributing cause to the occurrence of the response event. Event refers to an abstract elaboration of some perceived or imagined change in time and is always context dependent, i.e, it is like figure which stands out from ground. Transfer refers to the hypothesis that two response events are related in such a fashion that they share a stimulus event. Thus, the transfer expectation previously expressed in Table 1 that LZ learning for related languages yields greater positive transfer under CP conditions of acquisition than CR conditions (also shown in Figure I as Hypothesis 2) asserts that L2 as response events can be made to depend on stimulus events which are already related to L1. Note that the possibility of such a dependence relation is presumed possible by the assertion of a similarity between L1 and L2 as response events. In the absence of such claimed similarity (as in the case of unrelated languages) the attempt to make L1 response events depend on stimulus events already related to L2 Will be expected to hinder the acquisition of L2 (as in CP training with unrelated languages). It is important to realize that the notion of transfer as defined here refers to the kinds of expectations that the language teacher has on the basis of his understanding of the similarity relations between the response events and the stimulus events to which they are presumed to relate. On the other hand the observable occurrence of facilitative and inhibitory effects exhibited by the
learner depend on the similarity relations that he perceives (with or without his explicit awareness) and on the events that are in fact stimulus events for him (as determined by his past and ongoing experiences). Thus failure of observation to confirm expected transfer effects indicate that the teacher has misconstrued the effective learning situation as it in fact takes place. This way of looking at the problem sheds new light on some old controversies. For instance, consider the long standing debate of whether or not one should teach grammatical rules explicitly. Arguments for the affirmative include the following: knowledge of a language is in fact knowledge of grammatical rules, hence rules ought to be taught directly; when general principles are understood, the learning is more thorough and can better be applied to new instances; it is simpler and more effective to teach a rule explicitly than to leave it up to the learner to discover it. Arguments for the negative include: language use is mostly the automatization of habits; rules can be confusing when stated explicitly; rules are just hypotheses about linguistic relations and when these hypotheses are in error they con- fuse rather than help the learner. As I have defined the transfer problem it is clear that this controversy comes about by the fact that there is some confusion between the teacher's understanding of the task on the one hand and the learner's approach to it on the other hand. The former point of view justifies expectations of transfer effects when rules are taught explicitly, the latter point of view fails to confirm these expectations by empirical observations. The question that is most relevant in this connection is not "Are rules useful or not?" but rather "Under what conditions does the learner find rules helpful conditions under which rule learning is not expected to be helpful include: the learner does not understand the rules in the way intended by the teacher; the learner applies the rules to instances not covered by them; the learner spends an inordinate amount of time and effort in learning the rules; etc. Conditions under which rule learning is expected to be helpful include (in addition to the obverse of the preceding conditions): the learner supplies his own rules and generalizations and these are erroneous or inefficient hypotheses; the learner fails to supply his own generalizations and proceeds by rote learning of patterns; the learner already knows the rules by virtue of his L1 knowledge and the explicit state-
PROBLEMS IN ASSESSING PROFICIENCY 213
ment of the rules shows him how to use them in the applicable instances in L2; etc.
Stephens (1960) suggests some principles by which transfer effects can be maximized in any given situation:
(1) "Bring out the feature to be transferred." Whatever the "thing" that is to be transferred (a fact, a method, a general principle, an attitude, a way of life) it "should be highlighted and brought into focus" (p. 1542). Thus, the course should not "hope" that the student will see a relation, a similarity, or a difference: such things should be explicitly pointed out (e.g., L1- L2 differences or similarities in cognates, syntactic analogies between L1 and L2, phonological similarities in phonemic contrasts, etc.). When the two languages are related we can expect the learner to have a strong tendency to approach the task with an assimilatory strategy (as discussed previously). This can be quite advantageous for all those instances where the response events and stimulus-response associations are sufficiently similar in the two languages. Likewise, it is necessary to isolate and point out explicitly those instances in which the expectation of similarity does not hold. Failure to do so could not only lead to negative transfer but may also vitiate the positive effects of the assimilatory strategy by confusing the learner with respect to where he may and where he may not use his previous knowledge. Under such punishing conditions he may, as Wolfe (1967) points out, adopt an escapist strategy of choosing new forms which do not parallel either language. When the two languages are unrelated the student quickly learns the futility of the assimilatory strategy although a CP setting may unduly retard its discard.
(2) "Develop meaningful generalizations. "... Transfer is more likely to take place when the thing to be transferred is a generalization, a conscious insight, a constant error to be dealt with, or a rule that can be understood" (Stephens, 1960, p. 1542). In the context of grammatical rules or generalizations it is not clear how one is to characterize what is "meaningful" or "understood." In his review of the literature on the "transfer value of grammar" to writing skills, Meckel (1963) concludes that there is no good research evidence either to support or to negate the value of grammar as traditionally taught in developing writing skill. He argues that "commonly
accepted principles of transfer of training would not lead an experimenter to expect much transfer value from knowledge of grammar which has not included the knowledge and ability to apply grammatical principles to the construction of the pupil's own sentences" (1963, p. 981). Meckel does not preclude the possibility of transfer in this situation; his point is that adequate research on the question has not been carried out. He calls for research on (a) "what specific items of grammatical knowledge hold the greatest promise for improving composition skill" and (b) "what degree of mastery of this knowledge is required before we may reasonably expect transfer to writing skill" (p. 982).
Recent advances in linguistic theory are important in this context since they provide the teacher with more powerful generalizations than have been hitherto available in traditional grammar books. However, their formalized nature may not render them suitable for teaching many categories of students. The degree of adaptation necessary for transformational grammar to be a meaningful experience to the language learner and the kind of practice required to bring out their usefulness represent problems for which we do not at present have systematic knowledge for their solution. A student of linguistics who is also learning a second language would be more likely to profit from a transformational description of the language than a High School student who would have to spend an inordinate amount of time mastering the formalized metalanguage involved. There are also individual differences in grammatical aptitude to be taken into account. Two of the major abilities which Carroll and his associates have identified as important in foreign language aptitude consist of "grammatical sensitivity" and "inductive language learning ability" (see for example Carroll, 1966). One deals with the individual's ability to recognize the function of words in various contexts in the mother tongue; the other is the ability to "infer linguistic forms, rules, and patterns from new linguistic context with a mini- mum of supervision and guidance." These special abilities are likely to be a function of some early experience in the mother tongue and, when we come to isolate just what this experience is, we would no doubt be able to improve the success of second language teaching later in life.
PROBLEMS IN ASSESSING PROFICIENCY 215
(3) "Provide a variety of experiences." .. Whenever it is a principle or generalization that is to be transferred, it is most important to use a variety of experiences to develop the generalization" (Stephens, 1960, p. 1542). Many current applications of pattern drill seem to be based on the opposite notion of uniformity. The justification given for invariant patterns (with substitutions of equivalent parts of speech') is presumably that only one thing at a time' is to be taught and that, when considered over sets of lessons or even the course as a whole, all the patterns considered together do exhaust the important variations. However, one can argue that piecemeal presentation of patterns inhibits the attainment of broad generalizations and encourages the formation of restricted hypotheses which later have to be unlearned after many false starts. Wolfe (1967) gives the following example: practice on:
Where does he live?
Where did he go?
may interfere with later occurrences of:
Can you tell me where he lives?
Do you know where he went?
and points out the mistakes the earlier practice leads to:
"Can you tell me where does he live?
"Do you know where did he go?
Nevertheless, everything cannot be taught at once, so we need to develop more systematic knowledge with respect to the degree of "variability of experience" that maximizes generality of rules within the limits of demand for practice and drill required to develop fluency and automatization. Even if it may be true that such variability of experience may retard automatization of fluency, this con- sequence may in the long run be the better outcome. Ferguson (1956) cites evidence that "cognitive abilities play a more important role in the earlier stages of the learning of a motor task than in the later stages, when performance becomes organized in the form of a habituated psychomotor response pattern" (p. 127). Although we should heed his warning that generalization on this point may be precarious, since the distinction between cognitive and motor
abilities is arbitrary, I would like nevertheless to suggest that in language learning we can maximize transfer effects by concentrating in the early stages on the acquisition of general principles and only later should we worry about automatization of phonological habits, fluency, and correct pronunciation. (Note that this is the reverse of audiolingual practice.)
4.7.8 Transfer to Nonlinguistic Areas
In this section I want to consider briefly the kinds of transfer effects that we expect from learning a second language to areas of behavioral functioning that lie outside a direct interest in linguistic performance. A major justification for the teaching of foreign languages in school is the belief that such study has desirable consequences on the individual's intellectual capacities. It is argued that language is the carrier of culture and, therefore, language study provides an opportunity for contact with a culture other than one's own. Such contact, even if indirect, is supposed to broaden one's horizons, lead to a better understanding of one's own culture thus viewed in perspective, reduce ethnocentrism, and generally affect in a desirable way one's perception of the world.
I would like to point out that these various claims can be viewed as statements about the operation of transfer effects from one task (learning a foreign language) to another (understanding one's own culture, interpreting the behavior of other peoples, interacting with foreigners, etc.). As such, everything we have considered thus far about the nature of transfer is also relevant in this context. For example, CP training with unrelated cultures is expected to lead to strong interference effects which means that the second culture will not be understood as intended by the teaching goals. Or, CR training with related cultures that are highly similar will fail to capitalize on positive transfer effects that may occur in CP training. Furthermore, areas of transfer are to be considered separately, as they were in the case of linguistic processes, since they may vary in the specific transfer function that characterizes them. To give an example, consider the expectations we may have about transfer effects relating to perception of social distance in a particular foreign culture. With the study of a related culture in a CP setting, such as the study of French
PROBLEMS IN ASSESSING PROFICIENCY 217
in most American High Schools, the student is expected to learn quite easily the French system of address forms by transferring the English system which it closely resembles (see Brown, 1965), particularly when the relations involved are explicated by a sociolinguistic analysis. When the student is in a CR setting, such as learning French during a year of study abroad, the superficial differences between the forms of address that he may observe (such as the fact that in English there is no pronominal marker for formal versus intimate style) may interfere with the realization that both systems mark the same differentiations even though they use different methods (e.g., French Vous sometimes parallels English Title + Last Name). With unrelated cultures the transfer expectations under the two conditions are exactly reversed. Thus, the study of Korean in a CP setting makes it difficult to learn certain features of its system of address forms which differentiates among at least four levels of intimacy that attach special significance to such factors as the relative age of the addressee, his sex and marital status, and whether or not he is a classmate of the speaker (see Howell, 1968). CR training on the other hand minimizes the tendency to apply the English system in an environment that constantly belies similarities in culture.
The transfer effects of second language learning to nonlinguistic areas of functioning are likely to be greatly magnified under conditions of early childhood bilingual experience. This transfer problem is often discussed in the literature as the "balance effect," which refers to the expectation that the more time is spent on practice in L, the greater its detrimental effect on L, and on cognitive functioning in general. Thus the "balance effect" is essentially a negative transfer hypothesis. Macnamara (1966) has recently presented an extensive review of the literature on this question and has carried out a careful investigation of his own on the balance effect as it applies to the current bilingual instruction program in the public elementary schools in Ireland. His findings and conclusions lend strong confirmation to the negative transfer hypothesis: native speakers of English in Ireland who have spent roughly half of their school time learning Irish achieve a substantially lower standard in both written English and written Irish when compared with their native English or Irish counterparts whose education is monolingual. Furthermore, the bilingual children show a substantial handicap in problem arithmetic
tests, a fact which Macnamara (1966) relates to their linguistic difficulties (because numerical or mechanical arithmetic tests yield no differences between bilingual and monolingual children). In sharp contrast to these findings and conclusions, the study reported by Peal and Lambert (1962) shows that bilingual children in the French-Canadian setting of Montreal are distinctly superior in both verbal and non-verbal intelligence to their French speaking monolingual peers. These authors hypothesize that early bilingualism "might affect the very structure of intellect": "... a large proportion of an individual's intellectual ability is acquired through experience and its transfer from one situation to another" (p. 6). They argue that bilingual children are exposed to "wider experiences in two cultures" and these will give them "advantages which a monolingual does not en- joy": "Intellectually [the bilingual child's] experience with two lan- guage systems seems to have left him with a mental flexibility, a superiority in concept formation, and a more diversified set of
mental abilities.
There are many differences in the Irish and Montreal settings and somewhere in these differences is to be found the resolution to the contradictory positions which Macnamara and Peal and Lambert were led to espouse on the basis of their respective findings. Knowledge is lacking to resolve the issue at the present time.
4.7.9 Some Measures of Language Functioning
In this section I would like to review some techniques that are avail- able for the assessment of language functioning in relation to the various areas of transfer that were discussed in this chapter. Four general areas will be explored:
Functional relation between the two languages
Language functioning in the second language
Changes in nativeness in the first language
Attitudinal factors
4.7.9.1 Functional relation between the two languages.
This has been represented symbolically in the earlier discussion by the equation RA = f(XL1 - XL2) and refers to the interaction effects
PROBLEMS IN ASSESSING PROFICIENCY 219
between the learner's two language systems. Two types of tests appear appropriate: (a) translation and language switching tasks and (b) semantic distance tests. Included in the first type are latency measures for word translation, sentence translation, and language alternation for decoding and encoding tasks. Included in the second type are measures of affective distance by means of the semantic differential technique, cross-satiation index, and an associative interference task.
4.7.9.2 Language Functioning in the Second Language.
This has been symbolized earlier as RB = f(XL2 - XL2) and refers to the learner's knowledge of and behavior in the second language relative to native speakers. Five types of measures are appropriate here: (a) Mechanical Skills: these consist of standard proficiency and achievement tests within the four levels of skills: reading, writing, listening, and speaking; (b) Semantic Sensitivity: these include judgments of acceptability of participial and adverbial phrases which measure the speaker's sensitivity to implicit, aspectual, and metaphorical facets of meaning, as well as semantic differential measures which compare the learner's evaluations with those given by native speakers; (c) Utterance Meaning: these are intended to assess the learner's understanding of the functional communicative value of utterances both in terms of the intended message they carry and the implications they allow about the speaker (his intentions, psychological state, social background, etc.); (d) Communicative Competence. the purpose of these measures is to assess the learner's capacity to function in a communicative context through the medium of the second language. Various techniques are available with perhaps the "acting out" method being the most versatile and simple and one which reveals the extent to which the learner has command over various "codes" that are differentially appropriate to particular sociolinguistic contexts; (e) Cognitive Structure: the measures included under this heading are intended to reveal the extent to which the learner has incorporated the cognitive structure of the native speakers of the foreign language insofar as knowledge about one's culture is represented by lexical and semantic relations in language. Specifically, they are word-association structure, measures of semantic range, and word categorization behavior.
4.7.9.3 Changes in Nativeness in the First Language. This has been symbolized earlier as RC = f(XL1 - XL1) and refers to backlash interference: upon learning a second language the individual may come to adopt new cognitive, attitudinal, as well as linguistic modes of functioning. He is no longer a "typical" and representative speaker of his native language. Such changes may be quite subtle, even though significant in the long run, and can be assessed by com- paring the learner's word-association structure and semantic differential evaluations to monolingual speakers of his own culture. A test which can be quite sensitive to such changes is the "Cloze procedure."
4.7.9.4 Attitudinal Factors.
The subtle changes in nativeness just referred to can manifest themselves through feelings of "anomie" (which can be assessed) and changes in perceptual and cognitive orientations that are part of an individual's personality pattern. Depth interviews, measures of ethonocentrism, and the "personality differential" (which is an index of interpersonal perception) are techniques that might prove useful here. A different but yet highly relevant attitudinal factor relates to the sociolinguistic context of second language learning: What is the social status of the particular language being studied? How do the learner's reference groups (friends, parents, neighbors) view the second language and the culture with which it is identified? These and related questions can be incorporated in depth interviews and should be supplemented by ethnographic information that can be provided by knowledgeable resource persons within the community. Finally, the learner's strategy and orientation with respect to the learning task must be examined: where and how much does he use the assimilatory strategy? the separative strategy? What is his stated motivation for studying the second language? The latter question can be assessed by Lambert's Orientation Index; the former can be investigated by making observations of interferences and direct questioning during learning sessions.
PROBLEMS IN ASSESSING PROFICIENCY 221
Extrapolation from laboratory experiments to real life learning situations is not a scientifically rigorous process. Nevertheless, it is possible to be systematic without at the same time being rigid about the manner in which additions to knowledge are made. The analysis of transfer effects in second language learning should be molecular and have a systematic character that could potentially organize the knowledge language teachers have developed through their experience in the classroom. The analysis presented in this chapter out- lines some specific transfer expectations under four different conditions of second language acquisition: coordinate and compound training for related and unrelated languages. Some unexpected predictions are generated; for example, with related languages a compound setting will yield more positive transfer (hence be more facilitative) than a coordinate setting. Similarities between two languages in terms of their surface features are more relevant to the operation of transfer effects than deep structure relations. A distinction must be made between structural factors based on contrastive analyses and nonstructural factors pertaining to the learner's attitudes and the sociolinguistic context of the learning situation. Transfer effects operate at various levels of language functioning (e.g., mechanical skills, semantic sensitivity, communicative competence) and measures to assess these effects are suggested.