Production based instruction (PBI) that require learners to output language dominates language teaching methodology, language teaching course books, grammar books and teacher guides. This contrasts with the relatively smaller distribution of comprehension based instruction (CBI) which provide learners with input only. Natsuko Shintani, Shaofeng Li and Rod Ellis in 2013 used a meta-analysis to try to evaluate the relative effects of CBI and PBI on grammar learning.
Introduction and definitions
The meta-analysis consisted of 35 experiments on grammar learning from 30 studies over 19 years between 1991 and 2010. Grammar was defined as L2 morphology, syntax, or pragmatics. The target languages included English, French, Japanese and Spanish with mostly university adult learners in mainly foreign language learning contexts.
-
Coding
Independent variables – experimental and comparison groups coded as either CBI or PBI (based on primary study descriptions and the meta-analysis study definition of the two types of instruction).
Dependent variables – receptive tests (learners needed to comprehend oral & written input) and productive tests (learners had to output target forms). The tests were further divided into immediate post-test (taken within 7 days) and delayed post-tests (taken 8 days after).
Moderator variable 1 – does CBI use PI (Processing Instruction) or not;
Moderator variable 2 – does PBI use text manipulation or text creation
Processing instruction (PI) aims to focus attention of learners on those features that are overlooked due to processing constraints (the two main constraints are – learners process meaning before form and – learners tend to process the first noun or pronoun they see in a sentence as the subject or agent). Text manipulation activities include pattern drill, filling in blanks, transforming sentences from one pattern to another, or translating a sentence from the first language to the L2. Text creation activities include jigsaw tasks, information-gap tasks, and meaning-based production tasks. - Reliability of coding 94.5% between first and second author of study
- Effect sizes comparative effect size (between CBI and PBI); absolute effect size (CBI or PBI versus controls); pre-to-post effect sizes (pre-test vs post-test for CBI or PBI)
Results
Generally both comprehension based instruction and production based instruction had large effects on both receptive and productive tests. CBI had a more significant effect than PBI on immediate receptive tests. Both CBI and PBI were effective on immediate productive tests but PBI better on delayed productive tests. Processing instruction in CBI was found to be a significant moderator on receptive tests. The moderator variables of text creation and text manipulation did not play a significant role in the comparative effect of CBI and PBI.
Limitations
Only published studies were included so there may be a publication bias although a standard test showed no such overt bias. More serious is a lot of variability in the primary studies of how they operationalized CBI and PBI. Further it was difficult to control for any production that might have happened in CBI and conversely any comprehension that might have happened in PBI. There was also variation in explicit grammar information, learner age, learner proficiency levels, instructional context, research setting, target language and length of treatment. The authors argue that despite these drawbacks the constructs of comprehension and production based instruction is well founded. Further the many disagreements amongst researchers and teachers on the relative merits of input and output attest to the importance of this issue.
It is worth noting that some people have claimed it is better to see effect size as effect clarity that is – any difference between groups is clear not necessarily that the difference is large or important or educationally significant.
Conclusion
The authors posit that CBI has an advantage in the initial stages of acquisition while PBI may help in accessing partially acquired knowledge and consolidating such knowledge. They go on to speculate that CBI may be good for new grammar features and for beginners. And PBI may be suitable for more advanced learners to get more control over somewhat acquired knowledge. They further speculate about the benefit of combining comprehension and production. No such studies have done that yet but to teachers this mix will surely have featured in classrooms. So as with many a study there are no clear cut answers. However, carefully done meta-analysis do actually help to scout a path. For example both CBI and PBI in the primary studies used mainly intentional learning involving explicit instruction which has shown to be effective in grammar teaching.
References
