Historically, adult education as a field has been concerned with context. Scholars have long worked to acknowledge and understand how contextual factors affect adult learning and how they often alter the very definition of what is called adult learning. Open any formidable text in the field and you will likely encounter the word “context,” and along with it its synonyms, “background,” “situation,” and “conditions.” Select any manuscript from the field’s research journals (e.g., Adult Education Quarterly) and you’re likely to notice it emerging as a theme. Open the field’s leading specialty journals (e.g., Journal of Transformative Education) and your likely to see context or its referents mentioned in just about every paragraph! Clearly, for adult education researchers, context matters.
The field’s focus on context has emerged as an identity of sorts, fitting nicely with its adoption of constructivism as its principal epistemology. Constructivism is the idea that the meaning ascribed to any object or event is idiosyncratically constructed when a learner integrates it (relates it) with past experience and learning, an idea Fox (2006) has argued comes from the philosophy of descriptive contextualism and its analytic aims, description and understanding. From this perspective, researchers aim to describe and understand the rich contextual details of learning as it occurs in a particular time and place. They are not necessarily interested in predicting or influencing learning in regards to some pre-established aim across a variety of learners (the analytic aims of functional contextualism).
More useful investigations will aim to determine where something works, when something works, and with whom something works best.
This focus on context has been brought to the forefront with the field’s near uniform embrace of qualitative research methods (see Boeren, 2018; Daley, Martin, & Roessger, 2018), methods that dismiss the idea of generalization and what some might call “de-contextualized” truths. The culmination of this is that many of the field’s scholars now see as its mission the emancipation of adult learners rather than the prediction and influence of adult learning. Whereas the former fits easily into a milieu championing context, the latter is often seen as at odds with the idea that context matters. Scientific reports of adult learning, then, commonly come with gentle reminders to remember context.
Consider UNESCO’s decidedly quantitative report on adult and lifelong education (ALE): 3rd Global Report on Adult Learning and Education (UNESCO, 2016). In it the authors take special care to caution that “the plight of the most disadvantaged people may be hidden from view in overall statistics . . . a further reminder of the importance of looking beyond micro-level statistics and considering qualitative accounts of the value of [adult and lifelong education]” (p. 92). The report later concludes its discussion of the impact of ALE on employment by reminding readers of “the need to contextualize the evidence” (p. 100).
Intuitively this feels right (Context matters, doesn’t it?), particularly to researchers with little training in quantitative methods and statistics. But are such reminders warranted? Does a scientific investigation of adult learning necessarily ignore context?
The short answer is no; it doesn’t have to and it shouldn’t. Context matters.
Overly simplified scientific investigations of adult learning may aim to determine whether certain policies or practices work with little regard to their contexts. But more useful investigations will aim to determine where something works, when something works, and with whom something works best. These aims attempt to uncover what I call contextual generalizations; that is, findings that illuminate the contextual nature of adult learning but still provide researchers and practitioners with insights that help predict and influence adult learning in diverse settings.
So how does adult learning science do this and still make generalized causal inferences between interventions/policies and adult learning outcomes? The answer is to use methodologies that account for contextual variables and temporal change. By contextual variables I mean learner background variables (e.g., race/ethnicity, age, gender, SES, education history, cultural preferences, etc.) and learning setting variables (e.g., community SES, country GDP, neighborhood crime rate, school graduation rate, instructor experience, etc). By temporal change, I mean change that occurs in a particular time and place. Ways of accounting for contextual variables while still allowing for generalization are plentiful. I’ll discuss three that I think are particularly useful for experimental, quasi-experimental, and design-based research.
With whom does it work?
To understand with whom something works, scientific investigations of adult learning must consider interactions, a phenomena called moderating effects. Whereas main effects depict overall mean or slope differences between intervention/no-intervention groups and policy/no-policy groups, moderating effects depict how learner background variables influence the effects of practices and policies (i.e., how these variables interact). For example, consider a learning strategy that aims to engage adult learners in reflective dialogue to aid their transfer of course content to real-world settings. Let’s say a researcher randomly assigned learners to one of three professional development workshops. One of these workshops (the experimental group) used reflective activities along with instruction, modeling, practice, and feedback; whereas the other two (the control groups) used solely instruction, modeling, practice, and feedback. The content, instructor, and time of day were the same for each workshop. Transfer to real-world settings was measured by a blinded reviewer who rated each learner’s work products one week later for evidence of transfer. A main effect of reflective dialogue would simply describe the overall mean differences between the experimental and control groups. A significantly higher or lower mean score in any group would illustrate an effect. In this case, a researcher would find this effect using a simple one-way analysis of variance (ANOVA).
The number of learner background variables we could examine is only limited by what theory and research tell us is worth considering.
Although main effects may be interesting to researchers concerned with illustrating reflective dialogue’s overall impact, they largely ignore context. Principally, they don’t tell us with whom the intervention works best. Within the experimental condition, we may find that women show high transfer, whereas men show the same levels of transfer found in the control groups. Overall, this appears as a higher mean transfer score for the experimental group (women drive the average score up), but this effect is misleading; it ignores the differences between men and women. If we were to account for gender, we would likely find an interaction showing how gender moderates the effect of reflective dialogue. Similarly, we could consider things like a learner’s age, rank within the organization, educational level, race/ethnicity, salary, or region of birth. In each case, we could examine these background variables and how they interact with the learning strategy. We could even look at three-way interactions that considered, say, how a person’s age moderates the relationship between gender and reflective activities and its effect on transfer. It could be that older women respond very differently to reflective activities than younger women! For this hypothetical example, it’s complicated.
This analysis would likely take the form of factorial ANOVA, which is just another way of thinking about multiple regression with interaction terms. The number of learner background variables we could examine is only limited by what theory and research tell us is worth considering.
Where does it work?
A second way that scientific investigations of adult learning can consider context is to use multilevel models to examine nested data structures. Nested data structures occur when data from individual learners are not independent. Instead, they depend upon the contextual unit in which they are situated (i.e., nested). We might say learners are “nested” in classrooms, departments, work groups, organizations, neighborhoods, states, or even countries. Those belonging to a particular unit will tend to perform similarly to others within their unit. As a result, characteristics associated with the contextual unit itself can influence outcomes associated with the learner, and by simply knowing the contextual unit, researchers can make better than chance predictions about a learner’s outcome (often without any knowledge of the learner’s particular background variables). Greenberg and Phillips (2013) explain:
“Two subjects drawn randomly from the same neighborhood are more likely to be similar to one another than two subjects drawn randomly from the entire population of residents in a city, even when controlling for individual traits that are known to be important to the explanation of the outcome” (p. 220).
This is the essence of a nested data structure.
In nested data structures, researchers classify information according to level. In our case, the learner is the level-1 unit and the contextual unit is the level-2 unit. All salient variables pertaining to the learner (e.g., age, race/ethnicity, gender, SES) are level-1 predictors, and all salient variables pertaining to the contextual unit (e.g., group size, group composition, group SES, etc) are level-2 predictors. To analyze these data, researchers use multilevel models. Doing so helps them determine where something works best.
The idea here is that multilevel models can help researchers better understand where something works best and, more importantly, why it works best in that particular setting.
The simplest form of a multilevel model is a random-intercept model. Let’s consider what this would look like in practice. Say we were interested determining whether a GED prep software program helped improve adults’ GED testing scores. The software was distributed to GED study centers across the state. To use the software, learners must be enrolled in an in-person GED prep class, where traditional teaching strategies are also used. Learners are free to use the software as often as they like while in these classrooms. We want to know if there is a relationship between the time a learner spend on the program and the score a learner obtains on the GED. Because all learners must regularly attend a traditional class to have access to the program, we can assume that the time they spend in the classroom is constant.
A traditional approach using multiple regression would simply provide the average relationship between time on the program and GED score across all learners. Even if it incorporated learner background variables into the model (e.g., age, gender, race/ethnicity, SES), it would still ignore the differences in the social context in which these learners learn (e.g., the classroom and community). A random intercept model, though, would measure how learners GED scores differ across classrooms even after controlling for learners’ individual differences. After establishing that GED scores do differ across classrooms, researchers could then use a multilevel model to examine classroom variables that account for this variation (e.g., teacher experience, class size, average educational level in the class, or number of computers in the classroom). The possibilities for examining how level-2 variables affect level-1 outcomes are great, and an extensive discussion of all the ways multi-level modeling can do this is beyond the scope of this blog post. But the idea here is that they can help researchers better understand where something works best and, more importantly, why it works best in that particular setting.
When does it work?
To understand when something works, scientific investigations of adult learning must examine the role time plays in the unfolding of a practice or policy. There are numerous ways of doing this, and the choice of method depends on what serves as the primary unit of analysis (e.g., a person, a group of people, or several groups of people). In general, though, analyses that account for time will present graphical displays of the outcome on the y-axis and time on the x-axis. In this way, readers can visualize how the outcome changes over time, often in relation to the introduction of a practice or policy. Such methods are a considerable departure from cross-sectional designs that examine the effects of a practice or policy at a single moment in time.
Ensuring that researchers have the skill sets to attain contextual generalization will require that adult learning science programs incorporate advanced research design and statistics coursework that goes well beyond the traditional 2 or 3 courses in adult education graduate programs.
When a person is the primary unit of analysis, researchers can use single-case designs. These are very basic illustrations of how a person is performing in regards to some outcome over time. Generalization is accomplished through replication. Single-case designs are primarily analyzed graphically. However, there are ways to incorporate statistical analyses. A commonly used version of a single-case design is called a multiple-baseline design. In this design, measurements of a key outcome are taken repeatedly before and after the introduction of a learning intervention. Measures taken prior to the intervention are part of the baseline phase, whereas those taken after are part of what is often called the treatment phase. The introduction of the practice or policy is staggered across individual learners to minimize the possibility that an event concomitant with the intervention has caused the observed change. To further isolate the effects of the intervention, researchers generally wait until a learner’s baseline has stabilized before introducing it. We could use this design to examine, say, the effects of a book club on the number of pages a learner readers per week. An app like GoodReads could be used to measure the reading habits of a learner prior to joining the book club and during participation in the book club. In this way, we could see visually if participation had changed this learner’s reading habits.
Given the collaborative nature of adult learning, it is often difficult to find authentic applications for single-case designs. Fortunately, there are variations of this design that are used with units of analysis larger than a single person. When an aggregate unit is the primary unit of analysis (e.g., a school, a city, or even a classroom), researchers use time-series designs. Data from time-series designs are generally analyzed statistically—and require advanced statistical knowledge—but often communicated graphically to show aggregate changes over time. A possible application of a time-series design would be if an entire city started a book club and encouraged its citizens to track their progress using data-collection apps. The average number of pages read per week would serve as the repeatedly measured outcome. Researchers would collect a series of aggregate measures before the intervention (participation in the book club) and after it.
Similarly, latent growth models are used when there are multiple aggregate units (e.g., cities, classrooms, schools) participating in the practice or policy, and a researcher also wants to see how between-unit variables (e.g., learning setting variables) affect the outcome of interest. With latent growth models, the researcher can model the degree of change over time, as well as how learning setting variables contribute to the units’ variation in change over time. A comprehensive discussion of this approach is also beyond this post’s scope, but the main idea to take home is that latent growth models are another way researchers can determine when something works to better understand the context surrounding a particular practice or policy.
Conclusion
In this blog post I have tried to show how adult learning science can account for context while still aiming for causal inference and generalization. We do this by determining not only if a practice/policy works, but also where it works, when it works, and with whom it works. I have referred to this as contextual generalization. Ensuring that researchers have the skill sets to attain contextual generalization will require that adult learning science programs incorporate advanced research design and statistics coursework that goes well beyond the traditional 2 or 3 courses in adult education programs. As the adult learning sciences develops as a field, it will be critical to embrace some of the core values that adult education as a field has established so that the two can work together. I find much shared interest in the idea that context matters.
References
Boeren, E. (2018). The methodological underdog: A review of quantitative research in key adult education journals. Adult Education Quarterly, 68(1), 63-79.
Daley, B. D., Martin, L., & Roessger, K. M. (2018). A call for methodological plurality: Reconsidering research approaches in adult education. Adult Education Quarterly, 68(2), 157-169.
Fox, E. (2006). Constructing a pragmatic science of learning and instruction with functional contextualism. Educational Technology Research and Development, 54(1), 5-36.
Greenberg, D. F., Phillips, J. A. (2013). Hierarchical Linear Modeling of Growth Curve Trajectories Using HLM. In G. D. Garson (Ed.), Hierarchical linear modeling: Guide and applications (pp. 208-225). Thousand Oaks, CA: Sage.
UNESCO. (2016). 3rd global report on adult and lifelong education. Hamburg, Germany: UNESCO Institute for Lifelong Learning. Retrieved from http://unesdoc.unesco.org/images/0024/002459/245917e.pdf
Learning . . . Continued 