I am in Portland right now – presenting at the American Evaluation Association conference. It is interesting because there is a distinct and growing interest here in how cultural issues impact the field of evaluation. The growing interest hasn’t unfortunately translated into any kind of a comprehensive strategy for integrating cultural responsiveness into practice, but it has been nice to meet others who are working on the issue.
I presented with Dr. David Williams on this topic. A summary of our paper is as follows:
Cross-cultural Instructional Design for Online Teaching:
Implications for Evaluation
Paper presented at AEA, November 2006
A growing literature (Russon & Russon, 2004; Thompson-Robinson, Hopson, & SenGupta, 2004; and Wiesner, 1997) argues that culturally sensitive evaluation is essential in many contexts, including evaluation of online instructional design (Rogers, 2006). However, knowing how to evaluate and design sensitively is hampered by lack of information about challenges instructional designers/evaluators encounter.
This study, based on a dissertation (Rogers, 2006) that explores the experiences of twelve designers who have developed online instruction in cross-cultural contexts in over 80 countries, explores some ways evaluation could be used to enhance culturally sensitive evaluation and instructional design. Participants were interviewed using triangulation, negative case analysis, member checking, thick description, and an audit trail to enhance credibility, transferability, and dependability.
Interviewees noted three barriers to their ability to be responsive to cultural differences: (a) an over-focus on content development, (b) a relative lack of evaluation in real-world practice, and (c) the less than ideal roles instructional designer/evaluators assume in their organizational structures.
Building Evaluation Into Design:
These case studies clarify how increased sensitivity to cultural differences through evaluation could improve cultural practices of instructional designers, particularly by making evaluation integral to design by building several kinds of evaluation into the instructional design process. This presentation illustrates this view through use of Stufflebeam’s Context, Input, Process, Product evaluation model (2000), though many others might also be considered.
This study concludes that additional efforts are needed to educate and commit more stakeholders to engage in learner analysis and evaluation which will culturally sensitize them. Also additional instructional design models need to be identified and used that make evaluation and learner feedback more integral parts of the entire instructional design process from context or needs analysis, through input or alternative solution identification, to implementation and formative process evaluation to evaluation of the impact of the entire design effort. This is especially true for instructional design involving online learning because the normal means of adjusting to learner and cultural differences through face-to-face interactions are hampered by the medium.