Table of Contents
Introduction
Observation or Interaction-Based Approaches
Written Documents/Student Work
Institutional Documents and Records
Institutional Records
INTRODUCTION
Although this section is designed to review the utility and efficacy of qualitative methods of student assessment, "qualitative inquiry" represents more of a philosophical movement and form of understanding about the nature of meaning than a research methodology. In spite of the fact that much of this section of text is dedicated to identifying ways to use methods associated with qualitative inquiry, the discussion must begin with a brief review of concepts and philosophy. For the sake of brevity this discussion will not address many of the nuances and controversies surrounding qualitative inquiry within the research community (or for that matter the presence of diverse perspectives of traditional scientific methods). However, an understanding of the nature of knowledge and meaning, how this understanding is pursued within lines of qualitative inquiry, and the operation of methodological principles such as "validity" and "rigor" are important for both the appreciation and application of qualitative work.
The distinction between quantitative and qualitative inquiry is not simply that one involves the use of numeric data while the other relies upon text, or that the former is centered more in scientific principles and the latter based on intuitive judgment. Qualitative inquiry as a movement was a reaction against scientism and its dominance in research. Quantitative methods are based in the positivist tradition, the origins of which are found in enlightenment, empiricism, and the natural sciences. By contrast, qualitative inquiry draws from more postmodern philosophies such as hermeneutics, interpretivism, and constructivism, with broader use in the social sciences and humanities.
Scholars within each of the research traditions approach their work from a range of perspectives. While approaches to qualitative inquiry might be seen as existing on a continuum from empirical interpretivism to radical deconstructionist theory, some within the tradition would reject that dichotomization. A comparison of quantitative and qualitative traditions on beliefs about the nature of knowledge and its acquisition can provide a good entrée into the understanding of their differences and similarities.
How is knowledge acquired?
In the quantitative/empiricist tradition, knowledge is acquired through objective testing using the scientific method. The method was elevated over time to be the superior, if not the only, source of knowledge. Knowledge was thus defined by methodology. Knowledge became that which was empirically verifiable by an objective scientist. If something was not observable or testable, it could not be known. Complex phenomena were reduced to their individual components and concepts were operationalized in order to be tested.
In the qualitative tradition, knowledge is acquired through achieving understanding, and understanding is a way of being. Knowledge is not methodologically bound. Understanding is an ontological condition; humans are self-interpreting creatures, and understanding is something an inquirer pursues in order to gain knowledge. The scientific method represents only one of many ways to acquire understanding. In the qualitative tradition understanding is constructed, and thus for many is by definition dialogical. Understanding is attached to voice and the meaning-maker, and alternative voices are equally valid. In a similar vein, the interpretation of texts (particularly within the tradition of historical hermeneutics) requires their understanding within the context and culture of the author in order to ascertain the author’s meaning.
The distinction between researcher and subject is blurred in the qualitative tradition because pure objectivity is an illusion. The researcher and participant are inseparable and interactive. Interpreters are bound to their own traditions, and the question of whether the ‘outsider’ or the ‘insider’ is better at discerning understanding is a matter of debate. Qualitative inquiry, while it is evidence-based, requires more than empirical observation—it also involves the attempted discovery or revelation of shared meanings and collective understanding. Interviews and other forms of text and dialogue represent legitimate evidence because the researcher assumes that the individual’s perceptions and forms of meaning-making are important data for understanding human behavior. Acquiring knowledge begins with a broad, open-ended question rather than a testable hypothesis.
What constitutes evidence and valid research?
The nature of evidence and data represents another area of difference between the paradigms, with qualitative data being generally defined as empirical material, objects, or observations that are non-quantitative. Lincoln distinguishes between data and evidence by stating that data "is not evidence until two things happen: first, someone recognizes it as data, and second, an inquirer subjects it to some form of systematic analysis, which turns it into evidence directed toward some question or argument" (2002, p. 6).
Scholars have proposed a number of criteria for what constitutes qualitative evidence. For one, evidence should be systematically gathered in a manner consistent with the research tradition, and observable. Unlike with quantitative traditions, in qualitative inquiry research participants are partners in the research effort, and are given the opportunity to review, clarify, and elaborate on the evidence provided to the researcher. This "member checking" process is one of the ways that individuals who generate research data are "participants" and not "subjects." Further, qualitative data should be made available to the reader of resulting deliverables so that he or she can authenticate the researcher’s interpretations.
If one accepts the legitimacy of the qualitative inquiry paradigm, then the question of what constitutes valid evidence is less troubling and becomes a matter related to the methods employed for data collection and analysis. In qualitative research, data from multiple sources are compared for consistency in a process called triangulation. Data gathering, analysis procedures, and actual data (such as quotes from interviews) are reported in detail so that the reader can verify that established practices were followed and follow the path of analysis from participant to interpretation and presentation.
How can qualitative inquiry be useful in a liberal arts context?
Switching research paradigms can make an unanswerable question answerable and reasonable to ask. Because the nature of the research method used should follow from the questions asked, qualitative approaches are a natural choice to answer queries about the outcomes of a liberal arts education. Many of these outcomes are difficult to measure in quantifiable ways, particularly the broad descriptions often found in college mission statements. Student qualities and skills such as judgment and discernment, critical thinking, and synthesis and analysis are not easily quantifiable. For example, while basic writing skills can be quantified in the number of grammatical, spelling, and punctuation errors per words of text, the measurement of a students’ ability to produce original research or original thoughts, rather than simply summarizing the thoughts of others in a new way, is not as readily measured using numbers or statistics.
Wiggins suggests that liberal arts faculty have a "moral obligation to disturb students intellectually" (p. 547). If it is also true that the test drives the teaching, then what we measure will also be what is taught. In place of or in addition to exit examinations, the qualitative graduate interview is an example of one way to measure whether students were "intellectually disturbed" while at the institution. On selected campuses, structured interviews are used to assess the level of intellectual development for a sample of students at matriculation and again in a student’s final year. This developmental measure is combined with interview data about salient college experiences to determine relationships between campus programming and student development. Future studies could be built combining interview data with quantitative analyses of student program participation and other institutional data to examine outcomes such as student persistence and post-graduate activity.
Qualitative inquiry can also be tailored to specific campus contexts and questions. A limitation of externally-provided instruments is the frequent lack of institutional control over the questions asked for the unique context of that particular campus. Qualitative inquiry by definition understands knowledge as connected to its context. While transferability of knowledge occurs from one context to the next, qualitative researchers assume each context is unique.
In sum, qualitative inquiry and associated methodologies represent a means by which colleges and universities can assess student learning and other desired outcomes in a rich, campus-specific manner not available through traditional surveys and other quantitative approaches.
Recommended readings:
For a research design and "how to" text:
Creswell, J. W. (1998). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: Sage Publications.
Patton, M. Q. (1990). Qualitative evaluation and research methods. Newbury Park, CA: Sage.
For a philosophical understanding of qualitative inquiry:
Lincoln, Y. S., & Guba, E. G., (1985). Naturalistic inquiry. Beverly Hills, CA: Sage Publications.
As a resource for understanding qualitative terminology and jargon:
Schwandt, T. A., (1997). Qualitative inquiry: A dictionary of terms. Thousand Oaks: Sage Publications.
For an excellent discussion of evidence and validity:
Lincoln, Y. S. (2002). On the nature of qualitative evidence. Paper presented at the annual meeting of the Association for the Study of Higher Education, November 21-24 Sacramento, CA.
(If this piece it is not readily available, the following alternative is recommended: Lincoln, Y. S. (2002). Varieties of validity: Quality in qualitative research. In J. S. Smart, (Ed.), Higher education: Handbook of theory and research, Vol. 16 (pp. 25-72). New York: Agathon Press.
Next Section:
Observations or Interaction-Based Approaches