LiberalArtsOnline Volume 6, Number 5
May 2006
Are our students learning enough in college? A national literacy study might suggest that they are not. This month’s author, John Frazee, former vice president for Academic Affairs at Massachusetts College of Liberal Arts, asserts that the best determination of quality teaching is whether students are learning. But how can we actually measure this? With outside testing and pressures looming, Frazee argues that liberal arts colleges should take part in creating their own, distinctive assessments to ensure that satisfactory learning is taking place.
--Kathleen S. Wise, editor
-----------------------------------
Higher Education at Risk
John P. Frazee, Ph.D.
formerly Vice President for Academic Affairs at Massachusetts College of Liberal Arts
The results of the recently-released National Assessment of Adult Literacy (NAAL), a study conducted for the National Center for Education Statistics, should give pause to everyone interested in the future of higher education in the United States.
In 2003, NAAL assessed a large sample of American adults age 16 and older in three dimensions of literacy: quantitative literacy, prose literacy, and document literacy. To be considered "proficient"—the highest level—on the prose literacy scale, one would be able, for example, to compare viewpoints in two editorials. Proficiency in document literacy requires one to complete successfully such tasks as interpreting a table relating blood pressure, age, and physical activity. To demonstrate proficiency in quantitative literacy, one would be able to complete such tasks as computing the cost per ounce of food items and comparing the results. As these examples suggest, the literacy skills assessed in NAAL are functional rather than academic, the basic skills required of any adult living and working in the twenty-first century.
The "proficient" level defined by NAAL is at—or perhaps even below—what most higher education professionals would expect of entering first-year students. Even allowing for the fact that approximately 30% of students entering college require some remediation in basic skills, I suspect most faculty members and administrators are confident that students who successfully negotiate four years of college-level study would have little trouble with the comparatively simple tasks of the NAAL assessment. Were I to ask how their graduates would perform on the National Assessment of Adult Literacy, most college faculty and administrators would, I think, predict that something like 90% would be able to demonstrate proficiency.
Perhaps my hunch is overly optimistic. Still, it’s hard to imagine any college faculty member, much less any administrator, being comfortable with the idea that only three out of four graduates could demonstrate proficiency in these basic skills. A figure below 75% would surely be unthinkable to most higher education professionals.
In reality, however, our graduates’ performance on NAAL was much worse than anyone would likely predict. Only 31% of graduates of four-year colleges and universities demonstrated proficiency in prose literacy on the 2003 NAAL. The percentage was the same for quantitative literacy. Only 25% of college and university graduates showed proficiency in document literacy.
By any rational standard, these results are troubling. Could the news be any worse? Unfortunately, yes. In 2003, the percentage of college graduates demonstrating proficiency in prose literacy actually declined by nine percentage points compared to 1992, when NAAL was last administered. The percentage of college graduates demonstrating proficiency in document literacy also declined by twelve percentage points compared to 1992. Only quantitative literacy remained the same, with 31% demonstrating proficiency.
The NAAL results point to a conclusion that is as disturbing as it is inescapable: whatever else colleges and universities may be doing well, they are not adequately preparing graduates to meet the most basic challenges of contemporary life and citizenship. Furthermore, the situation is deteriorating.
While every corner of every college and university shares in the responsibility for addressing this problem, the ultimate burden rests squarely on the shoulders of liberal arts colleges—or, in a university setting, colleges of liberal arts. After all, those of us who are advocates of a liberal arts education pride ourselves on providing an education that transcends narrow specialization and builds the broader intellectual skills required of lifelong learners. Moreover, we get our shot at educating most students in four-year colleges and universities, either as a matter of institutional identity or through the general education requirements students take before moving on to their majors outside the liberal arts disciplines.
How should liberal arts colleges respond to this challenge? Given the scope of the problem revealed by the NAAL results, it would be foolhardy to look for quick or easy solutions. Fundamental changes are in order if colleges and universities are to retain the confidence—along with financial and political support—of the public. In what follows, I want to propose several ideas, both to suggest the magnitude of the changes I believe are required and to stimulate a much-needed discussion of this issue.
I begin with an obvious recommendation: teaching effectiveness must become a much higher priority across the four-year-college and university landscape. Of course, no college or university would have the temerity to admit that teaching is anything other than a high priority. As a practical matter, though, the reputation of most institutions rests on the research profile of the faculty, the size of the endowment, the selectivity of admissions, the success of the football team, or some combination of these. All are easier to measure than teaching effectiveness, but none relates meaningfully to the quality of teaching and learning.
Some colleges and universities have made more than a rhetorical commitment to teaching quality. For these institutions, teaching effectiveness is frequently the primary criterion in personnel decisions. Too often, however, their efforts to recruit for and reward high quality teaching founder because they lack valid and reliable measures. Most institutions rely on student evaluations as their primary gauge of teaching effectiveness. Faculty members routinely—and with considerable justification—criticize this method as a dubious measure of successful teaching. And even advocates of student evaluations concede that these instruments offer at best only an indirect estimation of teaching effectiveness. Institutions must re-think how they evaluate teaching.
The best measure of teaching effectiveness is whether students are learning. Colleges and universities should acknowledge and act on this obvious fact. They should undertake to assess whether their students are in fact learning what they have set out to teach. At the department level, this process might begin with faculty colleagues discussing a deceptively simple question: What do we want our graduates to know and be able to do when they graduate?
In my experience, this question has proven very difficult to answer. Faculty members are accustomed to thinking in terms of curriculum, not learning outcomes. If the curriculum "covers" the discipline, appropriate outcomes are assumed. Moreover, many faculty members believe that they "own" their part of the curriculum and the classes they teach. When faculty are accustomed to cultivating their own curricular turf without input or oversight from their colleagues, they are not likely to engage in serious—and often tough—department-wide discussions of learning outcomes. Yet without prior agreement about what the department’s overall learning outcomes should be, it becomes very difficult to ask, let alone challenge, whether a colleague is teaching his or her part of the curriculum effectively.
A similar question should be the subject of institution-wide discussion: What do we expect our students to know and be able to do once they have completed their general education requirements? Taking this question seriously would lead to a very different process of general education review and reform than the turf battles and political give-and-take that characterize most such efforts.
Difficult as the process of establishing desired learning outcomes is, it is only the first step in the process of making teaching a higher priority. With detailed, clear learning outcomes in hand, each institution needs to develop methods of assessing whether students are achieving those outcomes. I am not calling here for standardized or high-stakes testing. On the contrary, I want an assessment method that will preserve the ability of departments and institutions to develop and maintain their educational distinctiveness, an essential quality of American higher education. The value of that distinctiveness depends, however, on the ability of each department and of the college or university as a whole to demonstrate that students are in fact learning what they have set out to teach. Without valid and reliable measures of learning outcomes, curricula—and, for that matter, stated learning outcomes—are articles of faith at best. At worst, they are empty rhetorical exercises.
It should also be clear that I’m not advocating the kind of assessment program created for the purpose of satisfying regulators or accrediting bodies. In my experience, these too often devolve into exercises in compliance only. The learning outcomes assessment program I have in mind is of a different order altogether. It would be far more difficult to create and implement than the compliance exercises of the last two decades. It would also be far more valuable to the college or university because it would yield information that the institution, its departments, and its individual faculty members could use to improve teaching and learning.
A robust learning outcomes assessment program would be a much better tool for measuring the effectiveness of teaching and learning than those that currently exist. Moreover, it would provide information that general education committees, departments, and even individual faculty members could use to improve the design and delivery of their courses. Finally, such a program would also offer far more reliable data on the teaching effectiveness of individual faculty members than student evaluations or other input measures can.
I’m well aware that I have glossed over the many structural, financial, and political difficulties of designing and implementing an assessment program that would be both valid and reliable enough to use in making decisions about reappointment, tenure, and promotion. Nevertheless, I have confidence that the nation’s colleges and universities can overcome these difficulties. It’s a matter of priorities.
At the core of the reforms I envision is something of a seismic shift in the culture of the professoriate. I’m calling for an end to the cloistered classroom, the notion that what goes on in one’s classroom is sacrosanct; an end to the idea that faculty members own their classes; and an end to the assumption that passing grades are evidence of learning. This culture must give way to a far more open, transparent, and inquiry-based culture. We must recognize the mutuality of our responsibility for ensuring that those who teach have the skills to teach effectively, that those who don’t teach effectively receive help to develop the necessary skills, and that those who ultimately cannot demonstrate their ability to teach effectively not be rewarded with reappointment, tenure, and promotion.
Finally, we must extend this sense of transparency and mutuality to our colleagues in K-12 education. The NAAL results require us to acknowledge that, far from being above the challenges affecting primary and secondary schools, higher education shares them. Poorly-prepared high school graduates add to the cost and lower the quality of higher education. Poorly-prepared teachers contribute to the deficiencies in our public schools. Current practices in higher education yield unsatisfactory results for too many of our students. Unless we acknowledge our interdependence, we are unlikely to meet the challenges facing both the public schools and higher education. Schools and colleges should form sustained, mutually respectful partnerships to address the problems in teaching and learning that everyone faces. We are all in this together.
For now, higher education remains in control of its own destiny. If we act decisively to respond to the challenges implicit in the results of the National Assessment of Adult Literacy, we can ensure that our graduates are demonstrably well-prepared to lead productive and enlightened lives. If, however, we choose to ignore the weaknesses exposed by NAAL, our profession will have failed in its obligation to the public and played right into the hands of those who believe that higher education is out-of-touch, over-funded, and under-regulated. Those who may think I’ve overstated the dangers we face don’t need a crystal ball to see higher education’s future if we fail to act. They need look no further than the results of No Child Left Behind.
Direct responses to lao@wabash.edu. We will forward comments to the author.
--------------------------------------
For related articles, see:
Graduated but Not Literate, by Doug Lederman, Inside Higher Ed., 12/16/05
No College Left Behind, by Doug Lederman, Inside Higher Ed., 2/15/06
--------------------------------------
The comments published in LiberalArtsOnline reflect the opinions of the author(s) and not necessarily those of the Center of Inquiry or Wabash College. Comments may be quoted or republished in full, with attribution to the author(s), LiberalArtsOnline, and the Center of Inquiry in the Liberal Arts at Wabash College.
LiberalArtsOnline is a monthly publication that provides an online forum to promote inquiry and reflection on liberal arts education. We invite you to subscribe and to submit an essay. Past issues are available in the archives.