ASHE Higher Education Report, Volume 31, Number 2
Ernest T. Pascarella, Gregory C. Wolniak, Tricia A. Seifert, Ty M. Cruce, and Charles F. Blaich
The monograph can be purchased at the Jossey-Bass website.
Background and Purpose
Liberal arts education and the residential liberal arts college have a long and honored tradition in the American postsecondary education system (e.g., Kimball, 1986, Koblik & Graubard, 2000; Nussbaum, 2002; Rothblatt, 2003; Rudolph, 1962). At the heart of this tradition is the strongly held belief that undergraduate education should be a transforming experience that fundamentally changes the individual (Heath, 1968; Winter, McClelland, & Stewart, 1981). It is expected that liberal arts education, particularly as it transpires at residential liberal arts colleges, is most likely to provide the intellectual and interpersonal culture that leads to this transformation. There is indeed evidence to suggest the cultural distinctiveness of liberal arts colleges. In their review of the literature on college impact, Pascarella and Terenzini (1991) argue that there is a particular institutional ethos or social-psychological context that tends to maximize student development during college. The elements of this ethos would appear to be such things as a strong faculty emphasis on teaching and student development, a common valuing of the life of the mind, small size, a shared intellectual experience, high academic expectations, and frequent interactions in and outside the classroom between students and faculty and between students and their peers (Pascarella & Terenzini, 1998). Such environmental traits would appear to be most often found at small, private, liberal arts colleges, and particularly at selective liberal arts colleges (Astin, 1999; Chickering & Reisser, 1993; Heath, 1968; Pascarella & Terenzini, 1991; Winter, McClelland, & Stewart, 1981). A body of evidence from diverse, and frequently unconnected sources, has slowly begun to shed light on the various impacts of liberal arts colleges and liberal arts education on students. The predominant weight of this evidence suggests that, compared to other types of institutions, liberal arts colleges, and particularly academically selective liberal arts colleges, are more likely to foster student growth on a range of educational outcomes (e.g., Astin, 1993, 1999; Heath, 1968, 1976; McNeel, 1994; Pascarella & Terenzini, 1991; Umbach & Kuh, 2003; Winter, McClelland, & Stewart, 1981).
A close examination of this evidence, however, suggests important limitations. First, the studies tend to be disconnected with each focusing on a different aspect of the impact of liberal arts education. Consequently, the body of knowledge lacks any genuine systematic integration. Second, most of the investigations have serious methodological problems that lead to questions about the internal validity of the results. Even the methodologically most rigorous studies (e.g., Anaya, 1999; Astin, 1999; Umbach & Kuh, 2003) have the problems inherent in using student self-reported gains or objective measures, such as the Medical College Admissions Test, that are taken by only a small minority of students. This paper will report and discuss the results of a large, systematic study of the short- and long-term impacts of liberal arts colleges and liberal arts education. The results are based on analyses of two major longitudinal data sets that contain information on over 6,500 students and alumni from over 40 institutions located throughout the country.
The longitudinal nature of both data sets permitted us to carry out analyses in a way that avoids the major methodological problems characteristic of much of the existing research. Specifically, we were able to estimate the net effects of liberal arts colleges while controlling for the biasing influence of student characteristics and other potential confounding influences. For example, in estimating the effects of liberal arts colleges on intellectual and personal growth during college, we were able to take into account where students scored on each outcome measure when they entered college in addition to student background characteristics. Our review of the literature suggests that the paper will summarize the largest and most extensive investigation of the effects of liberal arts colleges and liberal arts education to date.
The study sought answers to the following questions:
The Study
The study reported in the paper was based on analyses of two longitudinal data sets: The National Study of Student Learning (NSSL) and the Appalachian College Association (ACA) Study. The NSSL was funded by the Department of Education and follows samples of students from 16 four-year colleges and universities for three years. (The 16 institutions vary widely in selectivity, are located in 13 different states, and include 5 liberal arts colleges, 7 regional colleges and universities, and 4 research universities.) The NSSL contains extensive information on students’ cognitive and personal development (using standardized instruments) and measures a wide range of students’ academic and non-academic experiences during each of the three years of college. The data were collected from each student on four separate occasions: precollege, and at the end of the first, second, and third years of college. Altogether, there are between 500-600 variables on each NSSL participant.
The Appalachian College Association study was funded by the Mellon and Spencer Foundations and the data collection was conducted by ACT. The ACA study follows three separate samples of alumni from 26 institutions for 5 years, 15 years, and 25 years after graduation, respectively. (The 26 institutions are located in 4 different states and include 10 private baccalaureate liberal arts colleges, 11 private masters-level institutions, and 5 public regional universities.) The ACA study contains not only extensive information on each participant prior to college (e.g., ACT composite scores, background and demographic data, and aspirations), but also a wide variety of information on participants’ perceptions of college impact and their personal and occupational lives after college.
Sample Descriptions
The core samples for the National Study of Student Learning consisted of 1,957 students through the first follow-up (580 at liberal arts colleges, 544 at research universities, 833 at regional institutions), 1,341 students through the second follow-up (419 at liberal arts colleges, 373 at research universities, 549 at regional institutions), and 936 students through the third follow-up (299 at liberal arts colleges, 259 at research universities, and 378 at regional institutions). These three follow-up samples represent 72.2%, 49.5%, and 34.6%, respectively, of the original random sample of 2,709 entering students at the 16 institutions in the NSSL. Because of attrition from the original sample over the course of the study, several steps were taken to improve and verify the representativeness of the samples. First, we developed a separate sample weighting algorithm for each of the three years of the study to adjust for potential response bias by sex, ethnicity, and institution. Within each of the 16 institutions, participants were weighted up to that institution’s end-of-year population by sex (i.e., male or female), and race/ethnicity (i.e., White, Black, Hispanic, Other). For example, if an institution had 100 Hispanic men in its first-year class and 25 Hispanic men in the sample, each Hispanic male in the first-year sample was assigned a weight of 4.00.
The core sample for the Appalachian College Association Study consisted of 5,786 alumni of the 26 participating institutions (2,007 at private baccalaureate liberal arts colleges, 1,990 at private masters-level institutions, and 1,789 at public regional universities). Overall, the sample represented about 12.6% of the population of alumni of the 26 institutions who graduated between 1974–76, 1984–86, or 1994–96. Using estimated population figures provided by ACT, the alumni sample was weighted up to the total population of alumni of the 26 participating institutions by sex and institution, within each of the three graduation cohorts. The weighted sample was quite similar to the estimated population of alumni by sex, race, ACT English and Mathematics scores, secondary school grades, precollege educational degree plans, and expectations of the need for financial aid during college. The three graduation cohorts were almost equally represented: 33.9% of the weighted sample was from the 1974–76 cohort, 31.4% was from the 1984–86 cohort, and 34.7% was from the 1994–96 cohort.
Variables
The extensive set of outcome variables addressed in the study precludes describing them all in a proposal. However, the NSSL data included 11 measures of student intellectual and personal development during college (e.g., standardized measures of critical thinking, reading comprehension, science reasoning, and the like, as well as graduate degree aspirations and scales measuring such dimensions as openness to diversity, locus of control, and attitude toward literacy). Complete operational definitions and psychometric properties of all measures will be shown in the full paper. The ACA data included extensive measures of graduates’ professional and personal lives, as well as their retrospective perceptions of the impact of their undergraduate education. Operational definitions of all outcomes are provided in the full paper. The outcomes included such dimensions as: graduate degree attainment/goal and graduate degree plans for children; retrospective perceptions of the impact of one’s undergraduate college and overall satisfaction with the undergraduate experience; labor market experiences; community, social, and political involvement; savings behavior and charitable donations; involvement in continuing education; health status and health-related behaviors; satisfaction with life and sense of control over life events; region of the country in which one lives and works; and the importance of different skills and values in one’s current endeavors.
Analyses
Various forms of linear and logistic regression were employed to estimate all effects. The exact regression specifications (i.e., the potential confounding influences controlled) are presented in the full paper.
Major Findings and Conclusions
Of 11 measures of intellectual and personal growth, liberal arts colleges (as compared to other institutions) had no significant net influence on 4 measures (i.e., reading comprehension, critical thinking, graduate degree plans, and locus of control), negative effects on 2 measures (i.e., mathematics and science reasoning), mixed effects on 2 measures (i.e., positive attitude toward literacy and preference for higher order cognitive tasks), and positive net effects on openness to diversity, learning for self-understanding, and writing skills.
We hypothesized that the reason for the mixed effects of liberal arts colleges on student intellectual and personal development is that institutional type (e.g., liberal arts colleges, research universities, regional institutions) may be a structural characteristic that is too general and remote to adequately capture the full impact of liberal arts education. What may count more in the explanation of student intellectual and personal growth are environmental characteristics and individual experiences that provide a more proximal and construct-valid expression of a liberal arts education than simply attending a liberal arts college. To test the validity of this "hypothesis," we used the theoretical and empirical literature to construct several psychometrically-reliable scales operationalizing salient components of a liberal arts education or experience (e.g., extensive interaction between students and faculty, faculty emphasis on effective teaching, academic challenge and high expectations, an integrated intellectual experience, extensive student extracurricular involvement and interaction with peers, emphasis on a residential experience). These were two aggregate-level measures of an institution’s "liberal arts emphasis," and an individual-level measure of students’ "liberal arts experiences." (Complete operational definitions of all three scales are provided in the full paper.) Net of student background characteristics, all three measures were strongly predicted by attendance at a liberal arts college—evidence of their construct validity. However, these measures of an institution’s liberal arts emphasis and students’ liberal arts experiences were generally more consistent and positive net predictors of student intellectual growth in college than was institutional type. Moreover, these estimated effects persisted in the presence of statistical controls not only for student precollege characteristics, but also for institutional selectivity and institutional type. This suggests that, although a liberal arts emphasis and liberal arts experiences are most likely at liberal arts colleges, they are not exclusive to those institutions. When implemented and nurtured at research universities and regional institutions, they have important impacts on student intellectual and personal growth.
Analyses of net long-term impacts indicated that, in comparison to alumni of other types of institutions, graduates of baccalaureate liberal arts colleges reported that their undergraduate experience had a significantly stronger impact on their learning and intellectual development, development of leadership skills, personal and spiritual development, and development of responsible citizenship. Similarly, liberal arts college graduates had higher levels of graduate degree attainment, were more likely to be employed in a nonprofit organization and to report that their undergraduate experience prepared them well for their first and current jobs, donated a larger percentage of income to charity, took more continuing education courses, and were more likely to report that learning and intellectual development and responsible citizenship were important in their current lives.
We uncovered no evidence that the net effects of attending a liberal arts college on intellectual and personal development differed in magnitude for different kinds of students. However, that was not the case for an institution’s liberal arts emphasis and students’ liberal arts experiences. The clear weight of evidence indicated that the influence of these variables may function in a compensatory manner. That is, an institution’s liberal arts emphasis and students’ liberal arts experiences tended to have stronger positive effects on intellectual and personal growth for students who began college relatively low on these dimensions than for their counterparts who started postsecondary education with relatively high levels of intellectual and personal development. Though less extensive, there was also evidence to suggest that the cognitive benefits of an institution’s liberal arts emphasis and students’ liberal arts experiences accrued more to students of color than to their White peers.
The final section of the paper discusses the implications of the findings for institutional policy and planning, as well as for future inquiry on the impact of college on students. It also delineates the limitations of the study.
References
Anaya, G. (1999, April). Within-college curricular and co-curricular correlates of performance on the MCAT. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada.
Astin, A. (1993). What matters in college? San Francisco: Jossey-Bass.
Astin, A. (1999). How the liberal arts college affects students. Daedalus, 128(1), 77-100.
Chickering, A., & Reisser, L. (1993). Education and identity (2nd ed.). San Francisco: Jossey-Bass.
Heath, D. (1968). Growing up in college: Liberal education and maturity. San Francisco: Jossey-Bass.
Heath, D. (1976). What the enduring effects of higher education tell us about a liberal education. Journal of Higher Education, 47(2), 173-190.
Kimball, B. (1986). Orators and philosophers: A history of the idea of liberal education. New York: Teachers College Press, Columbia University.
Koblik, S., & Graubard, S. (Eds.). (2000). Distinctively American: The residential liberal arts colleges. Brunswick, NJ: Transaction Publishers.
McNeel, S. (1994). College teaching and student moral development. In J. R. Rest & D. Narvaez (Ed.), Moral development in the professions: Psychology and applied ethics (pp. 26-47). Hillsdale, NJ: Lawrence Erlbaum.
Nussbaum, M. (2002, November). Liberal education and global responsibility. Paper presented at the Council of Independent Colleges, CAO Conference, Santa Fe, NM.
Pascarella, E., & Terenzini, P. (1991). How college affects students. San Francisco: Jossey-Bass.
Pascarella, E., & Terenzini, P. (1998). Studying college students in the 21st century: Meeting new challenges. Review of Higher Education, 21(2), 151-165.
Rothblatt, S. (2003). The living arts: Comparative and historical reflections on liberal education. Washington, DC: Association of American Colleges and Universities.
Rudolph, F. (1962). The American college and university: A history. New York: Vintage Books.
Umbach, P., & Kuh, G. (2003, May). Student experiences with diversity at liberal arts colleges: Another claim for distinction. Paper presented at the annual meeting of the Association for Institutional Research, Tampa, FL.
Winter, D., McClelland, D., Stewart, A. (1981). A new case for the liberal arts: Assessing institutional goals and student development. San Francisco: Jossey-Bass.