Abstract
The National Survey of Student Engagement (NSSE) collects information from thousands of college students nationwide about their participation in programs and activities related to student learning and personal development. Using this survey, colleges can find out detailed information regarding the frequency that students engage in activities such as communicating with instructors, discussing classroom material with other students, and attending cultural activities. Although most questions on the survey relate to college experiences, several questions relate to outcomes of interest, such as students’ satisfaction with their college experience and the extent of impact they believe their college experience is having on their knowledge, skills, and personal development in 16 specified areas. Campuses that are members of a multi-institution consortium can add extra items to the standard NSSE, and although not described in this review, the Faculty Survey of Student Engagement (FSSE) can be administered in conjunction with the NSSE to provide a more comprehensive examination of the teaching and learning environment on a college campus. The NSSE is typically administered to first-year and senior students concurrently in the spring of the academic year. The survey can be administered annually or at longer intervals for the benefits of over-time data. The NSSE provides rich information for campus self-study and improvement initiatives, and many institutions have incorporated select findings into their public relations and marketing efforts. Although NSSE was not developed to specifically assess liberal arts educational processes and outcomes, its comprehensive scope and focus on "best practices" in undergraduate education and its widespread, successful application for institutional improvement in scores of liberal arts college settings suggest its appropriateness for this purpose.
Introduction
The National Survey of Student Engagement (NSSE) is a survey that gathers information on student participation in programs and activities offered by colleges and universities for students’ learning and personal development. [a] NSSE was developed by researchers at Indiana University with the sponsorship of the Carnegie Foundation for the Advancement of Teaching and the Pew Forum for Undergraduate Learning. Designed for use with undergraduates attending four-year public and private institutions of higher education, the NSSE provides an estimate of what students do with their time both in and out of class, and how students view the curricular and co-curricular aspects of their institution.
Technically, the survey consists of 30 items, of which the last 16 pertain to student background characteristics such as race/ethnicity, full- or part-time enrollment status, and average college grades. However, given that the first 11 of the 30 are multiple-part items, the actual number of answers is closer to 100.
This review includes information about the administration of the NSSE, provides a brief summary of its development and uses, and evaluates the appropriateness of the NSSE for assessing liberal arts education experiences.
Survey Administration
As of spring 2004, the National Survey of Student Engagement [c] had been administered at over 730 different public and private four-year colleges and universities since 2000, representing all 50 states and Puerto Rico. Random samples, typically ranging from 450 to 1,000 depending on institutional size, are selected from first-year and senior-level undergraduates. Together, this information can generate a picture of the overall collegiate experience from a point near institutional entry to a point where students are at least well immersed in their academic majors, if not nearing graduation [b].
The survey can be administered as-is, or questions can be added to provide supplemental information to probe more deeply into a given topical area (note: questions can only be added when the institution chooses to partner with other schools to make up a consortium). A web option, in addition to paper administration, is available. The cost of the survey is determined primarily by undergraduate enrollment size, and the fee structure is graduated to accommodate its use in very small schools. Fees are also impacted by whether an institution desires to administer the survey to more students than the standard sampling guidelines, and whether the additional sampling is administered by NSSE or locally [b]. For an institution with an undergraduate enrollment of 4,000-15,000, the cost of the standard survey is $4,900 [d].
Campuses typically administer the NSSE during the spring semester. Initial and follow-up contacts by email or U.S. mail are made to encourage students to complete the survey. In the fall following the survey administration, campuses receive a report summarizing the collective characteristics of participating institutions and students. The report includes some general information on overall response rates, describes the students who completed the survey, and presents preliminary analyses of patterns of participation among various groups of students as well as suggestions for interpreting the data presented in the report. Campuses also receive national and local benchmark data for five clusters of activities related to effective educational practice: level of academic challenge, active and collaborative learning, student and faculty interaction, enriching educational experiences, and supportive campus environments. The national comparisons are based on survey administrations since 2000. The information provided makes norm-referenced (with peer institutions or by Carnegie classification) and criterion-referenced (relative to the standards an institution establishes for itself) comparisons possible. NSSE also provides some written instructions to a campus for interpreting their data and returns a campus’ own data in a format suitable for further statistical analyses. Supplemental analyses are available from the survey administrators for an additional fee. More complete information about the report that NSSE provides is available on their website [e].
Potential Users and Uses of the NSSE
The National Survey of Student Engagement can provide valuable campus-specific and comparative information for a variety of interested education stakeholders such as academic and student affairs administrators, potential students and their parents, and education oversight and advisory groups, both on and off campus. Originally designed as an alternative measure of institutional quality, the survey can provide descriptive information for academic and student affairs administrators and practitioners, faculty, and researchers about the experiences and self-perceived gains of college students on a given campus. To the extent that student participation in educationally purposeful activities constitutes an appropriate measure of institutional quality, the instrument garners a wealth of information about the quality of a liberal arts experience on a particular campus. Such detailed data can provide empirical support and direction for an institution’s self-improvement efforts related to teaching and learning and can help the institution know how well it is using its resources. Radford University, for instance, used its 2000 data on five key benchmarks of student learning, such as level of academic challenge and supportiveness of the campus environment, as the focus of campus-wide discussions. In addition, the data informed many of the strategies that were implemented by the university in the development of its "Quality Enhancement Plan," a self-study component of the accreditation process [f]. General guidelines on how campuses can use their survey data for accountability and improvement can be found in the report entitled, "Converting Data into Accountability," which is available from the survey developers.
In addition, the data provided to a campus in its feedback reports can identify gaps between commonly held perceptions of the campus experience and students’ reports of their experiences and guide the institution’s efforts to reshape the environment. The instrument may be particularly helpful in identifying subpopulations of students whose experiences may be less favorable than others. For example, the experiences of commuter students [6], transfer students [4], students of color (particularly on predominantly white campuses), students within certain majors, and white males [10] may merit particular examination in this regard. Finally, campuses may find their NSSE data to be particularly informative as they consider issues of student performance and persistence, because these seem to go hand in hand with student engagement [4].
Off-campus consumers of data such as accrediting agencies, state oversight agencies, potential students and their families, alumni, and the media may also be interested in NSSE data. The data collected using this instrument reflect "new" information about institutional quality, different from U.S. News & World Report ratings [10], and in that respect can contribute additional perspectives about an individual institution and about the higher education experiences of our nation’s students as a whole. In this latter respect, data can serve as a national barometer of the quality of our country’s institutions of higher education, providing fodder for educational advocates of various capacities.
Individual institutions are free to use their own NSSE data for institutional purposes. They can also use the survey findings to highlight specific aspects of their campus to the public or to inform change and strategic planning [b].
Development and History of NSSE
The National Survey of Student Engagement is grounded in research literature that explains how students develop cognitively and personally as they experience college and university life. Several fundamental premises about college student development provide a theoretical base for the instrument. First, the time and energy students devote to educationally purposeful activities is the single best predictor of their learning and personal development. [1] Second, institutions that are characterized by the principles for good practice in undergraduate education [3] and are perceived to be inclusive and affirming are more likely to direct students’ energies toward high levels of student participation. The seven principles include student/faculty contact, cooperation among students, active learning, prompt feedback, high expectations, time on task, and respect for diverse talents and ways of learning. Thus, the extent to which institutions of higher education can develop and maintain these environmental conditions, and the extent to which students are willing to partake of these experiences, will have a direct and linear relationship with students’ personal and learning gains in college. This perspective on the college experience has been offered as an alternative measure of institutional quality to more traditional measures such as U.S. News rankings, graduation rate, reputation, and number of degrees granted. In addition to its theoretical grounding, the content of the NSSE is also supported across the broader higher education community—most of the questions have been used in other long-running, respected college student research programs, such as UCLA’s Freshman Survey (CIRP) and Indiana University’s College Student Experiences Questionnaire (CSEQ) [5].
Technical Qualities
The information in this section is summarized in a report by the survey developer [5] available online.
As a self-report instrument, NSSE is subject to criticisms that respondents may be untruthful or inaccurate in their responses, but this concern is minimized by the omission of socially sensitive areas of inquiry. In addition, instructing respondents to situate their estimates of time spent on various activities within the context of the recent past (i.e., the current school year) encourages more accurate responses. Survey respondents are selected randomly from an institution’s enrollment to enhance claims that the reported data accurately reflect the general pool of students from which the sample was selected.
The survey has undergone multiple drafts and revisions throughout its development, external reviews by focus groups of students [7] and professional survey researchers, and field testing at 75 selected colleges and universities to ensure that the statements were clear and easily understood. The developers also performed statistical tests that suggest the underlying constructs are accurately represented by the survey items. The data collected to date indicate that there may be small differences between respondents and non-respondents in that the latter may be slightly more engaged than the former and similarly, students who take the web version of the survey may also be slightly more engaged than those who take the paper version [2]. Re-administration of the survey from year to year suggests reasonable correspondence of responses within a given institution.
Using NSSE to Assess Liberal Arts Outcomes
The NSSE survey focuses on curricular and co-curricular experiences that promote student learning. It is not designed to assess outcomes, per se. However, the NSSE includes indirect measures that assess the extent to which students engage in educational practices associated with high levels of learning and development. For example, the following are all examples of practices that contribute to student learning:
Research done by the Center of Inquiry in the Liberal Arts suggests that the degree to which students experience a combination of these practices during their college career correlates positively to liberal arts outcomes. For example, a student who experiences high levels of faculty interaction; supportive relationships with faculty, administrators, and peers; and challenging coursework in college will likely demonstrate growth in areas like openness to diversity and critical thinking, among others. This same research suggests that these gains will be more pronounced for women, high-risk students, and minority students.
Most NSSE questions look directly at the best practices discussed above. For example, question 11 asks students to assess the extent to which their experiences on a given campus have contributed to their knowledge, skills, and personal development in acquiring a broad general education; acquiring work-related knowledge and skills; writing clearly and effectively; speaking clearly and effectively; thinking critically and analytically; analyzing quantitative problems; using computing and information technology; working effectively with others; voting in local, state, or national elections; learning effectively on one’s own; understanding oneself; understanding people of other racial and ethnic backgrounds; solving complex real-world problems; developing a personal code of values and ethics; contributing to the welfare of the community; and developing a deepened sense of spirituality. Although these items shed no light on how much students have gained or developed over the course of their college experience at a given institution, they do provide information on the extent to which students attribute their developmental gains to their collegiate experiences. Thus, they can offer insight into the strength of association (from very little to very much) between various campus environmental features and student outcomes.
Another area of the survey deals with acquisition of higher order thinking skills. Students are asked to estimate how frequently their coursework emphasized memorizing facts, analyzing basic elements of an idea, synthesizing and organizing information and ideas in more complex ways, making judgments about the value of arguments, and applying theories or concepts to practical problems or new situations. (The NSSE also collects data on several dimensions of the liberal arts experience not found in the Pascarella et al. list, including the development of student citizenship and community service, fitness, spirituality, and personal codes of values and ethics.)
In addition to using the survey questions themselves to understand the relationship between campus environment and outcomes, campuses can use their NSSE data to develop statistical prediction models. Institutional research offices or NSSE researchers can readily provide these additional analyses. For example, a campus can identify which aspects of the environment and which engagement experiences are the best predictors of students’ satisfaction, willingness to repeat their college-choice decision, or grades.
Data from the NSSE survey is intended to be interpreted at an institutional level. Survey results provide institutions with information about the quality of the undergraduate student experience and can suggest the presence or absence of a liberal arts environment. However, capturing the broader intellectual character of a school can be challenging when using aggregate survey data gathered from student perceptions of curricular and co-curricular practices. Such information is certainly useful, but additional surveys, and qualitative approaches should be used to provide a more in-depth and rich representation of institutional character.
Conclusions
The National Survey of Student Engagement provides a widely used and easily administered tool for assessing levels of student participation in the curricular and academically pertinent co-curricular dimensions of the undergraduate experience. While not specifically developed to explore the liberal arts experience, the NSSE appears to capture detailed information on the fundamental dimensions and characteristics of the liberal arts experience—and more. Cross-sectional data collected at one point in time and longitudinal data collected over time can be used to create institutional profiles of students as a whole and document first- to senior-year changes in participation and perceptions, both of which are helpful in identifying foci for a campus’ self-improvement efforts.
Unfortunately, most campuses do not have baseline information about their students’ inclinations at college entry toward engagement or participation in campus experiences and opportunities. This limitation is, of course, not exclusive to the NSSE, but the absence of such data makes it nearly impossible to separate the institution’s influence in channeling students toward educationally purposeful and meaningful activities and outcomes, and those tendencies that students brought with them as they first stepped foot on a given campus. Apparent variations in the college environment may simply be differences in the students’ openness to the impact of college that existed at the point of entry [8]. Neither does the NSSE provide an actual assessment of the level of students’ proficiency in desirable liberal arts outcomes. Despite these limitations, the NSSE appears to be a comprehensive instrument suitable for some self-assessment of liberal arts education experiences. It has additional benefits associated with sustained use across a large number of institutions, such as rich cross-sectional and longitudinal data for peer institutions and national comparisons.
Links
