Introduction
The National Survey of Student Engagement (NSSE) is a survey that gathers information on student participation in programs and activities offered by colleges and universities for students’ learning and personal development. (a) NSSE was developed by researchers at Indiana University with the sponsorship of The Carnegie Foundation for the Advancement of Teaching and The Pew Forum for Undergraduate Learning. Designed for use with undergraduates attending four-year public and private institutions of higher education, the NSSE provides an estimate of what students do with their time both in and out of class, what students perceive their institutions emphasize, and how much students perceive that their institution has contributed to their knowledge, skills, and personal development in specified areas (b).
Technically, the survey consists of 30 items, of which the last 16 pertain to student background characteristics such as race/ethnicity, full- or part-time enrollment status, and average college grades. However, given that the first 11 of the 30 are multiple-part items, the actual number of answers is closer to 100.
In this review, we include information about the administration of the NSSE, provide a brief summary of its development and uses, comment on the relationship between NSSE and the Input-Environment-Output framework of the college experience, and evaluate the appropriateness of the NSSE for assessing liberal arts education experiences.
Survey Administration
As of spring 2004, the National Survey of Student Engagement (c) had been administered at over 730 different public and private four-year colleges and universities since 2000, representing all 50 states and Puerto Rico. Random samples, typically ranging from 450 to 1,000 depending on institutional size, are selected from first-year and senior-level undergraduates. Together, this information can generate a picture of the overall collegiate experience from a point near institutional entry to a point where students are at least well immersed in their academic majors, if not nearing graduation (b).
The survey can be administered as-is or, if an institution is part of a consortium of six or more other institutions in the survey, questions can be added to provide supplemental information to probe more deeply into a given topical area. A web option, in addition to paper administration, is available. The cost of the survey is determined primarily by undergraduate enrollment size, and the fee structure is graduated to accommodate its use in very small schools. Fees are also impacted by whether an institution desires to administer the survey to more students than the standard sampling guidelines, and whether the additional sampling is administered by NSSE or locally (b). For an institution with an undergraduate enrollment of 4,000-15,000, the cost of the standard survey is $4,900 (d).
Campuses typically administer the NSSE during the spring semester. Initial and follow-up contacts by email or U.S. mail are made to encourage students to complete the survey. In the fall following the survey administration, campuses receive a report summarizing the collective characteristics of participating institutions and students. The report includes some general information on overall response rates, describes the students who completed the survey, and presents preliminary analyses of patterns of participation among various groups of students as well as suggestions for interpreting the data presented in the report. Campuses also receive national and local benchmark data for five clusters of activities related to effective educational practice: level of academic challenge, active and collaborative learning, student and faculty interaction, enriching educational experiences, and supportive campus environments. The national comparisons are based on survey administrations since 2000. The information provided makes norm-referenced (comparisons with peer institutions or by Carnegie classification) and criterion-referenced (relative to the standards an institution establishes for itself) comparisons possible. NSSE also provides some written instructions for a campus in interpreting their data and returns a campus’ own data in a format suitable for further statistical analyses. Supplemental analyses are available from the survey administrators for an additional fee. More complete information on the report that is provided is available HERE (e).
Potential Users and Uses of the NSSE
The National Survey of Student Engagement can provide valuable campus-specific and comparative information for a variety of interested education stakeholders such as academic and student affairs administrators, potential students and their parents, and education oversight and advisory groups, both on and off-campus. Originally designed as an alternative measure of institutional quality, the survey can provide descriptive information for campus academic and student affairs administrators and practitioners, faculty, and researchers about the experiences and self-perceived gains of college students on a given campus. To the extent that student participation in educationally purposeful activities constitutes an appropriate measure of institutional quality, the instrument garners a wealth of information about the quality of a liberal arts experience on a particular campus. Such detailed data can provide empirical support and direction for an institution’s self-improvement efforts related to teaching and learning and can help the institution know how well it is using its resources. Radford University, for instance, used its 2000 data on five key benchmarks of student learning, such as level of academic challenge and supportiveness of the campus environment, as the focus of campus-wide discussions. In addition, the data informed many of the strategies that were implemented by the university in the development of its Quality Enhancement Plan, a self-study component of the accreditation process (f). General guidelines on how campuses can use their survey data for accountability and improvement can be found in the report entitled, "Converting Data into Accountability," which is available from the survey developers HERE.
In addition, the data provided to a campus in its feedback reports can identify gaps between commonly held perceptions of the campus experience and students' reports of their experiences and guide the institution’s efforts to reshape the environment. The instrument may be particularly helpful in identifying subpopulations of students whose experiences may be less favorable than others. For example, the experiences of commuter students (6), transfer students (4), students of color (particularly on predominantly white campuses), students within certain majors, and white males (10) may merit particular examination in this regard. Finally, campuses may find their NSSE data to be particularly informative as they consider issues of student performance and persistence, because these seem to go hand in hand with student engagement (4).
Off-campus consumers of data such as accrediting agencies, state oversight agencies, potential students and their families, alumni, and the media may also be interested in NSSE data. The data collected using this instrument reflect "new" information about institutional quality, different from US News & World Report ratings (10), and in that respect can contribute additional perspectives about an individual institution and about the higher education experiences of our nation’s students as a whole. In this latter respect, data can serve as a national barometer of the quality of our country’s institutions of higher education, providing fodder for educational advocates of various capacities.
Individual institutions are free to use their own NSSE data for institutional purposes, and many post-select findings from the NSSE administration on their campus to highlight to the public specific aspects of their campus (b).
Development and History of NSSE
The National Survey of Student Engagement is grounded in research literature that explains how students develop cognitively and personally as they experience college and university life. Several fundamental premises about college student development provide a theoretical base for the instrument. First, the time and energy students devote to educationally purposeful activities is the single best predictor of their learning and personal development (Astin, 1993; Pascarella & Terenzini, 1991; Pace, 1980). Second, institutions that are characterized by the principles for good practice in undergraduate education (3) and are perceived to be inclusive and affirming are more likely to direct students’ energies toward high levels of student participation. The seven principles include student-faculty contact, cooperation among students, active learning, prompt feedback, high expectations, time on task, and respect for diverse talents and ways of learning. Thus, the extent to which institutions of higher education can develop and maintain these environmental conditions, and the extent to which students are willing to partake of these experiences, will have a direct and linear relationship with students’ personal and learning gains in college. This perspective on the college experience has been offered as an alternative measure of institutional quality to more traditional measures such as U.S. News rankings, graduation rate, reputation, and number of degrees granted. In addition to its theoretical grounding, the content of the NSSE is also supported across the broader higher education community--most of the items have been used in other long-running, respected college student research programs, such as UCLA’s Freshman Survey (CIRP) and Indiana University’s College Student Experiences Questionnaire (CSEQ) (5).
Technical qualities
The information in this section is summarized in a report by the survey developer (5) available online.
As a self-report instrument, NSSE is subject to criticisms that respondents may be untruthful or inaccurate in their responses, but this concern is minimized by the omission of socially sensitive areas of inquiry. In addition, instructing respondents to situate their estimates of time spent on various activities within the context of the recent past (i.e., the current school year) encourages more accurate responses. Survey respondents are selected randomly from an institution’s enrollment to enhance claims that the reported data accurately reflect the general pool of students from which the sample was selected.
The survey has undergone multiple drafts and revisions throughout its development, external reviews by focus groups of students (7) and professional survey researchers, and field testing at 75 selected colleges and universities to ensure that the items were clear and easily understood. The developers also performed statistical tests that suggest the underlying constructs are accurately represented by the survey items. The data collected to date indicate that there may be small differences between respondents and non-respondents in that the latter may be slightly more engaged than the former and similarly, students who take the web version of the survey may also be slightly more engaged than those who take the paper version (2). Re-administration of the survey from year to year suggest reasonable correspondence of responses within a given institution.
Relationship of the NSSE to the Inputs/Environment/Outcomes Model
The Inputs-Environment-Outcomes (I-E-O) model is a framework that can guide assessments in higher education using data on student inputs, the educational environment, and student outcomes (1). In this model, Inputs are defined as "personal qualities the student brings initially to the education program, and include the student's initial level of developed talent at the time of entry" (p. 18). It is important to know what students "look like" at entry to make a stronger case for cause and effect between the campus environment and student outcomes.
In the model, Environment "refers to the student's actual experiences during the educational program" (1 p. 18), including participation in specified practices, programs, and interventions. The individual Input and the campus Environment to which they are exposed are presumed to interact in ways that bring about certain Outcomes of interest, which in this case are described as the "'talents' we are trying to develop in our educational program" (1 p. 18).
The majority of questions on the NSSE focus on the Environment phase of the I-E-O model, in which students indicate the frequency at which they participate in a wide range of educationally purposeful activities. The questions address outward behaviors as well as some mental processes, attitudes, and impressions of the campus environment, such as the friendliness of other students, emphasis the campus places on studying, support for academic and non-academic responsibilities, and provision of a culturally enriching environment.
Fewer questions on the NSSE pertain to the Inputs and the Outcomes components of an I-E-O framework. Information about students’ age, gender, ethnicity, and parental education levels can appropriately be characterized as Input variables, because students possess these characteristics at college entry. Other items such as those pertaining to class level, full- or part-time attendance status, and housing, may loosely fall into the category of Inputs, but should, perhaps, be more appropriately described as intervening variables that can indirectly influence the impact of the Environment on student Inputs.
In this application of the I-E-O model, student Outcomes can be described as the consequences or end results of the college experience. Desirable outcomes for the liberal arts experience might include reading comprehension, critical thinking, writing skills, openness to diversity/challenge, and plans to obtain a graduate degree (9). The NSSE does not assess students’ actual levels of academic or personal proficiency in any of these areas, nor does it even ask students to report self-perceived gains. However, students’ responses to several questions on the survey could be considered as Outcomes of the college experience, albeit not the most typical ones of interest for a liberal arts experience. Two survey questions ask students to rate their level of satisfaction with the quality of academic advising and with their overall educational experience at the university, and a third asks students whether they would attend the institution again if they could start over. Finally, one question asks students to report their cumulative average grades.
A final set of items on the NSSE (question 11) probes the relationship between the Environment (defined as "your experience at this institution") and select Outcomes. Specifically, students are asked to assess the extent to which their experiences on a given campus have contributed to their knowledge, skills, and personal development in acquiring a broad general education; acquiring work-related knowledge and skills; writing clearly and effectively; speaking clearly and effectively; thinking critically and analytically; analyzing quantitative problems; using computing and information technology; working effectively with others; voting in local, state, or national elections; learning effectively on one's own; understanding oneself; understanding people of other racial and ethnic backgrounds; solving complex real-world problems; developing a personal code of values and ethics; contributing to the welfare of the community; and developing a deepened sense of spirituality. Although these items shed no light on how much students have gained or developed over the course of their college experience at a given institution, they do provide information on the extent to which students attribute their developmental gains to their collegiate experiences and thus can offer insight into the strength of association (from very little to very much) between various campus environmental features and student Outcomes.
In addition to using the survey questions themselves to understand the relationship between campus Environment and Outcomes, campuses can use their NSSE data to develop statistical prediction models. Institutional research offices or NSSE researchers can readily provide these additional analyses. For example, a campus can identify which aspects of the Environment and which engagement experiences are the best predictors of students’ satisfaction, willingness to repeat their college-choice decision, or grades.
Using NSSE for liberal arts education assessment
In this section, we consider the appropriateness of the NSSE for assessing the liberal arts experience. For the purposes of this review, a liberal arts education is defined as occurring when students attend college full-time, experience effective teaching, have high levels of interaction with faculty and peers both within and outside the classroom, are encouraged to have high levels of academic expectation and effort, learn in settings that emphasize integration of ideas, and take courses that are characterized more by liberal arts perspectives than by vocational or technical perspectives (9). The National Survey of Student Engagement was not designed specifically as an instrument for assessing the liberal arts experience, but instead has its roots in more general "best practices" for effective undergraduate education (3). There appears to be a considerable amount of overlap between the two, as evidenced by single or multiple survey items pertaining to all six essential characteristics of a liberal arts education cited above. For instance, in one area of the survey dealing with acquisition of higher order thinking skills, students are asked to estimate how frequently their coursework emphasized memorizing facts, analyzing basic elements of an idea, synthesizing and organizing information and ideas into more complex ones, making judgments about the value of arguments, and applying theories or concepts to practical problems or new situations. The NSSE also collects data on several dimensions of the liberal arts experience not included in the Pascarella et al. list, including the development of student citizenship and community service, fitness, spirituality, and personal codes of values and ethics.
Finally, the NSSE asks students about several dimensions of campus life which are typically not associated with a liberal arts education, such as students’ acquisition of job or work-related knowledge, skills, and work or practicum experiences, their use of information technology, career plans, and the quality of their relationships with administrative personnel and offices.
Conclusions
The National Survey of Student Engagement provides a widely-used and easily administered tool for assessing levels of student participation in the curricular and academically-pertinent co-curricular dimensions of the undergraduate experience. While not specifically developed to reflect the liberal arts experience, the NSSE appears to capture detailed information on the fundamental dimensions and characteristics of the liberal arts experience--and more. Cross-sectional data collected at one point in time and longitudinal data collected over time can be used to create institutional profiles of students as a whole and document first- to senior-year changes in participation and perceptions, both of which are helpful in identifying foci for a campus’ self-improvement efforts.
Unfortunately, most campuses do not have baseline information about their students’ inclinations at college entry toward engagement or participation in campus experiences and opportunities. This limitation is, of course, not exclusive to the NSSE, but the absence of such data makes it nearly impossible to separate the institution’s influence in channeling students toward educationally purposeful and meaningful activities and outcomes, and those tendencies that students brought with them as they first stepped foot on a given campus. Apparent differences in the college environment may simply be differences in the student’s openness to the impact of college that existed at the point of entry (8). Neither does the NSSE provide an actual assessment of the level of students’ proficiency in desirable liberal arts outcomes. Despite these limitations, the NSSE appears to be a comprehensive instrument suitable for some self-assessment of liberal arts education experiences. It has the additional benefits associated with sustained use across a large number of institutions, such as rich cross-sectional and longitudinal data for peer institutions and national comparisons.
References