by Glenda Droogsma-Musoba, Indiana University
Amelia Noël-Elkins, Indiana University
Chris Rasmussen, University of Michigan
August 10, 2004
Each of the following describes the classroom assessment tool in terms of its strengths and weaknesses.
Portfolios
Definition: A compilation of the work produced by a student that will allow for his or her reflective assessment of the broader educational gains from the assignments. Portfolios are most frequently associated with artists and represent a sample of their work.
How to: Throughout a pre-defined education segment (e.g., semester, academic year, four years) a student compiles the assignments in a central location. The process of gathering the work and reflecting on prior assignments highlights educational gains and progression. Compilation and systematic analyses of the data enable assessment of process and educational outcomes over time or between individual students or groups.
What it does not measure: This method does not measure the final standing of a student against a content benchmark. Rather, it measures the progression of educational gains.
Benefits: The process encourages and develops self-reflection skills in students and provides valid information to assist them in assessing their intellectual competencies. Institutions benefit as a result of readily available data to help inform the adjustment of classroom assignments.
General challenges: Depending on the length of the predetermined education segments, portfolio collection and analysis can be a long process with relatively small gains over an extended period of time. In addition, the student’s ability to appreciate educational gains may require supervision. As an institutional assessment of general student outcomes over the course of enrollment, the process requires matching student work over several years of study in multiple departments.
Integration with other research/assessment: On the most basic level, this is integrated with the normal evaluative process in the classroom setting.
Journals
Definition: As an academic assessment tool, a journal is a written record of a student’s self-reflective process that can be integrated into the regular classroom curriculum. Journaling values the student perspective of the educational experience as a valid source of data, rather than relying on only the professor or researcher’s perspective.
How to: On a regular basis during classroom sessions, students are asked to reflect on a portion of the subject matter or academic experience. This reflection can be directed by the instructor to achieve specific goals. The instructor can also make the journals either private or public to achieve different academic objectives. Private journals can be used to provide a forum for student self-reflection and/or to increase the amount of writing a student does, thereby developing writing skills. Journals can be shared with the instructor to gauge whether students are grasping the subject matter. They can be shared among the entire class to stimulate classroom discussion. Frequency and length of journal entries are dependent on the goals of the task. One-minute journaling at the end of a class period can assess students’ comprehension, reflection, and unanswered questions. More extensive, and often less frequent, journaling (including, but not limited to, assigned journaling outside of the classroom) can be used to assess students’ synthesis of various perspectives or course concepts.
What it does not measure: Journals are not particularly effective at measuring mastery of subject matter or skills outside of writing and thinking.
Benefits: Journaling can be integrated easily into the daily curriculum. If shared with the instructor, journals provide immediate feedback that can be used to improve the quality of teaching. Journals are particularly effective when students are expected to integrate theory into practice (such as in a course with a laboratory, service learning, or practicum component) or to integrate subject matter across disciplines.
General challenges: Depending on the structure of journal assignments, little opportunity may exist for the instructor to assess the quality of the student’s reasoning and expression. In secondary analysis of journals, the quantity of data can quickly become unmanageable. Protecting the confidentiality of student work or making students aware that their work will be read by others without identification is important for research integrity.
Integration with other research/assessment: Journaling compliments the normal evaluative process in the classroom.
Course Assignments
Definition: Course assignments include any work product expected of students for evaluation of their learning and skill development during a course or series of courses.
How to: In addition to their utility for evaluation of student outcomes, regular course assignments or essay exams can be useful for institutional or departmental assessment. Looking at a class’s overall performance on an assignment can indicate the general skill level of the group.
What it does not measure: Because of the diversity of options and flexibility that faculty possess with designing evaluative measures, it is difficult to identify a student outcome or skill that could not be assessed at least in part through course assignments. While areas such as moral development and character are less amenable to assessment, even something as simple as the institutional tracking of plagiarism on course assignments could be accomplished through the use of student course assignments.
Benefits: Course assignments represent a rich, readily available source of data. The participation problems of survey research are virtually removed because of grading. Assignments that both contain educational benefits and provide secondary institutional data do not require additional faculty and student time, in contrast to external assessments.
General challenges: Faculty may initially resist adjusting course assignments for the assessment of institutionally defined outcomes. Respecting the privacy of student work may require informed consent or other alterations in the classroom. As with all secondary document analysis, the quantity of material in course assignments from multiple students may become unwieldy, making analysis time intensive. Ease of data collection is countered with the time commitment required for analysis.
Integration with other research/assessment: Classroom assignments can easily be combined with other forms of data for cross validation. Assignments can be used in concert with other research tools to meet institutional needs.
Capstone Projects/Theses
Definition: At the conclusion of a predetermined educational time period (e.g., semester, academic year, four years) a final paper is written or project completed that employs the skills and knowledge acquired throughout the educational process and represents a synthesis of the student’s learning. The topic is usually of the student’s choice, and the paper or project is executed under the supervision of a faculty member who can assist the student in honing writing and argumentation skills.
How to: As part of a senior course either within the student’s major or across disciplines, the capstone paper or project can be integrated as a component of graduation requirements. Ideally, the paper or project is evaluated for product as well as content. Evaluation may be improved if done in part by faculty outside of the student’s academic major. Secondary analysis of student work against a benchmark standard can provide rich evidence of developed academic ability. Earlier courses in a student’s educational career can help him or her develop the skills necessary to write an extensive, rigorous research paper which is either original or involves a novel interpretation of a subject matter.
What it does not measure: While the assignment is broad in its measurement scope, it may not be the most efficient assessment mechanism for outcomes such as computational or grammatical skills, or for subject matter recall or mastery of writing mechanics.
Benefits: The capstone project or thesis requires the student to produce substantial written research that may contribute to the body of knowledge in a given area. In this, the student develops academic competencies while preparing for graduate study or a professional role. When secondary analyses of text are done for demonstration of higher order skills, student capstone papers or projects can provide documentation of outcomes like argumentation and critical thinking, to name only two.
General challenges: Extensive long-term preparation is necessary because the capstone project or thesis represents a significant undertaking for both the student and the supervising faculty member. In addition, faculty mentoring is required for assistance in and evaluation of student work. At smaller schools or departments, the project or thesis could be required for all students. However, on a larger scale, the time and effort necessary may make universal participation logistically impossible. If only some students participate in capstone projects, assessing outcomes on an institutional level will be complicated by inherent sampling bias.
Integration with other research/assessment: Because this is a culmination of previous academic work, earlier course work and evaluation should prepare a student for the skills and knowledge necessary to write a capstone or thesis. For institutional analyses purposes, pre and post comparisons of students’ first-year work and their capstone papers can provide a measure of educational development.
Presentations
Definition: Presentations, as defined for the purposes of assessment, are any type of formal or informal opportunity for individuals or groups of students to publicly share their work or assume a designated role in teaching or leading the class in a particular topical area or theme. Presentations can be readily incorporated as a component of a larger in-class evaluative process. Presentations can replace or supplement written assignments in order to delve deeper into a particular subject matter, and provide a showcase for students with diverse learning styles. Presentations fit particularly well within the lecture-style classroom environment by placing students in a position to teach their peers.
How to: As one element of the course evaluation process, students can be required to orally present their findings on a particular area within the overall class focus or course subject. Alternatively, students can be required to teach a class session or debate another student or student team during class. Presentations can be videotaped, offering students the opportunity for self-reflection on their own communication competencies.
What it does not measure: While presentations focus more on oral than written communication skills, a certain amount of writing is necessary to develop the presentation. Instructors can build writing into the assignment by requiring the development of a formal outline or note cards, or requiring a reflective paper addressing the process of developing and giving the presentation.
Benefits: Oral presentation and the ability to concisely and confidently express oneself to an audience are critical components of the educational process. In addition, the practice requires students to reason effectively and spontaneously when asked to respond to challenges or questions raised during the presentation.
General challenges: The biggest challenge associated with presentations is the fear many people have about public speaking. Second, some instructors question the value of using classroom instructional time for delivery of course material by "non-experts" when content mastery is important for professional fields.
Integration with other research/assessment: Presentations are readily integrated into the overall evaluation process. If the presentation is connected to a larger research project or paper, the additional workload need not be significant. It also provides a different medium through which an instructor can discern a student’s grasp of the material.
General/Exit Examinations
Definition: A comprehensive examination given near the conclusion of the student’s academic career (usually during the final semester immediately prior to graduation). The exam is generally given to gauge a student’s acquisition and application of a particular type or form of knowledge or skill, as well as his or her ability to integrate knowledge from various disciplines. The exam can be written, oral, or a combination thereof.
How to: The structure of the exam should be broad enough so that a student has the opportunity to express and apply knowledge from both his or her specific field of study and the general or liberal studies curriculum. The exam should also be structured so that students are required to formulate an argument and support that argument with specific examples from their course of study.
What it does not measure: This approach represents a one-time picture of a student’s ability to develop and support an argument under the pressure of a time limit within a relatively artificial context. It does not suffice as a comprehensive measure of academic progress or advancement toward educational gains, but can be used in concert with multiple methods implemented through the student’s career.
Benefits: The process can be as simple or elaborate as an institution desires. In addition to its pedagogical value, the general exam can become a distinguishing ritual of the institution that students look back upon with satisfaction or fondness. General exams with an oral component provide a final opportunity for faculty/student interaction and a personal means for instructors to help and observe students integrate and apply knowledge gained in college.
General challenges: Coordination of the effort to administer and evaluate exams is the biggest challenge. General exams that include an assessment of specific discipline-based knowledge may not be as effective with non-traditional or part-time students whose educational career extends beyond four or five years. Because of the unique nature of different academic majors, exams must be tailored to meet the educational goals of specific programs. Comparison across academic disciplines would be difficult, but not impossible. Analysis of the written component of a general exam could be time consuming.
Integration with other research/assessment: Since the exam represents a comprehensive measure of what has been learned over the college career, it serves as a culmination of the academic objectives of various in-class exercises and methods of approaching in-class assignments.
Entrance and Exit Interviews
Definition: Simply put, interviews are conducted with students when they enter college and when they leave—either through graduation or early departure. These interviews can be designed to measure not only the academic objectives of the institution or department, but also the campus culture that fosters or hinders the achievement of those objectives. Interviews can also be designed to measure a host of other outcomes, such as moral or intellectual development, although interviews incorporating these objectives are often much more extensive, requiring greater interviewer skill, and may compete with other purposes of the interview.
How to: Interviews can be approached from both a quantitative and a qualitative perspective. Interviews can take the form of a survey instrument that can produce quantitative measures of the differences between individuals or groups of students when they begin and/or complete college. Interviews can also be structured qualitatively to supplement quantitative research. Exit interviews or surveys of students who choose to leave the institution before completing their degrees can be useful in determining reasons for departure.
What it does not measure: This approach may not be as useful in assessing the specific topical or content knowledge of a student, either in the broad sense or in a specific subject area. The combined entrance/exit interview strategy is more a measure of growth and change than of achievement.
Benefits: The entrance and exit interviews provide a "pre-test" and a "post-test" that can measure development over time and correlate that development with elements of the educational environment. The institution can closely tailor interview questions to college or university goals.
General challenges: For some, the time and resources necessary to conduct and process the results of the interviews may be prohibitive. Transcribing, coding, and interpreting interview data is labor intensive. Exit interviews or surveys of early departure students are limited by each student’s perceived ability to speak freely with someone associated with the institution he or she chose to leave, as well as by low response rates.
Integration with other research/assessment: The objectives of entrance/exit interviews must be widely stated in order to integrate data with other institutional assessment.