Portfolios
Definition: A compilation of the work produced by a student that will allow for his or her reflective assessment of the broader educational gains from the assignments. Portfolios are most frequently associated with artists and represent a sample of their work.
How To: Throughout a pre-defined education segment (e.g., semester, academic year, four years) a student compiles the assignments in a central location. The process of gathering the work and reflecting on prior assignments highlights educational gains and progression. Compilation and systematic analyses of the data enable assessment of process and educational outcomes over time or between individual students or groups.
Application to Liberal Arts: One of the goals of a liberal arts education is to advance higher level thinking? Portfolios allow the student to become more critical of his or her approach and execution of similar assignments, thereby enhancing critical thinking skills. In addition to the self-reflective benefits for the student, the institution or faculty member can use portfolios to evaluate students’ progress. Institutions can evaluate students’ written work for degree of skills in critical thinking, argumentation, synthesis of the writings and beliefs of others, and original thought. The pieces can be evaluated, sometimes by someone outside the discipline, for improvements in higher order thinking separate from students’ command of the subject matter. Faculty members can use a group of students’ portfolios on their own or with colleagues to evaluate process-level outcomes of teaching.
What it does not measure: This method does not measure the final standing of a student against a content benchmark. Rather, it measures the progression of educational gains.
Benefits: The process encourages and develops self-reflection skills in students and provides valid information to assist them in assessing their intellectual competencies. Institutions benefit as a result of readily available data to help inform the adjustment of classroom assignments.
General challenges: Depending on the length of the predetermined education segments, portfolio collection and analysis can be a long process with relatively small gains over an extended period of time. In addition, the student’s ability to appreciate educational gains may require supervision. As an institutional assessment of general student outcomes over the course of enrollment, the process requires matching student work over several years of study in multiple departments.
Integration with other research/assessment: On the most basic level, this is integrated with the normal evaluative process in the classroom setting.
Journals
Definition: As an academic assessment tool, a journal is a written record of a student’s self-reflective process that can be integrated into the regular classroom curriculum. Journaling values the student perspective of the educational experience as a valid source of data, rather than relying on only the professor or researcher’s perspective.
How To: On a regular basis during classroom sessions, students are asked to reflect on a portion of the subject matter or academic experience. This reflection can be directed by the instructor to achieve specific goals. The instructor can also make the journals either private or public to achieve different academic objectives. Private journals can be used to provide a forum for student self-reflection and/or to increase the amount of writing a student does, thereby developing writing skills. Journals can be shared with the instructor to gauge whether students are grasping the subject matter. Journals can be shared among the entire class to stimulate classroom discussion. Frequency and length of journal entries are dependent on the goals of the task. One-minute journaling at the end of a class period can assess students’ comprehension, reflection and unanswered questions. More extensive and often less frequent journaling can be used to assess students’ synthesis of various perspectives or course concepts.
Application to Liberal Arts: Journaling provides the students with extensive writing assignments designed to promote self-reflection and hone their written communication skills. Repeated work on writing skills increases students’ ability to think and express themselves critically. Analysis of journals can be conducted for institutional assessment of student outcomes such as writing skills, thinking skills, and integration of theory and practice or integration across disciplines.
What it does not measure: Journals are not particularly effective at measuring mastery of subject matter or skills outside of writing and thinking.
Benefits: Journaling can be easily integrated into the daily curriculum. If shared with the instructor, journals provide immediate feedback that can be used to improve the quality of teaching. Journals are particularly effective when students are expected to integrate theory into practice such as in a course with a laboratory, service learning, or practicum component, or to integrate subject matter across disciplines.
General challenges: Depending on the structure of journal assignments, little opportunity may exist for the instructor to assess the quality of critical thinking and expression. In secondary analysis of journals, the quantity of data can quickly become unmanageable. Protecting the confidentiality of students' work or making students aware that their work will be read by others without identification is important for research integrity.
Integration with other research/assessment: Compliments the normal evaluative process in the classroom.
Course Assignments
Definition: Course assignments include any work product expected of students for evaluation of their learning and skill development during a course or series of courses.
How To: In addition to their utility for evaluation of student outcomes, regular course assignments or essay exams can be useful for institutional or departmental assessment. Looking at a class’ overall performance on an assignment can indicate the general skill level of the group.
Application to Liberal Arts: While faculty members need no introduction to course assignments or essay exams, some may appreciate assistance in adjusting course assignments to better measure higher order skills in addition to or in place of simple content mastery. Assignments can be useful assessments of liberal arts outcomes when they require synthesis of the thoughts of others, evaluation or discernment about those thoughts, argumentation and proof, or original ideas, rather than simply recall of facts or summary of literature.
What it does not measure: Because of the diversity of options and flexibility faculty possess with designing evaluative measures, it is difficult to identify a student outcome or skill that could not be assessed at least in part through course assignments. While areas such as moral development and character are less amenable to assessment, even something as simple as the institutional tracking of plagiarism on course assignments could be accomplished through the use of student course assignments.
Benefits: Course assignments represent a rich, readily available source of data. The participation problems of survey research are virtually removed because of grading. Assignments that both contain educational benefits and provide secondary institutional data do not require additional faculty and student time, in contrast to external assessments.
General challenges: Faculty may initially resist adjusting course assignments for the assessment of institutionally-defined outcomes. Respecting the privacy of student work may require informed consent or other alterations in the classroom. As with all secondary document analysis, the quantity of material in course assignments from multiple students may become unwieldy, making analysis time intensive. Ease of data collection is countered with the time commitment required for analysis.
Integration with other research/assessment: Classroom assignments can easily be combined with other forms of data for cross validation. Assignments can be used in concert with other research tools to meet institutional needs.
Classroom Instructional Techniques
Definition: Within the classroom setting, different approaches can be used to develop students' critical thinking skills and to gauge whether the objectives of the course are being accomplished. These can take the form of daily quizzes, short reading and writing assignments, reflection, and brief student-to-student discussions.
How To: If structured correctly, classroom techniques can encourage students to think at a higher level about class material. If an instructor wants students to reflect more deeply about a particular reading assignment, a question can be developed to ask students to do just that. For example, students can be asked to critique the author’s argument or approach to the subject matter, rather than simply repeating or summarizing what the author said. Classroom techniques are usually used in combination, often to accompany or accentuate lectures. The right techniques can motivate passive learners to assume a more active role in the classroom.
Application to Liberal Arts: Classroom instructional techniques can be useful tools to get students to move beyond rote memorization and arrive at a point of more critical, higher order thinking.
What it does not measure: Classroom techniques are most often used to develop student skills and for temporary assessment of comprehension rather than institutional assessments of broader skills.
Benefits: The primary benefit is a repeated reinforcement of what is expected from the instructor in terms of the quality of work and student involvement in the learning process. This communicates to students throughout the course the type and quality of work they will be expected to complete.
General challenges: Certain classroom techniques and instructional methods are more time intensive in planning and execution. Some instructors may also resist efforts to modify classroom practice. Academic freedom is another concern, as instructors may be encouraged but not necessarily required to adopt certain classroom techniques.
Integration with other research/assessment: Consistently rigorous smaller assignments can naturally lead to the student thinking toward a larger assignment, such as a paper or final exam.
Capstone Projects/Theses
Definition: At the conclusion of a predetermined educational time period (e.g., semester, academic year, four years) a final paper is written or project completed that employs the skills and knowledge acquired throughout the educational process and represents a synthesis of the student’s learning. The topic is usually of the student’s choice, and the paper/project is executed under the supervision of a faculty member who can assist the student in honing writing and argumentation skills.
How To: As part of a senior course either within the student’s major or across disciplines, the capstone paper/project can be integrated as a component of graduation requirements. Ideally the paper or project is evaluated for product as well as content. Evaluation may be improved if done in part by faculty outside of the student’s academic major. Secondary analysis of student work against a benchmark standard can provide rich evidence of developed academic ability. Earlier courses in a student’s educational career can help him or her develop the skills necessary to write an extensive, rigorous research paper which is either original or involves a novel interpretation of a subject matter.
Application to Liberal Arts: When poorly designed or evaluated, the capstone project/thesis is little more than a summary of other people’s thinking without reflection or synthesis. When well designed, the effort requires original thought, application of critical thinking, and higher order skills such as analysis, investigation, argumentation, and evaluation, as well as the ability to synthesize these elements into a cohesive, persuasive final product.
What it does not measure: While the assignment is broad in its measurement scope, it may not be the most efficient assessment mechanism for outcomes such as computational or grammatical skills, or for subject matter recall or mastery of writing mechanics.
Benefits: The capstone project/thesis requires the student to produce substantial written research which may contribute to the body of knowledge in a given area. In this, the student develops academic competencies while preparing for graduate study or a professional role. When secondary analyses of text are done for demonstration of higher order skills, student capstone papers/projects can provide documentation of outcomes like argumentation and critical thinking, to name only two.
General challenges: Extensive long-term preparation is necessary because the capstone project/thesis represents a significant undertaking for both the student and the supervising faculty member. In addition, faculty mentoring is required for assistance in and evaluation of student work. At smaller schools or departments, the project/thesis could be required for all students. However, on a larger scale, the time and effort necessary may make universal participation logistically impossible. If only some students participate in capstone projects, assessing outcomes on an institutional level will be complicated by inherent sampling bias.
Integration with other research/assessment: Because this is a culmination of previous academic work, earlier course work and evaluation should prepare a student for the skills and knowledge necessary to write a capstone/thesis. For institutional analyses purposes, pre and post comparisons of students’ first year work and their capstone papers can provide a measure of educational development.
Writing as Process and Product
Definition: In classes where written expression is the primary form of course evaluation, written assignments can be structured so that the focus is not exclusively on the final product, but also on the process by which that product is developed.
How To: In this approach, the writing process is the ultimate objective. The student is challenged by the development of a topic, the revision of drafts, the argumentation provided in the paper to support his or her positions, and finally, the finished product.
Application to Liberal Arts: This process allows for the development and refinement of critical thinking and argumentation skills.
What it does not measure: This is one of the more comprehensive measures of critical thinking and the ability to express thoughts in a coherent manner. However, in contrast to some other student products, the writing process does not measure the student’s ability to produce quick responses to subject matter challenges.
Benefits: A focus on the process of writing has benefits to both faculty and students. For the faculty member, the process-orientated assignment allows for a periodic review of each student’s progression in the course, and creates a structured opportunity to assist students who are not meeting expectations. For students, the nature of the assignment forces them to focus more on how and what they are doing rather than concentrating only on the final product. The process also encourages students in on-going evaluation of their own work and helps to develop their sense of writing competency.
General challenges: Because of the process-oriented nature of the procedure, this has the potential to be time consuming to the instructor.
Integration with other research/assessment: Because this procedure is based on the already established grading procedure within the class, it is naturally integrated.
Presentations
Definition: Presentations, as defined for the purposes of assessment, are any type of formal or informal opportunity for individuals or groups of students to publicly share their work or assume a designated role in teaching or leading the class in a particular topical area or theme. Presentations can be readily incorporated as a component of a larger in-class evaluative process. Presentations can replace or supplement written assignments in order to delve deeper into a particular subject matter, and provide a showcase or opportunity for students with diverse learning styles. Presentations fit particularly well within the lecture-style classroom environment by placing students in a position to teach their peers.
How To: As one element of the course evaluation process, students can be required to present orally their findings on a particular area within the overall class focus or course subject. Alternatively, students can be required to teach a class session. Presentations can be videotaped, offering students the opportunity for self-reflection on their own communication competencies.
Application to Liberal Arts: Presentations can develop a student’s ability to express ideas in a structured and supportive forum. In addition, presentations require a student to synthesize material, discover the most pertinent issues to communicate to the class, and express those points coherently. Presentations provide a measure of verbal communication competencies that is not available through written class assignments. As with secondary analysis of written documents, videotaped presentations can be used as institutional research data measuring the level of verbal competency of advanced students. Aggregate data can provide institutional measures of student outcomes such as verbal competency, rhetorical skills, argumentation, and synthesis. When combined with early samples of the same students’ presentation skills through videotaping in the first year, students are able to reflect on their own development, while institutions are able to assess the institutional value-added component by comparing later efforts to students’ initial skills.
What it does not measure: While presentations focus more on oral than written communication skills, a certain amount of writing is necessary to develop the presentation. Instructors can build writing into the assignment by requiring the development of a formal outline or note cards, or requiring a reflective paper addressing the process of developing and giving the presentation.
Benefits: Oral presentation and the ability to concisely and confidently express oneself to an audience are critical components of the educational process. In addition, the practice requires students to think critically and spontaneously when asked to respond to challenges or questions raised during the presentation.
General challenges: The biggest challenge associated with presentations is the fear many people have about public speaking. Second, some instructors question the value of using classroom instructional time for delivery of course material by "non-experts" when content mastery is important for professional fields.
Integration with other research/assessment: Presentations are readily integrated into the overall evaluation process. If the presentation is connected to a larger research project or paper, the additional workload need not be significant. It also provides a different medium through which an instructor can discern a student’s grasp of the material.
General/Exit Examinations
Definition: A comprehensive examination given near the conclusion of the student's academic career (usually during the final semester immediately prior to graduation). The exam is generally given to gauge a student’s acquisition and application of a particular type or form of knowledge or skill, as well as his or her ability to integrate knowledge from various disciplines. The exam can be written, oral, or a combination thereof.
How To: The structure of the exam should be broad enough so that a student has the opportunity to express and apply knowledge from both his or her specific field of study and the general or liberal studies curriculum. The exam should also be structured so that students are required to formulate an argument and support that argument with specific examples from their course of study.
Application to Liberal Arts: The general format allows a student to exhibit his or her ability to develop and support an argument. If the curriculum is constructed in a way that progressively challenges students throughout college, the general exam becomes a final assessment of one’s ability to gather, evaluate, and use evidence; weigh differing perspectives; consider social, historical, and cultural contexts; and use persuasion in an attempt to convince others of one’s point of view. Student exit exams can be qualitatively analyzed to measure content mastery, integration of learning from multiple disciplines, and the ability to substantiate an assertion. When the exam is designed to measure these competencies, the secondary analysis of multiple student exams can provide evidence of aggregate graduate competency.
What it does not measure: This approach represents a one-time picture of a student’s ability to develop and support an argument under the pressure of a time limit within a relatively artificial context. It does not suffice as a comprehensive measure of academic progress or advancement toward educational gains, but can be used in concert with multiple methods implemented through the student’s career.
Benefits: The process can be as simple or elaborate as an institution desires. In addition to its pedagogical value, the general exam can become a distinguishing ritual of the institution that students look back upon with satisfaction or fondness. General exams with an oral component provide a final opportunity for faculty/student interaction and a personal means for instructors to help and observe students integrate and apply knowledge gained in college.
General challenges: Coordination of the effort to administer and evaluate exams is the biggest challenge. General exams that include an assessment of specific discipline-based knowledge may not be as effective with non-traditional or part-time students whose educational career extends beyond four or five years. Because of the unique nature of different academic majors, exams must be tailored to meet the educational goals of specific programs. Comparison across academic disciplines would be difficult, but not impossible. Analysis of the written component of a general exam could be time consuming.
Integration with other research/assessment: Since the exam represents a comprehensive measure of what has been learned over the college career, it serves as a culmination of the academic objectives of various in-class exercises and methods of approaching in-class assignments.
Entrance and Exit Interviews
Definition: Simply put, interviews are conducted with students when they enter college and when they leave—either through graduation or early departure. These interviews can be designed to measure not only the academic objectives of the institution or department, but also the campus culture that fosters or hinders the achievement of those objectives. Interviews can also be designed to measure a host of other outcomes, such as moral or intellectual development, although interviews incorporating these objectives are often much more extensive, requiring greater interviewer skill, and may compete with other purposes of the interview.
How To: Interviews can be approached from both a quantitative and a qualitative perspective. Interviews can take the form of a survey instrument which can produce quantitative measures of the differences between individuals or groups of students when they begin and/or complete college. Interviews can also be structured qualitatively to supplement quantitative research. Exit interviews or surveys of students who choose to leave the institution before completing their degrees can be useful in determining reasons for departure.
Application to Liberal Arts: Interviews can be structured to address the specific educational goals and objectives of the institution, including desired liberal arts outcomes. Interviews can also be used to measure students’ experiences and perceptions, such as interaction with faculty, developmental growth resulting from an overseas study experience, or other salient growth experiences provided by the institution. When combined with other measures, the interview can be used to assess the relationship between processes and student outcomes. If appropriately structured, it may also be used to assess character development.
What it does not measure: This approach may not be as useful in assessing the specific topical or content knowledge of a student either in the broad sense or in a specific subject area. The interview is more a measure of growth and change than of achievement.
Benefits: The entrance and exit interview provide a "pre-test" and a "post-test" that can measure development over time and correlate that development with elements of the educational environment. The institution can closely tailor interview questions to college or university goals.
General challenges: For some, the time and resources necessary to conduct and process the results of the interviews may be prohibitive. Transcribing, coding and interpreting interview data is labor intensive. Exit interviews or surveys of early departure students are limited by the student’s perceived ability to speak freely with someone associated with the institution they chose to leave, as well as by low response rates.
Integration with other research/assessment: The objectives of entrance/exit interviews must be widely stated in order to integrate data with other institutional assessment.
Next Section:
Institutional Documents and Records