High-Impact Practices 3. Honors as a Platform for Undergraduate Research Excellence - An Integrated Framework for Evaluating Undergraduate Research Experience Across Disciplines
Chaonan Liu, Scholar in Residence for Undergraduate Research, Purdue University John Martinson Honors College
Jordan Meyer, Data Manager, Purdue University John Martinson Honors College
Undergraduate research experiences have been reported to have positive effects on student academic learning and professional development. This is especially true for Honors colleges and programs, where research often plays a key role in the curriculum and is seen as a highly rewarding experience for students. At Purdue University’s John Martinson Honors College (JMHC), students are required to complete a scholarly project as part of the curriculum. One of the most common ways to fulfill the requirement is by conducting research under the guidance of a faculty mentor. While there is much literature regarding the benefits and practice of undergraduate research, most is discipline-specific and therefore outside the scope of Honors programs and colleges that aim to provide students with a unique interdisciplinary experience. Given the interdisciplinary nature of the JMHC, where students from 190+ diverse majors engage with research in various disciplines, the effective evaluation of the student research experience presents a unique challenge.
To assess how well the JMHC students’ scholarly projects align with the intended learning objectives, this study employs a combination of qualitative and quantitative methods. We analyzed 150 student scholarly projects that were randomly selected from those submitted between 2019 and 2021, representing diverse majors across nine colleges at Purdue University. By integrating the Research Skill Development framework (Willison and O’Regan, 2006/2015) and the JMHC scholarly project learning objectives, and through content analysis of the projects, a comprehensive framework was developed to assess student research experiences. The framework emphasizes the nine competencies essential for students’ development throughout the entire undergraduate research process, from generating clear initial research questions to effectively communicating research findings. When coding the scholarly projects with the framework, students’ skills were aligned with the four mastery levels in each competency, from beginning (1) to mastery (4). Students’ competency levels were then compared between different colleges using statistical analysis. By far, our preliminary findings indicate that, compared to the performance of those in other colleges, students in the business school and in the College of Engineering have lower scores in several competencies including: identification of research question(s), evaluation, and interpretation of findings and, demonstrated knowledge of disciplinary and/or interdisciplinary skills. These findings guided further in-depth qualitative analysis exploring these potential shortcomings, indicating the need for additional support for students’ development.
Beyond presenting the study and results, we also want to introduce our novel program evaluation process. We invited JMHC students to this evaluation process where students learned and engaged in hand-on education evaluation. Our experience could serve as a valuable example for institutions and programs seeking to assess and enhance their students’ undergraduate research experience.
Willison, J. and O’Regan, K. (2006/2015), “Research skill development framework”, available at: www. adelaide.edu.au/rsd
