2021 Assessment Practice and Discovery Mini Grant Award Recipients

Program: Apparel Merchandising and Management
Faculty Lead: Claire Whang

Title: Revisiting AMM Department Program Learning Objectives

During our first 3-year assessment cycle, we faced several challenges in collecting, assessing, and interpreting data due to broadly and vaguely written program SLOs.

During the summer, we (1) reviewed the institutional learning objectives (ILOs) and other colleges/departments’ program SLOs, (2) collected past 3-years of AMM curriculum changes data, and (3) assessed all of AMM’s program SLOs using the following five criteria: a. Do our SLOs align with the university's core value? b. Do our SLOs reflect the recent curriculum updates? c. Are our SLOs applicable for hybrid/online courses? d. Do our SLOs reflect option-specific nuances? e. Are our SLOs measurable? After carefully reviewing the collected data, we (4) developed a revised proposal for the AMM department program SLOs. Further, (5) a curriculum matrix and data collection timeline were created based on the new program SLOs.

This project will facilitate future assessment work in the program in many ways. First, the updated SLOs from this project will be directly applied to the upcoming assessment phase. We are at the end of our 3-year assessment cycle, so the next assessment phase can be planned based on the findings of this project. Second, the updated program SLOs can be used to evaluate our curriculum and course-level SLOs to ensure our courses align well with the program goals. Lastly, the updated Program SLOs can be used to develop program-level assessment rubrics. These rubrics can then be used to capture students’ performance systematically, which can solve issues faced during the assessment process.

Program: College of Engineering
Faculty Lead: Ellips Masehian

Title: Improving the engineering ethics scores of CPP engineering students and graduates on the national Fundamentals of Engineering (FE) exam through designing supplementary teaching and assessment materials.

Recent analyses of the performance of CPP undergraduate engineering students and graduates on the national Fundamentals of Engineering (FE) Examination have shown that they have been scoring lower than the national average on the Ethics topics on the exam for the past 4-5 years. Such poor results have also been observed during the prior ABET assessment despite the fact that most CPP undergraduate engineering students pass the “IME 4020: Ethical Concepts in Technology and Applied Science” 3-unit required course.

This project exposes the students to the rationale and testing methods of the FE exam by providing them with supplementary training materials such as PowerPoint slides, recorded video lectures (10 online modules,) and assessment quizzes. In addition, a specific rubric was developed to help assess the students’ level of understanding in engineering ethics issues across five criteria. Practice quizzes were also developed as a supplement so that students can prepare and score better on the ethics section of the FE exam. The designed rubric is also incorporated into the Canvas as an auxiliary and convenient assessment tool on the Canvas.

The next step is to officially add these online components to the existing IME 4020 course. The importable file will be available to all instructors who teach the engineering section of the course for repeated use. More questions will be added over time to the quizzes to further boost the students’ preparation level.

Program: Communication
Faculty Lead: Ivana Cvetkovic

Title: Enhancing the Assessment of Student Learning Outcomes: Development of new rubrics and exit survey.

The Communication Department Assessment Committee developed rubrics to assess the key indicators of our Program Learning Outcome that addresses students’ 1) critical thinking; and 2) ability to communicate and collaborate with individuals from diverse backgrounds. The Critical Thinking Assessment Rubric addresses students’ demonstration of the ability to conceptualize, analyze, and evaluate communication processes and the impact they have on their personal lives and our society. The Diversity Perspective Assessment Rubric addresses students’ abilities to evaluate, critique, and theorize the ways different diversity concepts affect individuals and the society. Additionally, we updated an existing survey to assess and measure the two SLOs.

Our next steps include:

Presentation of the Exit Survey results to the Communication Department faculty Presentation of the two Assessment Rubrics to the faculty.
After receiving the faculty feedback, the finalization of the Assessment Rubrics: Critical Thinking SLO and Diverse Perspectives SLO.
Assessment of the student signature assignments using the new rubrics. Completion of the Assessment reports – Critical Thinking SLO and Diverse Perspectives SLO.
Comparison of the committee’s assessment with the survey data and the discussion of areas for improvement, and possible program adjustments.

Program: Computer Science
Faculty Lead: Hao Ji

Title: Closing the Assessment Loop for the ABET-Accredited Bachelor of Science Program in Computer Science

The ABET-accredited Bachelor of Science program in Computer Science was recently evaluated by the ABET team in Fall 2020. With the feedback received from ABET, we developed an improved assessment plan that presents a new yearly assessment cycle to timely collect and evaluates student outcomes. The purpose of this project is to close the assessment loop for AY2020-21, in which direct and indirect assessment data were collected through a number of core courses and surveys. In this project, we followed the revised assessment process and implemented the proposed assessment activities. We analyzed the collected assessment data and drafted an annual assessment report with a new assessment baseline. The evaluation results of this project will be presented to the department, as an important input for the program's continuous improvement to strengthen student learning.

Program: Education
Faculty Lead: Sara W Juarez

Title: Rubric Development to Ensure Equitable and Effective Assessment of Teacher Candidates

Prior to this mini-grant, a common rubric for Key Assessments aligned with CA State Teacher Performance Expectations (TPEs) was not developed for consistent use across courses. As part of this work, ES program faculty discussed Key Assessments in each course, revised the assessments required to meet the new state TPEs, and created a common rubric that will be used to evaluate Key Assessments across each course. The Key Assessment Rubric reflects the vision and values of the ES program across courses, especially as related to application, evidence-based support, structural organization, and style & mechanics. Descriptors in the rubric reflect principles of asset-based teaching and learning, to support students’ development of growth mindsets.

We believe the success of this project will impact students in multiple ways: (a) setting a clearer purpose for course learning outcomes through stronger alignment to the program learning outcomes and new TPEs, (b) providing students with more explicit and well-defined mastery criteria for Key Assessments across courses, and (c) ensuring students can identify each element of the mastery criteria through common language for each level of achievement in the Key Assessment rubric. Ultimately, this will ensure candidates master all TPEs by the end of the program through a developmental approach.

The new rubric will be used beginning in Fall 2021 to evaluate course learning outcomes and student competencies on the key assessments for each course. We will collaboratively analyze key assessment results at the end of each semester, seeking feedback from tenured, tenure-track, and adjunct faculty who teach in our program. Recommendations to close the loop by adjusting or improving program content based on these results will be made by program faculty.

Program: Engineering Management
Faculty Lead: Shokoufeh Mirzaei

Title: Closing the loop on assessment findings of Master’s of Engineering Management

In this project, our goal was to close the loop of EMT external assessment that was conducted during the academic year 2020-2021 by assessing the program learning outcome through surveys from alumni, industry adviser council, and graduating students. As a result of this mini-grant, I was able to design three surveys to be sent out to: 1) Industry advisory council (IAC) regarding the current debates, trends, technologies, and latest developments in the field; 2) Graduating students regarding their general sentiment toward the program; and 3) Alumni regarding the connection between the program curriculum and their current job’s required skills Surveys are currently sent out to the department EMT committee for review and feedback.

Once the feedback is received and the surveys are finalized, they will be distributed among the stakeholders for data collection. After the data collection, we will analyze the results and define action items for improvement. Alumni and IAC surveys are scheduled to be sent out every two years. The graduating student's survey will be required to be taken by all graduating students. I wasn’t able to inquire the EMT department committee about the surveys during the summer. So, I sent out the surveys to them at the beginning of the Fall semester. Although the project budget and timeline have changed with respect to the initial proposal, we will be able to implement the steps as described in the proposal once we have the finalized surveys.

Program: Food Science and Technology
Faculty Lead: Yao Olive Li

Title: Coordinating two sets of program assessment matrices for FST program review with IFT and CPP

Food Science & Technology (B.Sc.) program had a successful 5-year program review in 2019, followed by accreditation approval (for its first time) from the Institute of Food Technologists (IFT) in 2020. Two sets of assessment materials are in place - one is derived from semester conversion and prepared for the institutional assessment prior to receiving the accreditation approval, and the other is prepared for IFT accreditation application using IFT's most recently updated standards. Although the two sets did not deviate much in program goals and SLOs, there were certain differences in data collection instruments, leading to confusion for individual instructors to align their course-level assessments accordingly.

Therefore, this mini grant was aimed to allow FST faculty to work over the summer 2021 to align the two sets of assessment instruments and to formulate a coordinated assessment plan to fulfill both CPP institutional and IFT accreditation requirements. The team had monthly meetings and attended the IFT 2021 virtual conference, especially the IFT Higher Education Board meeting to learn about new changes and requirements as set by the IFT. The outcomes from this summer mini-grant project include the updated FST curriculum; the updated alignment matrix allowing the 11 institutional PLOs to match the 11 IFT’s education outcomes; the updated course-level assessment instruments including revised ECOs for most courses that will be used for various data point collection, in which selected course-level outcomes (CLOs) in ECOs are highlighted to match the new set of PLOs. A revised timeline for data collection within a 5-year cycle to meet both CPP Institutional and IFT program review was also developed.

The continuing work include disseminating these revised assessment documents with all FST faculty including adjunct lecturers and to seek their input, specifically for the course activities and rubrics that they have used or will use to collect the assessment data under the course-alignment matrix. It has been a productive summer for the FST faculty team to complete most of the proposed deliverables – 4 out 5 are completed while the remaining is underway with the pending inputs from adjunct lecturers. This mini-grant is greatly appreciated to support our work over the summer 2021.

Program: Kinesiology
Faculty Lead: Andrea Metzker

Title: Let’s Assess So It Matters

The program set out to implement Social Emotional Learning (SEL) objectives into my Kinesiology class which teaches (liberal studies and kinesiology students) how to teach Elementary Physical Education. Because of obtaining this mini-grant, I was able to get several recommended books to develop and implement two components of SEL: 1) encouraging students to increasingly understand and assume responsibility for their well-being; and 2) designing a physical education program that is equitable to all races, genders, socioeconomic status, sexual orientation and exceptionality.

These practices which are currently being emphasized in California public schools and are a perfect fit in physical education classes where overall health (mental, physical and wellness) is a big component of the curriculum and, unfortunately, physical education is often a place where discriminatory practices (sexism etc.) have taken place in the past. Often these practices have been passed on through generations in the public school and this class gives me a great opportunity to be teach future teachers to teach just the opposite of these harmful practices.

This gave me a platform to interview physical educators about how they currently implement SEL into elementary and middle school physical education. My next steps are to continue with implementing the goals of this project while teaching the class throughout the semester and encouraging my students to incorporate Social Emotional Learning in the physical education curriculum when they become teachers.

Program: Landscape Architecture
Faculty Lead: Weimin Li

Title: Assessing Diversity, Equity, and Inclusion in the MLA Curricula

The program focused on collecting student artifacts for the examination of the nature of student work in different courses in relation to the achievement of learning outcome concerning equity and social justice, diversity, and inclusion. Student artifacts present strong evidence that students in the MLA program have been exposed to societal issues related to diversity, equity, and inclusiveness at local sites, neighborhoods, and regional contexts. Artifacts reveal that students have the theoretical and technical tools and methods to collect data, analyze them, and investigate how different populations, especially those in disadvantaged communities have been affected. Students were able to identify new opportunities and develop strategies and practical solutions for issues raised.

This work will further inform the development and finalization of the learning outcome for diversity, equity, and inclusiveness and add it to the program learning outcome based on findings from current artifacts and the faculty’s vision for new changes. Then once it is established, it will be used to assess all program courses.

Program: Philosophy
Faculty Lead: Cory Aragon

Title: Re-defining Philosophy Department Learning Outcomes

The Philosophy Department has been engaged in a multi-year curricular redesign project. This project is designed to improve student meta-cognition of the value of their philosophical education and better prepare students to be able to articulate this value to future employers. Consequently, a major goal of this project is to clearly identify the central skills-based learning outcomes of our program and to find specific "homes" for each of the skills PLOs in the departmental curricula. Doing so carves out a designated space in our curricula to engage in the "development" stage of skills PLO in a focused manner. Students will have a clearer understanding of the primary skills PLOs and be better able to articulate the value of these skills and name the specific ways they have grown the skill. Second, it also designates a space for assessment of skills PLO development.

The mini-grant funded our proposed faculty retreat to discuss what exactly we were aiming for in re-designing our PLOs; different ways to articulate what these PLOs are, what they require for development, and what makes them valuable; and how we would design space in our program curricula to both develop skills and assess skills PLOs.

We made considerable progress in the retreat, leaving with a broader project of developing a broader framework for understanding and naming the value of our PLOs. We also began the work of attaching up PLOs to specific skills PLO "labs" that we will pilot in the Spring of 2022. In the end, the retreat enabled us to take a pretty significant step forward in our ongoing redesign project, and we will continue the work we started in the retreat throughout the next year and maybe beyond.

Program: Plant Science
Faculty Lead: Eileen Cullen

Title: Aligning Program Assessment with Curriculum Design

This project strengthened linkages between Program Learning Outcomes (PLOs) and Student Learning Outcomes (SLOs) in the BS Plant Science program. Working as a group for the first time to chart our assessment course, a total of six new/updated PLOs were finalized including a new PLO for equity & social justice, and diversity & inclusion. We reviewed SLOs and discussed ideas to update SLOs to correspond to the updated PLOs. The PLO process was successful because faculty had the opportunity to systematically focus on the primary framework for our undergraduate program. Deliverables include a revised set of 6 PLOs with 2 new, and 4 substantially revised, PLOs.

Next steps are to use the PLOs from this project as a framework to guide and align curriculum redesign with program assessment. We will present the new PLOs, and partially revised SLOs, to faculty and lecturer staff at the April 2022 retreat to guide curriculum re-design. Specifically, guidance will be to map course learning objectives (CLOs) to draft SLOs outputs from this project. Plans to complete SLOs will be incorporated into the curriculum redesign retreat and require follow up work.

Program: Political Science
Faculty Lead: Mario Guerrero and Robert Nyenhuis

Title: Designing Assessment Instruments for Lower-Division Political Science Courses

Over the last few years, the Political Science department revised its research methods sequence, spurred by discussions from our Faculty Methods sub-committee. Specifically, we added a methods sequence, encompassing two semesters: Introduction to Research Methods (PLS 2051) and Advanced Research Methods (PLS 3055) to allow students to focus on basic issues of research design prior to being exposed to qualitative and quantitative methods. As such, we set out to examine what effect this sequencing has had on student learning outcomes and our department’s broader course learning outcomes. We collected syllabi from all instructors who taught the methods sequence, reviewed Senior exit survey data, and also considered some of the advice given by two external reviewers in our department’s Program Review this past academic year.

We explored whether the following department learning outcomes were being met:

Students can demonstrate the ability to conduct library research on political science topics using a variety of scholarly and current resources.
Students can demonstrate the ability to analyze quantitative and qualitative data.
Students can effectively present arguments, both about research and political positions, orally and in writing.

To start our project, we collected existing syllabi and course assignments from our instructors, and examined each instructor’s syllabus for the introduction, and implementation of skills associated with each learning outcome (LO) addressed above. Overall, we scored most of our instructors’ efforts as “excellent”, indicating that they introduce and allow students to develop mastery of these skills in a host of assignments and class activities.

From reviewing our Senior Exit survey data, we identified that1) students appreciated the methods sequence and thought the two courses were very valuable for preparing them for their capstone projects; and 2) students expressed difficulty registering for these vital courses in a timely fashion. The PLS department will have an opportunity to hire a new faculty member in academic year 2021/2022, and the job description has a requirement for the candidate to teach quantitative methods courses.

Program: Regenerative Studies
Faculty Lead: James A. Blair

Title: Closing the Loop and Regenerating an Action Plan to Assess the Lyle Center MSRS Program

The main purpose of the summer assessment mini grant for the MS program in Regenerative Studies was to close the loop on an updated and more relevant assessment plan as part of the action item response to the 2020 program review report. This was accomplished, first, through the development of a new action plan that corresponded to a thorough written response to the program review. Second, the program sought to better engage equity, social justice, diversity and inclusion by updating the curriculum matrix, rubric and assessment plan to fully align with the SLOs. This was accomplished through revision of the curriculum matrix, assessment plan and especially the detailed criteria of the rubric to be more consistent with the SLOs posted on the website.

Specifically, SLO#7 was missing from the assessment materials. To address this significant gap, a detailed rubric was created for that particular SLO, and edited and reorganized to provide more inclusive and less potentially biased or ableist language, and to avoid the “deficit” framing of student success throughout the document. Participation in the Assessment Summer Institute was particularly helpful for workshopping the revised rubric in anticipation of data collection. The next step for this item will be to reevaluate the language of SLOs. This will require further discussion and consensus among staff.

Third, the program began to re-consider the pacing of SLOs in the assessment plan, but the number of SLOs assessed in a single year remains excessive. Now that the rubric and other materials better address all SLOs, the next step will be to ensure that the plan follows a more coherent logic in relation to potential assessment data collection, analysis and reporting the near future. Ideally, this will include goals or benchmarks for measuring student success. Finally, the program began to consider potential curriculum tracks or research areas that would align with the SLOs for students to connect with potential advisors, faculty members and one another. The next step for this activity will be to fully develop and implement these proposed curriculum tracks or research areas across the program.

Program: Special Education
Faculty Lead: Heather Taylor

Title: Providing Access and Equity in the MA.Ed Program

While enrollment has grown with our combined credential and MA program, students are not adequately prepared for their culminating activity in terms of research and writing skills. Because students feel underprepared for the culminating activity, some choose not to complete their MA degree.

Prior to this grant, the program did not have a mechanism in place to assess the preparation of our MA students for the culminating activity. Using current comprehensive exam rubrics for the past 5 years, strengths and areas of need were determined for students in the areas of academic writing and research. Based on this analysis, three additional rubrics were created: (1) to assess future projects for the strengths and areas of need in the areas of academic writing and research; (2) to assess future theses for the strengths and areas of need of academic writing and research; and (3) to assess student academic writing and research during coursework in core program MA courses. In addition, these areas of need were identified as focus areas for course instructors for AY 2021-2022.

Next steps include implementing the rubric in all program MA courses, as well as piloting the rubric with projects and theses that are submitted in AY 2021-2022. This data will then be reviewed at the end of Fall and Spring semesters to determine common areas of need among candidates’ research and writing skills. Recommendations will be created for adjusting or improving program content based on these results for AY 2022-2023. The hope is that this assessment loop will become common practice in the program.