Blog

In the first blog of a two-part series, we explored how University of Colorado Denver (CU Denver) and University of New Mexico (UNM) demonstrated their commitment to partnering with students and communicating information and opportunities with transparency. With these foundations firmly in place at each respective institution, a culture of improvement with students began to take shape.

The foundations of coproduction (see Figure 1) help create effective and sustainable structures for deeper partnerships with student leaders. In this second part of the series, we will illustrate how the subsequent foundations of coproduction helped establish effective, sustainable programs and practices for improving the student experience at both CU Denver and UNM. These foundations of coproduction build upon each other to create a deep, sustainable practice of improvement.  Each of their stories provide key lessons for others who want to engage in coproduction to improve student experience.

1. Commit to partnering with students
2. Communicate information and opportunities with transparency
3. Consult students as experts
4. Cultivate shared language, approach, and agreements
5. Collaborate, centering on equal opportunities and power with faculty and students

Consult with students as experts

Students have rich and valuable perspectives to share about how to best support and engage other students. Universities often seek to build their capacity to truly listen to students and incorporate feedback about the support students need, but may not know how to create the systems that support this culture of partnership. When campuses elevate the voices of students, they can help inform priorities and build a practice of authentic coproduction.

University of Colorado Denver: Inclusive Excellence Interns

Drs. Laurel Hartley and Amanda Beyer-Purvis, inspired by the partnerships happening between students and faculty through the Learning Assistants (LA) program, saw an opportunity to encourage the power of student-led improvement within an entire department. In Fall 2020, they piloted a student-led program in the biology department, hiring five undergraduate students as Inclusive Excellence Interns. These students were tasked to improve the student experience in introductory biology and carried out their project with guidance from faculty advisors only when needed.

Interns initiated all meetings and determined when they wanted feedback, support, or advice. Together, the interns decided that creating a sense of community among incoming first-year students was the most meaningful way to help students find support, persist, and succeed in and beyond their introductory biology course. So, they developed a student-centered “Welcome to Biology” webpage that included resources and community forums for all intro biology students. The interns acted as moderators, answering questions, and providing community-building opportunities such as games, peer interactions, and community nights that developed a sense of community for students who were not experiencing the traditional peer connections on campus.

The interns found this to be a priority project because incoming first year students would be completely remote. Having a centralized Canvas (CU Denver’s student learning management platform) shell that served as an academic, social, and FAQ resource was something they considered key to improving equitable outcomes for first year remote learning.

“I think that’s the biggest thing that we worked on…was just creating this kind of network, this little hub for all students to feel included and be able to have a safe place to go and communicate with each other,” says Mike Swing, a fourth-year student who served as a LA and Inclusive Excellence Intern.

University of New Mexico: Supporting the Whole Student

Peer Learning Facilitators (PLFs) are assigned to classes of at least 50 students, with a focus on larger classes; the goal is to have at least one PLF for every 50 students, if possible. In addition to PLFs, UNM also offers virtual student learning support though Online Learning Assistants (OLAs) who, much like PLFs, work with both faculty and student, only fully remotely. In the last year UNM had 38 PLFs supporting 43 courses which included 2,700 students.

PLFs support equity in courses with large class sizes in a variety of ways. PLFs increase access to the instructional team both during and outside class time, assist instructors in using active learning strategies shown to positively impact success for diverse groups of students, and serve as “near-peer” mentors who normalize personal and academic challenges to encourage help-seeking behaviors. PLFs are key to boosting equitable learning outcomes for students in general education courses.

“I think PLFs assist with a lot more than just material; they encourage students to be confident and help with their self-efficacy,” says Nell Johnson, an alum of UNM and former OLA for Dr. Kimran Buckholz. “We definitely are an emotionally supportive component of the class which is super important for persistence in STEM classes.”

Dr. Buckholz adds: “I can confirm that [students] often say ‘I really appreciated being taught by someone who already took the class.’ It’s a very different experience than being taught by graduate TAs. We have the opportunity to bring in students who actually took the class and understand what the most difficult concepts are and the anxiety and stress that students go through. Graduate TAs and professors cannot understand this as meaningfully as PLFs can.”

Cultivate shared language, approach, and agreements

Cultivating partnership includes building capacity, developing a shared language and approach, and policies and agreements for working with students. Creating training opportunities and compensation for work is critical. Fair compensation for students’ time and expertise helps to ensure more equitable access to leadership opportunities and representation of student voices, since many students are balancing work and school requirements and are unable to volunteer for unpaid activities. This requires that universities allocate budgets and resources to building student leadership opportunities.

University of Denver Colorado

The five students recruited as Inclusive Excellent Interns had previously served as Learning Assistants which helped build upon their previous teaching training and experience. Still, to prepare for this new role, students participated in a free Coursera course on inclusive leadership to encourage their voice and agency to lead as students. The course emphasized that people can lead from any place, helping students find their lever of power from any position.

Given the impact Learning Assistants and Inclusive Excellent Interns have on the learning environment, CU Denver ensures these roles are accessible to many students. For example, the LA role is a paid position. LAs are funded by the university’s budget, not by tuition or fees, at the Denver minimum wage of approximately $16/hour for 9 hours/week. CU Denver hires 80-100 LAs per semester to facilitate learning in lecture and lab courses in biology, chemistry, math, and physics. Inclusive Excellence interns worked on average five hours per week and earned $16/hr. Funding was provided through HHMI (Howard Hughes Medical Institute).

Additionally, all CU Denver LAs take a 2-credit course to prepare them for the leadership positions and offers a place to learn from other LAs. CU Denver pays all or part of the tuition for the training course, because “paying people a good wage is important,” says Dr. Laurel Hartley. “This way, there isn’t a barrier to participation.”

Investing in student leadership is crucial to positioning students as equal partners in driving improvement efforts. Beck Harrott, an alum of CU Denver, former LA, and current medical student saw the LA role directly influence the class structure. “I helped initiate a pilot with a professor where we introduced concept maps to a physiology course. The integration was slow; we wanted to be sure we were adding content that helped clarify concepts student often struggled with. When classes moved to fully remote, we transitioned the concepts maps online. Given the positive feedback we received from students and the need to adapt to online learning, we decided to bring the concepts maps to a general biology course and implemented them immediately. Though it was difficult to restructure traditional lecture courses, the improvement we saw in student grades told us our efforts made an impact.”

University of New Mexico

To prepare students for the leadership and teaching required for their role, all PLFs enroll in a 3-credit EDPY (Educational Psychology) class called “Facilitating College Learning.” This class, led by Dr. Carolyn Hushman, is a co-learning space, a place to learn and share experiences among other students navigating similar challenges and successes. Specifically, students study learning theories in education and gain practical strategies they can immediately use in their courses. Taken together, this seminar is designed to build a community of practice where students and faculty are both learners and teachers.

PLFs are paid for hours spent in class and for time supporting students outside of class. They typically work around 10 hours per week at $12/hr. PLF salaries are paid through university fees.

PLFs report the experience enhancing their communication skills and helping them see the student community more positively. They also report the EDPY course being instrumental to their success. To support students in this role, they also received a once-a-week training with all PLFs about topics including networking, communal group work and professional development, e.g., resume/CV presentations.

Lyndsey Englemann, student and current PLF at UNM, used the knowledge and training she received to influence the classroom learning environment, and felt comfortable taking this initiative with the support of her faculty lead. “Once I received an email from a student who was going to class but didn’t understand the material. They felt comfortable enough to tell me they were too afraid to ask questions in class. I knew this wasn’t a unique experience, and I wanted to find a way to help more students. So, I spoke to my faculty lead and recommended we reserve time at the beginning of class to share challenges and successes. We agreed this would give students time to reflect on their experiences and normalize not understanding the course content on the first try.

Collaborate, centering on equal opportunities and power between faculty and students

Authentic collaboration distributes leadership in a way that creates equal opportunities for students to lead and sustain authentic partnerships. This requires evaluating hierarchies and power dynamics and creating opportunities where imbalances of power are minimized. This way, all collaborators have equal opportunity to engage in the improvement process.

University of Colorado Denver

Faculty understood they needed new models for the faculty-student relationship to build authentic partnership with students.  To truly engage in the coproduction process, Dr. Hartley wanted “LAs that can say what needs to be said and talk as colleagues with faculty. [We have to] set up an environment where we can listen to the LAs and give them agency to do what they need to do.”

One of the ways faculty and students built this collaborative environment is through weekly one-on-one meetings to prepare for in-class instruction. Mike Swing explains: “We’re able to meet with our faculty every week and have important discussions about what went well, and what didn’t. We’re able to debrief every single week and have conversations that are meaningful with our students about what’s going well, that’s really crucial for us and the role we play in student learning.”

University of New Mexico

The majority of instructors (84%) report not being able to use strategies for active learning that are essential for student engagement during class without the support of the PLFs. The PLF is important in lowering the student-to-instructor ratios, increasing the opportunity for more students to receive personalized instruction and feedback in the classroom.

“I think the biggest difference here at UNM is that our PLFs are part of that instructional team, that they are meeting with the instructor of the course. It’s an opportunity for them to say, ‘yeah, I struggled with this unit, can I ask you a few questions before I go out there and try to help others,’” Dr. Hushman shares. ”It’s a partnership between the instructor as well as the PLF student. And I’m not aware of other programs on campus that are bringing the instructor to the table in quite the same way.”

“Learning assistants are like equalizers, because they’re in the middle and help break down power dynamics,” said Nell Johnson. “We learn from students what professors might not see happening in the classroom, and we can then go to professors and say what we heard from students. We work with professors to make sure changes are made based on what students say they need.”

Ready to Coproduce More Equitable Outcomes?

Learn more about how to create opportunities for coproduction in your context with a free resource, Foundations for Authentic Coproduction from Shift—an SEP learning partner. It lays out a framework for coproduction that goes over the five foundations mentioned here, including tips and tools for each foundation.

Authentic coproduction with students, when done with intention, prioritizes their voices and experiences. This means learning from and connecting with a diverse array of students. Shift’s free video resource Building a Culture of Empathy & Capacity for Change goes more into how to do this.

The SEP has a catalog of learning from university and learning partners in its SEP Resource Hub. These resources are designed primarily for campus leadership, administrators, and staff to utilize innovative, evidence-based practices in their daily work, including ways to engage faculty to improve student experience in the classroom.

---

Authors:

Shift:

• Theresa Todd, Improvement Advisor
• Karen Zeribi, Founder and Chief Visionary

University of Colorado Denver:

• Beckston Harrott, Alum, Department of Biology
• Mike Swing, Student, Department of Biology
• Amanda Beyer-Purvis, Former Project Manager, Office of Inclusive Excellence in STEM
• Laurel Hartley, Associate Professor, Integrative Biology

University of New Mexico:

• Avery Bachman-Rhodes, Alum, Women, Gender and Sexuality Studies Program
• Lyndsey Engelmann, Student, Department of Biology
• Nell Johnson, Alum, Department of English Language and Literature
• Kimran Buckholz, Principal Lecturer III, Department of Biology
• Carolyn Hushman, Associate Professor of Educational Psychology
• Sushilla Knottenbelt, Principal Lecturer III, Department of Chemistry

The authors wish to thank the following for their contributions to this blog series: Shay Bluemer-Miroite, Director of Programs at Shift for her insights and suggestions; Gaby España and Sara Veltkamp of Minerva Strategies, for facilitating author interviews and their significant input shaping the content.

Universities are where students learn, grow, build relationships, and tackle both academic and personal challenges. To be successful, students must be supported by their institutions through accessible learning and leadership opportunities, responsive systems, and inclusive policies. Ultimately, students must feel like they belong and can thrive both academically and personally.

A sense of belonging is crucial. It affects retention and academic achievement, especially for students from structurally disadvantaged or historically underrepresented groups. Given the importance of the student experience on long-term success, universities can benefit from implementing a culture of improvement where students are not only listened to but partnered with to implement effective systemic change.

The Student Experience Project (SEP) aims to improve the student experience for and with students through coproduction. Coproduction is a term used in systems improvement with the following key tenets: Everyone within a system is interconnected and has a role to play in improving it; those most affected by a system are often best positioned to redesign or improve it, and; the greatest opportunity for more equitable outcomes is when those affected by the system (e.g., students) combine efforts with those who have professional expertise (e.g., faculty) (see Figure 1).

Developing authentic coproduction at higher education institutions includes creating opportunities for students to build agency and leadership as equal partners with faculty and others charged to serve students.

The SEP, a collaborative effort between six university partners[1] and seven national learning partners[2], worked to integrate student voice, experience, and perspective within the national network and at each participating university. Examples include surveys, focus groups, and student panels. All universities also participated in a student co-led initiative,  We Belong in College. This social media campaign—in partnership with GetSchooled—gathered students’ rich contributions about why they belonged in college and shared their unique insights. These combined efforts to elevate student voice played an essential role in motivating and directing the efforts of the SEP.

Two SEP university partners—University of New Mexico (UNM) and University of Colorado Denver (CU Denver)—explored the power of student leadership even further. Student leaders worked with their teams to help plan, implement, study, and act upon new strategies for improving both the student and instructor experience. At both institutions, faculty and students partnered to improve the student experience in large, introductory STEM courses. Research shows that it is within these courses—often during their first year—that students face incredible challenges with belonging, building a growth mindset, and knowing how to get the support they need for academic success.

These universities demonstrate the foundations of coproduction; all necessary elements for building authentic partnerships with students. Shift, SEP Learning Partner and expert in systems improvement, include five foundations of coproduction applied to higher education (Figure 2):

1. Commit to partnering with students
2. Communicate information and opportunities with transparency
3. Consult students as experts
4. Cultivate shared language, approach, and agreements
5. Collaborate centered on equal opportunities and power with faculty and students

Shift’s five foundations of coproduction build upon each other to create an authentic practice of organizational improvement. In this first of a two-part series, we explore how CU Denver and UNM demonstrated their commitment to partnering with students and communicating information and opportunities with transparency. While both excelled, they engaged in these foundations differently based on the differences in their programs. With these foundations firmly in place at each respective institution, a culture of improvement began to take shape.

In the second part of this series, we illustrate how the subsequent foundations of coproduction helped establish effective, sustainable programs and practices for improving the student experience at both CU Denver and UNM. Each of their stories provide key lessons for others who want to engage in coproduction to improve student experience.

Commit to partnering with students

For authentic coproduction to occur within institutions of higher education, faculty and campus leadership must first commit to partnering with students and elevating the value of their diverse, unique perspectives.

At the core of SEP is a commitment to transforming the college student experience and creating equitable learning environments through innovative, evidence-based practices to increase degree attainment. Both participating universities support efforts that realize this mission through their peer-support initiatives.

University of Colorado Denver: Learning Assistants

Since 2012, undergraduate Learning Assistants (LAs) at CU Denver have supported students as they navigate STEM courses. Among the goals of the LA program is to join faculty as part of the instruction team. LAs engage with students inside and outside the classroom and meet weekly with the instructor to make lesson plans.

This commitment to partnership is inherent in the way faculty value student input and leadership. “Without students, what really would a university be? Not having student agency and input through back-and-forth conversations, it just doesn’t make sense,” says Dr. Laurel Hartley, STEM lead for the SEP team and co-director of the Learning Assistant program.  “I wouldn’t be able to operate in or understand a university where we didn’t talk with students and bring them into what it means to learn and what it means to shape the environment that [they’re] in.”

University of New Mexico: Peer Learning Facilitators

Peer Learning Facilitators (PLFs) are undergraduate students who are invited to join instructional teams in courses where they have previously been successful due to their help-seeking behaviors. Help-seeking behaviors include asking questions during and after class and attending student drop-in hours.

The PLF program was first introduced in 2010 through a grant-funded initiative and was later terminated when the funding ended. Dr. Sushilla Knottenbelt, a faculty lead of the UNM SEP team, advocated for its return and in Fall 2020, the program restarted with its first cohort, including Dr. Knottenbelt, along with Dr. Carolyn Hushman and Dr. Kimran Buckholz—among other faculty and instructors.

For the dedicated faculty who advocated for the return of the PLF program, a commitment to student-faculty partnership was too important to retire. This partnership was important not only for students, but for faculty, too.

“It’s a win-win strategy,” says Dr. Kimran Buckholz. “I hope that I never have to teach without PLFs. They are an amazing team of collaborators.”

Communicate information and opportunities with transparency

To ensure equitable outcomes through coproduction, communication must be multifaceted and inclusive to build authentic relationships with those most affected by a system. This includes sharing accessible information, creating space for questions, detailing priorities, and listening. This means co-creating a vision of successful partnership that engages new champions.

University of Colorado Denver

The Learning Assistant program works to improve not only student achievement, but also student relationships and sense of belonging. “LAs are a big part of the inclusivity mission,” says Dr. Laurel Hartley. “This is important in large first-year classes where faculty can’t get to know everyone—the Learning Assistants help bring insight to the faculty members.”

Beck Harrott, an alum of CU Denver, former LA, and current medical student points out that “the role is not a ‘one shoe fits all’ model.”  LAs helped with classes in a variety of ways—in and outside of the classroom—and are given a lot of flexibility to help students, which he found helpful. While working as an LA, Beck recounts one time where a student in the process of a gender transition mentioned they didn’t feel comfortable working in a group.

Beck, understanding that his role required advocating for his peers and building a culture where all students feel included, felt comfortable sharing this with the course instructor.  “[B]eing able to just like say, ‘Hey, I know you really want these small groups, but I don’t think this person is going to be able to learn or even show up to class if they’re forced to be in a group.’ Listening to me, the professor was able to make accommodations. And none of the other students really knew or it wasn’t a big deal. But being able to slide that in there was super helpful.”

University of New Mexico

PLFs are recruited in many ways—recommendations can come from faculty, instructors, and teaching assistants; from other PLFs; and from appeals to students through student organizations and campus communication channels. The only criterion for prospective PLFs is that they must have demonstrated academic success in the course they want to support (typically “B” or better).

Dr. Buckholz’s PLFs are a daily presence in her classes. She reminds students to attend PLF help sessions and integrates PLFs as mentors and leaders into her active learning activities in lecture. Students are inspired to become PLFs and reach out to Dr. Buckholz to join her team. Each semester, Dr. Buckholz also solicits recommendations from her current team of PLFs. “One of the things I’ve struggled with is our team is female-biased and it’s hard to recruit males and non-binary students, although our team has included all of these students over the years.” Dr. Buckholz maintains an on-going list of potential PLFs to reach out to—those who she knows or have been recommended by current PLFs. “I make efforts to reach out to a variety of students, both by gender and ethnicity.”

These efforts must be transparent, so students are aware of the opportunities available to them and that their expertise is essential and desired by faculty. It establishes students as important partners in transforming the student experience. Avery Bachman-Rhodes, a former PLF and current employee at UNM, recalls the impact of having a faculty member recognize her potential and personally seek her out for a leadership role. “I was ecstatic to be invited by Dr. Buckholz to apply as a PLF. I felt recognized as having the potential to be an effective part of creating a productive and inclusive learning environment for my peers.”

Read how UNM and CU Denver continued their commitment to establishing effective, sustainable improvement with students in the second blog post in this series on Authentic Coproduction in Action.

Ready to Coproduce More Equitable Outcomes?

Learn more about how to create opportunities for coproduction in your context with a free resource, Foundations for Authentic Coproduction from Shift—an SEP learning partner. It lays out a framework for coproduction that goes over the five foundations mentioned here, including tips and tools for each foundation.

Authentic coproduction with students, when done with intention, prioritizes their voices and experiences. This means learning from and connecting with a diverse array of students. Shift’s free video resource Building a Culture of Empathy & Capacity for Change goes more into how to do this.

The SEP has a catalog of learning from university and learning partners in its SEP Resource Hub. These resources are designed primarily for campus leadership, administrators, and staff to utilize innovative, evidence-based practices in their daily work, including ways to engage faculty to improve student experience in the classroom.

---

Authors:

Shift:

• Theresa Todd, Improvement Advisor
• Karen Zeribi, Founder and Chief Visionary

University of Colorado Denver:

• Beckston Harrott, Alum, Department of Biology
• Mike Swing, Student, Department of Biology
• Amanda Beyer-Purvis, Former Project Manager, Office of Inclusive Excellence in STEM
• Laurel Hartley, Associate Professor, Integrative Biology

University of New Mexico:

• Avery Bachman-Rhodes, Alum, Women, Gender and Sexuality Studies Program
• Lyndsey Engelmann, Student, Department of Biology
• Nell Johnson, Alum, Department of English Language and Literature
• Kimran Buckholz, Principal Lecturer III, Department of Biology
• Carolyn Hushman, Associate Professor of Educational Psychology
• Sushilla Knottenbelt, Principal Lecturer III, Department of Chemistry

The authors wish to thank the following for their contributions to this blog series: Shay Bluemer-Miroite, Director of Programs at Shift for her insights and suggestions; Gaby España and Sara Veltkamp of Minerva Strategies, for facilitating author interviews and their significant input shaping the content.

[1] University of Toledo, University of North Carolina at Charlotte, University of Colorado Denver, Portland State University, University of New Mexico, Colorado State University

[2] Shift, Association of Public Land Grant Universities, Coalition of Urban Serving Universities, College Transition Collaborative, PERTS, EducationCounsel, funded by the Raikes Foundation

 

During the Fall 2020 term, instructors in the SEP improved students’ overall experiences of their learning environments significantly. One aspect of the student experience, however, proved to be more challenging for instructors to move than others: student’s feelings of self-efficacy, or the extent to which an individual believes in their ability to do well on a specific task or in a specific domain (Bandura, 1997). 

Understanding how to support students’ feelings of self-efficacy is important because when students have developed self-efficacy in a course, they show greater motivation and better performance (Bandura & Locke, 2003). Past research has demonstrated the critical role of self-efficacy for these key outcomes in STEM domains and for groups who have been traditionally viewed as being less interested, confident, or successful in STEM (e.g., women; Pajares, 2005). While self-efficacy is important to attend to, it can also feel like one of the more difficult aspects of the student experience to support. That is because students’ feelings of self-efficacy are often informed by multiple factors outside of instructors’ control. For example, when students participating in SEP courses are asked to explain the way that they respond to self-efficacy questions in our surveys, they commonly cite performance in past courses, mental health challenges, and responsibilities outside of school as factors that contribute significantly to their feelings of self-efficacy in their current courses. With so many outside factors impacting students’ feelings of self-efficacy, it’s reasonable to wonder – how much can instructors really do?

Past research, and feedback from students in SEP participating courses indicates that while there are outside factors contributing to students’ self-efficacy, self-efficacy can still be fostered and bolstered within the classroom (Corbett & Hill, 2015; Dennehy & Dasgupta, 2017; Hill et al., 2010). In this blog post, we share 5 instructional practices that can positively impact students’ feelings of self-efficacy in their current courses, along with resources that instructors can use to incorporate these approaches into their teaching. 

Instructional practices that contribute to higher levels of self-efficacy in students’ current courses:

1. Communicate care and belief in all students’ ability to succeed: One of the most common themes in student feedback about what positively impacts students’ feelings of self-efficacy is when they feel confident that their instructors believe that all students in the course are capable of learning and growing their abilities during the term. In other words, when students believe that their instructor has a growth mindset about students’ abilities, they are more likely to believe that they personally can be efficacious within that course. Instructors can take a number of steps to build growth mindset cultures in their courses, including using effective growth mindset messaging to establish expectations at the start of the term, emphasizing throughout the term that ability is developed over time by being responsive to to feedback and utilizing new strategies for learning, and by countering internalized fixed mindset beliefs at the end of the term. 
2. Provide clear course content, expectations, and feedback: students tell us that when instructors have clear expectations, consistent policies, and provide actionable and timely feedback, even fast paced and heavy workload courses can feel more manageable. The SEP Syllabus Review and Policy Review guides support instructors in reviewing their course structure and messaging with an eye toward ensuring that course policies and practices support student learning and growth. Utilizing wise feedback framing statements and providing students opportunities to reflect on their learning through the use of assessment wrappers further support self-efficacy by helping to ensure that critical feedback is delivered in a way that engenders trust and promotes continued engagement and learning.
3. Provide and normalize the use of resources for learning: There are many resources available on college campuses to support students through academic challenges, but even when those resources are known, students can be reluctant to use them because of the stigma felt when reaching out to utilize them. Instructors can help normalize the use of resources as a standard part of succeeding in college by promoting resources to support all students’ success in the syllabus or course website, and during in class remarks throughout the term.
4. Highlight self-relevance and connect content to students’ sense of purpose: Research shows that when students see their coursework as relevant to their lives and connected to their sense of purpose, it increases students’ motivation to persist through challenges, which also boosts feelings of self-efficacy. Instructors can encourage students to develop a sense of self-relevance in their coursework by providing opportunities for students to connect their own interests and goals to their coursework.
5. Highlight the accomplishments of others: students are more likely to believe that they, themselves, can do things to improve when they see others (and those like themselves) succeed. One way to do this is by highlighting role models that share students’ backgrounds and goals in course lectures (for examples, see the Acknowledging Diverse Identities in the Ensuring Classroom Identity Safety resource), or sharing how you (or others) have found a place within your discipline by sharing a Belonging Story.

 

Authors: 

• Kathryn Boucher, PhD, Associate Professor, College of Applied Behavioral Sciences, University of Indianapolis, and Principal Investigator, College Transition Collaborative 
• Krysti Ryan, PhD, Director of Research, College Transition Collaborative 

 

References: 

Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W. H. Freeman and Company. 

Bandura, A., & Locke, E. A. (2003). Negative self-efficacy and goal effects revisited. Journal of Applied Psychology, 88, 87–99.

Corbett, C., & Hill, C. (2015). Solving the Equation: The Variables for Women’s Success in Engineering and Computing. American Association of University Women. Washington, DC.

Dennehy, T. C., & Dasgupta, N. (2017). Female peer mentors early in college increase women’s positive academic experiences and retention in engineering. Proceedings of the National Academy of Sciences, 114(23), 5964-5969.

Hill, C., Corbett, C., & St. Rose, A. (2010). Why so Few?: Women in science, technology, engineering, and mathematics. Washington, D.C.

Pajares, F. (2005). Gender Differences in Mathematics Self-Efficacy Beliefs. In A. M. Gallagher & J. C. Kaufman (Eds.), Gender differences in mathematics: An integrative psychological approach (pp. 294–315). New York: Cambridge University Press.

The last ten years have seen an explosion of interest in growth mindset and belonging among institutions of higher education. Since 2017, about 100 institutions have annually made use of free resources created by the College Transition Collaborative (CTC) and the Project for Education Research that Scales (PERTS), including the Growth Mindset for College Students program and the Social Belonging for College Students program. Despite the availability of free, easy-to-implement, scientifically validated programs for students, institutions still frequently approach PERTS and CTC in search of something else: measurement tools.

They are right to do so. Research has pointed to institutions’ critical role in creating culture and policies to help all students feel confident that they belong in college and that their institution believes in their potential. These factors can be important predictors of excellent and equitable outcomes (Murphy & Reeves, 2019; Canning et al., 2019). Programs to help students approach college with a growth mindset and clear beliefs about belonging — while valuable — are unlikely to help institutions change their cultural norms and practices and see more equitable outcomes (Murdock-Perreira, Boucher, Carter, & Murphy, 2019). It is unfair for the burden to be placed solely on students to “persevere” through conditions that actually send signals that they don’t belong. Systemic and sustainable change necessitates a structural approach. 

Institutional leaders understand that measurement is key to meaningful change. Without measurement, you can try all the evidence-based, psychologically attuned practices you want — and some of them might even work. But you’ll never know what was effective, or where. With institutional resources and bandwidth increasingly limited, it is more important than ever to understand the return on investment for any new initiative. And once the most effective practices are identified, data helps build a case to administrations about how and where to spread them. 

Likewise, measurement alone without a theory of change and aligned evidence-based change ideas can leave those leading systems change adrift. Determining how to change systems from scratch can be daunting and burdensome. And building buy-in for systems change without confidence in why the change is important or specificity in what the change will entail is usually a non-starter. A theory of change organizes the efforts of many diverse actors around a single definition of success, so that everyone is pointing in the same direction and with a shared set of measures and change ideas. (For free resources on developing theories of change and designing continuous improvement efforts, visit https://www.shift-results.com/online-learning/.) 

 

The SEP Approach

In the Student Experience Project (SEP), we co-designed a theory of change, which includes a shared and measurable aim, a theory for how to accomplish that aim, and measurements including longer-term academic outcomes as well as more frequent measures to assess student experience across multiple terms. This framework was designed collaboratively by leading social psychologists and continuous improvement experts, alongside teaching faculty and staff from diverse fields such as chemistry, mathematics, public health, biology, and the social sciences. The six universities in the SEP cohort used this framework over two years to continuously improve students’ experiences in their academic courses, while sharing data and learning to learn faster together. These institutions were able to reduce disparities in positive student experiences, which predicted gains in academic outcomes as well. (See Boucher et al., 2021.)

 

Measuring Experiences

While most institutions already track academic outcomes (e.g., DFW’s, persistence), few track student experiences systematically (we hope this will change). We developed a portfolio of shared measures to learn about student experience in depth and over time, as well as to support day-to-day improvement work with student experiences in classes. We adapted survey measures from research literature to provide evidence-based, actionable, and practical measures for busy instructors to use in their classes.

Evidence-based. We chose measures to comprehensively address the most important factors known from research to shape learning and achievement outcomes, including social belonging, institutional growth mindset, and self-efficacy, as well as lesser explored but equally important contributors to outcomes such as identity safety and trust in the fairness of institutional practices. We started with a list of several dozen constructs, all pulled from research literature showing firm links to academic outcomes. We whittled this list down as a community, based on (1) initial data analyses showing which constructs were most important at each institution, (2) which constructs were likely to be sensitive to change in the time-course of a semester based on previous research; and (3) which were most meaningful to institutional stakeholders. We arrived at a list of seventeen items, designed to provide a global assessment of the experiences that matter most to student learning, appropriate for repeat sampling.

Actionable. A measure’s unit of analysis must match its unit of action. Institutions commonly turn to large-scale climate surveys to measure student experiences. But while climate surveys can be useful as snapshots across departments or a whole campus, they rarely get data into the hands of the people who can actually improve students’ experiences in the classroom directly. In the SEP, our data had to be relevant and accessible to the people who would actually use it: in this case, instructional faculty and community of practice (CoP) facilitators. All measures use language that focuses students’ attention to their immediate context (“in this class…”), rather than their global college experience, because class experiences are within instructors’ spheres of influence. Instructors and CoP facilitators furthermore received data early enough in the semester to make changes: the Ascend program allowed CoPs to survey students and receive comprehensive data reports within a week’s time. Reports are delivered directly (and confidentially) to faculty, and include recommendations for practical, evidence-based approaches that instructors can try. This allowed instructors to be nimble and responsive to how students’ experiences were changing week-to-week, rather than waiting until the end of the semester — or longer — when the data are no longer directly actionable.

Practical. Finally, we knew that minimizing the time and resources needed for students to engage with the survey, and for faculty and administrators to understand the data, would be critical to ensure buy-in. Our seventeen-item survey was brief enough to be completed in under 10 minutes, and the Ascend program handled all the data collation, imputation, and summarization so that busy instructors could prioritize learning from data and improving their practice. 

 

Student Experiences are a “Global Health” Measure for Higher Education

The SEP student experience measures give educators a series of comprehensive yet straightforward snapshots of their students’ experiences throughout the semester. While this idea may be novel in higher education, we adapted lessons from a revolution occurring in medicine over the last 30 years. The field of medicine is working to move away from its sole reliance on clinical outcomes, to a more patient-centered approach that also incorporates patients’ own experience of their condition as indicators of overall health and well-being. The Patient Reported Outcome Measures (PROMIS) has helped enable this revolution by allowing diverse groups of clinicians to measure and improve patients’ quality of life using an industry-standard measurement framework (Forrest et al., 2014). Like student experience, facets of patient well being are diverse and include multiple constructs. The PROMIS measures provide both breakdown measurement by construct as well as a “global” measure to indicate overall wellbeing. 

We have adapted this concept for education, with strong conviction that an evidence-based, actionable, and practical framework of measuring students’ experiences in college can help revolutionize higher education in a similar way. The SEP measurement framework includes both individual constructs of student experience as well as a “global” measure which weaves them together into a singular measure, which the SEP Community named the Student Experience Index. We believe that it is important to broaden institutional focus beyond objective (and more distal) outcomes like DFWs and retention, to also include students’ own experiences of their learning which are clearly predictive of these outcomes. 

 

Resources and Next Steps

Going forward, we will continue to validate and develop the SEP measurement framework in order to further simplify it, maximize its value as a leading indicator of educational outcomes, and increase its practicality and scalability. Our goal is to create a measurement tool that is valuable to educators and can guide efforts to improve student experience and equitable outcomes across broad swaths of the higher education landscape. The measurements are currently available as a stand-alone instrument, or for use within the Ascend program. As we continue to refine and validate them, we will make updates available to the public as well.

We encourage those using these (or any) instruments to measure students’ experiences to be thoughtful about how the data are used. Data should be used as a tool for improvement by faculty to improve their practice, not wielded as an evaluative framework to judge or penalize faculty. These evaluative efforts often backfire as they do not result in authentic change, or belief about why the changes are important. Furthermore, evaluative measures such as course evaluations are known to increase bias against women and faculty of color. This risk is mitigated with the SEP measures because they focus attention on things like the pace of the course, the instructor’s flexibility, etc., rather than the instructor’s intellectual ability and competence (which trigger negative stereotypes about women and faculty of color). Nevertheless, the SEP measures are much more powerful when used in communities of practice for improvement. For this reason, the Ascend program keeps individual instructor data confidential — for faculty to share at their sole discretion. The power of using these in an improvement community (such as an improvement network or community of practice) is for faculty to share their data and exchange ideas for how to improve. We recommend a similar procedure for any institution looking to measure and improve student experiences.

 

A Student-Centered Revolution

While thoughtfulness is needed in this (or any) kind of higher education reform, the big picture is that institutions of higher education seem to be “awakening” to the importance of students’ experiences for their learning and achievement. One resource available to this revolution is the domain expertise of higher education practitioners. In fact, most data analysis techniques in wide use today were developed by such instructors. The capacity of higher education practitioners to use data for professional development is therefore high. We hope that more institutions will add frequent and consistent measurement as well as practices to learn directly from students and instructors. These snapshots of how the learning environment is being experienced speaks volumes of what is working, and what can be improved now.

 

Authors:

• Sarah Gripshover, PhD, Director of Research, PERTS
• Kathryn Boucher, PhD, Assistant Professor, College of Applied Behavioral Sciences, University of Indianapolis, and Principal Investigator, College Transition Collaborative
• Krysti Ryan, PhD, Director of Research, College Transition Collaborative 
• Karen Zeribi, Founder and Executive Director of Shift

 

Many students experience difficulty adjusting to college, which can make them question whether they belong and are capable of succeeding in college, as well as wonder how they are viewed by their school. While institutions offer many resources to support students when they experience academic difficulty, students may not be aware that these resources exist, or may be hesitant to utilize them if they believe that seeking help may be perceived negatively by their instructors or peers. Research demonstrates that when institutions communicate that academic difficulty is not uncommon, that seeking support is a key strategy for success rather than a sign of weakness, and that all students are capable of growing their abilities and succeeding in college, students are more likely to take advantage of campus resources to support their success (Canning et. al., 2019, Brady, Kroeper, Henderson, Ozier, et. al. (In Progress)). They are also less likely to feel shame and stress about experiencing academic difficulty (Brady et. al, 2018). 

Institutions have an opportunity to destigmatize academic difficulty and connect students with resources through their early warning or early alert practices. Early warning practices refer to a variety of processes that allow instructors to identify students who, based on course performance in the opening weeks of the term, may not be on track to receive a passing grade, and provide interventions that can help students improve their performance. Campuses participating in the Student Experience Project have been testing numerous changes to their early warning practices to support student belonging and communicate an institutional growth mindset. Institutions in the SEP have been primarily focused on ensuring that communications sent to students who have an early warning referral or alert from their instructors are attuned to students’ concerns, and thus directly and adaptively answer the questions on their mind. Campuses are investigating whether adjusting their messaging to normalize academic difficulty, destigmatize the use of academic support resources, and convey a growth mindset about student abilities will lead to more students engaging with academic support resources. 

Ensuring that the early alerts process is designed to support equitable student experience can take time and require input from a variety of stakeholders on campus. Here are a few steps that institutions can consider taking to set the foundation for effective early alerts:

• Engage instructors in communicating about early alerts in a psychologically-attuned way: Instructors often submit early alerts for their students behind the scenes, but don’t always communicate with students about the purpose of this process. This can leave students feeling uncertain about what receiving an early alert means for their ability to succeed in the course, or embarrassed about the difficulty they are experiencing. To ensure that students perceive early alerts as helpful and supportive, rather than punitive, instructors can normalize these challenges and encourage students to seek support from the instructional team or other campus resources. The SEP team at University of Colorado – Denver worked closely with instructors to craft student-attuned statements about early alerts that they can share during class time and in the syllabus. For more tips on developing psychologically-attuned messages about early alerts, click here. 

 

• Encourage deans and chairs to remind their faculty to participate in the early alerts process: Providing support and resources to students who need it requires high participation among faculty in the early alerts process. While instructors may get messages from central campus offices about participating in early alerts, academic leaders can reinforce this messaging in the context for success for their college or department. Tracking and publishing data on early alerts participation can provide opportunities for deans and chairs to encourage participation and celebrate success. 

 

• Consider the name of your early alerts process: While “early alert” and “early warning” are commonly used terms among faculty, staff, and administrators across higher education, these phrases may cause unnecessary confusion or alarm for students. These terms may also lead campus stakeholders to approach the process solely as a notification system, rather than a more holistic opportunity to connect students with resources. CU Denver recently adopted the name “Early Action” for this process, to signal to students that they can take steps to utilize resources and succeed in their courses. 

 

• Have the right messenger: Many campuses use automated systems to send early alert notification messages to students, allowing notifications to be distributed quickly and efficiently. Students may perceive these messages as cold and impersonal, particularly if they are coming from unfamiliar departments or email addresses. Several SEP campuses surveyed students about their perceptions of early alert notification emails; the SEP team at UNC Charlotte found that students much preferred and were more likely to take action on notifications that came from their instructors, advisors, or other familiar people. Investigate the settings in your early alerts software to understand what your personalization options might be.

 

• Review any early alert template messages for stigmatizing language: Do a quick scan of any centralized early alerts notifications or email templates for language that might make students feel shame or stigma about receiving an early alert. Consider reframing or revising these statements to be more attuned to students’ experiences. For example, if your messages reference that students receiving early alerts should take action so they do not fail a course, changing the framing to emphasize early alerts as an opportunity to utilize resources designed to help them succeed would be more supportive of student experiences. 

 

• Ensure students are being directed to resources that can help them: While you’re checking your messages, make sure that information about relevant campus resources, such as advising, tutoring, etc., are up-to-date and easily accessible to students. If your institution offers both in-person and virtual services, explicitly stating this in the early alert notification may help more students connect with these resources. 

 

• Include student perspectives of experiences with early alerts: Studies of attuned academic probation letters show that including student stories into academic standing communications is beneficial for students (Brady et. al. (In Progress)). The University of North Carolina at Charlotte has several student stories on their Early Alerts website that demonstrate the kind of perspectives that can be highlighted in your early alert communications. For more information on the importance of including student stories, example student stories that you can adapt for your institutions, and suggestions for gathering perspectives from students, check out this free toolkit for improving communications about academic standing from the College Transition Collaborative. 

 

References:

Brady, S. T., Kroeper, K. M., Ozier, E. M., Henderson, A. G., Walton, G. M. & the College Transition Collaborative (2018). Academic probation and the role of notification letters [Research Brief]. Retrieved from http://collegetransitioncollaborative.org/content/sass_toolkit_researchbrief_final.pdf

Canning, E.A., Muenks, K., Green, D.J., & Murphy, M.C. (2019). STEM faculty who believe ability is fixed have larger racial achievement gaps and inspire less student motivation in their classes. Science Advances, 5(2).

 

Resources:

How You Say It Matters: A Toolkit for Improving Communications About Academic Standing by the College Transition Collaborative

 

AUTHOR: 

Samantha Levine, Associate Director, Coalition of Urban Serving Universities and APLU Office of Urban Initiatives 

Several members of the SEP Cohort contributed to this blog post.