efoley

During the Fall 2020 term, instructors in the SEP improved students’ overall experiences of their learning environments significantly. One aspect of the student experience, however, proved to be more challenging for instructors to move than others: student’s feelings of self-efficacy, or the extent to which an individual believes in their ability to do well on a specific task or in a specific domain (Bandura, 1997). 

Understanding how to support students’ feelings of self-efficacy is important because when students have developed self-efficacy in a course, they show greater motivation and better performance (Bandura & Locke, 2003). Past research has demonstrated the critical role of self-efficacy for these key outcomes in STEM domains and for groups who have been traditionally viewed as being less interested, confident, or successful in STEM (e.g., women; Pajares, 2005). While self-efficacy is important to attend to, it can also feel like one of the more difficult aspects of the student experience to support. That is because students’ feelings of self-efficacy are often informed by multiple factors outside of instructors’ control. For example, when students participating in SEP courses are asked to explain the way that they respond to self-efficacy questions in our surveys, they commonly cite performance in past courses, mental health challenges, and responsibilities outside of school as factors that contribute significantly to their feelings of self-efficacy in their current courses. With so many outside factors impacting students’ feelings of self-efficacy, it’s reasonable to wonder – how much can instructors really do?

Past research, and feedback from students in SEP participating courses indicates that while there are outside factors contributing to students’ self-efficacy, self-efficacy can still be fostered and bolstered within the classroom (Corbett & Hill, 2015; Dennehy & Dasgupta, 2017; Hill et al., 2010). In this blog post, we share 5 instructional practices that can positively impact students’ feelings of self-efficacy in their current courses, along with resources that instructors can use to incorporate these approaches into their teaching. 

Instructional practices that contribute to higher levels of self-efficacy in students’ current courses:

1. Communicate care and belief in all students’ ability to succeed: One of the most common themes in student feedback about what positively impacts students’ feelings of self-efficacy is when they feel confident that their instructors believe that all students in the course are capable of learning and growing their abilities during the term. In other words, when students believe that their instructor has a growth mindset about students’ abilities, they are more likely to believe that they personally can be efficacious within that course. Instructors can take a number of steps to build growth mindset cultures in their courses, including using effective growth mindset messaging to establish expectations at the start of the term, emphasizing throughout the term that ability is developed over time by being responsive to to feedback and utilizing new strategies for learning, and by countering internalized fixed mindset beliefs at the end of the term. 
2. Provide clear course content, expectations, and feedback: students tell us that when instructors have clear expectations, consistent policies, and provide actionable and timely feedback, even fast paced and heavy workload courses can feel more manageable. The SEP Syllabus Review and Policy Review guides support instructors in reviewing their course structure and messaging with an eye toward ensuring that course policies and practices support student learning and growth. Utilizing wise feedback framing statements and providing students opportunities to reflect on their learning through the use of assessment wrappers further support self-efficacy by helping to ensure that critical feedback is delivered in a way that engenders trust and promotes continued engagement and learning.
3. Provide and normalize the use of resources for learning: There are many resources available on college campuses to support students through academic challenges, but even when those resources are known, students can be reluctant to use them because of the stigma felt when reaching out to utilize them. Instructors can help normalize the use of resources as a standard part of succeeding in college by promoting resources to support all students’ success in the syllabus or course website, and during in class remarks throughout the term.
4. Highlight self-relevance and connect content to students’ sense of purpose: Research shows that when students see their coursework as relevant to their lives and connected to their sense of purpose, it increases students’ motivation to persist through challenges, which also boosts feelings of self-efficacy. Instructors can encourage students to develop a sense of self-relevance in their coursework by providing opportunities for students to connect their own interests and goals to their coursework.
5. Highlight the accomplishments of others: students are more likely to believe that they, themselves, can do things to improve when they see others (and those like themselves) succeed. One way to do this is by highlighting role models that share students’ backgrounds and goals in course lectures (for examples, see the Acknowledging Diverse Identities in the Ensuring Classroom Identity Safety resource), or sharing how you (or others) have found a place within your discipline by sharing a Belonging Story.

 

Authors: 

• Kathryn Boucher, PhD, Associate Professor, College of Applied Behavioral Sciences, University of Indianapolis, and Principal Investigator, College Transition Collaborative 
• Krysti Ryan, PhD, Director of Research, College Transition Collaborative 

 

References: 

Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W. H. Freeman and Company. 

Bandura, A., & Locke, E. A. (2003). Negative self-efficacy and goal effects revisited. Journal of Applied Psychology, 88, 87–99.

Corbett, C., & Hill, C. (2015). Solving the Equation: The Variables for Women’s Success in Engineering and Computing. American Association of University Women. Washington, DC.

Dennehy, T. C., & Dasgupta, N. (2017). Female peer mentors early in college increase women’s positive academic experiences and retention in engineering. Proceedings of the National Academy of Sciences, 114(23), 5964-5969.

Hill, C., Corbett, C., & St. Rose, A. (2010). Why so Few?: Women in science, technology, engineering, and mathematics. Washington, D.C.

Pajares, F. (2005). Gender Differences in Mathematics Self-Efficacy Beliefs. In A. M. Gallagher & J. C. Kaufman (Eds.), Gender differences in mathematics: An integrative psychological approach (pp. 294–315). New York: Cambridge University Press.

The last ten years have seen an explosion of interest in growth mindset and belonging among institutions of higher education. Since 2017, about 100 institutions have annually made use of free resources created by the College Transition Collaborative (CTC) and the Project for Education Research that Scales (PERTS), including the Growth Mindset for College Students program and the Social Belonging for College Students program. Despite the availability of free, easy-to-implement, scientifically validated programs for students, institutions still frequently approach PERTS and CTC in search of something else: measurement tools.

They are right to do so. Research has pointed to institutions’ critical role in creating culture and policies to help all students feel confident that they belong in college and that their institution believes in their potential. These factors can be important predictors of excellent and equitable outcomes (Murphy & Reeves, 2019; Canning et al., 2019). Programs to help students approach college with a growth mindset and clear beliefs about belonging — while valuable — are unlikely to help institutions change their cultural norms and practices and see more equitable outcomes (Murdock-Perreira, Boucher, Carter, & Murphy, 2019). It is unfair for the burden to be placed solely on students to “persevere” through conditions that actually send signals that they don’t belong. Systemic and sustainable change necessitates a structural approach. 

Institutional leaders understand that measurement is key to meaningful change. Without measurement, you can try all the evidence-based, psychologically attuned practices you want — and some of them might even work. But you’ll never know what was effective, or where. With institutional resources and bandwidth increasingly limited, it is more important than ever to understand the return on investment for any new initiative. And once the most effective practices are identified, data helps build a case to administrations about how and where to spread them. 

Likewise, measurement alone without a theory of change and aligned evidence-based change ideas can leave those leading systems change adrift. Determining how to change systems from scratch can be daunting and burdensome. And building buy-in for systems change without confidence in why the change is important or specificity in what the change will entail is usually a non-starter. A theory of change organizes the efforts of many diverse actors around a single definition of success, so that everyone is pointing in the same direction and with a shared set of measures and change ideas. (For free resources on developing theories of change and designing continuous improvement efforts, visit https://www.shift-results.com/online-learning/.) 

 

The SEP Approach

In the Student Experience Project (SEP), we co-designed a theory of change, which includes a shared and measurable aim, a theory for how to accomplish that aim, and measurements including longer-term academic outcomes as well as more frequent measures to assess student experience across multiple terms. This framework was designed collaboratively by leading social psychologists and continuous improvement experts, alongside teaching faculty and staff from diverse fields such as chemistry, mathematics, public health, biology, and the social sciences. The six universities in the SEP cohort used this framework over two years to continuously improve students’ experiences in their academic courses, while sharing data and learning to learn faster together. These institutions were able to reduce disparities in positive student experiences, which predicted gains in academic outcomes as well. (See Boucher et al., 2021.)

 

Measuring Experiences

While most institutions already track academic outcomes (e.g., DFW’s, persistence), few track student experiences systematically (we hope this will change). We developed a portfolio of shared measures to learn about student experience in depth and over time, as well as to support day-to-day improvement work with student experiences in classes. We adapted survey measures from research literature to provide evidence-based, actionable, and practical measures for busy instructors to use in their classes.

Evidence-based. We chose measures to comprehensively address the most important factors known from research to shape learning and achievement outcomes, including social belonging, institutional growth mindset, and self-efficacy, as well as lesser explored but equally important contributors to outcomes such as identity safety and trust in the fairness of institutional practices. We started with a list of several dozen constructs, all pulled from research literature showing firm links to academic outcomes. We whittled this list down as a community, based on (1) initial data analyses showing which constructs were most important at each institution, (2) which constructs were likely to be sensitive to change in the time-course of a semester based on previous research; and (3) which were most meaningful to institutional stakeholders. We arrived at a list of seventeen items, designed to provide a global assessment of the experiences that matter most to student learning, appropriate for repeat sampling.

Actionable. A measure’s unit of analysis must match its unit of action. Institutions commonly turn to large-scale climate surveys to measure student experiences. But while climate surveys can be useful as snapshots across departments or a whole campus, they rarely get data into the hands of the people who can actually improve students’ experiences in the classroom directly. In the SEP, our data had to be relevant and accessible to the people who would actually use it: in this case, instructional faculty and community of practice (CoP) facilitators. All measures use language that focuses students’ attention to their immediate context (“in this class…”), rather than their global college experience, because class experiences are within instructors’ spheres of influence. Instructors and CoP facilitators furthermore received data early enough in the semester to make changes: the Ascend program allowed CoPs to survey students and receive comprehensive data reports within a week’s time. Reports are delivered directly (and confidentially) to faculty, and include recommendations for practical, evidence-based approaches that instructors can try. This allowed instructors to be nimble and responsive to how students’ experiences were changing week-to-week, rather than waiting until the end of the semester — or longer — when the data are no longer directly actionable.

Practical. Finally, we knew that minimizing the time and resources needed for students to engage with the survey, and for faculty and administrators to understand the data, would be critical to ensure buy-in. Our seventeen-item survey was brief enough to be completed in under 10 minutes, and the Ascend program handled all the data collation, imputation, and summarization so that busy instructors could prioritize learning from data and improving their practice. 

 

Student Experiences are a “Global Health” Measure for Higher Education

The SEP student experience measures give educators a series of comprehensive yet straightforward snapshots of their students’ experiences throughout the semester. While this idea may be novel in higher education, we adapted lessons from a revolution occurring in medicine over the last 30 years. The field of medicine is working to move away from its sole reliance on clinical outcomes, to a more patient-centered approach that also incorporates patients’ own experience of their condition as indicators of overall health and well-being. The Patient Reported Outcome Measures (PROMIS) has helped enable this revolution by allowing diverse groups of clinicians to measure and improve patients’ quality of life using an industry-standard measurement framework (Forrest et al., 2014). Like student experience, facets of patient well being are diverse and include multiple constructs. The PROMIS measures provide both breakdown measurement by construct as well as a “global” measure to indicate overall wellbeing. 

We have adapted this concept for education, with strong conviction that an evidence-based, actionable, and practical framework of measuring students’ experiences in college can help revolutionize higher education in a similar way. The SEP measurement framework includes both individual constructs of student experience as well as a “global” measure which weaves them together into a singular measure, which the SEP Community named the Student Experience Index. We believe that it is important to broaden institutional focus beyond objective (and more distal) outcomes like DFWs and retention, to also include students’ own experiences of their learning which are clearly predictive of these outcomes. 

 

Resources and Next Steps

Going forward, we will continue to validate and develop the SEP measurement framework in order to further simplify it, maximize its value as a leading indicator of educational outcomes, and increase its practicality and scalability. Our goal is to create a measurement tool that is valuable to educators and can guide efforts to improve student experience and equitable outcomes across broad swaths of the higher education landscape. The measurements are currently available as a stand-alone instrument, or for use within the Ascend program. As we continue to refine and validate them, we will make updates available to the public as well.

We encourage those using these (or any) instruments to measure students’ experiences to be thoughtful about how the data are used. Data should be used as a tool for improvement by faculty to improve their practice, not wielded as an evaluative framework to judge or penalize faculty. These evaluative efforts often backfire as they do not result in authentic change, or belief about why the changes are important. Furthermore, evaluative measures such as course evaluations are known to increase bias against women and faculty of color. This risk is mitigated with the SEP measures because they focus attention on things like the pace of the course, the instructor’s flexibility, etc., rather than the instructor’s intellectual ability and competence (which trigger negative stereotypes about women and faculty of color). Nevertheless, the SEP measures are much more powerful when used in communities of practice for improvement. For this reason, the Ascend program keeps individual instructor data confidential — for faculty to share at their sole discretion. The power of using these in an improvement community (such as an improvement network or community of practice) is for faculty to share their data and exchange ideas for how to improve. We recommend a similar procedure for any institution looking to measure and improve student experiences.

 

A Student-Centered Revolution

While thoughtfulness is needed in this (or any) kind of higher education reform, the big picture is that institutions of higher education seem to be “awakening” to the importance of students’ experiences for their learning and achievement. One resource available to this revolution is the domain expertise of higher education practitioners. In fact, most data analysis techniques in wide use today were developed by such instructors. The capacity of higher education practitioners to use data for professional development is therefore high. We hope that more institutions will add frequent and consistent measurement as well as practices to learn directly from students and instructors. These snapshots of how the learning environment is being experienced speaks volumes of what is working, and what can be improved now.

 

Authors:

• Sarah Gripshover, PhD, Director of Research, PERTS
• Kathryn Boucher, PhD, Assistant Professor, College of Applied Behavioral Sciences, University of Indianapolis, and Principal Investigator, College Transition Collaborative
• Krysti Ryan, PhD, Director of Research, College Transition Collaborative 
• Karen Zeribi, Founder and Executive Director of Shift

 

Many students experience difficulty adjusting to college, which can make them question whether they belong and are capable of succeeding in college, as well as wonder how they are viewed by their school. While institutions offer many resources to support students when they experience academic difficulty, students may not be aware that these resources exist, or may be hesitant to utilize them if they believe that seeking help may be perceived negatively by their instructors or peers. Research demonstrates that when institutions communicate that academic difficulty is not uncommon, that seeking support is a key strategy for success rather than a sign of weakness, and that all students are capable of growing their abilities and succeeding in college, students are more likely to take advantage of campus resources to support their success (Canning et. al., 2019, Brady, Kroeper, Henderson, Ozier, et. al. (In Progress)). They are also less likely to feel shame and stress about experiencing academic difficulty (Brady et. al, 2018). 

Institutions have an opportunity to destigmatize academic difficulty and connect students with resources through their early warning or early alert practices. Early warning practices refer to a variety of processes that allow instructors to identify students who, based on course performance in the opening weeks of the term, may not be on track to receive a passing grade, and provide interventions that can help students improve their performance. Campuses participating in the Student Experience Project have been testing numerous changes to their early warning practices to support student belonging and communicate an institutional growth mindset. Institutions in the SEP have been primarily focused on ensuring that communications sent to students who have an early warning referral or alert from their instructors are attuned to students’ concerns, and thus directly and adaptively answer the questions on their mind. Campuses are investigating whether adjusting their messaging to normalize academic difficulty, destigmatize the use of academic support resources, and convey a growth mindset about student abilities will lead to more students engaging with academic support resources. 

Ensuring that the early alerts process is designed to support equitable student experience can take time and require input from a variety of stakeholders on campus. Here are a few steps that institutions can consider taking to set the foundation for effective early alerts:

• Engage instructors in communicating about early alerts in a psychologically-attuned way: Instructors often submit early alerts for their students behind the scenes, but don’t always communicate with students about the purpose of this process. This can leave students feeling uncertain about what receiving an early alert means for their ability to succeed in the course, or embarrassed about the difficulty they are experiencing. To ensure that students perceive early alerts as helpful and supportive, rather than punitive, instructors can normalize these challenges and encourage students to seek support from the instructional team or other campus resources. The SEP team at University of Colorado – Denver worked closely with instructors to craft student-attuned statements about early alerts that they can share during class time and in the syllabus. For more tips on developing psychologically-attuned messages about early alerts, click here. 

 

• Encourage deans and chairs to remind their faculty to participate in the early alerts process: Providing support and resources to students who need it requires high participation among faculty in the early alerts process. While instructors may get messages from central campus offices about participating in early alerts, academic leaders can reinforce this messaging in the context for success for their college or department. Tracking and publishing data on early alerts participation can provide opportunities for deans and chairs to encourage participation and celebrate success. 

 

• Consider the name of your early alerts process: While “early alert” and “early warning” are commonly used terms among faculty, staff, and administrators across higher education, these phrases may cause unnecessary confusion or alarm for students. These terms may also lead campus stakeholders to approach the process solely as a notification system, rather than a more holistic opportunity to connect students with resources. CU Denver recently adopted the name “Early Action” for this process, to signal to students that they can take steps to utilize resources and succeed in their courses. 

 

• Have the right messenger: Many campuses use automated systems to send early alert notification messages to students, allowing notifications to be distributed quickly and efficiently. Students may perceive these messages as cold and impersonal, particularly if they are coming from unfamiliar departments or email addresses. Several SEP campuses surveyed students about their perceptions of early alert notification emails; the SEP team at UNC Charlotte found that students much preferred and were more likely to take action on notifications that came from their instructors, advisors, or other familiar people. Investigate the settings in your early alerts software to understand what your personalization options might be.

 

• Review any early alert template messages for stigmatizing language: Do a quick scan of any centralized early alerts notifications or email templates for language that might make students feel shame or stigma about receiving an early alert. Consider reframing or revising these statements to be more attuned to students’ experiences. For example, if your messages reference that students receiving early alerts should take action so they do not fail a course, changing the framing to emphasize early alerts as an opportunity to utilize resources designed to help them succeed would be more supportive of student experiences. 

 

• Ensure students are being directed to resources that can help them: While you’re checking your messages, make sure that information about relevant campus resources, such as advising, tutoring, etc., are up-to-date and easily accessible to students. If your institution offers both in-person and virtual services, explicitly stating this in the early alert notification may help more students connect with these resources. 

 

• Include student perspectives of experiences with early alerts: Studies of attuned academic probation letters show that including student stories into academic standing communications is beneficial for students (Brady et. al. (In Progress)). The University of North Carolina at Charlotte has several student stories on their Early Alerts website that demonstrate the kind of perspectives that can be highlighted in your early alert communications. For more information on the importance of including student stories, example student stories that you can adapt for your institutions, and suggestions for gathering perspectives from students, check out this free toolkit for improving communications about academic standing from the College Transition Collaborative. 

 

References:

Brady, S. T., Kroeper, K. M., Ozier, E. M., Henderson, A. G., Walton, G. M. & the College Transition Collaborative (2018). Academic probation and the role of notification letters [Research Brief]. Retrieved from http://collegetransitioncollaborative.org/content/sass_toolkit_researchbrief_final.pdf

Canning, E.A., Muenks, K., Green, D.J., & Murphy, M.C. (2019). STEM faculty who believe ability is fixed have larger racial achievement gaps and inspire less student motivation in their classes. Science Advances, 5(2).

 

Resources:

How You Say It Matters: A Toolkit for Improving Communications About Academic Standing by the College Transition Collaborative

 

AUTHOR: 

Samantha Levine, Associate Director, Coalition of Urban Serving Universities and APLU Office of Urban Initiatives 

Several members of the SEP Cohort contributed to this blog post.