Technology Tools

Student Success Innovations to Watch

If you scroll down to the end of the above infographic, which is a great summary of NGLC’s very first grant program, Building Blocks to College Completion, you’ll see five “Innovations to Watch.”

These innovations are changing higher education because of their success in one or more of these dimensions:

• Reach: The number of students, faculty, and institutions using their innovation
• Improved student outcomes: Significant positive effects on course completion, course grades, and persistence
• Extension: Adoption of the innovation continues to grow and the innovation has extended to new projects or new users

NGLC created brief profiles of each of these innovations that detail their designs, results, and the “secrets” of their success.

Innovations to Watch

 Blended Learning Toolkit: University of Central Florida

This toolkit is a free and open-source online resource for creating blended learning courses. The project team worked with over 100 faculty at 20 institutions during the grant period, and the toolkit has since been accessed over 100,000 times by 75,000 visitors from across the globe. The team continues to add open and freely available resources to the toolkit and offers a “BlendKit” MOOC each year. Read more.

 E²Coach: University of Michigan Department of Physics

Using the E²Coach learning analytics tool and tailored messaging to students in introductory physics resulted in a significant positive effect on students’ success compared to a control group. The program has expanded to other disciplines and has triggered other learning analytics efforts at the University of Michigan with significant funding support over the next five years from the university’s IT Council. Read more.

 Hybrid Lab Courses: California State University, Northridge

Demonstrated evidence that the Hybrid Lab model works—with improved student outcomes in traditionally high failure rate, multisection, gateway courses—is spurring continued adoption of the model across both the California State University System and the California Community College system. The CSUN team blew past its targets for scale during the initial grant period, and as of last count has served nearly 10,000 students. The team also found net savings of $5K per semester at CSUN. Read more.

  U Do the Math: Chattanooga State Community College

In this redesigned developmental math program that closes income-based achievement gaps and reduces the college’s costs, students are more likely to complete their course and complete the developmental math sequence more quickly. Students also do better in the college-level math courses they take later. U Do the Math continues to thrive at all three partner institutions and now flourishes in high schools through the Tennessee Higher Education Commission’s SAILS program, with similarly impressive outcomes for high school seniors. Read more.

 U-Pace: Society for the Teaching of Psychology and the University of Wisconsin-Milwaukee

An independent external evaluation of the grant project conducted by SRI International found fairly large effects of U-Pace on course completion, for low-income students as well as their more advantaged peers. Since their NGLC grant, U-Pace has been recognized with awards, expanded to courses outside of psychology, and is now in the midst of a four-year study supported by the U.S. Department of Education’s Institute of Education Sciences. Read more.

But What about the Other Student Success Innovations that NGLC Funded?

There were 29 innovations in all that received “Building Blocks for College Completion” grants. They utilized OER, blended learning, deeper learning, or learning analytics, and focused on developmental and introductory core courses which are typical roadblocks to persistence and college completion. To learn more about these innovations, visit our recently redesigned and updated Higher Ed Tech Innovation grant recipient pages.

Evaluation Findings

Here I offer you a quick summary of the evaluation findings from SRI International, an independent non-profit research and innovation center, commissioned by the Bill & Melinda Gates Foundation to conduct an independent evaluation of the grant projects. The goal was to determine whether the projects achieved their expected results in terms of scale and student outcomes.

1. Overall, results suggest that technology-enabled innovations can promote student achievement.

Across the NGLC projects, 82% of students using an NGLC innovation finished their courses successfully. The subject matter mastery rate averaged 66% across projects. The corresponding figures for deeper learning and persistence to the next academic term were 49% and 72%. (These numbers are based on available data and do not include all students that participated.)

• These outcomes were generally consistent with outcomes for similar students taking the same courses without the innovation. And for eight projects, the evaluation produced evidence of positive impact on student outcomes.

It’s important to consider that the technology-enabled innovations were at an early stage of development and the time frame for adoption was short given the 15-month grant period. According to SRI, these outcomes are not the ultimate conclusion on the NGLC projects’ effectiveness because most of the innovations were still relatively new and not fully mature.

2. On average, students in developmental and gateway math courses using the innovations had better outcomes than the comparison group.

3. With 36 percent of student participants in grantee innovations considered low-income, SRI found that the projects were as effective for low-income students as they were for those from more affluent backgrounds.

Scale vs. Student Achievement

SRI evaluated the projects’ scalability and concluded:

1. The projects met the majority of the targets they set for scaling.

Scaling to support larger numbers of students at multiple institutions was a primary objective of the grant program. By the end of the grant period, course-related innovations reached 282 campuses, 1,152 instructors, and 69,277 students. In addition, OpenStudy was used over 130,000 times and the University of Hawaii’s STAR course planning system registered 23,509 logins.

2. The innovations improved student course outcomes when implemented at the grant recipients’ own campus, but effects were not found at partner institutions during the grant term.

SRI found that scaling to a large number of colleges and universities in the short timeframe of the grant did not immediately translate to improved academic achievement at partner institutions. SRI attributed the difference to two factors.

• It was difficult for some project teams to quickly address differences in the nature of the students and the environments at partner institutions in the rapid adoption timeframe of the grant—15 months.
• Some project teams needed to make significant—and in some cases unanticipated—technology improvements in order to make it possible for other colleges to adopt their innovations. That drew project teams’ full time and attention away from implementation.

The Way Forward

Colleges that adopt a student success innovation should therefore understand the intrinsic facilitating factors of the innovation, consider its applicability to new institutional settings and student populations, prepare to invest in the time needed for the innovation to flourish and produce results, and consider adopting the seven practices that emerged from the “Building Blocks for College Completion” grant:

1. Achieve greater impact with “whole-course” models.
2. Design student success innovations with active, self-paced, data-driven learning.
3. Cultivate the involvement of faculty with early engagement and ongoing training and resources.
4. Engage students as designers and facilitators of an innovation as well as learners.
5. Join project communities to accelerate adoption and efficiency.
6. Address a student success innovation’s functionality as well as its academic requirements.
7. Create enduring impact through long-term planning and design.