Readings

Denise Troll Covey, Using Data to Persuade: State Your Case and Prove It (download accessible version)

Although this article is nearly 20 years old, it lays out some timely, creative, and practical strategies for persuading administrators in a higher education environment. Here is a taste: “Perfect data are impossible to obtain. Near-perfect data can take so long to obtain that the opportunity will pass you by or the problem will engulf you. Settle for good enough data to get the job done.”

Jake Cohen, The Pedagogical Value of Failure

Cohen encourages educators to learn from mistakes, unexpected outcomes, and other “failures” in teaching, as a valuable opportunity to improve pedagogical skill.

Key Characteristics of the Scholarship of Teaching and Learning (SoTL)

SoTL is a “systematic investigation of a teaching/learning issue that is shared for review, dissemination and possible some action that changes what is done in the classroom.”^[1] While we are not asking participants to develop a formal research project for your Action Plan, the thinking and principles foundational to SoTL are helpful to consider as you determine how you will evaluate your plan’s effectiveness.

Evaluation Strategies

Why Evaluate?

First and foremost: remember that we are not asking you to conduct a formal, scientifically rigorous evaluation of your Action Plan! We hope that mindful, intentional evaluation will help you achieve one or more of the following goals.

Determine what actually works for students

Our #1 goal here is to increase the equity of student outcomes. The extensive research we’ve provided in this FLC offers a solid foundation, but it is very broad, and not every approach will be the right fit for your specific students, discipline, or course. Knowing what truly works will require some trial and error, and the more intentional you are in seeking evidence of effectiveness (or learning from ineffective attempts), the better you’ll know what to do next.

Identify patterns for your future practices

It may turn out that you are most effective when you build in automated supports using D2L tools. Or perhaps you’ve found a terrific support person on your campus who can strategize with you. Knowing how you work best, and what kinds of approaches you enjoy taking, will increase the likelihood that you’ll integrate effective equity practices into your teaching on a regular basis. As with the first bullet point, don’t forget about The Pedagogical Value of Failure; it’s almost as helpful to know what doesn’t work for you as it is to know what does.

Get buy-in from colleagues and campus leaders

As Denise Troll Covey points out in “Using Data to Persuade,” determining ahead of time what kinds of data will convince and persuade your target audience can help you identify the “good enough data” that you need. The more evidence you have that these equitable practices make a difference, the more likely you are to create a culture of equity on your campuses, and the more subsequent support you and your colleagues will have in your efforts.

Identify Your Frame

BetterEvaluation, a non-profit organization dedicated to collaborative improvement of evaluation, recommends that you start by framing the boundaries for an evaluation.

• Why are you evaluating?
• How will success look?
• Who will use the evaluation?
• What questions are you seeking to answer?

Resources for Data

We are “satificing” rather than seeking true scientific rigor with these projects. That said, there is some practical information, both quantitative and qualitative, that you can gather to determine your project’s evaluation.

• The Minnesota State Equity Scorecard (offered through the the Equity by Design initiative) offers improved/increased access to much of this information in ways that may enable you to drill down to your specific class(es).
  Your D2L course data may provide some helpful insights, particularly if you can pull comparative information from a prior semester to compare to the semester where you implemented your Action Plan.
• Consider either surveying your students directly about your Action Plan practice or seeking feedback within your student evaluations regarding the practice your implemented.
• Keep your eyes and ears open for stories from individual students. Sometimes the best evidence that a practice is effective comes from comments like, “Now I understand!” or “I did so much better after you…”

Possible Evaluation Measures

• An increase in course/assessment outcomes: a higher grade average for either a course or a specific assessment that has been redesigned for equity
• An increase in course/assessment completion: more students complete a course (the D/F/W rates go down) or submit a specific assessment that has been redesigned for equity
• An increase in attendance/engagement: student attendance increases, or student engagement with an activity increases
• A decrease in course costs: the savings for students when you opt to use low- or no-cost resources (like Open Educational Resources or materials available through the library) in place of expensive texts or software
• A decrease in grade, completion, or engagement disparities: particularly between white and underrepresented students, for a single activity, assessment, course, or program