Write better feedback for better math learning.
Research suggests that Black and Latino students pay attention to what teachers say about their work in math class, and that teacher feedback can affect their learning, motivation, and sense of belonging. Learn how to give better feedback with AI.
Overview
With support from the Bill and Melinda Gates Foundation, Amplify researched the features of effective feedback for underrepresented minority students and used that research to create a generative AI tool that helps teachers give students feedback about math.
This project resulted in the following work:
- Two literature reviews about feedback in mathematics, especially for underrepresented minority students.
- A prompt for a large language model (LLM) that would take in written feedback and offer feedback on the feedback, helping its author learn principles of effective feedback using principles of effective feedback.
- A professional development activity in the Desmos Classroom platform that engages teachers in writing feedback messages about student thinking and receiving feedback on their feedback via an API call to OpenAI’s ChatGPT.
Literature reviews
We asked researchers to review literature about feedback, especially for underrepresented minority students. The researchers compiled the following literature reviews:
LLM prompt
We wrote an LLM prompt designed to take in the following context:
- The text of a math problem
- A student’s response to the problem
- A teacher’s feedback to the student
- The principles of effective feedback we distilled from the literature reviews
The prompt gives teachers feedback on their feedback, helping them learn to give effective feedback using the principles of effective feedback. The prompt and sample response:
You are a helpful and patient mathematics instructional coach who needs to support teachers who are trying to give feedback to their students on mathematics problems.
You will give feedback to teachers helping them give more effective feedback to their students. Your goal is to help teachers improve their feedback using several principles. It is important that you also use these principles when you give feedback to the teachers.
First, feedback must communicate high standards and the message that a person is capable of meeting high standards. Those responses might look like:
- “I am giving you these comments because I have very high expectations and I know that you can reach them.”
- “The expectations in this course are high and I know you can do great work. The feedback here is designed to help you get there.”
Second, feedback must emphasize mastery goals rather than performance goals. Feedback that incorporates mastery goals:
- Is specific and substantive rather than one that is comparative/fosters competition.
- Encourages perseverance, focuses on development of competence on the relevant topic and/or domain.
- Acknowledges (praises) effort, improvement, and tells the person what is right about their work.
- Emphasizes achieving a standard relative to mastery of content and prior performance, rather than focusing on relative performance (comparison to others).
Third, feedback needs to be in language that the person receiving it can easily understand. Feedback to students should be written at a level that a grade 6 student can read and understand. Feedback to teachers should be written at a level that a grade 12 student can read and understand.
The feedback should be concise. It should not tell the teacher the exact feedback to offer. Rather it should ask a question designed to help them think more about the principles of effective feedback we have described above.
For context, you will receive a [mathematics problem], [student response], [teacher feedback message]. Here is an example of how you can support a teacher with providing feedback that follows the principles of effective feedback:
[mathematics problem] Write the equation of this line. The line is given by the equation y = ⅔ x + 4
[student response] y = 3/2 x + 4
[teacher feedback message] That is incorrect. Try again.
Your feedback to the teacher might be:
Giving effective feedback is challenging and all of us can learn it. Here you told the student their answer is incorrect, which is important information, but doesn’t tell them what part of the task they got right. Can you change your feedback to emphasize an area where they are correct or, if you don’t see anything correct, to give them a first question to help them start?
Given the following a [mathematics problem], [student response], and [teacher feedback message], write a three-to-five sentence message to the teacher that gives them feedback on their feedback and provides an example.
Teacher professional development activity
The OpenAI ChatGPT interface could help teachers develop more effective feedback, but working with it is more complex too. Teachers must input a math question, a student answer, and their original message into the context window.
We wanted to simplify that process by embedding an API call to ChatGPT in an environment that automatically collects and injects the context information. To do that, we used the Desmos Classroom Activity Builder, the same platform we use to author our Desmos Math 6-A1 curriculum.
In the professional development activity we created, teachers think about how students might respond to common math questions in middle school. They brainstorm their own responses to common wrong answers. And then they receive feedback from a helpful generative AI agent.
Prototype: AI-enhanced Feedback Support
Desmos Math contains an interface that lets teachers write feedback to students through its software platform. We created an activity that would a) help teachers learn how to give asset-based mathematics feedback, b) introduce educators to the affordances of AI in their practice.
To use the activity:
- Click this activity link.
- Log in with a Google or Amplify account.
You will interact with ideas about feedback and several incorrect student answers. When you give your feedback on those student answers, you’ll receive AI-generated feedback on your feedback. You can revise and resubmit multiple times.
User Testing: Report of Results
We invited middle school mathematics teachers to participate in this teacher feedback activity through one of our email communications. 95 teachers responded affirmatively and 44 of them ultimately participated. They accessed the activity and completed a survey about their experience.
Overall, we have significant qualitative evidence in survey responses that teachers are interested in learning more about giving effective feedback and that they are at least curious about generative AI. This particular instantiation has significant room for improvement and we will continue to investigate ways to improve the kinds of feedback Black and Latino students receive from their teachers. Read more in the below report.
Asset-oriented AI feedback for Black & Latino students at scale: Results of user testing