Step 2 in a Reflective Data Analysis Protocol: Analyze Why It Happened

When looking at the needs of our students throughout year, teachers must consistently and deliberately monitor the recovery of the learning. And this monitoring must include looking beyond the numbers process to analyze why students did or did not learn what they needed to.

“Numbers are important, but they can’t provide educators everything, especially when we’re looking for root causes of students’ learning difficulties. Teachers must see that data stretch beyond what’s expressed on test company spreadsheets. The concept of data encompasses many kinds of information that help teachers know their students, and themselves as practitioners, in depth—and data can be interpreted in many nuanced ways.” (Morrison, 2008)

A reflective data analysis protocol will help you when it comes to reteaching to the specific needs of your students. That’s why we’ve provided this series of articles that explain, step by step, what a reflective data analysis is, how to implement it, and why it’s valuable for your students.

We began the series with an article that introduced a reflective data analysis protocol. And, then continued the series with a focus on each of the three steps:

IDENTIFY: What Happened

• • What were the students demonstrating?
  • What were the students’ successes and challenges?

REFLECT: Why It Happened

• • How did the format impact student demonstration of learning?
  • How did previous decisions in lesson design and delivery impact student demonstration of learning?

IDENTIFY: What Happens Next

• • Regarding successes, how can previous decisions in lesson design and delivery be replicated to ensure continued student learning?
  • Regarding challenges, how can previous decisions in lesson design and delivery be revised to ensure student learning?

In other articles, we provided an in-depth look at the first step—IDENTIFY: What Happened and an in-depth look at the third step—IDENTIFY: What Happens Next.

This article takes an in-depth look at the second step…

REFLECT: Why It Happened

Author’s Note: Before reflecting on why it happened, make sure you’ve determined the success and challenges as shown in the data analysis during the first step—IDENTIFY: What Happened.

Now, we can begin to think about why our students demonstrated their learning the way that they did. This is reflective piece.

We need to look at two areas—the format of the assessment and the previous decisions of the teaching and learning experience. How could these have impacted how our students demonstrated their learning on the assessment?

First, let’s review the different types of items that students could encounter on assessments.

• Traditional selected response items prompt students to select one response from a set of options.
• Multiple selected response items prompt students to select one or more responses from a set of options. All correct responses must be selected for the student to be given credit.
• Non-traditional response questions take advantage of computer-based administration to assess a deeper understanding of content and skills than would otherwise be possible with traditional item types. These kinds of questions might include hot text, drag-and-drop, editing text, or drawing an object.
• Constructed-response questions prompt students to produce a text or numerical response in order to collect evidence about their knowledge or understanding of a given assessment target. These questions include short answer responses.
• Performance tasks measure a student’s ability to demonstrate critical-thinking and problem-solving skills. Performance tasks challenge students to apply their knowledge and skills to respond to complex real-world problems. They can be best described as collections of questions and activities that are coherently connected to a single theme or scenario. These activities are meant to measure capacities such as depth of understanding, writing and research skills, and complex analysis, which cannot be adequately assessed with traditional assessment questions. The performance tasks are taken on a computer (but are not computer adaptive) and will take one to two class periods to complete.

Now, let’s reflect on why it happened.

ASK: How did the format impact student demonstration of learning?

In other words, were your students completely aware of the format of the assessment, each item type and what it was asking the students to demonstrate, and the cumulative effect of this assessment on their learning? When reflecting on the format, remind yourself of the skill being assessed, the type of item, the Depth of Knowledge level, the vocabulary and technology used, and the possible impact each of these had on student results.

As you are analyzing the format of the assessment, your task is to discover examples of both successes and challenges. Below is an example of a teacher reflecting on the format when students were successful and identifying why.

Here’s another example. This time it is from a teacher who is reflecting on the format when the students were less successful and then identify why the format was a challenge.

Click here to download a blank copy for you to use for your students.

Once you’ve analyzed the format of the assessment, you’ll want to review how the skill was previously taught. This is where you will be reflecting on your practice.

ASK: How did the previous decisions in lesson design and delivery impact student demonstration of learning?

You will want to discover which of your instructional decisions are supportive of student learning and which of your instructional decisions are less supportive of student learning. You’ll want to look at both results—successes and challenges. If your students were successful—this tells you to replicate your instructional decisions. If your students were challenges—this tells you to revise your instructional decisions.

A self-reflection process on a success:

1. 1. Identify a standard/skill in which your students were successful.
   2. Review when you taught this standard/skill. What was the unit? What was the lesson?
   3. Reflect on how you taught this standard/skill through the lens of lesson design and delivery.
   4. Determine which decisions were supportive for your students and their learning.
   5. Think about ways to replicate these decisions.

A self-reflection process on a challenge:

1. 1. Identify a standard/skill in which your students were challenged.
   2. Review when you taught this standard/skill. What was the unit? What was the lesson?
   3. Reflect on how you taught this standard/skill through the lens of lesson design and delivery.
   4. Determine which decisions were not supportive for your students and their learning.
   5. Think about ways to revise these decisions.

Utilize a Focused Learning Model to Reflect and Plan

Elevated Achievement’s Learning Model is a schema you can use to reflect on lesson design and delivery. It is driven by the learner and answers the overall question—What does a learner need to know in order to better own his learning?

The Learning Model is made up of five student-centered phases: setting the Learning Context, stating the Learning Outcome, engaging in the Learning Process, producing the Learning Demonstration, and implementing the Learning Application.

The Learning Model supports lesson planning and reflection that develops learnership by ensuring teachers answer the following questions:

• Why is the learning important? The Learning Context makes connections to the real world and to the final prompt of the unit.
• What will my students learn? The Learning Outcome uses the standards to develop the outcome of the lesson (what students will learn) and the demonstration of their learning (how students will show mastery).
• How will my students learn it? The Learning Process determines which instructional strategy or methodology will most effectively and efficiently teach your current students the outcome of the lesson.
• How will my students show that they have learned it? The Learning Demonstration determines how students will show mastery of the outcome of the lesson.
• How will my students continue to use what they learned? The Learning Application makes connections to how students will use today’s learning to produce a stronger final product and how they will use this learning in other classes and situations.
• What is the student’s role in the learning? Answering these questions during planning and then sharing this information with the students will clarify for them their role in the learning.

Again, ASK: How did the previous support in lesson design and delivery impact student demonstration of learning?

Below is an example from a teacher who reflected on the previous instructional decisions in regard to a success and a challenge. This will help them determine the best way to reteach the skills that students did not master.

Click here to download a blank copy for you to use for your students.

So, What Now?

Once teachers have completed the first two steps of the protocol—IDENTIFY: What Happened and REFLECT: Why It Happened—they are ready to craft a reteaching plan for those skills that students are still needing to master. This is the last step of the protocol—IDENTIFY: What Happens Next—and this is the topic of the next article in the series.