Educators knew that last March’s pandemic-prompted disruption to in-person schooling would impact student learning–they just didn’t know how much impact the move to emergency distance learning would have.
Wilton Public Schools officials found a way they believe measures not only where academic progress was interrupted but also gives teachers information on how to shape the curriculum going forward and target specific ways to address individual students’ strengths and areas for improvement.
They’ve begun using a diagnostic tool called IXL Diagnostics, one of the components in the IXL online program suite the district has used for years for skill and practice review. District administrators reviewed what they learned during last Thursday evening’s (Oct. 22) Board of Education meeting.
In introducing the presentation, Assistant Superintendent for Curriculum Chuck Smith told Board members that the information they’ve gathered is better than he anticipated–but that not everything is positive.
Overall what they found was that they didn’t see gaps they were expecting to find in English Language Arts (ELA), but gaps were present in the area of Math–compounding deficits that already existed in the district.
“A lot of people were making some very dire predictions about what the impact of the pandemic would have on student achievement. So I was worried that we would see a bigger lift than what we’re seeing in the data right now, at least with this data. I don’t want to sugar coat it–there are areas that are in need of improvement,” Smith said, adding that he and his team would present about how the district will “support the teachers in using this information to address those areas of concern.”
Explaining IXL Diagnostics
Because both teachers and students are familiar with the IXL platform, it was an easy step for them to work with the diagnostics program. IXL Diagnostic assessments are “criterion-referenced,” which means they compare student performance against a standard skill set, rather than against other students’ performance (the way a MAP test does).
“We can get a sense of how much of the curriculum they have mastered. It’s an important distinction. This information is actually more useful for teachers because it tells them what students need to learn,” Smith explained.
Karen Brenneke, the district’s humanities curriculum coordinator who presented alongside Smith, said there was another benefit: it allows teachers to identify where students had “unfinished” learning and learning regression. While those types of student gaps are familiar to teachers after summer breaks in typical years, they expected this year’s gaps would be more substantial, thanks to inefficiencies of remote, emergency e-learning.
“As our emergency e-learning bled into the summer vacation months, we started to brace ourselves for potential substantial gaps in knowledge, understanding and skills–some of which may have been due to the structure of the e-learning environment itself and or student participation or the lack thereof,” Brenneke explained.
The diagnostic tests can be administered in multiple, shorter sessions until a student has completed about 110-125 questions, and a pinpoint score is generated. Students are assessed in six strands in math and four strands in ELA. The three-digit score that’s generated gives an immediate picture of the student’s progress: the first digit represents the grade level at which the student is performing, and the second two numbers represent the percentage toward mastery of end-of-year standards. (Another way to look at the second number is by aligning it with the months of the year–for example, a ‘6’ means the student is doing work commensurate with the 6th month of the school year.)
In addition to an overall score, each student also receives a fuller assessment report, displaying both strengths as well as areas to refine and grow. The tool also reveals the upcoming learning and what the student is ready to learn directly to the student, which Brenneke said “[offers] a high degree of agency when it comes to setting goals for his or her own learning.”
Brenneke illustrated the scoring with sample student reports. In the first example, she shared a 7th grader’s report for math, with an overall math score of 770. This student is performing within his/her grade level–shown by the first digit, ‘7’; and the second two numbers–’70’–represent the percentage toward mastery of the end-of-year standards (or approximately the 7th month of the 7th grade school year).
The second example she shared was an ELA report for a 4th grader:
Thus far, IXL Diagnostic tests have been administered to students in Cider Mill and Middlebrook. Smith’s team said that there are plans to administer the IXL Diagnostic tests at Wilton High School. Miller-Driscoll students will take the NWEA’s MAP testing for primary grades later this year.
For ELA, students are assessed in four areas: reading strategies, writing strategies, vocabulary, and grammar and mechanics. They also receive an overall reading score.
Brenneke shared the overall average ELA scores by grade level for Wilton students in grades 3-8 (below). Based on what the scores indicated, she was pleasantly surprised–on average, students are performing at grade level either at or ahead of schedule (where they should be by the 3rd-5th month of their school year).
“As you can see this tool is not showing us the gaps that we anticipated,” she said, adding that the data will help teachers put together small group instruction as well as help them match reading texts with students.
Brenneke also pointed out that while K-grade 2 students did not complete IXL Diagnostics, the district knows that the move to emergency e-learning had an impact.
“Some of [the district’s] youngest learners in 1st and 2nd grades may be evidencing more unfinished learning on some reading assessments, particularly Running Records, which allow us to spy in on a reader’s mistakes so we can analyze what they are ready to learn next. This potential unfinished learning should not be surprising given that our students learning (along with every kindergarten, first grade, [and] second grade student last year), their learning was interrupted at a critical juncture while they were still building their foundational skills,” she said
Brenneke said that teachers are prepared to address those skill gaps while also working on at-grade level curriculum. The district will also continue to gather more data next month with the NWEA’s MAP test.
The findings were less positive for Wilton students’ overall performance in math. Trudy Denton, the district’s math curriculum coordinator, told that story to the BOE members, explaining that e-learning may not have been sufficient and may have compounded gaps that existed even before the district went all-remote last March. The IXL results showed that the gaps seen in past years were still present.
“Math trends as shown here [above] are consistent with prior year’s data, in that learning gaps that often begin in the elementary school do grow over time. In typical years, and to a somewhat greater extent this year, we reduce instructional time exactly when the curriculum becomes more challenging in grades six, seven, and eight. In addition, the learning models used in middle school during emergency e-learning may not have met the needs of some students,” Denton said, although she added it was difficult to determine if students who didn’t actively participate in e-learning was a factor.
In other words, the average pinpoint math scores showed that starting as early as 4th-grade students were falling behind and not performing at their grade level. By 7th and 8th grades, not only were students a year behind, but they were only doing the work typically taught in the first month of that year.
“That is typically how learning gaps progress–they start small and, if they’re not fully addressed, they grow over time,” Denton said.
She cautioned BOE members to remember that the scores reflect an average. “There are students who are significantly above this and some students who are below it.”
Denton also reiterated that each student not only had a three-digit score but a more complete report showing where students skills are on multiple strands. From this, teachers are able to target what skills individual students need to work on. “This is a starting point,” she noted.
Denton said she’s tried to break down the scores by building and grade to look for patterns and trends on major areas and concepts where gaps exist. “[The learning skills about] measurement always seems to lag and that’s kind of across the grades. So that’s an overall area that probably needs to be addressed.”
The data shows that the district needs a plan to address the gaps across all learning models, given that the district may move from one learning model to another–between hybrid, in-person, and remote.
Denton added that teachers need time to collaborate together “to identify, analyze, and plan for addressing the learning gaps,” and that schedules need to carve out time both for that teacher collaboration as well as “necessary instructional time.”
Key to what the district needs to address is how teachers will continue to teach the current grade’s curriculum as well as incorporate any prior learning students may have missed that is part of that current content.
Board member Glenn Hemmerle expressed his frustration with what he was hearing.
“I just continue to be troubled about our math performance. It has been historically here in this district, a weak spot. We’ve always excelled in the English language arts area; math continues to be our weak spot. I just don’t understand, with all the emphasis that we put on it, the attention we paid to it, I am just deeply troubled that we cannot get this thing moving in the right direction,” he said.
Board chair Deborah Low agreed. “You’re absolutely right. And I think we’re all concerned,” she said, adding that she would like to see it discussed at length at an upcoming meeting.
“This is revealing news that we really need more time to sort of delve into,” Low said.
Brenneke clarified that the IXL Diagnostic test is “designed to help a teacher identify what a student is ready to learn next rather than to measure an achievement level. That’s an important distinction to make.”