Understanding Student Learning Levels by Analyzing Data

Ann Mariano, Director of Technology at Framingham Public Schools

In the fall of 2019, the Framingham Massachusetts school district wasstarting tobuild out systems for data reporting. A new technology director was hired in January 2020, and her focus in previous roles had been on using data to improve student outcomes. She was hoping to use her prior knowledge to build data systems in Framingham. But six weeks into her tenure, the world shifted, schools closed, and 8,500 students and nearly 2,000 employees,who were all home, needed devices and learning management systems to survive the closure of schools. Technology shifted, and the tech team worked overtime to get devices out and teaching tools like Canvas, ScreenCastify, Zoom, and Google Meet onto everyone’s screens.

When the pandemic wound down and the school day returned to ‘normal,' the tech team had rolled out thousands of devices, distributed thousands of hotspots, trained thousands of students and teachers, and now had to shift the focus to getting everyone back to as normal a situation as possible. As students returned, it became apparent that the learning loss was severe and the behaviors were challenging. The district needed a way to objectively see how the students were performing.

Because of the prior focus of the new technology director on data, she found that this would be the new focus of her world going forward. She needed to go back to building a data dashboard that would allow the district and building teams to see where students were and eventually find how the district could focus in on individual student outcomes.

In the summer of 2022, Ann Mariano, the director of technology,and Nick Sweeney, the district data analyst, implemented a new data dashboard system by a company called AnalylticVue. In the months prior, Mariano and her team had issued an RFP to find a data system to meet the district’s needs. The district had been using a free dashboard, Google DataStudio, which was incredibly flexible, to produce reports. But in the spring of 2022, word had reached Massachusetts technology directors that Google would not sign a data privacy agreement for Data Studio, and as a result, district staff had to stop using it almost immediately. The RFP for a new product was produced and shared. About ten responses come back. After a review by multiple staff members, AnalyticVue won the contract.

The good news for Framingham was that most of their data exportshad been written from their student information system (Aspen from Follett) to an SFTP server. Using the same exports to AnalyticVue allowed the district to be up and running with the dashboard in about six weeks.

The first goal was to create a dashboard that allowed district leaders to monitor trends in the district, such as suspension data, absentee rates, and reading and math scores. The best part aboutAnalyticVue is that one can easily create custom roles for data consumption so that once the district view was created, the data could be parsed on a school identifier, and each school would automatically only see that school's data. It made for a smooth transition so that data could be secured by the building and only building administrators could see it. The system easily allowed the district to filter the data by students who are emergent bilinguals, have special needs, or may be homeless, in addition to many other demographic categories. The dashboard also allowed the staff to look at language as an indicator since over 70 languages are spoken in the district.

During the winter of 2022, the district began to share the data with elementary teachers in what is known as a roster view. Teachers can see the data for only their students based on their class rosters in Aspen. It’s also possible to look in more depth at a student’s performance in what is called the learner analysis view. That view lets the district/teacher see anything that is known about a particular student's assessment scores, attendance, and behavior, including historical data.

During the Fall and Winter, other constituents were also looking for reports, such as the high school to see who was skipping classes and the equity department to monitor children who attend after-school programs. The requests were fast and furious and caused the district to split the technology department into two parts: the Department of Educational Technology with all data and instructional technology and the Department of Technology with all devices and networking. The shift allowed the head of educational technology, Ann Mariano, and her small two-person team to focus on the many data requests.

In September of 2023, the district and AnalyticVue added an MTSS module to the system to track individual student plans. They believe that this will allow the staff even more capacity to see student performance. During the launch of the module, in a span of two weeks,254plans were created by coaches and interventionists using a system designed by Mariano, the Framingham coordinatorof MTSS,Sara DeLuca, and AnalyticVue. Next up is the ability to review data from classroom visit rubrics.

The district plans to continue the progress it has made going forward. There is a roadmap over several years as to what can be done with the dashboard. Framingham is fortunate to have intelligent and dedicated educators who can take the knowledge provided by the dashboard and use it to help students.