IB's Secret Algorithm

Updated: Sep 15

Original article written by Tom Simonite

Teen regrets about grades aren’t unusual, but the way the foundation behind the International Baccalaureate Diploma Programme calculated this year’s grades was. This year's results were determined by a formula that IB, the foundation behind the program, hastily deployed after canceling its usual springtime exams due to Covid-19. The system used signals including a student’s grades on assignments and grades from past grads at their school to predict what they would have scored had the pandemic not prevented in-person tests.

More than 15,000 parents, students, and teachers have signed an online petition asking IB to “take a different approach with their grading algorithm and to make it fairer.” One math teacher at a school in the Middle East says IB should disclose the full workings of its model for outside scrutiny. He and a colleague have been puzzling over its design since several students lost scholarships to top universities, after receiving results much lower than expected by their teachers. Some students caught out are now unsure how they’ll pay for college. “My only guess is a flawed model,” he says.

In normal times, IB diploma students select six subjects, from options such as physics and philosophy, and receive final grades determined in part by assignments but mostly by written tests administered in the spring. When Covid-19 seized hold of the world in March, IB canceled all tests and said it would calculate each student’s final grades using a method developed by an unnamed educational organization that specializes in data analysis.

IB, headquartered in Geneva, opted to use a statistical formula instead—adding to the growing list of tech fixes proposed to automate away fallout from the pandemic. The workings of the IB diploma—and the timing of the results—proved particularly harmful for IB students applying to US colleges. Unlike AP tests, which are typically separate from high school grades, the IB results are intended to reflect a student’s work for the year. IB students are often granted college admission based on predicted grades, and they submit their final results when they become available over the summer. Some colleges, including NYU and Northeastern, warn on their admissions pages that students whose IB results don’t get close enough to those predictions may lose their place.

The idea was to use prior patterns to infer what a student would have scored in a 2020 not dominated by a deadly pandemic. IB did not disclose details of the methodology but said grades would be calculated based on a student’s assignment scores, predicted grades, and historical IB results from their school. The foundation said grade boundaries were set to reflect the challenges of remote learning during a pandemic. For schools where historical data was lacking, predictions would build on data pooled from other schools instead.

One visual arts teacher at a US school says what she and coworkers have seen suggests it wasn’t well tailored. “When I saw the marks, I was floored,” she says. “I am always conservative in my predicted grades, but every single student except one were downgraded.” Of 15 students she works with, four have to rethink their plans for this fall, because they missed out on college places, something she didn’t expect for any of them.

Suresh Venkatasubramanian, a professor at the University of Utah, says “The burden of proof should be on the system to justify its existence.” He goes on to say that basing a student’s grades on past trends at their school, potentially unrelated to the student’s own school career, could be unfair. Using data from other schools—as IB did for schools with little track record—is a “red flag,” he says, because it would mean some students’ grades were calculated differently than others.

Constance Lavergne, whose son in the UK received lower-than- expected IB grades and missed out on his preferred college, is one of many parents struggling to understand what happened. She says her experience working closely with data analysts in the tech industry makes her suspicious of IB’s methodology. It would naturally generate noisier results for smaller classes, like her son’s, because they offer fewer past data points, she suggests. “There’s something wrong with the algorithm,” Lavergne says.

The math teacher in the Middle East said he believed his school had suffered because of how IB announced and calibrated its model. Students at the school submitted their assignments before IB said those assignments would help steer the grading model. Some IB students at other schools had not yet submitted those assignments, allowing them to put in extra effort, aided by knowing they didn’t have to prepare for exams. This weekend, he plans to work with his math PhD colleague and a software package to probe where the IB formula may have gone wrong.

Kathe Lee © 2020