When we started our fact-check, we thought it would be a fairly simple task. The claim was about the number of journalists killed in different wars. Find the numbers, compare them, and determine whether the claim was true or false. However, the deeper we went into the research, the more we realised that it was not nearly that straightforward.
As part of EU Fact-Checking Week, our group investigated a claim from an Al Jazeera article stating that more journalists had been killed since the beginning of the Gaza war than in several major wars combined, including both World Wars.
At first, this seemed like a classic fact-check. The claim was based on numbers, so it should have been possible to verify them. But during our research, we quickly realised that the numbers themselves were not the main problem.
Who is actually being counted?
The deeper we looked into the sources, the more often we found ourselves asking the same question: who actually counts as a journalist?
In the Gaza war, some datasets included media workers alongside journalists. This category can include translators, drivers, fixers, and other people who support media organisations. Historical data from the World Wars presented different challenges. Some of the people listed as journalists were also soldiers, while other cases were only partially documented.
As a result, it became clear that we were not always comparing the same thing. The numbers looked comparable, but they were often based on different definitions.
When numbers turn into questions
Originally, we wanted to find out whether the claim was true or false. Instead, we spent a large part of our research trying to understand how the numbers had been produced in the first place.
This was probably the biggest surprise for us. Statistics often seem objective and straightforward. But once we started looking more closely, we realised that every dataset is based on choices. Who is included? Who is excluded? What definition is being used?
Those choices can have a major impact on the final result.
What does this mean for fact-checking?
At several points during our research, it felt as if we were no longer fact-checking the claim itself, but the data behind it. Looking back, we would argue that this is exactly what factchecking is about.
Fact-checking is not only about confirming or debunking numbers. It is also about questioning whether those numbers actually measure what they claim to measure.
This is especially important in political debates. Many misleading claims are based on real statistics. The problem is often not that the numbers are invented, but that they are taken out of context or compared in ways that are not methodologically sound.
What we learned
The most important lesson from this fact-check was not whether a particular number was right or wrong.
What we found more interesting was how easily statistics can create an impression of certainty. Behind every number are definitions, decisions, and interpretations that shape what the number actually means.
Our fact-check showed us that statistics should never be accepted at face value, even when they appear convincing. Sometimes the most important question is not whether a number is correct, but whether it is being used in a meaningful way.
RESEARCH | ARTICLE © Marius Moch, Jade University of Applied Sciences Wilhelmshaven, Germany.
Photo: AI-generated by Google Gemini
Leave your comments, thoughts and suggestions in the box below. Take note: your response is moderated.
