The Scientific Method Comes To Journalism
But will it be its saviour? #6 of the Indian Journalism Project.
It is the year 2005. The news magazine Outlook is at the height of its powers and influence under its legendary editor Vinod Mehta. The magazine’s bureau chief in South India, based in Bangalore, is Sugata Srinivasaraju. Sugata1, as he is known, has just seen some explosive documents related to tech giant Infosys grabbing land.
Sugata tells the story in his recent conversation with Amit Varma on the latter’s podcast The Seen and The Unseen:
I did the very big story in 2005 on the Infosys land scam, when everybody was going gaga over Infosys…when they were making petitions for more land and when they had not actually used what was already with them. I wanted to do the story because I had some documents. [They] had still not reached me, but I had seen them. I suggested the idea [to Vinod], and he said, “yeah go ahead, no problem”.
Sugata continues:
I wrote this story and then he [Vinod] called me. He said, “are you sure of all that you've written?” By then he had started getting calls from various quarters and he said, “I've been getting calls, are you sure of what you've written?”
I said, “yes sir”.
He would then insist that I call him “Vinod” but see, I could never do it, so I said, “sir, I'm pretty sure” and I looked up my facts.
He said, “do you have the documents which support all that you're saying?”
I said, “sir, I have seen all of it. This person has promised that he'll give it to me on Friday.”
“But”, he said, “we go to press on Thursday, what shall we do? Have you seen it? Are you very sure? Are you pucca?
I said, “sir, I’m pretty much sure.”
He said, “theek hain, then I’ll run it.” Because he gave himself this reason, “too many people are calling, so it must be correct. We go down, we go down together. Don’t worry, I’ll run it.”
The story, which is published, kicks up a row and surfaces all kinds of questions about Infosys. Sugata continues:
That was a great moment, and then on the Friday, I got all the documents and I faxed them. Then comes Saturday and I get a call from him [Vinod]. He’s laughing.
He said, “have you restored the balance?” I said, “yes sir.”
“Everything is quiet and silent and people are appreciating what you have written, so it must be true”, he said, “don't stop with the story, do two more. Let's make it a series.”
So I started scrambling for more documents and then realised that S M Krishna had preferential shares in Infosys…
Sugata eventually reports on and writes a series of stories, prompting the Infosys management to request a sit-down with him. He meets one of the firm’s top executives, Mohandas Pai, and says, “I stand by what I wrote”.
The reporting, in the end, is not contested. “So it was correct”, concludes Sugata in his conversation with Amit.
The thing that stands out for me in the narration is the line, “have you got the documents?” Vinod Mehta repeatedly asks this question of Sugata Srinivasaraju, and each time he does so, he is reassured by the reporter. Emboldened by these reassurances, the editor takes a risk and runs the story. A day later, the reporter gets the documents. This is important, because without the documents, there is no story, no matter how true it is.
This attitude, that there is no publishing any story without documentary evidence is the credo by which many reporters and editors do their work. It is a good system, designed to keep the journalists accountable. (In contrast, most of the journalism we see today doesn’t involve evidence, but eyewitness reports. We’ll revisit this point a little later in the piece.)
In today’s era of plunging trust levels, the need for proof is even more important. But it’s not just proof alone, there is now an increasing need to throw light on the very process of fact-finding. In other words, the ‘scientific method’, with its emphasis on replicability of results and evidence, is increasingly important.
This question captured the interest of Katie Palmer, a former science and health editor at Wired Magazine. In 2018, when she was a John S. Knight journalism fellow at Stanford University, she wrote,
Everywhere I look, I see parallels between science and journalism — in the ethics that guide our work, in the foibles that impair the best of that work, and in the technologies that promise to either upend or turbo-charge our fields.
The most common scientific practice nominated for journalistic use? Replicability: the ability for other people to re-do your work and see what happens for themselves. Every scientific paper ideally comes with step-by-step directions so another scientist can repeat an experiment and make sure its result is sound.
In journalism, the equivalent is radically transparent sourcing. A reader or other journalists would be able to see who you talked to — even the unquoted sources — read full transcripts, and dive into the source material. (This is how rigorous fact-checking works, too.)
Jeff Jarvis, professor of journalism innovation at the Newmark School of Journalism is also excited by the scientific method:
One…problem is the methodology of journalism, which is built to regurgitate events and opinions around those already in power without accountability for outcomes. Imagine — as one of my former students, Elisabetta Tola, is— journalism in the scientific method, beginning with a hypothesis, seeking data to test it, calling on experts to challenge it, and recognizing — as scientists do and journalists do not — that knowledge does not come in the form of a final word but instead as a process, a conversation.
One might be tempted to conclude: all this talk of scientific method is very well, but is it really practical?
It’s a question that has been answered by fact-checkers around the world. At BOOM, where Media Buddhi is housed, each fact-checked article needs to show both evidence and the process. (BOOM is a signatory to the International Fact Checking Network’s Code of Principles.) The idea is, anyone who reads our fact-checks can examine the way we do them and retrace our steps. Like us, there are more than 100 other fact-checkers around the world who follow the same principles.
While it may be desirable to have more of the scientific method in journalism, is it 100% necessary? If we insisted on documentary evidence to the exclusion of everything else, what happens to anonymous sourcing when needed and eyewitness accounts, which are crucial to journalism and many other fields including human rights and the law?
And what about the fact that the idea of evidence itself might be flawed?
Frederick Schauer is a professor of law at the University of Virginia. In his 2022 book, The Proof: Uses of Evidence in Law, Politics, and Everything Else he writes,
Much that we know, we learn from what others tell us. The image of Sherlock Holmes in his deerstalker hat searching with an outsized magnifying class for physical cues is endearing and enduring, but much of the evidence we use comes not from what we see or find but from the statements—the testimony—of others.
Schauer gives an example of this. “I know the date of my birth”, he writes, “because my mother told me and because it is written on a document prepared by a public official…of course, I did not see, hear, taste, smell, or touch my own birth.”
He continues:
I know when I was born not because of direct perception, but because of what others have told me.
Schauer also gives us a journalistic example from the excellent reporting on the mob invasion of the Capitol building in the US on 6 January 2021.
A lot of what we know credibly, is because of first-person accounts or hearsay, “even though those who provided the accounts typically provided them to journalists, who in turn provided them to the public. Even if we trust the journalists to report accurately on what the participants and observers told them, as we generally do, we are still expected to form our beliefs about what happened on the basis of accounts offered by the people who were there, people whose testimony we cannot observe or cross-examine, and whose accounts we are asked to simply assume are both honest and accurate.”
The point is that we wouldn’t “know much about the world…without this kind of hearsay.”
Schauer continues:
[There is] is good reason to think twice about each step in the chain when we evaluate a hearsay statement, but it is important to remember that hearsay evidence is still evidence. And it is usually pretty good evidence at that.
If, as journalists, we insisted upon documentary evidence for every story we published, we would fall far short of the mission of journalism. Even without documentary evidence, we can still do the stuff that happens around it, such as interviews, cross-referencing, and self-testing for bias. The scientific method will continue to creep into journalism (where it is very much welcome). But perhaps, it’s not the scientific method that we need so much as greater transparency.
As someone said once, transparency is the new objectivity.
As part of the Indian Journalism Project, we are matching mentors and mentees in a structured mentorship program. If you are interested in mentoring a journalist, or if you are interested in being mentored, do fill up this short four-question survey.
The Indian Journalism Project is a 100-day effort by BOOM and Media Buddhi starting on World Press Freedom Day (3rd May) and ending on India’s Independence Day (15th August). Read the inaugural piece on the four challenges of Indian journalism.
Do subscribe and share with your friends!
When I reference someone by their first name after introducing them, it is because I know them personally. When I don’t know them, I refer to them by their surname. This is a somewhat irregular ‘style’ I know, but I’m sticking with it for now.