By Alan Schwarz, Walt Bogdanich and Jacqueline Williams, New York Times

The National Football League was on the clock.

With several of its marquee players retiring early after a cascade of frightening concussions, the league formed a committee in 1994 that would ultimately issue a succession of research papers playing down the danger of head injuries. Amid criticism of the committee’s work, physicians brought in later to continue the research said the papers had relied on faulty analysis.

Now, an investigation by The New York Times has found that the NFL’s concussion research was far more flawed than previously known.

For the last 13 years, the NFL has stood by the research, which, the papers stated, was based on a full accounting of all concussions diagnosed by team physicians from 1996 through 2001. But confidential data obtained by The Times shows that more than 100 diagnosed concussions were omitted from the studies — including some severe injuries to stars like quarterbacks Steve Young and Troy Aikman. The committee then calculated the rates of concussions using the incomplete data, making them appear less frequent than they actually were.

After The Times asked the league about the missing diagnosed cases — more than 10 percent of the total — officials acknowledged that “the clubs were not required to submit their data and not every club did.” That should have been made clearer, the league said in a statement, adding that the missing cases were not part of an attempt “to alter or suppress the rate of concussions.”

One member of the concussion committee, Dr. Joseph Waeckerle, said he was unaware of the omissions. But he added: “If somebody made a human error or somebody assumed the data was absolutely correct and didn’t question it, well, we screwed up. If we found it wasn’t accurate and still used it, that’s not a screw-up; that’s a lie.”

These discoveries raise new questions about the validity of the committee’s findings, published in 13 peer-reviewed articles and held up by the league as scientific evidence that brain injuries did not cause long-term harm to its players. It is also unclear why the omissions went unchallenged by league officials, by the epidemiologist whose job it was to ensure accurate data collection and by the editor of the medical journal that published the studies.

In 2013, the NFL agreed to a $765 million settlement of a lawsuit in which retired players accused league officials of covering up the risks of concussions. Some players have appealed the settlement, asking for an examination of the committee’s concussion research.

Some retired players have likened the NFL’s handling of its health crisis to that of the tobacco industry, which was notorious for using questionable science to play down the dangers of cigarettes.

Concussions can hardly be equated with smoking, which kills 1,300 people a day in the United States, and The Times has found no direct evidence that the league took its strategy from Big Tobacco. But records show a long relationship between two businesses with little in common beyond the health risks associated with their products.

In a letter to The Times, a lawyer for the league said, “The NFL is not the tobacco industry; it had no connection to the tobacco industry,” which he called “perhaps the most odious industry in American history.”

Still, the records show that the two businesses shared lobbyists, lawyers and consultants. Personal correspondence underscored their friendships, including dinner invitations and a request for lobbying advice.

In 1997, to provide legal oversight for the committee, the league assigned Dorothy C. Mitchell, a young lawyer who had earlier defended the Tobacco Institute, the industry trade group. She had earned the institute’s “highest praise” for her work.

A co-owner of the New York Giants, Preston R. Tisch, also partly owned a leading cigarette company, Lorillard, and was a board member of both the Tobacco Institute and the Council for Tobacco Research, two entities that played a central role in misusing science to hide the risks of cigarettes.

The NFL’s concussion committee began publishing its findings in 2003 in the medical journal Neurosurgery. Although the database used in the studies contained numerical codes for teams and players, The Times decoded it by cross-referencing team schedules and public injury reports.

The NFL’s concussion studies have faced questions since they were published, but even the league’s harshest critics have never suggested, and no evidence has ever arisen, that the underlying data set could be so faulty.

“One of the rules of science is that you need to have impeccable data collection procedures,” said Bill Barr, a neuropsychologist who once worked for the New York Jets and who has in the past criticized the committee’s work.

By excluding so many concussions, Barr said, “You’re not doing science here; you are putting forth some idea that you already have.”

The Work Begins

In an introduction to the first of the concussion committee’s papers, the league’s commissioner at the time, Paul Tagliabue, acknowledged the need for “independent scientific research” to better understand the risks of concussions.

“As we looked more deeply into the specific area of concussions, we realized that there were many more questions than answers,” Tagliabue wrote.

The committee’s chairman, Elliot Pellman, the team physician for the Jets, emphasized that his group aimed to produce research that was “independent” and “meticulous.”

In fact, most of the dozen committee members were associated with NFL teams, as a physician, neurosurgeon or athletic trainer, which meant they made decisions about player care and then studied whether those decisions were proper. Still, the researchers stated unambiguously — in each of their first seven peer-reviewed papers — that their financial or business relationships had not compromised their work.

The committee said it analyzed all concussions diagnosed by team medical staffs from 1996 through 2001 — 887 in all. Concussions were recorded by position, type of play, time missed, even the brand of helmet.

The committee’s statements emphasized the completeness of the data.

“It was understood that any player with a recognized symptom of head injury, no matter how minor, should be included in the study,” one paper said.

And in confidential peer-review documents, the committee wrote that “all NFL teams participated” and that “all players were therefore part of this study.”

Those statements are contradicted by the database.

The Times found that most teams failed to report all of their players’ concussions. Overall, at least 10 percent of head injuries diagnosed by team doctors were missing from the study, including two sustained by Jets receiver Wayne Chrebet, who retired several years later after more concussions. Pellman, the Jets’ physician, led the research and was the lead author on every paper.

According to the research papers, team physicians were to fill out forms specially designed for the studies to submit information about concussions — a system that went well beyond the league’s standard injury-reporting protocols. In one paper, the committee wrote, “The Commissioner of the NFL mandated all team physicians to complete and return forms whenever they examined a player with a head injury.”

But after The Times described how it had identified missing concussions, the NFL said this week that the studies, in fact, “never purported” to include all concussions.

Teams were “not mandated” to participate, the league said, only “strongly encouraged.” And some teams, a spokesman said, “did not take the additional steps of supplying the initial and/or follow-up forms.” He did not explain why some teams had not included all concussions identified by medical personnel.

The league explained, as did the papers, that some concussions went undiagnosed in the first place because players are known to occasionally hide their symptoms of concussion from team doctors; that symptoms of concussion can be so brief that no one notices; and that doctors might have used different criteria to make concussion diagnoses.

But the vast majority of omitted concussions identified by The Times were included in the NFL’s public injury reports, meaning that medical staffs had made the diagnoses and reported them to the league. Some of the omitted concussions were reported by the teams to the news media after a game but do not appear on the injury reports, presumably because the player’s status for the next game was not in doubt.

The database does not include any concussions involving the Dallas Cowboys for all six seasons, including four to Aikman that were listed on the NFL’s official midweek injury reports or were widely reported in the news media. He and many other players were therefore not included when the committee analyzed the frequency and lasting effects of multiple concussions.

Several other teams have no concussions listed for years at a time. Yet the committee’s calculations did include hundreds of those teams’ games played during that period, which produced a lower overall concussion rate.

A Cowboys spokesman, Rich Dalrymple, said the team had participated, but he declined to say how many cases were reported and which players were involved. He said he did not know why the Cowboys’ data did not appear in the studies. A San Francisco 49ers spokesman did not return messages seeking comment about Young.

Dr. Robert Cantu, one of the peer reviewers who at the time criticized the committee’s analyses, said, “It should be an unmistakable red flag that a team does not report any concussions over multiple years.”

Some injuries were more severe than what was reflected in the official tally. According to committee records, St. Louis Rams quarterback Kurt Warner sustained a concussion on Dec. 24, 2000, that healed after two days. But Warner’s symptoms continued, and four weeks later he was ruled out of the Pro Bowl with what a league official described as lingering symptoms of that head injury.

The NFL declined to make Pellman available for an interview. The study’s epidemiologist, John Powell, who no longer works on behalf of the league, did not respond to interview requests. Michael L. J. Apuzzo, editor of Neurosurgery when the papers were published, did not respond to interview requests.

The committee and the NFL have long claimed that the papers were vetted through a rigorous, confidential peer-review process before publication, which legitimized their methods and conclusions. But more than a dozen pages of anonymous back-and-forth between reviewers and the committee show some reviewers almost desperate to stop the papers’ publication while the authors brushed aside criticism.

One reviewer wrote, “Many of the management of concussion suggestions are inappropriate and not founded on facts.” Another said the committee’s assertion that the league was handling concussions too cautiously was not proved and was therefore “potentially dangerous.”

An author of the NFL studies responded, “If the truth is dangerous, then I suppose our results are dangerous.”

Overlapping Interests

In 1992, amid rising concerns about concussions, Tisch — the Giants and Lorillard part owner — asked the cigarette company’s general counsel, Arthur J. Stevens, to contact the NFL commissioner at the time, Tagliabue, about certain legal issues.

Stevens was not just any tobacco lawyer; he was a member of the industry’s secretive Committee of Counsel, which helped direct tobacco research projects. In a letter obtained by The Times, Stevens referred Tagliabue to two court cases alleging that the tobacco and asbestos industries had covered up the health risks of their products.

In one case, the family of a dead smoker sought internal documents that the tobacco industry had withheld on the grounds of lawyer-client privilege — which does not apply if invoked to cover up a crime. The judge in the case reacted angrily after reading those internal records.

“The documents speak for themselves in a voice filled with disdain for the consuming public and its health,” the judge, H. Lee Sarokin of U.S. District Court in New Jersey, wrote earlier in 1992. Tobacco lawyers succeeded in having Sarokin removed from the case.

Why an influential tobacco lawyer would recommend legal cases to the NFL is not known, because neither Stevens nor Tagliabue would agree to be interviewed. Tisch died in 2005.

Joe Lockhart, a league spokesman, said that the cases involved potential bias of judges and that there was no evidence that the letter “was taken seriously.”

Even so, records show that, in the legal arena, the league and the tobacco industry sometimes intersected.

Before joining the NFL, Mitchell, a young Harvard Law School graduate, had been one of five lawyers at Covington & Burling who had provided either lobbying help or legal representation to both the NFL and the tobacco industry, sometimes in the same year. Tagliabue had been a partner at the firm before becoming the NFL’s commissioner.

In 1992, Mitchell defended the Tobacco Institute against a smoker’s lawsuit. She also worked on behalf of the institute in a landmark secondhand smoke case, as well as for other nontobacco clients. Mitchell said she was not responsible for legal strategy in the tobacco cases.

At the NFL, said Brian McCarthy, a league spokesman, Mitchell’s work for the concussion committee was administrative. “She did not have any responsibility or any role in directing the research,” he said.

Waeckerle, the concussion committee member, offered a different view.

He said Mitchell asked committee members: “How can this affect us? How can this be studied? How should we view it? Is this a legitimate concern, or is this part of somebody’s zeal, and do we need to be concerned?”

Waeckerle praised her for bringing a nonmedical voice that made members consider the risks, benefits and “what are the intended and nonintended consequences of whatever we were discussing.” He said, for example, that she wanted to ensure that the committee’s work applied only to the NFL, not to college or youth football.

Mitchell said in an interview that she left the NFL after six years for personal reasons, unrelated to her work, and that she did not recall much about the committee’s work.

“I don’t think I saw any reports,” she said. “It was in the early stages.”

Mitchell added that, as the league’s assistant secretary, she had broad responsibilities beyond health and safety issues.

Her contributions to the concussion committee won her thanks in five research papers — three by the NFL and two by a Canadian company that did contract work for the NFL. The committee wrote that she worked “tirelessly to initiate” its research and that “her efforts paved the way for successful completion of the research.”

On at least two occasions in the 1970s and 1980s, the NFL hired a company whose client list included the Tobacco Institute to study player injuries. The league also hired a company — for a matter unrelated to player safety — that had performed a study for the tobacco industry that played down the danger of secondhand smoke.

The NFL’s ties to tobacco are reflected in other ways. When Congress was considering legislation that dealt with when a team owner could relocate a franchise, Joe Browne, a league official sought lobbying advice from a representative of the Tobacco Institute.

“I would like to take the opportunity to sit down and discuss this bill with you further,” Browne said in a 1982 letter to the institute’s president, Sam Chilcote.

Neil Austrian, a former NFL president, had previously run an advertising agency that under his leadership reversed its ban on taking tobacco clients. He called Philip Morris “an honorable company that sets high standards.” It was during his tenure at the NFL that the concussion committee was created.

Years later, when the committee’s work drew criticism during the peer review process, its members pushed back.

“We are aware the findings from the NFL confront some popular opinions,” they wrote, “but believe this study stands on its merits based on the physician evaluation of injury and quality assurance of the data.”

Decoding NFL Database to Find 100 Missing Concussions

From 1996 to 2001, the NFL gathered data in an effort to study the dangers of concussions. Whenever a concussion was diagnosed, the team physician was to record information like the player’s position, his symptoms and how much time he spent recovering.

As is common in such studies, players’ names and teams were represented only by codes, and the information is believed to have never been seen by more than a few committee members. There were 887 concussions recorded in all, forming a log that was consistently described as every head injury that teams had cared for during those six seasons.

The database served as the backbone for the committee’s 13 research papers.

The New York Times recently obtained the data, however, and was able to determine the identities of all of the teams and many of the players. Subsequent analysis confirmed that at least 100 concussions, including some serious ones to the game’s top players, were not included in the league’s studies.

To identify the teams, The Times examined the dates on which each team’s concussions occurred and whether the game was listed as home or away. That information was compared with the teams’ schedules during that time.

Many teams could fit only one pattern; other information, like whether the injury took place on natural or artificial grass, was used to determine and reconfirm the team identifications.

This process led to the conclusion that no Dallas Cowboys players appear in the database — most significantly quarterback Troy Aikman, whose four concussions during that period led him to retire early. Other teams have no listings for years at a time.

In some cases, contemporary news media articles helped with the identification of players who had sustained concussions that were announced by team officials but were apparently not considered significant enough to later list them on the NFL’s official injury reports. Such players were matched with possible database listings.

This process of identification and elimination identified more than 100 players whose concussions were publicly announced but still not included in the league’s research. It is unclear how many more concussions were diagnosed by teams but, because the symptoms ceased quickly, were never mentioned to outside outlets.

To Learn More:

Playing in the NFL is a Dangerous Job: 250 Players Dealing with Injuries after Second Week of Games (by Noel Brinkerhoff and Steve Straehley, AllGov)

NFL Players Sue League and Helmet Maker over Hidden Brain Injury Test Results (by Noel Brinkerhoff, AllGov)

NFL Players at Greater Risk for Dementia (by Noel Brinkerhoff, AllGov)