Can a Better Helmet Save Your Kid's Brain? - Patrick Hruby
Can a Better Helmet Save Your Kid's Brain?
Is a popular football helmet concussion safety rating system fatally flawed?
By Patrick Hruby | Washingtonian | December 2014
I've failed as a father.

Dean Mathews was watching a video of his son’s football game two years ago when the idea planted itself in his head.

On the screen, his son, Nick, then a sophomore wide receiver for Patriot High School in Prince William County, caught a short pass near the sidelines. He cut back across the middle of the field, stutter-stepping his way around three defenders. Then he collided with an opposing linebacker.

Just 15, Nick was only five-foot-nine and 140 pounds, tiny compared with the 225-pound defender. They crashed into each other helmet to helmet, and Nick spun like a revolving door before falling to the ground.

Mathews was in the stands when it happened. He feared a concussion, or worse. But Nick got up, as he always did: The boy wasn’t particularly fast, but he was determined and seldom made mistakes—two qualities that had earned him his starting spot on the varsity squad. And following a time-out, Nick returned to the field.

It was only the next day, while father and son reviewed the game film at home, that Mathews noticed something odd. Two plays after the hit, Nick’s teammates surged forward at the snap of the ball. But on camera, the boy lagged, motionless for a second, as if he were a video-game avatar and the person holding the control pad had forgotten to press the analog stick.

“Dude, pay attention to the ball,” Mathews chided his son as they watched the tape. “You gotta fire off!”

Nick’s response wasn’t what Mathews expected: “Dad, I don’t even remember the play.”

The boy, Mathews realized, had likely suffered a concussion.

Dean Mathews and his son, Nick. After Nick got a suspected concussion, Dean latched onto the Virginia Tech safety ratings. Photograph by Andrew Propp.

A county police officer and youth-football coach himself, Mathews was no shrinking violet. When he coached a five-year-old Nick in a youth league, he bought an old-school, smash-mouth football manual. “If you want to win, you use it,” he says. “But you have to understand that you will piss off parents. It’s not about babying kids.”

Still, with concussions dominating the national conversation about football, he’d been worrying about brain trauma all the same. He’d even tried to do something about it. Mathews had learned about a group of scientists at Virginia Tech who ranked helmets by how well they reduced concussion risk.

Using their rankings, he’d discovered that Nick’s team was outfitted in three-star helmets—“good” by Virginia Tech’s benchmark but not the “best available” five-star. So Mathews had gone out and bought Nick a top-rated model. He had it painted in Patriot’s dark-blue colors. Early in the season, he asked school officials if his son could wear it.

Nick “was still little,” he says, “going against kids who were basically men.”

But when the school told him no, Mathews didn’t push the issue. He didn’t want to cause trouble or jeopardize his son’s starting spot. Now, watching the collision on the film, Mathews suddenly felt he’d failed his child big-time. He wished he’d taken Nick to the doctor right after the hit, and that he’d fought harder for the five-star head protection.

“At that point,” Mathews says, “I had to get him in the better helmet.”

So he became engaged in a sort of activism that’s much more quiet—and probably much more common—than today’s newsier stuff about class-action lawsuits, suicides, and furious comparisons between all-American football and illegal, back-alley dogfighting. Mathews loved the sport, after all. He genuinely believed it was good for his kid. He just wanted to make it safer.

To do so, Mathews embraced the idea of a better helmet. He began pleading and wrangling, and this time around he wouldn’t take no for an answer. It took months of calling and e-mailing and one very impolitic question posed directly to the county’s risk-management department (“What if my kid gets hurt wearing our shitty helmet?”) before reluctant Patriot High administrators relented.

In the process, they joined school officials across the country. In just three years, the easy-to-grasp system devised at Virginia Tech—known by the acronym STAR—has become a kind of Good Housekeeping seal of approval, so influential that NFL teams now post the helmet rankings in locker rooms and manufacturers alter their designs to earn better scores. Prodded by alarmist local-media investigative reports, school systems have promised upgrades.

As a result, there’s been a shift in helmet sales, says Mike Oliver, executive director of the nonprofit National Operating Committee on Standards for Athletic Equipment. “I’m getting regular calls from athletic directors, parents, even players in some cases,” he says. “They tell me, ‘We’re demanding that our school system buys five-star helmets.’ Or I hear from the athletic directors saying, ‘What do I do? The parents are up in arms wanting five-star helmets.’ ”

What almost no one realizes, however, is that while Virginia Tech’s starring concept might come across as simple, the science behind it is decidedly not—and it’s far from settled. While Dean Mathews was butting heads with his son’s school, researchers who study brain trauma were battling over whether the ratings system he embraced was the closest thing yet to a panacea for the concussion crisis or something statistically meaningless.

And the dilemma that confronted Mathews two years ago—can I be a good dad and a football dad at the same time?—isn’t resolved. It’s just a lot more scientifically complicated.

Scientist Stefan Duma tests football helmets in the Virginia Tech lab where he pioneered his influential safety rating system. Photograph courtesy of Virginia Tech.

Stefan Duma, the researcher behind the STAR helmet-rating system, presides over a Virginia Tech lab that’s part museum, part torture chamber. A gas-powered piston and two metal anvils, used to punch and clobber rubber skulls, sit against one wall. Another is lined with dozens of helmets, including two old-school leather models from the 1950s. “We tested those,” Duma says. “Our system predicted that wearing them would result in 50 concussions per season. Per person.”

In 2007, Duma, a professor who runs the Virginia Tech-Wake Forest University School of Biomedical Engineering and Sciences, got a call from Lester Karlin, the longtime equipment manager for Virginia Tech’s football team. This was well before the NFL was forced to admit that nearly one-third of its retired players are likely to develop long-term neurological problems, including Alzheimer’s. But even then, behind closed doors, there was already growing concern about concussions. Hence the simple question Karlin had for Duma: Which helmet should we buy?

“I didn’t know,” Duma says. “There was no good information out there, nothing anybody could point to. That’s when we started our research.”

No helmet can prevent concussions. To understand why, imagine an egg wrapped in thick foam and surrounded by a sturdy plastic case. Strike the case with a baseball bat and both case and shell remain intact. The yolk, though, jiggles. The brain behaves the same way in an on-field collision. Neither the helmet nor the skull has to crack for the delicate, gelatinous mass floating inside to concuss.

Football is a game of perpetual yolk scrambling. And Duma knew concussions couldn’t be engineered out of it, any more than broken bones could be engineered out of car accidents. Still, he wondered if the right technology could lower the odds of harm.

Duma was uniquely positioned to look into the question. Safety is his specialty: He earned his doctorate researching side-mounted automotive air bags. Since arriving at Virginia Tech in 2000, he has worked on crash-proofing Black Hawk helicopters for the Army. But there was another reason, too. At the time of the equipment manager’s call, Duma was sitting on a few years’ worth of collision data from the school’s practices and games. Back in 2003, he had outfitted the players’ helmets with accelerometers, largely as a way to feed the team doctor real-time information about high-force impacts but also because, he says, no one had done a large-scale study of head impacts in football players by that time. This effectively gave Duma a database cataloging hundreds of thousands of hits.

Duma and other researchers knew that acceleration to the head correlates with brain injury. In general, the higher the former is, the more likely and severe the latter will be. The question facing Duma: How did the different rates of acceleration caused by different hits—helmet-to-helmet ones, whiplash-inducing blindside tackles, and so on—correlate with concussions? Was there a specific kind of hit that brought a higher probability of injury? Which were the riskiest?

Duma and his team dug into their database, which contained the location of and acceleration caused by each hit, plus a time stamp as well as records of concussions that had been diagnosed by Virginia Tech’s medical staff. And they examined the specific on-field hits that directly preceded those injuries. In an average hit, they discovered, players’ heads were subjected to 20 g-force units, or “Gs”—the rough equivalent of what a driver would experience in a low-speed fender-bender. But many hits in Duma’s data trove registered at 100 Gs or more. “We started to zero in on when most of these concussions happen, which is 80 to 120 Gs,” Duma says.

By comparing those hits with the ones that didn’t appear to cause concussions, Duma’s team created a mathematical model that predicted concussion risk. Their next step was to determine which helmets, if any, did a better job of mitigating the effect of those most dangerous hits. That’s where the anvils came in. Duma and his group placed different helmets onto dummy heads, then dropped them onto the anvils—120 times for each helmet, from different angles—while sensors measured the impacts.

Duma saw that different helmets that were dropped in the same way clocked in at different rates. Some never got up to that 80-to-120-G danger zone. Others did. He now believed he had an answer to Karlin’s question: Some helmets were definitely safer than others.

Duma and his colleagues ultimately came up with a STAR value (short for Summation of Tests for the Analysis of Risk), a metric that predicts the number of concussions a player could suffer in a season of college ball. Helmets with STAR values below 0.3—that is, fewer than 0.3 concussions per player per season—receive five stars; those that score below 1.0 but above 0.69 get one star. Helmets with STAR values at or above 1.0, or one concussion per season, receive zero stars and a “not recommended” rating.

What the Stars Mean
Virginia Tech professor Stefan Duma helped pioneer the STAR helmet rating system and published the first rankings in 2011. His lab outfits rubber dummy heads in helmets, then drops them onto anvils from different heights. Each helmet takes a total of 120 hits, in five spots on the head (the front, the top, the back, and each side). The testing is based on complex mathematical models derived by analyzing actual concussion and collision data from college practices and games, allowing Duma—a professor of biomedical engineering—to predict the likelihood of a concussion while a player is wearing a specific helmet. He then translates that probability into an easy-to-read starred score. Here's how 26 models ranked this year.
5 Stars "Best available"
0 to 0.299 predicted concussions per player per season
Riddell SpeedFlex, Schutt AiR XP Pro VTD, Schutt Vengeance VTD, Riddell 360, Rawlings NRG Quantum Plus, Rawlings NRG Tachyon, SG Version 2.0, Xenith EPIC, Xenith X2, Xenith X2E, SG Version 2.5, Riddell Revolution Speed
Four Stars "Very good"
0.3 to 0.399 predicted concussions per player per season
SG Version 1.0, Schutt ION 4D, Rawlings NRG Impulse, Xenith X1, Riddell Revolution, Rawlings NRG Quantum, Schutt Vengeance DCT, Riddell Revolution IQ
3 Stars "Good"
0.4 to 0.499 predicted concussions per player per season
Schutt AiR XP, Schutt DNA Pro+,Schutt AiR XP Ultra-Lite
2 Stars "Adequate"
0.5 to 0.699 predicted concussions per player per season
Schutt AiR Advantage
1 Star "Marginal"
0.7 to 0.999 predicted concussions per player per season
Riddell VSR4
NR "Not recommended"
One or more predicted concussions per player per season
Adams A2000 Pro Elite

“There’s a ton of science in the system,” Duma says. “The risk functions, the equations, the [player] exposure [to hits], the mapping in the lab—it’s very, very complicated. It spits out a simple answer on purpose, so people can understand that five-star is better than two-star.”

When the university released its first rankings in May 2011, the industry was skeptical. Manufacturers such as Schutt and Xenith were withering in their response. “Do not believe any overly simplistic claims, any short story,” Xenith’s CEO said in a statement, “and do not rely on anyone’s granting of stars. Stars are for kindergartners.”

Only one helmet, a Riddell, received five stars.

Just three years later, helmet makers have reversed themselves. Today they’re teaching to the Virginia Tech test, as it were. The newest models have larger shells and more interior padding, two design modifications that do more to reduce the force from high-impact hits.

And sure enough, in the university’s most recent rankings, Duma notes, 12 helmets received top scores—including three from Xenith, one of the big early critics. “The companies use the formula to make better helmets,” he says. “The differences are so dramatic—150 Gs in a no-star helmet versus 75 Gs in a five-star. I challenge any scientist to tell me that there is no difference.”

But no one is especially interested in grilling Stefan Duma about g-force statistical measurements in this moment of concussion panic. Instead, they want to see him—or at least his ratings—deployed to hound the coaches and administrators who oversee football.

One week this fall, two high-school players in Alabama and New York died during games. Last year, eight high-school players died, six from head injuries and two from neck damage. In 2012, autopsies on six dead teenage players showed signs of chronic traumatic encephalopathy (CTE), a neurodegenerative illness linked to repetitive blows to the head.

For some parents, the drumbeat of grim news means rethinking whether to let their kids play. But for others—last year, after four years of declines, the total participation in the high-school game ticked up—it has prompted a search for innovation. There’s a new cottage industry of equipment that purports to improve gridiron safety. Parents today can buy their child a sensor-equipped skull cap or chin strap that measures head acceleration and flashes a red light after hard hits. There’s even a helmet insert made of Kevlar.

And wherever the media raise alarms about safety gear, they cite STAR. Television stations in Indianapolis, Detroit, Dallas, Cleveland, Minneapolis, and Charleston, South Carolina, have all effectively burnished the system’s image by airing breathless investigative reports shaming schools for using low-scoring equipment. HELMET WARNING, read the headline for the Detroit report, introduced by an anchor promising “frightening facts” about schools using helmets rated “marginal.”

“What you don’t know about your kid’s helmet, some experts say, could come back to haunt you,” an anchor in Dallas intoned.

After Washington’s WUSA9 identified Arlington Public Schools as having a number of two-star models in its inventory, officials spent $68,000 to replace 308 of the district’s lower-ranked ones with a five-star Riddell. That was the right play, in Duma’s eyes. As he told the Indianapolis TV station, if schools can’t afford to pony up for the top-rated models, they “can’t afford to play football. It’s that simple. I don’t want to be the principal that says it’s okay to play in a one-star helmet.”

Neither Duma nor Virginia Tech makes money off the ratings, but both are getting plenty of acclaim—which in turn helps them launch new research ventures. By 2015, Duma’s lab plans to release hockey helmet ratings. Lacrosse, softball, and baseball eventually will follow. In September, the university announced that it was one of 16 chosen for a $30-million sports-concussions research project funded by the NCAA and the Department of Defense.

“For other sports, there’s dramatic room for helmet improvement,” Duma says.

Earlier this year, Duma and 14 colleagues published their largest study to date, in the Journal of Neurosurgery. Analyzing roughly 1.28 million blows to the head, they found that players wearing the one-star VSR4, made by Riddell, suffered a concussion rate more than twice as high as players wearing a four-star Revolution, also from Riddell.

The results seemed to validate STAR. Compared with the VSR4, the Revolution reduced risk by 53.9 percent; the earlier testing on dummy heads in Duma’s lab had predicted a 54-percent reduction. “We were surprised it came out so close,” Duma says. “That’s pretty amazingly accurate.”

But to a man named Don Comrie, the numbers looked off—way off. Comrie is CEO of PanMedix, a New York City company that helps drug firms design clinical trials and offers tests for concussions and dementia. He previously had studied brain injuries among American soldiers in Iraq and had consulted for the NFL Players Association. He knew his way around an academic study.

When he read Duma’s paper this year, one statistic stood out. Duma and his colleagues had tracked 1,833 players at eight colleges over six seasons. During that time, those athletes were diagnosed with just 64 concussions total. That’s roughly 1.3 concussions per team per season—a rate that struck Comrie as preposterously low.

Football, Comrie says, doesn’t have just a concussion problem. It has a concussion-counting problem. The reasons are many. The injury can be difficult to spot, because no single reliable diagnostic test exists and also because symptoms can be subtle or have delayed onsets. Moreover, pro and college teams often refuse to disclose concussions, and players at every level consistently hide the injury, both to abide by the sport’s macho, play-through-pain ethos and to avoid losing playing time, scholarships, and salaries.

How many football concussions go unreported to team officials? Estimates vary from one in two to nine out of ten. According to a Harvard and Boston University study released in August, college players on ten teams sustained six suspected concussions and 21 dizziness-inducing “dings” for every one concussion that was officially diagnosed, indicating that more than 80 percent may not have been disclosed. Last year, a study at Cincinnati Children’s Hospital Medical Center found that only 54 percent of high-school players would “always or sometimes” report concussion symptoms to a coach.

But even if everyone were accurately reporting injuries, Duma’s 1.3 concussions per team per season was still low. A survey of players from 25 colleges that was published in 2003 revealed a concussion rate twice that of Duma’s. Between 2000 and 2011, Columbia University’s football team counted more than 70 concussions, or at least 6.36 per season. “That’s five times more than [the Duma study],” Comrie says. “Something is wrong with his numbers.”

If Virginia Tech researchers are accepting concussion data from schools at face value, then they may not be accounting for all injuries, and thus the STAR system potentially has a fatal flaw. “Garbage in,” Comrie says, “garbage out.”

He explains it like this: Suppose you’re a researcher, and suppose the football concussions that schools are diagnosing and reporting to you are the obvious ones, big knockout blows that tend to happen when two players hit head-on and helmet to helmet, at high speed. You design your risk equations and lab tests accordingly, while helmet makers work to better protect players against those particular types of hits. All of this is good.

Now suppose schools are failing to diagnose and disclose concussions that are occurring, say, during lower-impact, glancing blows to the side of the helmet. If that’s the case, then what do your math models really measure? Are any of the helmets that perform well in lab tests actually safer in the real world? As Comrie puts it, “It’s like building on a sinkhole.”

“They started with a good premise,” Comrie says. “There’s a need for determining whether one helmet is better than another for concussions. But anything [the STAR system] says has to be couched in terms of incomplete data, probably inaccurate data, and data that may have no meaning.”

The lab at Virginia Tech's machinery can simulate traumatic tackles. Photographs courtesy of Virginia Tech.

Rival football players pummel each other by going helmet to helmet. Brawling scientists go head to head by firing off strongly worded letters to the editors of academic journals. That’s how Comrie and former NFL player Sean Morey took their beef with STAR public this past June, in the Journal of Neurosurgery. “We find it astonishing that [the journal] continues to publish articles on football concussion that only serve to retard scientific progress,” they wrote.

Duma and his colleagues fired back with a letter of their own, defending their study and repeatedly dismissing their opponents’ assertions as “wrong.” That triggered a second, harsher letter from Comrie and Morey, this one posted online, containing a long list of pointed critiques—including the fact that STAR’s development was based in part on data gathered by a since-discredited group of NFL-affiliated scientists.

Duma acknowledges that concussion underreporting is a problem across foot-ball. Nonetheless, he insists his research is sound. “The core argument,” he says, “is still that helmets that lower head acceleration will be better.”

Tom McAllister, an Indiana University psychiatry professor who has studied brain-injury recovery for more than 25 years and coauthored papers on STAR with Duma, says the safety rating program is definitely an improvement. “Is there a difference between helmets? The data suggests that there is,” McAllister says. “I would use that information if I was buying helmets for my kids.”

The problem is that some of STAR’s critics aren’t just obsessing about whether colleges report concussions accurately; they’re focusing on the head accelerations themselves.

Blaine Hoshizaki, director of the Neuro-trauma Impact Science Laboratory at the University of Ottawa, who also studies head impacts in football, says the problem with STAR is that its tests don’t do a good job of simulating some of the situations most responsible for concussions.

“You get hit in the head hard, you get a concussion,” Hoshizaki says. “You get hit harder, you get a catastrophic injury. That is what people say. But these injures are very sensitive to how you get hit.”

Football collisions produce two types of head acceleration, linear and rotational. Picture what happens when a boxer punches an opponent directly in the nose with a jab: His rival’s head pops back, basically in a straight line. That’s linear acceleration to the brain. Now picture a boxer throwing a left hook and hitting his opponent in the jaw: The blow comes from off center, and the head and brain spin and twist. That’s rotational.

As it turns out, spongy brain tissue is particularly vulnerable to insult via rotational acceleration. “It’s very hard for a boxer to knock someone out with a straight punch,” Hoshizaki says. “But a hook works.”

And here’s the thing: The STAR system doesn’t measure rotational acceleration, only linear. That means that the possibly more dangerous hits, according to Hoshizaki, aren’t even being accounted for.

Here’s the other thing: The kind of helmet designed to reduce linear acceleration—bigger, heavier, and with thicker interior padding—may also be the kind of helmet that increases rotational acceleration. Unfortunately, this is exactly what manufacturers have been making in order to improve their STAR scores.

“If you make a big, fat, soft helmet, you’re not just creating low linear acceleration,” says Hoshizaki, who helped design a Xenith helmet that received four stars in Virginia Tech's inaugural rankings. “You’re making a huge helmet that will increase the risk of getting hit and may create higher rotations. I don’t know if that’s a safer helmet.”

As soon as next year, Duma says, the STAR system will take rotational acceleration into account. While acknowledging criticism, he insists the current scores are valid and argues that an imperfect measure of risk is better than none at all—if automotive researchers hadn’t pushed ahead with seat belts and air bags decades ago despite a lack of scientific certainty at the time, he says, then important safety advances never would have been made.

“I think we are at 90 percent of where we are going to be,” Duma says of the potential for any further improvement in football-helmet safety. “About as good as we can get.”

When I tell Dean Mathews what I’ve learned about the Virginia Tech stars—that smart, well-meaning scientists are having a fierce debate over their reliability—he sounds surprised. First he’s heard of the fight, he says. “For me, the STAR system was a place to start,” he adds.

Nick is 17 now: two inches taller, 20 pounds heavier, and the leading high-school receiver in the Washington area.

It’s cool and dry on a Friday evening in Nokesville, with a brushstroke pink sunset fading into twilight. It seems the entire town has come to watch Nick and Patriot High take on visiting Hylton High from Woodbridge: young boys throwing wobbly passes behind the end zone, JV players guzzling Muscle Milk while wearing T-shirts that read pain is temporary, pride is forever, entire families clanging red-white-and-blue cowbells.

“He talks trash,” Dean Mathews says from his perch in the stands. “He’s not scared to death anymore.” Then the dad winces. Down on the synthetic turf, his son has just made a balletic sideline catch—leaping straight up, arms extended, football on his fingertips—that ended with a defender driving his helmet into Nick’s ribs. “He’ll be sore tomorrow,” Mathews says.

Nick catches a touchdown pass, his second of the game, and Mathews zooms in with his camera, pumping his fist. “I guess his ribs are all right,” he says. He thumbs his smartphone, and up comes a video of his son, silently colliding with a linebacker, spinning and falling. It’s the helmet-to-helmet hit from Nick’s sophomore year, the one that left the boy dazed and his father thinking: Oh, my God.

Nick, Mathews tells me, hasn’t suffered a concussion since.

“I believe the helmet is doing its job,” he says. “Better than what was on Nick’s head before.

“But I’m not a scientist. And I don’t even know if they know. I think there’s an element of faith here.”