SCIENTIFIC COMPUTING AND IMAGING INSTITUTE
at the University of Utah

An internationally recognized leader in visualization, scientific computing, and image analysis

by Gregory Scott Jones - NICS

BerzinsFig1
The crater resulting from the Spanish Fork Detonation.
First, the bad news: all across America, trucks and tractor-trailers are transporting industrial explosives on nearly every artery of the country's interstate and highway system. That's right, volatile explosives, including munitions, rocket motors, and dynamite, are moving at a high rate of speed down a roadway not too far from you.

Now, the good news: America's track record in transporting these materials is about as safe as they come. Very rarely, almost never in fact, are the potential dangers of these transports realized, largely due to instituted safeguards that seem to work very well.

However, accidents can happen. Take the August 2005 incident in Spanish Fork Canyon, Utah, for instance. A truck carrying 35,500 pounds of explosives—specifically small boosters used in seismic testing—overturned and exploded, creating a crater in the highway estimated to be between 20 to 35 feet deep and 70 feet wide according to the Utah Department of Transportation. But the damage wasn't solely financial. Four people, including the truck driver and a passenger, were hospitalized.

Thanks to the University of Tennessee's (UT's) Cray XT5 Kraken supercomputer and the hard work of a research team from the University of Utah, accidents such as the one in Spanish Fork Canyon may soon become better understood, and hopefully, one day, a near impossibility.

The team, led by Martin Berzins and Charles Wight, is using the processing power of Kraken to simulate burning and detonation processes in transportable explosives in hopes of one day making catastrophes like the one in Spanish Fork Canyon a thing of the past. Recent simulations have led the team to believe that the key to preventing these types of detonations is to pack the trucks transporting the explosives in such a way that temperatures and pressures are vented so as to avoid a chain-reaction explosion-to-detonation scenario.

Theoretically, the boosters in Spanish Fork Canyon should have just burned out or exploded one by one, said Wight. However, as evidenced by the size of the crater left at the scene, they instead detonated, with all of them simultaneously exploding and causing maximum damage.

The difference in burning—or deflagration—and detonation is a crucial one. In deflagration the rate of burning is strictly limited by the transfer of heat to each individual device. Detonation, however, is a phenomenon that occurs as the result of a shock wave that moves 1 million times faster than deflagration and in the end is approximately 1 million times more devastating.

This difference was evident in the Spanish Fork Canyon explosion. It is almost certain that the deflagration-to-detonation that occurred resulted from numerous separate devices being involved as opposed to one large device. Nearly all current knowledge surrounding explosions has evolved from experiments performed on single devices or at most three related devices. Berzins and his team are pioneers in the field of explosive arrays—the simultaneous explosion and resulting detonation of multiple devices in a single area.

"We want to really understand the nature of this kind of accident and devise ways to design trucks and package the explosives so that this doesn't happen again," said Wight regarding the Utah incident that spurred their explosives transport research.

Thanks in part to Kraken, the team has developed models for the transition from deflagration to detonation, and it is quickly reaching the point at which it can begin to apply these models to the full problem.

BerzinsFig2
An array of 64 explosive cylinders ignited in the lower left corner undergoing deflagration (left) and detonation (center) with an individual container fragmenting (right).


Team members are presently in the process of scaling up the models for explosions and detonations. That scaling up means simulating whole truckloads. For now the team is simulating only a few explosives at a time, a daunting task even with Kraken's 100,000-plus computing cores. This computing power especially comes in handy when dealing with detonations, in which the team believes that the phenomenon ultimately results from very fine-scale behaviors. "It's almost down to imperfections in the explosive where you can get high pressures and temperatures forming," said Berzins. "To properly model a full detonation, we have to get to the micrometer scale to model what's happening inside individual explosives."

This modeling across scales, from the micrometer inside explosives to the crater in the ground in Utah, is what the team refers to as "continuum-level science." Because of the relationships between the different scales, this type of problem specifically requires a system such as Kraken, which has the necessary processing power for simulating explosions and detonations from their microscopic beginnings to their larger cataclysmic finales.

Years in the making The University of Utah team used 3.5 million hours on Kraken in 2010 alone. The team previously used the National Science Foundation's PetaApps program to get its simulations to their current fidelity.

The team's weapon of choice in its quest to model a full-truckload detonation is Uintah, a very general-purpose scientific computing and engineering software. Developed with support from DOE, the Department of Defense, and the National Institutes of Health, Uintah is an all-purpose application that is currently putting out proverbial fires in areas from fluid flow to heat transfer to biochemistry to materials. In Berzin's case, it is excelling in every sense of the word.

"Uintah is scaling very, very well, up to our biggest runs of 100,000 cores," he said. "We've run our software on almost all of Kraken in a scalable way with good results with adaptive meshes—where all the action is, where there are high gradients of pressure and temperature." Those gradients require enormous computing power and are a major reason why Berzins and his team require a computer as powerful as Kraken to continue to move forward.

And moving forward they are. Not only does the team have codes that can use almost all of Kraken, therefore enhancing accuracy and expediting time to solution, but it has also developed fundamental science thanks to previous simulations and experimentalists. Furthermore, the simulations have given the group the computational knowledge to expand its algorithms to ensure that its codes continue to scale to tomorrow's even larger computing architectures. Applying models from numerous scientific arenas and using state-of-the-art computational science, Berzins estimates that the team will need an additional 2 years "to really pin this one down."

At that point maybe we'll all be able to rest a little easier knowing that although accidents will always happen, 30-foot craters and injured bystanders will possibly be no more than yesterday's news.

About NICS: The National Institute for Computational Sciences (NICS) is a joint effort of the University of Tennessee and Oak Ridge National Laboratory that is funded by the National Science Foundation (NSF). Located on the campus of Oak Ridge National Laboratory, NICS is a major partner in NSF's Extreme Science and Engineering Discovery Environment (XSEDE).