• Number 423  |
• September 29, 2014

Supercomputer turns the tide for consumer products research

Does your shampoo bottle say “Shake well before using”? Is your shower gel thin and runny? If you answered yes to these questions, you can begin to appreciate the challenges facing the developers of diverse consumer products. Companies need research and development tools that can accelerate the creation of globally competitive products. To that end, consumer-products giant Procter & Gamble (P&G) has turned to DOE's Oak Ridge National Laboratory and America’s fastest supercomputer to simulate microscopic processes that can threaten product performance and stability.

For this Fortune 50 firm founded in 1837, the stakes are high. Procter & Gamble generated than more $85 billion in revenue in 2013 from the global sale of everyday products like lotions, shampoos, toothpaste, and soaps, whose success in competitive markets depends on outstanding product performance. Downy, Head &Shoulders, Olay, and Crest are just some of P&G’s 22 pillar brands each generating more than a $1 billion annually.

To maintain brand dominance and generate new market leaders, P&G researchers need a deeper understanding of their product formulations. Whereas many product attributes are best understood through study of formulations at the molecular level, interesting and important molecular characteristics often cannot be observed experimentally. So researchers at P&G with colleagues at Temple University turned to ORNL’s Jaguar supercomputer, one of the fastest supercomputers in the world at the time (and since upgraded to become the Titan system), in an attempt to model and simulate microstructural arrangements of molecules that dictate the flow, thickness, performance, and stability of P&G products.

Using computers to improve the way products are made is nothing new for the company. “P&G fully understands the value of modeling and simulation,” said Tom Lange, associate director of modeling and simulation at P&G. “Over the last 15 years, researchers responsible for products like Pampers have used modeling and simulation to reduce, by over 50 percent, the number of experiments involved in their process design. We would like to achieve the same goal when it comes to designing formulated products like shampoos and fabric enhancers. Already, modeling and simulation has significantly improved the way our formulated products designers interact with our suppliers.”

Extraordinary science for ordinary products

Many of P&G’s products for the body and the laundry are comprised of systems of fat-soluble molecules called lipids. While some lipid types are prevalent in nature, for example making up a significant portion of our cell membranes, other lipid types play a major role in formulated products by, among other things, aiding in product deposition on skin, hair, and fabric. “For P&G, the motivation is to understand how products that are based on lipid systems perform and age,” explained P&G research scientist Russell DeVane. “To a large extent, whether you are designing formulated products or trying to understand nature, the same fundamental questions apply.” Many of these questions relate to the formation and time evolution of lipid vesicles.

Lipid vesicles are relatively large aggregates which form when lipid bilayers “roll” into spheres. Vesicles can also encircle each other like Russian nesting dolls. In the body, vesicles transport insulin, neurotransmitters, and other vital substances in their interiors and then fuse with cell membranes to deliver the contents. In manufactured products such as lotions and fabric softeners, man-made vesicles called liposomes encapsulate and suspend perfumes, dyes, and active ingredients. Over time vesicles can fuse with each other, causing phases to separate and otherwise threatening consistent product performance.

That concerns P&G. “Vesicle fusion is a fundamental problem that impacts many fields of research from neuroscience to physiology,” said Temple’s Michael Klein, a National Academy of Sciences member who has collaborated with P&G for 15 years and who led the simulation project. In addition to DeVane, Klein’s research partners include Temple’s Giacomo Fiorin and Axel Kohlmeyer and P&G’s Peter Koenig and Kelly Anderson.

“For Procter & Gamble, it is crucial to understand vesicle fusion if you want to extend the shelf lives of such products as fabric softeners, body washes, shampoos, lotions, and the like,” Klein continued. “Vesicle fusion is a very hard science problem.”

So hard in fact that related studies were linked to two Nobel prizes in 2013. A Nobel prize for chemistry was awarded for developing multiscale computational models of complex systems—models that Klein and team use for their P&G-related studies. And a Nobel prize for physiology was awarded for discoveries regarding how vesicle “traffic” is regulated in cells.

Compute to compete

Moreover, problems can pop up when perfumes or dyes are added. The addition of a lemon-fresh scent, for example, may alter the molecular structure of a formulation and turn a gel into a liquid, or vice versa. It takes time and money to design, manufacture, and test a formulation, only to find a flaw that forces an ingredient’s removal, replacement, or concentration readjustment.

“There’s a significant cost associated with stability problems,” DeVane said. “Despite our experimental and historical knowledge, we have no way of knowing when we’re going to have those problems. It would improve the bottom line if we had a better fundamental understanding of what’s going on.” One major cost is the time it takes to bring new formulations to market. Stability issues can double or triple project timelines. Fixing them often results in more expensive formulations or more expensive manufacturing processes. Slower delivery of products has significant market opportunity costs and deflects R&D effort otherwise intended for new technology discovery and development.

To speed discoveries about why vesicle systems have stability problems, the researchers knew they would need to simulate the reorganization of vesicles over time. Before vesicles fuse, their surfaces strain and break, exposing water-avoiding parts of molecules to an aqueous environment. Following every molecule in a system of fusing vesicles as they contort and combine would be a herculean feat. If only P&G researchers could use such simulations to predict performance attributes before a product is manufactured, they would be able to deliver high-quality products faster than competitors and earn greater revenues in the marketplace.

But the magnitude of the challenge exceeded the capabilities of even P&G’s powerful supercomputers. “We need extremely long simulations of extremely big systems,” DeVane said. “There are very few places where we can accomplish that.”

The need for significantly more computing capability drove the P&G funded-team of Klein and researchers from Temple University and P&G to apply for time on ORNL’s Jaguar. They obtained access through the Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program, which is jointly managed by the U.S. Department of Energy’s (DOE’s) Leadership Computing Facilities at Argonne and Oak Ridge national laboratories. “P&G won an allocation through the highly competitive INCITE program, the major means by which the scientific community gains access to our Oak Ridge Leadership Computing Facility,” said Suzy Tichenor, director of industrial partnerships for ORNL’s Computing and Computational Sciences Directorate.

The team used 69 million core hours on Jaguar over two years setting the stage for simulations of large, complex systems of lipid assemblies. To date no one has ever been able to simulate vesicle fusion in realistic systems, such as biological vesicles whose fusion is triggered by proteins. Klein and collaborators tried to trigger fusion of two 20-nanometer vesicles another way—by applying an electric field—and were similarly unsuccessful. The electric field gave the vesicles some kick, but not enough for fusion.

With Jaguar’s computational power, however, the researchers succeeded in artificially triggering the deformations of a single vesicle that happen before it fuses with another vesicle. The mathematical model applied an electrical field oscillating at 10 MHz to one vesicle in a simulation that tracked the movements of the 5,000 lipid molecules in that vesicle.

“It is truly a challenging problem,” DeVane said. “Sometimes the difficulty of a problem forces you to try things you might not normally consider. These simulations allowed us to investigate the very early events associated with the initiation of vesicle fusion. If you want a complete mechanistic understanding of a process, these details and the capability to study them are extremely important.”

This benchmark simulation allowed comparisons to experimental data that verified the simulation and the interpretation of the experiment. The simulation showed the induced dipole, the direction and magnitude of the electric field, and the deviation of the distorted vesicle’s shape from a perfectly spherical shape.

Race to revenue

To speed simulations Klein and several coworkers, notably DeVane, John Shelley (now at Schroedinger Inc.), and Wataru Shinoda (now at Nagoya University), developed a simplified coarse-grained model. Three strategies sped number crunching by a factor of 1,000, according to Temple’s Fiorin. First, the coarse-grain model accounted for only one of every ten atoms. Simulating this simplified system was computationally cheaper and faster than running a fully atomistic model. Second, mathematical approximations weighted nearby influences more than those farther away. Third, instead of running the simulation on 22,500 cores of all-CPU Jaguar continuously for a month, a GPU-equipped Titan prototype, TitanDev, that more fully exploited parallel processing shrank running time to 2 weeks. At Temple, where researchers share an 800-core cluster, the same simulation would have taken 5 or more years, Fiorin said.

Moreover, scientific-application-code changes sped the simulations. The researchers used an enhanced version of the LAMMPS molecular dynamics software code that Kohlmeyer and students from Temple’s Undergraduate Research Program developed. Its multi-threading technology made LAMMPS work in a more parallel fashion and overcame limitations due to message passing. With these changes, simulations efficiently used six times as many of Jaguar’s CPUs.

Further code changes harnessed the power of GPUs. The P&G group routinely uses LAMMPS in addition to another molecular dynamics code, NAMD. Whereas NAMD is better at simulating biological systems in full atomistic detail, LAMMPS is more suitable for coarse-grained simulations of large systems, explained former OLCF computational scientist W. Michael Brown, now of Intel. As the OLCF liaison for the collaborators, Brown aided in troubleshooting early software for Jaguar runs and supported development of LAMMPS for GPUs for TitanDev runs.

“The great thing about the GPU is not only the improved performance on Titan but also the benefit we get with our in-house computing resources. With a single GPU-enabled compute node, we are able to get the same performance we were getting out of 10 traditional compute nodes,” DeVane said. “This greatly extends our in-house resources. We can run more jobs at one time and get faster throughput. Moreover, the in-house work has relevance to the Titan work since it helps us more quickly set up simulations in house for production runs on Titan. Large-scale simulations provide the insights needed to develop simpler methods of measuring critical underlying physical and material properties.”

Next steps

All the computational resources of P&G working together couldn’t make progress on this problem. That doesn’t mean the problem isn’t worth solving. Rather, it means leadership computing facilities would be needed for simulations capable of showing how microscopic properties of vesicles influence macroscopic properties, such as phase separation as a product ages. Gaining that type of knowledge would have untold long-term benefits as it could help the company wash its hands of potential poor performers and more quickly embrace formulations that are more likely to grow its fortune. – Dawn Levy

https://www.olcf.ornl.gov/2014/08/25/oak-ridge-supercomputer-turns-the-tide-for-consumer-products-research/

Submitted by DOE’s Oak Ridge National Laboratory