Computational Engine Modeling
By Osman Yasar


Osman Yasar examines results of a computer simulation of a “cold-flow” experiment in which two pistons compress cold gas, slightly increasing its temperature in various regions. The study is part of a computational effort to design engines that use fuel more efficiently and produce lower emissions. Photograph by Tom Cerniglio.

The advent of massively parallel computers makes it possible to solve engine combustion problems in minutes, not months. ORNL has adapted the latest version of the KIVA engine simulation code for use on parallel computers. It should help American vehicle manufacturers design highly efficient and environmentally friendly cars more quickly.

American automakers must design more efficient, cleaner burning cars than ever, and quickly. The 1990 amendments to the Clean Air Act require a large reduction in pollutant emissions from U.S. cars and trucks by 2000, and the Comprehensive National Energy Policy Act of 1992 exerts pressure on vehicle manufacturers to improve fuel efficiency. At the same time, market competition demands a faster turn-around time than ever before for new vehicle designs.

Fortunately, American vehicle manufacturers are in a position to take advantage of new lightweight materials and better fuels, electronic controls, and ignition systems; and they have new tools to help them keep pace with government and market demands. One of these tools is computational engine modeling, in which manufacturers test new engine components on computers before creating expensive prototypes. Furthermore, the advent of massively parallel computers, such as ORNL’s Intel Paragon XP/S 150, has now made it possible to solve car-crash and engine combustion problems in a reasonable time—in minutes and hours, not days and months.

Internal combustion engines, which power most vehicles, are extremely complex energy systems. An internal combustion engine burns fuel within a group of cylinders containing movable pistons; the gases formed in combustion push the pistons, which ultimately turn the car’s wheels. The operation of these engines involves the coupled phenomena of combustion, turbulent fluid flow, turbulent flame propagation, radiative heat transfer, ignition and extinction, pollutant formation, and wall heat transfer—and in diesel and fuel injection engines, spray dynamics. Those phenomena are characterized by a number of different time and length scales.

Engine Simulation Codes Aid Efficiency

Because of the extreme computational demands, engine combustion modeling has been identified as a Grand Challenge problem (a complex, difficult problem that cannot be solved without the use of high-performance supercomputers). The goal of the modeling is to determine whether new designs will improve fuel efficiency and reduce emissions.

One of the most powerful multi-dimensional engine simulation codes is KIVA and its offshoots, KIVA-II and KIVA-3, which were developed by DOE’s Los Alamos National Laboratory (LANL) originally for CRAY computers. The success of KIVA simulations has led to wide use of the code in the past decade, and it has been implemented on other platforms by various institutions. At ORNL we developed a scalable distributed-memory parallel version of KIVA-3 that is proving to be very useful.

One typical KIVA simulation of just one engine cycle takes about 30 hours on mainframe supercomputers such as the CRAY Y-MP system. A processing time this long is not acceptable to the engine industry. Many medium-sized companies do not even have access to such computers. They typically do all of their engine development using simplified, zero-dimensional tools on PCs or workstations. It is hoped that ORNL’s parallel implementation of KIVA-3 on distributed-memory parallel computers or a cluster of workstations will reduce the time required for these simulations. Such parallelization would not only allow modelers to introduce more variables into their calculations to achieve higher grid resolution, but also provide them a scalability in performance and memory that is important to tackle large-scale, complex problems.

Involving a great deal of physics, KIVA analyzes coupled fluid dynamics, fuel spray dynamics, combustion and pollutant formation reactions, and heat transfer in an engine cylinder. It allows engine designers to see the effects of alterations to an engine’s geometry without actually building the engine. The user can combine the results with a computer graphics package to visualize the combustion process. The user can see how the fuel-air mixture is initially ignited and how the flame grows from the initial ignition point, spreading throughout the combustion chamber. Predictions of nitrogen oxide levels or other hazardous emissions can be obtained for optimum engine conditions.

KIVA has been a subject of much research and constant improvement since its first release in 1985. Besides the work by KIVA’s original authors, new submodels have been developed by other groups such as the Engine Research Center at the University of Wisconsin at Madison. Efforts are also under way at ORNL to add highly accurate spark-ignition, radiation heat transfer, and turbulence models.

The latest version, KIVA-3, addresses the inadequacies of earlier versions. It uses a block-structured grid, allowing modeling of more complex geometries. It also handles both intake and exhaust flows, permitting the computations to proceed simultaneously in the intake and exhaust manifolds as well as in the engine cylinder itself, both when each component is in physical contact with the other and when the flow paths are closed off by valve and piston motion. Using this new version, scientists from national laboratories and the automotive industry have demonstrated good agreement between the computations and actual experimental data. Thus, the new version can help the auto industry design cars that deliver increased fuel efficiency and reduced emissions.

Although the United States is a world leader in combustion technology, it is being challenged by aggressive new programs in Europe, Japan, and elsewhere. The health of the U.S. automotive industry is clearly critical to the overall economic health of the nation. The industry is faced with increasingly stringent emission and safety standards and strong overseas competition for its markets.

KIVA is helping to enhance the competitive position of the U.S. automative industry. Practically every major automotive and engine manufacturer has requested a copy of the KIVA computer code. One of the biggest users of the KIVA model is General Motors (GM), which has been heavily involved in its development since the beginning. To our knowledge, other users of KIVA include not only engine manufacturers such as Cummins Engine, Ford, Chrysler, Caterpillar, and John Deere, but also companies that make and use utility boilers in which coal is burned. In the United States, coal currently supplies 56% of our electricity and 17% of our total energy needs. KIVA is being used for coal combustion modeling needed to develop the next generation of utility boilers.

The international KIVA users group, which is headed by Rolf Reitz of the University of Wisconsin at Madison, holds its annual meetings in conjunction with annual meetings of the Society of Automative Engineers (SAE) Congress in Detroit.

To further strengthen the U.S. automotive industry’s competitive position, there are ongoing cooperative research and development agreements to advance state-of-the-art computer modeling to complement the experimentation. Participants include national laboratories (including ORNL), universities, and major car companies such as General Motors, Chrysler, and Ford. The enhanced capabilities in computer modeling using massively parallel computers are expected to help in designing new products and bringing them to market more quickly. The availability of such industrially important software tools is an indicator of the success of the federal High Performance Computing Program.

Computer Tools for Emission Control


Fig. 1. Display of the effect of spark energy on the nitrogen oxide (NOx) concentrations (g/cm3) for a baseline engine after ignition. Higher spark-energy deposition leads to an increase in the NOx level.

Developing better tools not only will help the auto industry improve fuel efficiency, but also will help it design engines with lower emissions of harmful by-products. During the past two decades, controlling emissions of nitrogen oxides (NOx) has become an issue of national importance because NOx contributes substantially to acid rain and photochemical smog. As a consequence, NOx emissions present the most widely spread detrimental impact on air quality, vegetation, and human health of any regulated emission. Current technologies to control NOx emissions either (1) modify the combustion zone in an effort to control the temperature, residence time, or stoichiometry, thereby lowering NOx emissions, or (2) use a reducing agent that reacts with the oxygen in the NOx molecule after combustion, producing nitrogen and water.

Other solutions could be developed by carefully studying the effect of input parameters on the emission, and the KIVA model might be useful in doing that. It should be noted, however, that there might be technological difficulties in controlling some of the input parameters, such as the spark energy. A recent spark-control experiment at ORNL’s Life Sciences Division (LSD) has been a source of new interest in using KIVA to study the effects of input spark energy characteristics on emissions (see Fig 1). Isidor Sauers and his colleague David Paul have recently been able to stabilize the spark breakdown voltage dramatically.


Isidor Sauers (standing) and David Paul examine results of a spark control experiment they developed at ORNL. They were able to stabilize the spark breakdown dramatically, opening new possibilities to lower emissions and engine cyclic variations.

We are all familiar with a spark discharge. You’ve felt one jab your hand after walking across a carpet. A more dramatic example is a lightning strike, which occurs when voltages become so high that the air can no longer sustain them so it starts to conduct electricity. In an automobile engine, a spark from the spark plug causes the same effect in the air-fuel mixture. The high voltage (up to 20,000 volts) from the ignition coil, which is generated in a very short time (on the order of millionths of a second), causes the electrical breakdown of the gas. The by-products of the breakdown depend not only on the intrinsic gas properties but also on random events such as the location and concentration of charged particles in the gas.

In an engine the energy from the ignition coil is transferred to the air-fuel mixture through a spark. How that energy gets dissipated (e.g., current and duration of the spark) in the air-fuel mixture can significantly affect the combustion process. Incomplete combustion leads to undesirable by-products such as paraffins, olefins, aromatics, aldehydes, ketones, carboxylic acids, acetylene, ethylene, polycyclics, carbon monoxide, hydrogen, nitrous oxides, sulfurous oxides, lead oxides (from impurities), various oxidants, and soot. To study the evolution of by-products to assess the accuracy of computational models, it is important to control in a systematic way the breakdown voltage and, hence, the energy delivered by the ignition coil to the gas.

Until now, it had not been possible to study the breakdown characteristics systematically because the randomness of the breakdown process results in breakdown voltage values that vary widely from spark to spark. Thus, determining the relationship between the input energy characteristics and output emissions has not been a topic of research. A joint effort among ORNL’s Center for Computational Sciences (CCS), Engineering Technology Division (ETD), and LSD is under way for improving KIVA-3 to probe this untapped field. This effort involves a close marriage of plasma physics and automotive engineering. Developing a new, highly accurate spark ignition model and validating it through experiments are among our primary goals. The experimental effort targets research engines employing the spark control mechanism developed earlier at ORNL.

The collaboration involving ETD (headed by Ron Graves), LSD (headed by I. Sauers), and CCS (headed by the author) is just one of the examples of ORNL’s multidisciplinary efforts in high-performance computing and high-precision spark-plug and engine diagnostics. A collaboration with General Motors and Cummins Engine on spark ignition modeling is also under consideration. As ORNL’s focal point for DOE’s high-performance computing program, CCS offers industrial researchers a mechanism through its outreach component—Computational Center for Industrial Innovation (CCII)—for taking advantage of ORNL’s modeling efforts on massively parallel computers.

Parallel Implementation of KIVA-3

KIVA-3 is a package consisting of the main code, a preprocessor, and a postprocessor. The preprocessor uses a block-structured mesh to discretize the physical domain and generates an input file for the main code. This input lists the grid point locations and the connectivity arrays that identify neighboring grids in all six directions. Each block is initially created independently using a tensor-product grid, but they are all patched together at the end with connectivity arrays describing the surrounding points for each mesh point. Because of connectivity information for each cell, the grid points do not have to be stored in any order in data arrays, making it possible to sort out the active and nonactive (ghost) cells and leading to shorter vector lengths in the computational loops.

The use of ghost cells, connectivity arrays, and cell-face boundary conditions in all directions creates a general recipe for physics, numerics, and boundary conditions that can be applied to a part of the domain as well as the whole domain, thereby providing for a convenient block-wise domain decomposition. In a block-wise distributed implementation, the ghost cells match the real cells of outer layers residing on other adjacent processors.

Spatial dependencies in the code extend only one layer in each direction and the presence of ghost cells and cell-face boundary arrays in the code is important in storing the neighborhood information that processors depend on. No dependency seems to be created through temporal differencing because variables are computed on the basis of quantities from the previous iteration or time step. Spatial differencing requires estimating variables and sometimes their gradients (diffusion terms) on the cell faces, leading to communication between adjacent processors sharing the cell face in question. Momentum cells around the boundary vertices are split between processors, requiring each to compute only their share of the vertex momentum contribution. Advection of mass, momentum, and energy involves fluxing through regular and momentum cell faces. Cell-face values that are evaluated via upwind differencing require physical quantities and their derivatives on both sides of the face. Spray dynamics requires some communication, and nonuniform distribution of particles and combustion could lead to load-balancing problems for some problems. Particles that cross processor boundaries must be created or destroyed, depending on the direction in which they move. The grid points on the shared faces between processors must have the same structure for predictable communication patterns.

Parallel implementation of KIVA-3 is done on the Intel XP/S 150 Paragon at ORNL. Although KIVA-3 uses the native NX Message Passing Library, the communication interface is kept modular and easily portable to other communication packages such as PVM and MPI. High parallel efficiencies (90%) have been measured for a baseline engine test case. The baseline engine problem has been successfully run on up to 1024 processors on the Paragon without any major input/output or communication bottlenecks. Each processor uses multiple input/output files. The speedup on a large number of nodes depends on the problem size, so it is necessary to keep the 100 grid-points/processor ratio of divisional work to operate on a high parallel efficiency. Though our tests so far involve relatively small problems, an advantage of the distributed-memory implementation would be the ability to run large problems. Solving a problem requiring 6 gigabytes of memory (millions of mesh points) is certainly feasible on our 1024-node Paragon, and our future work will attempt to demonstrate that.


Recommendations to develop advanced predictive capabilities for combustion and emission processes have come from recent workshops on the design of next-generation vehicles and environmentally responsive technologies. The available simulation software has been limited in its ability to represent the detailed physical processes and the complex geometries of engines and other industrial combustion systems. Trends such as increasing fuel economy and lowering emissions often conflict, so a balance must be achieved through careful optimization. Designers must address various parameters (i.e., piston bowl geometry, swirl, fuel injection pressure and rate, nozzle geometry, number of nozzles, compression ratio, spark ignition time and energy) to arrive at an optimized engine design that meets their targets.

Computational models not only provide tools for such a design study, but also might lead to a better understanding of the physics of engine combustion. New insights into the physical processes might lead to new ways of dealing with them. Our large-scale computational capabilities, based on a parallel implementation of KIVA-3 on a scalable system such as the Intel XP/S 150 Paragon, are expected to create a testbed for industrial optimization studies. As a result, more efficient and environmentally friendly automobiles may be designed and brought to market more quickly.


OSMAN YASAR is a staff scientist at ORNL’s Center for Computational Sciences and an adjunct associate professor at the Applied Mathematics Department of the State University of New York at Stony Brook. Before he joined the ORNL staff in 1994, he worked as a research staff member and a manager for the Parallel Computing Laboratory at the University of Wisconsin—Madison, where he earned his M.S. degrees in nuclear engineering and computer science and his Ph.D. degree in engineering physics. He also has taught at Inonu University and Hacettepe University (his alma mater) in Turkey. His areas of expertise are parallel computing, computational fluid and particle dynamics, engine combustion modeling, spark ignition, plasma and radiation hydro-dynamics, and adaptive mesh refinement. He has served as publication manager, conference chair, vice chair, and general chairman of the international Intel Supercomputer User's Group (ISUG), and is currently the advisory chair of ISUG. He is the founder and chairman of the international High-Performance Computing Users (HPCU) Group, a guest editor of the Journal of Computers and Mathematics with Applications, chief editor of the web-based Journal of High-Performance Computing Users, and an organizer of the High-Performance Computing Simulation conference. His Web site is at


Where to?

Next article | Contents | Search | Mail | Review Home Page | ORNL Home Page