When the nodes of the trial wavefunction are altered large weights are most likely to occur for configurations close to the zeros of the probability distribution . Large weights can also occur when varying the Jastrow factor if the number of electrons, , is large. For a small change in the one-body function, , the local energy changes by an amount proportional to , but the weight is multiplied by a factor which is exponential in , which can result in very large or very small weights if is large. A similar argument holds for changes in the two-body term, which shows an even more severe potential instability because the change in the two-body term scales like . For high order correlations - three-body and higher terms - the potential instabilities increase in severity. This difficulty in optimising high order correlations indicates a problem with current methods of wavefunction optimisation: the optimisation process becomes increasingly fragile as higher orders of correlation are added. Until this problem is resolved it is likely that wavefunctions including high order correlations will be restricted to small systems.
The instability due to the weights has been noticed by many researchers. In principle one could overcome this instability by using more configurations, but the number required is normally impractically large. Various ways of dealing with this instability have been devised. One method is to limit the upper value of the weights [53] or to set the weights equal to unity. [81,46] Schmidt and Moskowitz [81] set the weights equal to unity in calculations for small systems in which the nodes were altered. An alternative approach is to draw the configurations from a modified probability distribution which is positive definite, so that the weights do not get very large. [82] For the wavefunctions in this thesis, which include calculations for large systems of up to 1000 electrons, [83] the weights were set equal to unity while optimising the Jastrow factor.
When using the correlated sampling approach, whether or not the weights are modified, better results are obtained by periodically regenerating a new set of configurations chosen from the distribution , where is the updated parameter set. This helps the convergence of the minimisation procedure by reducing any bias due to the small sample of configurations used in the optimisation procedure. One can also restrict the allowed variation in the parameters before regenerating a new set of configurations, but this can slow the convergence. It has been found [46] that setting the weights to unity allowed the parameters to be altered by a larger amount before the configurations must be regenerated. Typically, after three or four regenerations, the parameters converge to stable values giving a small variance and low energy in subsequent VMC calculations.
These strategies can often overcome the numerical instability. For QMC methods to be applied to large systems with many inequivalent atoms, wavefunctions with many variable parameters are required. It is likely that the determinantal part of the wavefunction as well as the Jastrow factor will eventually be optimised on a regular basis. This has only recently been attempted for solids [84] in a limited form. It is also desirable that excited states as well as ground states be optimised. In order to accomplish these aims, improved optimization techniques are essential. In the remainder of this chapter, a deeper understanding of the issues of numerical stability is developed by analysing different energy and variance minimisation techniques.