Efficient standard function minimization algorithms for optimization
based on NEWTON steps require to calculate and store, among other arrays,
the matrix of second derivatives with respect to the parameters, or the
inverse matrix. For a large number n of parameters of the order of 100 000
this would require a very large memory space, larger than available
on a standard PC. The gradient of the function (vector of first order
derivatives) requires much less space. Simple steepest-descent minimization
algorithms, requiring only the gradient, are usually rather slowly convergent
or not converging at all. The L-BFGS (J. Nocedal) algorithm for
large-scale minimization is a matrix-less quasi-NEWTON method.
It allows to
take advantage of the good convergence properties of the NEWTON methods
without having to store the full matrix of second derivatives. It requires
the calculation of the gradient and needs memory space for a small number of
LVMINI is a program package based on the L-BFGS algorithm. It requires
a memory space for example of 52 Mbyte for 100 000 parameters.
For a small number of parameters a calculation of parameter errors is
included. It can
be used for a small number of parameters down to 2 parameters, where it
needs 34 double precision words (including the covariance matrix).
Until now there is only a limited experience, and the program may fail for
certain classes of problems
Large-scale optimization with the limited-memory BFGS program LVMINI: The program code
The software can be freely used for research and education. We expect that all publications describing work using this software quote at least one reference.
Disclaimer: This software is provided without any expressed or implied warranty. In particular there is no warranty of any kind concerning the fitness of this software for any particular purpose.
The LVMINI program package and a small test program, a makefile and the manual is in the file lvmini.tgz. Save the tgz file in a fresh subdirectory on your system. To install, first type
tar -xzf lvmini.tgzto unpack, and then the test program is compiled and linked by
makeand the testprogram is executed by
./lvtest(Rosenbrock function and straight-line fits).