lbfgsmin

When the problem is very large, a limited-memory bfgs algorithm may be needed, if bfgsmin is not feasible due to memory limitations. lbfgsmin is called as

$\displaystyle [theta,\, value,\, convergence]=lbfgsmin(''f'',\,\{ args\},\,\{ control\})$

The first argument $ ''f''$ is a string variable that holds the name of the function to be minimized. The second argument, $ args$, is a cell array that hold the arguments of $ f$. The third argument $ control$ is an optional cell array of $ 5$ elements. The elements of $ control$ are the same as for bfgsmin, except that there is one more element that controls how many iterations are used to form the quasi-Hessian matrix (this is the memory of the method). The control vector is fully described in Table 3. You can easily modify the above example to use the lbfgsmin method.

Table 2: Controls for lbfgsmin
Element Purpose Default Value Other possible values
1


It is possible that lbfgsmin can outperform bfgsmin even when memory is not an issue. Remember that both of these algorithms are approximating the Hessian matrix using previous gradient evaluations. If the true Hessian is changing rapidly, then a limited memory approximation may be better than a long memory approximation. The Rosenbrock function is such a case. The program lbfgsmin-example.m minimizes a 200-dimensional Rosenbrock function using both algorithms. The output

/home/sh/Dokumenter/octave/octave-forge/main/optim/doc/lbfgsmin-example.out
shows that the limited memory algorithm uses significantly more iterations that the ordinary BFGS algorithm, but it is almost 4 times as fast. In general, though, the ordinary BFGS algorithm is recommended when memory limitations are not a problem.

Søren Hauberg 2008-04-29