Modeling

Use analytical/automatic derivatives

This is the single most important piece of advice we can give to you. It is tempting to take the easy way out and use numeric differentiation. This is a bad idea. Numeric differentiation is slow, ill-behaved, hard to get right, and results in poor convergence behaviour.

Ceres allows the user to define templated functors which will be automatically differentiated. For most situations this is enough and we recommend using this facility. In some cases the derivatives are simple enough or the performance considerations are such that the overhead of automatic differentiation is too much. In such cases, analytic derivatives are recommended.

The use of numerical derivatives should be a measure of last resort, where it is simply not possible to write a templated implementation of the cost function.

In many cases it is not possible to do analytic or automatic differentiation of the entire cost function, but it is generally the case that it is possible to decompose the cost function into parts that need to be numerically differentiated and parts that can be automatically or analytically differentiated.

To this end, Ceres has extensive support for mixing analytic, automatic and numeric differentiation. See CostFunctionToFunctor.

Do not return the norm or squared norm of a residual vector from a CostFunction

A CostFunction should output the signed components of the residual vector \(r(x) \in \mathbb{R}^k\) directly, never its Euclidean norm \(\|r(x)\|\) or squared norm \(\|r(x)\|^2\). Ceres already forms the sum of squared components \(\frac{1}{2} \|r(x)\|^2 = \frac{1}{2} \sum_{i=1}^k r_i(x)^2\) internally.

Collapsing a \(k\)-dimensional residual vector into a scalar \(\tilde{r}(x) = \|r(x)\|\) leaves the objective value unchanged, but replaces the \(k \times n\) Jacobian \(J(x)\) with the single row \(\frac{r(x)^\top}{\|r(x)\|} J(x)\). The Gauss-Newton Hessian approximation then drops from \(J(x)^\top J(x)\) to the rank-1 matrix \(\frac{1}{\|r(x)\|^2} J(x)^\top r(x) r(x)^\top J(x)\), discarding curvature information in all directions orthogonal to \(r(x)\) and stalling convergence. Worse, \(\|r(x)\|\) is not differentiable at \(r(x) = 0\), producing NaN Jacobians at the exact point the solver is trying to reach. Returning \(\|r(x)\|^2\) avoids the NaN but changes the objective to \(\frac{1}{2}\|r(x)\|^4\), whose Jacobian \(2 r(x)^\top J(x)\) vanishes as \(r(x) \to 0\), making the Gauss-Newton system singular near the solution.

Do not pack multiple independent measurements into a single residual block when using a LossFunction

A LossFunction operates on the squared norm \(s = \|f_i(x)\|^2 = \sum_{j=1}^{k_i} f_{ij}(x)^2\) of an entire residual block, not on its individual scalar components \(f_{ij}(x)\) separately. For a multi-dimensional measurement such as a 2D image reprojection error, this is desirable: \(\rho(\|f_i(x)\|^2)\) is isotropic (rotationally invariant in measurement space) and down-weights both coordinates together whenever the feature match is an outlier.

Conversely, you should not pack multiple independent measurements into a single CostFunction if you intend to use a non-trivial LossFunction. A single outlier component will inflate \(\|f_i(x)\|^2\) and cause the LossFunction to down-weight every other inlier measurement in the same residual block. Independent measurements that can fail independently should be added as separate residual blocks.

When using Quaternions, consider using QuaternionManifold

Quaternions are a four dimensional parameterization of the space of three dimensional rotations \(SO(3)\). However, the \(SO(3)\) is a three dimensional set, and so is the tangent space of a Quaternion. Therefore, it is sometimes (not always) beneficial to associate a local parameterization with parameter blocks representing a Quaternion. Assuming that the order of entries in your parameter block is \(w,x,y,z\), you can use QuaternionManifold.

Note

If you are using Eigen’s Quaternion object, whose layout is \(x,y,z,w\), then you should use EigenQuaternionManifold.

How do I solve problems with general linear & non-linear inequality constraints with Ceres Solver?

Currently, Ceres Solver only supports upper and lower bounds constraints on the parameter blocks.

A crude way of dealing with inequality constraints is have one or more of your cost functions check if the inequalities you are interested in are satisfied, and if not return false instead of true. This will prevent the solver from ever stepping into an infeasible region.

This requires that the starting point for the optimization be a feasible point. You also risk pre-mature convergence using this method.

How do I solve problems with general linear & non-linear equality constraints with Ceres Solver?

There is no built in support in ceres for solving problems with equality constraints. Currently, Ceres Solver only supports upper and lower bounds constraints on the parameter blocks.

The trick described above for dealing with inequality constraints will not work for equality constraints.

How do I set one or more components of a parameter block constant?

Using SubsetManifold.