How to Use Ceres-Solver Library for Book-Fitting

Before and after applying Ceres-solver on book fitting

We applied Ceres-Solver for book fitting (you may check how to use Ceres-Solver for curve-fitting). For solving non-linear equation, Google has released a library called Ceres-Solver that is used in Google Street View and Maps. Another sample of using Ceres-Solver is written here. 

Firstly, we model a curve of a book. Our model generally is not precise due to noise. Using Ceres-Solver, we can minimize the error caused by the noise. There are many ways to model the bended shape of a paper (many publications explain this technique). For simplicity, I am using the cubic bezier for modelling the curve.

So we have
1. Captured points : points that form the edge (edge can be extracted using canny method)
2. Model : cubic bezier curve (1 curve is represented by 4 points: 2 end points and 2 control points)
3. Ceres solver
What we want to achieve is having 2 control points that make our model fit to the edge of the paper.

In Ceres-Solver, in order to do this, we design a cost-function:
error = |model - data|

The model is the points that are formed by cubic bezier equation (parameter : 2 control points). Data is the captured points. Ceres-Solver minimizes the error and we will have 2 control points so that our model can fit to the edge of the paper. The following images describe how ceres-solver work. The ceres-solver is applied to the top and bottom edge. 

before fitting and after fitting

Ceres-Solver Curve Fitting in animated GIF