Tomasi–Kanade factorization

This article has multiple issues. Please help or discuss these issues on the . (Learn how and when to remove these template messages) This article may be too technical for most readers to understand. Please to make it understandable to non-experts, without removing the technical details. (August 2018) (Learn how and when to remove this template message) The topic of this article may not meet Wikipedia's general notability guideline. Please help to demonstrate the notability of the topic by citing reliable secondary sources that are independent of the topic and provide significant coverage of it beyond a mere trivial mention. If notability cannot be shown, the article is likely to be merged, redirected, or deleted. Find sources:  –  ·  ·  · scholar · JSTOR (April 2020) (Learn how and when to remove this template message) This article relies largely or entirely on a single source. Please help by introducing citations to additional sources. Find sources:  –  ·  ·  · scholar · JSTOR (April 2020) This article relies too much on references to primary sources. Please improve this by adding secondary or tertiary sources. (April 2020) (Learn how and when to remove this template message) (Learn how and when to remove this template message)

The Tomasi–Kanade factorization is the seminal work by Carlo Tomasi and Takeo Kanade in the early 1990s.^[1] It charted out an elegant and simple solution based on a SVD-based factorization scheme for analysing image measurements of a rigid object captured from different views using a weak perspective camera model. The crucial observation made by authors was that if all the measurements (i.e., image co-ordinates of all the points in all the views) are collected in a single matrix, the point trajectories will reside in a certain subspace. The dimension of the subspace in which the image data resides is a direct consequence of two factors:

1. The type of camera that projects the scene (for example, affine or perspective)
2. The nature of inspected object (for instance, rigid or non-rigid).

The low-dimensionality of the subspace is mirrored (captured) trivially as reduced rank of the measurement matrix. This reduced rank of measurement matrix can be motivated from the fact that, the position of the projection of an object point on the image plane is constrained as the motion of each point is globally described by a precise geometric model.

Method[]

The rigid-body factorization introduced in provides a description of 3D structure of a rigid object in terms of a set of feature points extracted from salient image features. After tracking the points throughout all the images composing the temporal sequence, a set of trajectories is available. These trajectories are constrained globally at each frame by the rigid transformation which the shape is undergoing, i.e., trajectory of every point will have similar profile.

Let the location of a point j in a frame i be defined as p_ij = (x_ij, y_ij)^T where x_ij and y_ij are horizontal and vertical image co-ordinates respectively .

A compact representation of the image measurements can be expressed by collecting all the non-homogeneous co-ordinates in a single matrix, called the observation matrix P such that

${\displaystyle \mathbf {P} =\left({\begin{array}{ccc}x_{11}&\cdots &x_{1N}\\\vdots &\ddots &\vdots \\x_{F1}&\cdots &x_{FN}\\y_{11}&\cdots &y_{1N}\\\vdots &\ddots &\vdots \\y_{F1}&\cdots &y_{FN}\\\end{array}}\right)}$

P is a 2F × N matrix, where F is the number of frames and N the number of feature points. Ideally, the observation matrix, should contain perfect information about the object being tracked. Unfortunately, in practice, most state-of-art trackers can only provide point tracks that are incomplete (due to occlusion) and inaccurate (due to sensor noise) if placed in an unstructured environment.

As mentioned earlier, the central premise behind the factorization approach is that a measurement matrix P is rank limited. Further, it is possible to factor P into two sub-matrices: a motion and a shape matrix, M and S of size 2F × r and N × r respectively.

${\displaystyle \mathbf {P} =\mathbf {M} \mathbf {S} ^{T}.\,}$

The size and structure of S generally depends on the shape properties (for example whether it is rigid or non-rigid) and M depends both on the type of camera model we assume and the shape properties. The essence of factorization method is computing

The optimal r-rank approximation of P with respect to the Frobenius norm can be found out using a SVD-based scheme.

References[]

See also[]

• Structure from motion