Projective ambiguity

• Although the two projection matrices define ${\bf F}$ the converse is not true.
• Suppose ${\bf P}$ and ${\bf P'}$ are two matrices consistent with ${\bf F}$ , then
  $${\bf x} = {\bf P} {\bf X}$$    and $${\bf x'} = {\bf P'} {\bf X'}$$

• But, if ${\bf H}$ is any arbitrary homography of 3-space, then
  $${\bf x} = ({\bf PH^{-1}}) ({\bf HX})$$    and $${\bf x'} = ({\bf P'H^{-1}}) ({\bf HX'})$$

• Thus, we can only recover projective structure modulo an unknown homography.

How to recover ${\bf H_{PE}}$ , the homography that upgrades the projective structure to Euclidean?