Orthogonal Matrix for Transforming One Unit Vector to Another

By 苏剑林 | June 05, 2021

In this article, we discuss a practical linear algebra problem:

Given two $d$-dimensional unit (column) vectors $\boldsymbol{a}, \boldsymbol{b}$, find an orthogonal matrix $\boldsymbol{T}$ such that $\boldsymbol{b} = \boldsymbol{T}\boldsymbol{a}$.

Since the two vectors have the same magnitude, it is clear that such an orthogonal matrix must exist. So, how do we find it?

Two Dimensions

It is not hard to imagine that this is essentially a problem of vector transformation (such as rotation or reflection) within the two-dimensional sub-plane formed by $\boldsymbol{a}$ and $\boldsymbol{b}$. Therefore, let us first consider the case where $d=2$.

Schematic of orthogonal decomposition

As shown in the figure above, through orthogonal decomposition, we can obtain a vector $\boldsymbol{b} - \boldsymbol{a}\cos\theta$ which is perpendicular to $\boldsymbol{a}$. After normalization, we can obtain an orthonormal basis:

where $\theta$ is the angle between $\boldsymbol{a}$ and $\boldsymbol{b}$. In this coordinate basis, the coordinates of $\boldsymbol{a}$ are $(1,0)$ and the coordinates of $\boldsymbol{b}$ are $(\cos\theta, \sin\theta)$, i.e.,

Therefore,

Here, there are two choices for $\boldsymbol{R}$:

From a geometric perspective, the former corresponds to a rotation of the vector, while the latter corresponds to a reflection. Both lead to slightly different final results, but from a purely mathematical standpoint, they are both orthogonal matrices that satisfy the requirements. Equation $\eqref{eq:ba}$ implies that the sought orthogonal matrix is:

Multi-Dimensions

For readers who have understood the above process, the logic for the multi-dimensional case is already quite clear. We similarly choose an orthonormal basis first and then transform the problem into a simpler case. Since $d > 2$, $\boldsymbol{a}$ and $\frac{\boldsymbol{b} - \boldsymbol{a}\cos\theta}{\Vert \boldsymbol{b} - \boldsymbol{a}\cos\theta\Vert}$ are not enough to form a complete basis. However, theoretically, we can find $\boldsymbol{e}_3, \cdots, \boldsymbol{e}_d$ such that

forms an orthonormal basis, where

In this case,

The definition of $\boldsymbol{R}$ remains the same as before. Thus, the matrix we are looking for is:

Simplification

Expressing the matrix $\eqref{eq:final-1}$ using block matrices $\boldsymbol{Q}, \boldsymbol{R}, \boldsymbol{E}$, the result is $\boldsymbol{Q}\boldsymbol{R}\boldsymbol{Q}^{\top}+\boldsymbol{E}\boldsymbol{E}^{\top}$. Noticing that we also have $\tilde{\boldsymbol{Q}}\tilde{\boldsymbol{Q}}^{\top}=\boldsymbol{I}_{d\times d}$, this means $\boldsymbol{E}\boldsymbol{E}^{\top}=\boldsymbol{I}_{d\times d} - \boldsymbol{Q}\boldsymbol{Q}^{\top}$. Therefore, the transformation $\eqref{eq:final-1}$ can finally be written as:

One surprising point about this result is that $\boldsymbol{E}$, which carries a degree of randomness, is eliminated, resulting in a deterministic outcome. We can further substitute the concrete forms of $\boldsymbol{Q}$ and $\boldsymbol{R}$ to simplify the result significantly:

It is worth noting that the second matrix is a symmetric orthogonal matrix (orthogonality is necessary, symmetry is not)! This means that for the same orthogonal matrix $\boldsymbol{T}$, it can transform $\boldsymbol{a}$ into $\boldsymbol{b}$ and also transform $\boldsymbol{b}$ into $\boldsymbol{a}$:

This is quite an interesting result, so we select it as our final answer. Noting that $2(1-\cos\theta)=\Vert\boldsymbol{a} - \boldsymbol{b}\Vert^2$, this result can also be written as:

This is the Householder transformation with $\boldsymbol{a} - \boldsymbol{b}$ as the mirror plane. Therefore, if one is already familiar with the Householder transformation, this result can be derived easily.

Using the following logic, we can also obtain a more formally symmetric $\boldsymbol{T}$:

To obtain $\boldsymbol{T}$ such that $\boldsymbol{b}=\boldsymbol{T}\boldsymbol{a}$, one can first find $\tilde{\boldsymbol{T}}$ such that $-\boldsymbol{b}=\tilde{\boldsymbol{T}}\boldsymbol{a}$, and then let $\boldsymbol{T}=-\tilde{\boldsymbol{T}}$.

In other words, by substituting $\boldsymbol{b} \to -\boldsymbol{b}$ in result $\eqref{eq:final-2}$ and then negating the entire expression, we can also obtain a transformation that meets the requirements. Applying this logic to the second solution in $\eqref{eq:final-2}$, we get:

This is what the author considers to be the simplest form of the solution. Note that this is a new solution, generally not equal to either of the two solutions in $\eqref{eq:final-2}$. This means we have provided three feasible solutions so far.

Code

Code verification gives us more confidence in the correctness of the theoretical results. Below is a reference verification code:

#! -*- coding: utf-8 -*-
import numpy as np

def orthonormal_matrix_for_a_to_b(a, b):
    """Find orthogonal matrix T such that Ta is in the same direction as b
    """
    a = a / np.linalg.norm(a)
    b = b / np.linalg.norm(b)
    ab = (a + b).reshape((-1, 1))
    return ab.dot(ab.T) / (1 + a.dot(b)) - np.eye(a.shape[0])

a = np.array([1, 2, 3, 4, 5])
b = np.array([9, 8, 7, 6, 5])
T = orthonormal_matrix_for_a_to_b(a, b)
assert np.allclose(T.dot(T.T), np.eye(a.shape[0])) # Verify orthogonality
r = T.dot(a) / b
assert np.allclose(r, r[0]) # Verify if parallel
assert r[0] > 0 # Verify if in same direction
r = T.dot(b) / a
assert np.allclose(r, r[0]) # Verify if parallel
assert r[0] > 0 # Verify if in same direction

The experimental results show that result $\eqref{eq:final-3}$ is indeed correct. Of course, one can also start from $\eqref{eq:final-3}$ and directly calculate $\boldsymbol{T}\boldsymbol{T}^{\top}=\boldsymbol{I}_{d\times d}$ as well as $\boldsymbol{b}=\boldsymbol{T}\boldsymbol{a}, \boldsymbol{a}=\boldsymbol{T}\boldsymbol{b}$ to ensure the result is correct.

Summary

In this article, we solved a linear algebra exercise: finding an orthogonal matrix that transforms one unit vector into another. We ultimately arrived at a rather simple and interesting result. This transformation can often simplify coordinate-independent problems into a specific case for handling, making it quite valuable in practice.