By 苏剑林 | July 16, 2026
Yesterday, I saw classmate @Ji_Ha_Kim share an interesting paper "Beyond Cosine Similarity", which proposes constructing new, more relaxed similarity metrics based on the rearrangement inequality. After reading it, I found it quite interesting and decided to briefly record it here.
Cosine Similarity
For two vectors $\boldsymbol{x}, \boldsymbol{y} \in \mathbb{R}^d$, a commonly used normalized similarity metric is cosine similarity:
$$\cos(\boldsymbol{x}, \boldsymbol{y}) = \frac{\boldsymbol{x} \cdot \boldsymbol{y}}{\Vert\boldsymbol{x}\Vert \Vert\boldsymbol{y}\Vert}$$
Setting aside the geometric meaning and looking at it from a purely algebraic perspective, cosine similarity is a normalized inner product constructed based on the Cauchy-Schwarz inequality:
\begin{equation}-\Vert\boldsymbol{x}\Vert\, \Vert\boldsymbol{y}\Vert \leq \boldsymbol{x}\cdot \boldsymbol{y}\leq \Vert\boldsymbol{x}\Vert\, \Vert\boldsymbol{y}\Vert\end{equation}
When $\cos(\boldsymbol{x}, \boldsymbol{y}) = \pm 1$, it means there exists $k > 0$ such that $\boldsymbol{y} = \pm k\boldsymbol{x}$. This indicates that cosine similarity is insensitive to magnitude; it measures the degree of directional similarity, or linear correlation, between two vectors. In particular, from the AM-GM inequality, we have $\Vert\boldsymbol{x}\Vert\, \Vert\boldsymbol{y}\Vert \leq \frac{1}{2}(\Vert\boldsymbol{x}\Vert^2 + \Vert\boldsymbol{y}\Vert^2)$. Substituting this into the Cauchy-Schwarz inequality for further bounding yields:
\begin{equation}-\frac{1}{2}(\Vert\boldsymbol{x}\Vert^2 + \Vert\boldsymbol{y}\Vert^2) \leq \boldsymbol{x}\cdot \boldsymbol{y} \leq \frac{1}{2}(\Vert\boldsymbol{x}\Vert^2 + \Vert\boldsymbol{y}\Vert^2)\end{equation}
The first equality holds when $\boldsymbol{x} = -\boldsymbol{y}$, and the second holds when $\boldsymbol{x} = \boldsymbol{y}$. From this, we can define a new normalized inner product:
\begin{equation}\newcommand{dcos}{\mathop{\text{dcos}}}\dcos(\boldsymbol{x},\boldsymbol{y}) = \frac{\boldsymbol{x}\cdot \boldsymbol{y}}{\frac{1}{2}(\Vert\boldsymbol{x}\Vert^2 + \Vert\boldsymbol{y}\Vert^2)}\end{equation}
In this case, $\dcos(\boldsymbol{x}, \boldsymbol{y}) = \pm 1 \Leftrightarrow \boldsymbol{x} = \pm\boldsymbol{y}$, meaning $\dcos$ is a metric for the degree of exact identity between two vectors.
Rearrangement Inequality
For the inner product $\boldsymbol{x}\cdot \boldsymbol{y}$, the rearrangement inequality also holds:
\begin{equation}\boldsymbol{x}^{\downarrow}\cdot \boldsymbol{y}^{\uparrow}=\boldsymbol{x}^{\uparrow}\cdot \boldsymbol{y}^{\downarrow}\leq \boldsymbol{x}\cdot \boldsymbol{y} \leq \boldsymbol{x}^{\uparrow}\cdot \boldsymbol{y}^{\uparrow} = \boldsymbol{x}^{\downarrow}\cdot \boldsymbol{y}^{\downarrow}\end{equation}
Here $\boldsymbol{x}^{\downarrow}, \boldsymbol{x}^{\uparrow}$ refer to the new vectors obtained by sorting the components of $\boldsymbol{x}$ in descending/ascending order (likewise for $\boldsymbol{y}$). Simply put, it means "reversed order inner product $\leq$ mixed order inner product $\leq$ same order inner product". The condition for the first equality to hold is that $\boldsymbol{x}, \boldsymbol{y}$ are completely oppositely ordered, i.e., $(x_i - x_j)(y_i - y_j) \leq 0$ for all $i, j$. The condition for the second equality is that $\boldsymbol{x}, \boldsymbol{y}$ are completely similarly ordered, i.e., $(x_i - x_j)(y_i - y_j) \geq 0$.
Obviously, reordering does not change the magnitude, i.e., $\Vert\boldsymbol{x}^{\downarrow}\Vert = \Vert\boldsymbol{x}^{\uparrow}\Vert = \Vert\boldsymbol{x}\Vert$. Therefore, combining this with the Cauchy-Schwarz inequality, we get:
\begin{equation}-\Vert\boldsymbol{x}\Vert\, \Vert\boldsymbol{y}\Vert \leq \boldsymbol{x}^{\uparrow}\cdot \boldsymbol{y}^{\downarrow}\leq \boldsymbol{x}\cdot \boldsymbol{y} \leq \boldsymbol{x}^{\uparrow}\cdot \boldsymbol{y}^{\uparrow} \leq \Vert\boldsymbol{x}\Vert\, \Vert\boldsymbol{y}\Vert\end{equation}
This shows that for $\boldsymbol{x}\cdot \boldsymbol{y}$, the rearrangement inequality is tighter than the Cauchy-Schwarz inequality. The standard way to prove the rearrangement inequality is the method of local adjustment, starting from the identity:
\begin{equation}(x_i - x_j)(y_i - y_j) = (x_i y_i + x_j y_j) - (x_i y_j + x_j y_i)\end{equation}
The first term on the right is the inner product of $(x_i, x_j)$ and $(y_i, y_j)$, and the second term is the inner product after swapping $x_i, x_j$ or $y_i, y_j$. If $x_i > x_j$ and $y_i > y_j$ (same order), then the left side is greater than 0, meaning that swapping from same order to opposite order will decrease the inner product. Thus, the maximum is achieved in the case of complete same ordering, and similarly, the minimum is achieved in complete opposite ordering.
Rank Similarity
If it is known that $l(\boldsymbol{x}, \boldsymbol{y}) \leq \boldsymbol{x} \cdot \boldsymbol{y} \leq u(\boldsymbol{x}, \boldsymbol{y})$ and both equality bounds are reachable, then one can generally construct a similarity metric:
\begin{equation}2\cdot\frac{\boldsymbol{x}\cdot \boldsymbol{y} - l(\boldsymbol{x},\boldsymbol{y})}{u(\boldsymbol{x},\boldsymbol{y}) - l(\boldsymbol{x},\boldsymbol{y})} - 1 = \frac{2\cdot\boldsymbol{x}\cdot \boldsymbol{y} - l(\boldsymbol{x},\boldsymbol{y}) - u(\boldsymbol{x},\boldsymbol{y})}{u(\boldsymbol{x},\boldsymbol{y}) - l(\boldsymbol{x},\boldsymbol{y})} \in [-1, 1]\end{equation}
Substituting the rearrangement inequality, we get:
\begin{equation}\newcommand{rcos}{\mathop{\text{rcos}}}\rcos(\boldsymbol{x},\boldsymbol{y}) = \frac{2\cdot\boldsymbol{x}\cdot \boldsymbol{y} - \boldsymbol{x}^{\uparrow}\cdot\boldsymbol{y}^{\downarrow} - \boldsymbol{x}^{\uparrow}\cdot\boldsymbol{y}^{\uparrow}}{\boldsymbol{x}^{\uparrow}\cdot\boldsymbol{y}^{\uparrow} - \boldsymbol{x}^{\uparrow}\cdot\boldsymbol{y}^{\downarrow}}\end{equation}
Thus, $\rcos(\boldsymbol{x}, \boldsymbol{y}) = 1$ means $\boldsymbol{x}, \boldsymbol{y}$ are completely in the same order, and $\rcos(\boldsymbol{x}, \boldsymbol{y}) = -1$ means $\boldsymbol{x}, \boldsymbol{y}$ are completely in opposite orders. If cosine similarity describes linear correlation, then $\rcos$ to some extent describes some degree of non-linear correlation. $\rcos$ remains invariant under translation and positive scaling, i.e.:
\begin{equation}\rcos(a \boldsymbol{x} + b\boldsymbol{1}, c\boldsymbol{y} + d\boldsymbol{1}) = \rcos(\boldsymbol{x}, \boldsymbol{y})\end{equation}
However, this is not true for general strictly monotonic component-wise transformations (such as component-wise cubing); $\rcos$ preserves and utilizes the numerical magnitudes rather than relying solely on the rank.
Connections and Differences
It should be noted that while this article was inspired by "Beyond Cosine Similarity", the rank similarity constructed here is slightly different from the one in the original paper. The construction method in the original paper is ($\newcommand{recos}{\mathop{\text{recos}}}\recos$ is also the notation from the original paper):
\begin{equation}\recos(\boldsymbol{x},\boldsymbol{y}) = \frac{\boldsymbol{x}\cdot\boldsymbol{y}}{\|\boldsymbol{x}^{\uparrow}\cdot\boldsymbol{y}^{\updownarrow}\|},\qquad \boldsymbol{y}^{\updownarrow} = \begin{cases}\boldsymbol{y}^{\uparrow}, & \boldsymbol{x}\cdot\boldsymbol{y} > 0\\ \boldsymbol{y}^{\downarrow}, & \boldsymbol{x}\cdot\boldsymbol{y} < 0\end{cases}\end{equation}
That is, it selects either the same-order or opposite-order bound based on the sign of the inner product. Although this definition also restricts the result to $[-1, 1]$, the sign of the inner product can no longer be used to distinguish between same-order and opposite-order, which I believe is less reasonable than the definition in this article.
For example, take $\boldsymbol{x}=(1,2,3), \boldsymbol{y}=(6,5,4)$. These two vectors are completely oppositely ordered. We believe a rank similarity metric should at least yield a negative result. However, clearly $\recos(\boldsymbol{x}, \boldsymbol{y})$ yields a positive result, whereas $\rcos(\boldsymbol{x}, \boldsymbol{y})$ can yield a negative result. The definition in this article directly uses the double-sided bounds of the rearrangement inequality for normalization, anchoring $\pm 1$ to complete same-ordering and complete opposite-ordering, which I think is semantically more reasonable.
Of course, which of $\rcos$ or $\recos$ is more appropriate in practice depends on the specific scenario; the claim of being "more reasonable" here is only in regards to the semantic meaning of the term "rank similarity."
Rank Correlation Coefficients
If we treat $\boldsymbol{x}, \boldsymbol{y}$ as a series of 2D data points $(x_1, y_1), (x_2, y_2), \dots$, then cosine similarity is related to their Pearson correlation coefficient:
\begin{equation}\mathop{\text{pearson}}(\boldsymbol{x}, \boldsymbol{y}) = \cos(\boldsymbol{x}-\bar{\boldsymbol{x}}, \boldsymbol{y}-\bar{\boldsymbol{y}})\end{equation}
where $\bar{\boldsymbol{x}}, \bar{\boldsymbol{y}}$ are the means of the components of $\boldsymbol{x}$ and $\boldsymbol{y}$. Based on the Pearson correlation coefficient, the Spearman's rank correlation coefficient was derived:
\begin{equation}\mathop{\text{spearman}}(\boldsymbol{x}, \boldsymbol{y}) = \mathop{\text{pearson}}(\boldsymbol{r}_x, \boldsymbol{r}_y)\end{equation}
$\boldsymbol{r}_x, \boldsymbol{r}_y$ are the rank vectors of $\boldsymbol{x}$ and $\boldsymbol{y}$. That is, if $x_i$ is the $k$-th smallest component of $\boldsymbol{x}$, then $(\boldsymbol{r}_x)_i = k$. From the definition, it can be seen that the rank correlation coefficient completely ignores the magnitude of the data itself and only cares about the order. We previously used this metric when evaluating semantic similarity. Interestingly, the Spearman rank correlation coefficient can also be viewed as a special case of $\rcos$:
\begin{equation}\mathop{\text{spearman}}(\boldsymbol{x}, \boldsymbol{y}) = \rcos(\boldsymbol{r}_x, \boldsymbol{r}_y)\end{equation}
I leave it to the reader to verify this equality, so I won't expand on it here. Thus, $\rcos(\boldsymbol{x}, \boldsymbol{y})$ can be seen as a generalized rank similarity that preserves original magnitude information.
In a Nutshell
This article introduced the approach of constructing similarity metrics based on double-sided inequalities of the inner product from the perspective of "similarity = normalized inner product," with a focus on the rank similarity metric constructed from the rearrangement inequality.