Stop Guessing Perceptual Dimensionality: An Occam's Razor for Ordinal Embeddings

Based on our ICLR 2026 paper: LORE: Jointly Learning The Intrinsic Dimensionality and Relative Similarity Structure from Ordinal Data

Introduction

Measuring and mapping human perception quantitatively is hard. If you want to measure physical distance, you use a ruler. But if you want to measure how “sweet” a cake is, or how “aesthetic” a painting is, no absolute physical ruler exists.

Historically, researchers relied on absolute queries, like asking subjects to rate a stimulus on a 1-5 Likert scale. But absolute scales are fundamentally flawed: my “moderately sweet” might be your “cloyingly sweet”. Moreover, humans are quite inconsistent in their absolute judgments giving very different answers to the same questions at different times [1]. To get around the lack of an absolute ruler, early psychophysics [2], [3], [3] and later, machine learning [4], [5], [6] leveraged relative queries. These ask subjects to compare stimuli against each other (e.g., “Is stimulus A more similar to B than to C?”) instead of rating them in a vacuum. (Check out Episode 1 of my blog series for more details on how relative queries allowed psychophysics to escape the absolute ruler.)

However, converting these relative judgements into a latent geometry is non-trivial. The field developed Multidimensional Scaling (MDS) algorithms [7], which yield exact solutions but require an exhaustive pairwise distance matrix, and Ordinal Embedding (OE) algorithms [5], [6], [8], which are highly data-efficient but non-convex. Despite their challenges, these methods allowed researchers to finally build multidimensional perceptual maps instead of flat rulers.

But there’s a massive catch. Both MDS and OEs require you to specify the dimensionality of the map before you even start building it.

The Problem of Dimensionality

Consider a thought experiment. Imagine trying to map the perceptual geometry of food. How many dimensions do you think you need? Sweetness and sourness give us two. Texture adds a third. Maybe temperature adds a fourth?

Regardless of your intuition, manually choosing the dimensionality means you are making an arbitrary guess about the shape of the space. This is a major roadblock when the entire goal of your research is to discover the underlying structure of an unknown percept, which is the case for much of psychology!

Because existing OEs require you to specify this intrinsic rank upfront, you are forced into a blind guessing game that leads to two major problems:

• Underfitting: If you guess too few dimensions, you crush complex relationships. The model might combine “spicy” and “hot temperature” into a single confusing axis.

• Overparameterizing: If you play it safe and guess 15 dimensions, the algorithm will happily spread the data across all of them. A 15D model might satisfy all your triplet comparisons perfectly, but it fractures simple concepts across multiple axes, making the resulting map totally uninterpretable to scientists.

Scientific discovery demands Occam’s Razor: we want the simplest model that explains the data. A 2D taste map is infinitely more useful than a 10D one if the underlying percept only varies in two ways. Yet, historically, we had no scalable way to let the data tell us its own complexity.

LORE: Letting the Data Tell You the Intrinsic Rank

To solve this, we introduce LORE (Low Rank Ordinal Embedding), a new algorithm that jointly learns the intrinsic rank and the relative similarities directly from the data.

The core intuition behind LORE is to apply Occam’s Razor mathematically. We want to balance fitting the similarity structure (which existing OEs do quite well) with penalizing unnecessary dimensions. Strictly minimizing the number of dimensions (the matrix rank) is computationally NP-Hard [9]. The standard workaround is to use a convex relaxation called the nuclear norm [10].

However, there is a problem: the nuclear norm uniformly shrinks all singular values [11], [12] not just the lower order ones. While this works well empirically for standard matrix completion [10], it often fails to recover the true intrinsic rank of perceptual spaces because it over-penalizes the largest, most important dimensions that actually define the space [13].

Instead LORE regularizes using the nonconvex Schatten \(p\) Quasi-norm (\(0 < p < 1\)) [13], [14]. This specific penalty is much more forgiving to the large, dominant singular values but aggressively crushes the smaller, noisy ones to zero.

By balancing a smoothed ordinal embedding loss with this Schatten \(p\)-quasi-norm penalty, LORE automatically prunes away unnecessary dimensions during training. \[\begin{array}{rlr} \underset{\mathbf{Z}}{\text{min}}\text{ }\Psi(\mathbf{Z}) & = \sum\limits_{(a,i,j) \in T}\log(1 + \exp(1 + d\left( \mathbf{Z}_{a,:},\mathbf{Z}_{i,:} \right) - d\left( \mathbf{Z}_{a,:},\mathbf{Z}_{j,:} \right)))) & + \lambda\sum_{i = 1}^{\min\left\{ N,d' \right\}}\sigma_{i}\left( \mathbf{Z} \right)^{p}. \end{array}\]

Because this objective is highly non-convex, standard optimization methods often fail. To solve this, we use an efficiently scaled, iteratively reweighted Singular Value Decomposition (SVD) algorithm [15]. Even with the inherent non-convexity, this guarantees convergence to a stationary point. And because stationary points in OE landscapes are generally known to be nearly as good as global optima [16], [17], LORE reliably yields robust, high quality embeddings.

Does it work? Yes!

Evaluating LORE against existing methods is tricky because, for real human data, we don’t actually know the “true” intrinsic rank. Therefore, we first had to leverage synthetic data where the ground-truth rank is known.

We benchmarked LORE against state-of-the-art OEs across synthetic environments, as well as LLM-generated proxy perceptual spaces (using SBERT [18] to model human alignment. It has been found that LLMs are able to capture human perceptual similarity [19]). As seen in the plot below, the results were stark: LORE was the only method that accurately tracked and recovered the true intrinsic rank while maintaining near-optimal test triplet accuracy.

But the most exciting results came from real, noisy human data, including the Food-100 dataset [20], which contains crowdsourced triplet ratings of 100 different food items based on perceived similarity of taste.

Given a maximum allowable dimension of 15, standard OEs blindly used all 15 dimensions, creating a completely tangled perceptual space. LORE, acting as an automated Occam’s Razor, seamlessly compressed the embedding down to roughly 3.3 dimensions without sacrificing accuracy.

The ultimate test of any perceptual map is whether its dimensions actually mean something. Remarkably, even though LORE was trained purely on relative similarities, the resulting axes organically aligned with human interpretable features. The first axis naturally separates sweet from savory; the second contrasts dense foods with light ones; and the third distinguishes carb-heavy items from proteins and vegetables.

The Takeaway

When we model human perception, we aren’t just trying to maximize predictive accuracy on a holdout set. Instead, we are trying to uncover the latent geometry of the mind. By jointly inferring relative similarities and intrinsic dimensionality, LORE bakes Occam’s Razor directly into the learning process. It removes the need to blindly guess the dimensionality, ensuring we neither underfit nor overparameterize the underlying perceptual space.

Code

Code to reproduce our results and for your own ordinal datasets is available at https://github.com/vivek2000anand/lore_iclr. We are in progress of integrating LORE into the open source python comparison based machine learning library cblearn which would make calling LORE as easy as a call to a standard sciki-learn model. Stay tuned!

References