From Rotating Cubes to Manifold Hacks
Back in my final year for my master's degree, I was handed a deceptively simple challenge: "Make sense of high-dimensional data by staring at 2-D pictures of it."
The classic trick for that is the Grand Tour - an algorithm that rotates an n-D dataset so quickly that a 2-D screen feels like a window into higher dimensions (imagine animating a cube until the 3-D shape pops out). Fun, but painfully slow. I wanted something faster and smarter.
That curiosity dragged me down a rabbit-hole of geometry called the Stiefel manifold. Think of it as the space of all possible "camera angles" you can point at an n-D dataset, more formally they are matrices that represent orthonormal frames that you can use to transform the data. Instead of blindly spinning the cube, I could optimise over those angles and jump straight to the most informative view.
First brush with machine learning
To judge which view was "informative," I used a linear Support Vector Machine hinge-loss as my cost function. That one decision - bolting a tiny ML model onto a physics-style optimisation problem - was my gateway drug to machine learning. Debugging gradients felt exactly like checking Lagrangians in mechanics, just with Python instead of pen-and-paper.
Four algorithms went into the ring
- Vanilla gradient descent – nice and deterministic, but loves local minima.
- Stochastic gradient descent – same idea, plus a random kick to escape ruts.
- Particle-swarm optimisation – a flock of candidate projections chasing the best performer.
- Genetic algorithm on the Stiefel manifold – crossover via geodesics, mutations via random tangent kicks.
That genetic approach was the real breakthrough: blending two parent projections along the manifold's "straightest path" meant every child already satisfied the orthogonality constraints. It felt less like numerical hacking and more like sculpting in curved space.
Why it still feels like physics
The maths is pure differential geometry - geodesics, Cayley transforms, even the Sherman-Morrison-Woodbury trick for fast matrix inverses. Swapping "phase space" for "manifold of frames" kept the project firmly in my physics comfort zone while delivering ML-style results.
Results in two bullet points
- Wine dataset (13 features, 3 classes) – found a 2-D projection with 100% linear separability in seconds.
- Breast-cancer dataset (30 features, 2 classes) – 98.2% accuracy on unseen data, matching the full 30-D SVM while throwing away 28 dimensions.
Take-aways for the tech folks
- Geometry matters. Sometimes the best optimisation landscape isn't Euclidean.
- Physics intuition transfers. If you've ever minimised an action, you already grok gradient descent on a manifold.
- Creativity lives in constraints. By respecting the Stiefel manifold's rules, the genetic algorithm turned a clunky search into an elegant dance.
I walked into this project thinking "data visualisation." I walked out hooked on machine learning, armed with a toolbox of manifold tricks, and convinced that the border between physics and ML is mostly paperwork.