I have a vested interest in this subject more than in my usual posts, so please bear with me if I am too biased. My interest is disclosed in the context of the article.
Celeste is a very young baby and will go very far: it aims for the sky. The brainchild of Tim Nugent, currently Bioinformatic PhD student at University College of London, United Kingdom, Celeste is at the bleeding edge of improved panorama making.
It takes the stitching process to the next level. Today’s process is quite simple: identify control points (CP) all over the pictures to register their position relative to each other. Blindly optimize using statistics. Warp. Blend.
But what when statistics fails? All panorama makers have had that difficult stitch. Not all CPs are born equal, and CPs on features moving between shots are particularly bad, especially if there are enough of them to skew the statistics.
The first place where bad CPs are located is the sky. Clouds, airplanes, birds – almost everything in the sky moves. And this is what Celeste aims at: to identify and prune out CPs that are on clouds, using support vector machines.
Tim has trained an SVM to identify the clouds in the sky, and after more than 600 CPU hours of training on a sample of images with the clouds masked out, Celeste has achieved maturity with an accuracy of 82% (false positive rate of 14% and false negative rate of 22%), which is good enough for the application at hand. If 80% of bad CPs in the sky can be weeded out this way, traditional statistics will take care of the the rest.
I’ve been Tim’s mentor along this project and enjoyed an easy task. He is self-sufficient, highly motivated, and knows where he’s going. He verified with me a couple of ideas on how to integrate Celeste in the stitching process, but other than that I was in the lucky situation of just watching him go.
Tim was in Toronto for a congress a few weeks ago and I made the detour to meet him. He’s an outgoing person and fun to hang around with, well traveled with many interest, some of which are shared, including scuba diving and dance music. Tim considers himself a biologist and he is dedicated to science, despite knowledge of the better pay-packages offered in industry and financial services for people with his skills. He’ll finish his PhD soon and is looking for an opportunity to continue bioinformatic research.
On the weekend I took him to Niagara Falls, one of the most famous local attractions. The weather was uncooperative, but the falls are always an amazing sight. Last year I had better luck with the weather.