That’s me! I just can’t seem to get into the habit of blogging on a regular basis. Quite a bit has happened since my last post. I’ve just delivered a pretty major project at work which went well I think. It was a massive amount of work though, and really draining long days which has left me desperately in need of a holiday. Luckily Anthea and I have the next two weeks off to trip around northern Italy! We are both really looking forward to it and have been hearing amazing stories from our friends about places to visit.

I have also returned to my computer vision project. I got talking about it to a mate a few weeks back and found myself getting really enthused again. Of course the reality of the situation is that it is still bloody hard work, but I am taking a much more systematic approach this time and taking more time to try and fully understand the math before coding. My first goal is to reliably find the fundamental matrix. Once I have that nailed, 3d reconstruction shouldn’t be too hard.

I got another LG 19″ monitor this week so I now have a cool dual head setup. I’ll take a pic and post it sometime soon.

So it turns out this 3d reconstruction stuff is a lot more complicated than I initially thought. I am trying to get the essential matrix for my camera system, however I am having real trouble getting something that makes sense. I have code that produces an output, it’s just wrong.

It would help if I had a really good text that explained this stuff in baby language so a math retard like me could understand it but unfortunately I don’t at the moment so I am doing a lot of googling. I have ordered Multiple View Geometry in Computer Vision from Amazon as this seems to be the authoritative text, however its got a 4-6 week delivery time.

Looks like this could take a while.

Well, I was really hoping that my next post regarding the my vision project would be to announce that I have phase one working correctly. Alas that is not the case. After some hard debugging over the last couple of weeks I found and fixed a number of bugs and problems.

I am now much more confident in my matching between stereo images, and also in my interframe matching. I rewrote this stuff to be much more in line with the work of Se et al. During the testing process I found a major bug in my kd-tree range search implementation which must have been having a major impact on the number of good quality matches I was getting.

I find it quite difficult to test this stuff as I can’t think of a good way of automating the testing of the quality of matching between images. At the moment I simply annotate the images with the SIFT feature locations and then check for matches by eye.

I got really excited about a week ago when I thought I have finally cracked it and was getting reliable interframe matching and ego motion estimation. In fact I still think that for the most part it is working correctly. However, when I checked the 3d world coordinates the system was calculating for the observed features I realised they were completely wrong. Something that was in reality 1 meter away from the center of the camera system was registering as being nearly 3 meters away, and something that was in fact over 2m away came back less than a meter. I think the problem is in the way I am calculating the disparity and I am not taking all the camera intrinsic and extrinsic parameters into account properly.

To that end I am now working on calculating the essential matrix from the fundamental matrix, and then I should be able to much more accurately calculate the relative position of the observed features.

Time will tell.

The epipolar geometry turned out to be much harder than I anticipated. I now have enough of it done to be able to check the epipolar constraint successfully. I am now in the process of reworking my stereo matching algorithm to include the epipolar constraint along with disparity, orientation, scale and uniqueness constraints. I’m hoping this will improve my matching algorithm and reduce any false positives. That in turn should improve my localisation accuracy as there will be fewer outliers and less error to minimise.

It’s been pretty hard to find time to work on this stuff recently as we have been so busy socially. I guess it’s that time of year with Christmas parties, catching up with friends etc. We are going to Germany this weekend and are off to India for two weeks from the 25th of Dec so I won’t really get time to do any decent hacking on this for the next month or so. I am really hoping that I have enough of the pieces in place that I’ll be able to make some good progress in the new year and actually be able to report some reasonable results.

I have been doing a little bit of reading about FPGA design lately and am getting quite excited about that stage of the project. I expect that will be immensely difficult but should be quite interesting too.

I’m finally making progress on my Vision project again. After spending a couple of months trying to figure out how to do nonlinear least squares minimisation I think it is now working. I still have quite a bit of testing to do before I am sure it is giving sensible results but so far it looks good. I couldn’t have gotten this working without the tireless help of Robin Hewitt who (via the Seattle Robotics Society mailing list) coached me through the process.

I have now returned to the beginning of the project again. Before I go any further I have decided to rework what I have done so far. My understand of computer vision concepts has come a long way since I started this project and I am sure I can improve my original code.

To that end I am writing a camera calibration application that will more accurately calibrate my camera system. This program will be able to calculate the intrinsic parameters of each camera and save and load these parameters from file. It will also be able to calculate the essential matrix of the stereo system and hence will be enable me to far more accurately match SIFT features as I will be able to include the epipolar constraint which I am currently not using.

I am hoping this calibration work will have a drastic improvement on the accuracy of the system, but only time will tell I guess.