Finally managed to get an hour or so of hacking in tonight. I’m trying to figure out how to use the nonlinear least squares minimisation in the GNU Scientific Library. It’s pretty tricky and I think it may take a while before I get it sussed out.
I’m hoping to get some time to finish hacking on my changes to the SourceEditorDisplayBinding class in MonoDevelop. This will add the drop down select boxes for jumping to a method definition in code.
Maybe I might get some time for this on Sunday…
Had a great time in Berlin last weekend. It wasn’t really what I expected and the really famous sites (check point charlie, Brandenburg gate) left me a bit disappointed. We did see a lot of neat stuff though and we really lucky with the weather. The highlight of the weekend was probably the Berlin Dom which was very cool.
The trip was also the first time I had used my new Sigma 28-200mm lens for my SLR. I found it a real pleasure to use and am really looking forward to seeing what the photos are like. The films should be developed this week so I hope to get some photos up on the website by early next week.
Life is still extremely busy and I haven’t had a chance to do any hacking for ages. Doesn’t look like this week will provide too many opportunites with me busy nearly every night!
I got my first lot of photos developed to day since I started using my Canon SLR again. I have been doing heaps of reading and trying to learn as much as I can about photography in the last few weeks. I’ve decided to finally get serious about something that I have had a bit of a side interest in for years. It turns out that taking good photographs is a lot more difficult than I thought!
I shot a couple of rolls of film over the last couple of weekends playing around with exposure priority, apeture priority, and various other settings on my camera. Pretty much every photo sucks. It has been a lesson in humility and I can see that this will take some practice!
I’ve put a couple of photos up on my web site, mostly so people will have something to refer back to in time to come and be able to see how much I have improved 😉
Here they are:
There are various things wrong with all of them. The most obvious being the photo of the underground train being very blurry but I like it anyway. I intend to retake that shot once I have my new tripod in a month or so.
I got the films processed and digitally scanned at Jessops and am not very happy with the results. I realise these aren’t the greatest photos, but the quality is quite grainy and I know my camera and the kodak film I was using produce better results than this. Next time I think I’ll try Snappy Snaps as I hear they are pretty good.
Well, I haven’t written anything now for over a month. I very occasionally get technology overload. It’s funny when it happens as it is the thing that I am probably most passionate about, but like all good things it is possible to have too much of it. I just find that after coming home from work the last thing I feel like doing is more coding or tinkering with photos in the Gimp or anything that requires too much thinking. I pretty much revert to couch potato status and watch lots of TV, or surf the web as that can be pretty brainless as well 😉
Anyway, the latest bout is over and I am gaining enthusiasm to get started again on some of my projects. I have been well and truly stuck on my vision project for a while so it’s time to sit down and nut out this problem and start moving forward again. There’s also been a new release of MonoDevelop and I am pretty keen to tidy up some work I did a while back and get it submitted.
I have finally managed to update the website with our Scotland photos (many months after our actual trip!). So head over to www.jamesfitzsimons.com and check them out.
We are off to Berlin next Friday so I’ll probably have more photos to post soon. I dug my Canon SLR out while I was at home and finally learning how to use it properly. So, the next lot of photos will be a little different as they will have been taken on film using a much more sophisticated camera, and hopefully with slightly better technique!
Well, looks like we are all packed for our trip back to New Zealand. I reckon it will take us about 36hrs door to door so am already looking forward to getting there. I have all the important things like my laptop sorted so I can do some hacking on the plane on the way over. This is when I wish I had one of those laptops with a 5-6 hr battery life instead of the measly 2hrs I’ll probably get from mine.
I can’t wait to get home now!
I really need to post more often!
I now think I understand where the errors come from that I discussed in my last post. From what I have read, I am trying to use SVD as if I am doing a rigid body transformation – which I’m not. The fact that I am using a linear method to try and find errors in a nonlinear system means that I am getting a less exact answer than I would like.
As a result I’ve spent the last week trying to understand how nonlinear least squares minimisation works. With the help of some great people on the Seattle Robotics Society mailing list I think I am now getting a handle on it. The next step is trying to figure out the practical application of the mathematical theory to my work. This mostly entails trying to decipher how to call the appropriate functions in the GNU Scientific Library.
I also figured out that there should be no error accumulation. That was due to my misunderstanding of how the ego motion is calculated. I was transforming my SIFT feature coordinates to world coordinates one step too early and was accidentally cumulating the error over each subsequent frame.
Hopefully with the fix to the latter problem and the incorporation of a nonlinear method to calculate the ego motion I should be getting much more accurate results.
In other news, I had a great weekend watching NZ win the Tri-nations with an exciting win over Australia. Had a great BBQ at our place on Saturday night. And to top it all off, watched my Hockey team gain a convincing win in their first game of the season (GO RACERS!) 😉
So close and yet so far. That is how it feels at the moment. I found the bug I mentioned in the last post, and had hoped would be the last piece of the puzzle before I had camera ego motion calculations working, but it wasn’t to be.
I now have a problem with the build up of error over consecutive frames. My sanity check to see if ego motion calculation is working is to run the system completely stationary ( the only option at the moment anyway ). Therefore the ego motion calculation between frames should render a 0, 0, 0, 0 result. i.e No movement in any of the 3 spatial dimensions and no rotation. Unfortunately I am getting results that look more like this after one iteration:
—————————————–
New position after ego motion estimation:
X = -0.044358336825996
Y = 0.025654321647698
Z = -0.003337081060976
Theta = 0.431281320335258
Obviously subsequent iterations compound this error and it quickly spirals out of control.
Now I realise in the real system there will be the odometery feeding into this and the odometery and ego motion will be combined in a kalman filter to get the actual estimated motion. In this way the accumulative error won’t matter so much as it will be dampened by the odometery each iteration.
My concern is that this error is far too big to start off with. I can accept (and was expecting) some error between frames, but in the above example the system believes it has moved 4cm to the left. 4cm is too much. What I have to do now is identify the source of this error. I suspect it may be something to do with excessive accuracy in the SIFT feature 3d position calculations. Any error between calculated feature locations could be being exacerbated by the least squares minimisation. That at least is my uneducated guess at this point. More investigation will hopefully shed some light on this issue.
At the risk of jinxing myself completely I think I am nearly at the point where my vision system is correctly calculating camera ego motion! Now if that didn’t make you excited it should because it is very very cool.
I managed to spend a bit of time hacking on the weekend and made a huge amount of progress. Now that I have identified what I suspect (but haven’t yet confirmed) are libsifts short comings and moved to using Lowe’s example binary to extract SIFT features things are starting to work. The matching algorithm I was using to match features between stereo images and between frames was also a bit broken, so I ported Lowe’s example matching algorithm from C to C# and that greatly improved matching performance. So now I have one bug to squash and ego motion calculations will be working. I’m pretty sure I know where the problem lies so hopefully I’ll get that nailed in the next day or two.
At that point I will have one remaining task before I can declare phase one complete. I still have to figure out how to use the camera extrinsic parameters to give more accuracy to the stereo triangulation calculations. At that point the system will be able to accurately build a map of SIFT features observed in it’s environment, and simultaneously localise itself within that environment.
Well, about Wednesday night I was looking at the output of one of the sift annotated images from my vision system and I finally snapped. It just looked wrong. It had been bothering me for a while that it didn’t seem to be selecting very interesting features and taking an unreasonable amount of interest in the bedroom wall, so I broke out the example binary provided by Professor Lowe himself (the inventor of the SIFT algorithm) for comparison.
As I suspected running the two SIFT implementations against identical images yielded completely different results. Lowe’s implementation finding many more and far more interesting features than the version I had been using for months.
This discovery was simultaneously bad and good news. Bad in that it meant I had a bunch of work to do, but good in that I had finally discovered the source of all my frustration.
My new plan of attack has been to write a bunch of C# code to invoke Lowe’s binary out of process and parse the resulting key file back into my program. It’s all a bit messy really but this whole project is still just a proof of concept so I’m not too stressed about performance at the moment. There is no doubt in my mind at all that C# is magnitudes slower than C for this sort of work, but development time is significantly less and for a prototype that’s what matters.
I still have a bunch of work to do before I fully understand stereo vision and the associated camera calibration requirements but I am getting there.
It’s been ages since I did any work on my vision project and I find myself struggling to find the motivation. I have been stuck on a tricky problem for a while now and I think I need a breakthrough to get some interest back. The problem is that the calculations I am using to calculate the real world 3d coordinates of features are giving completely nonsensical answers. I am having trouble establishing whether the formulas I am using are wrong, or if the feature matching isn’t working properly and I am trying to work out pose information for a bad stereo feature match…
If only I could find a site on the web somewhere that had a definitive discussion of calculating real world 3d coordinates from stereo images that would at least help discount the first option.
Jamesfitzsimons.com 