I am trying to write a program to correlate 2 images from a stereoscopic set of cameras at a known distance and angle, etc. However, i have a question regarding the correlation of objects in the image.

My thought on how to approach this is to have an image, run an edge detect, find which points on each image correlate (i.e, of the same object) and find the total amount these points moved, and calculate the depth. My problem is the correlation.

Image: Flickr Photo Download: 97Edge

The red lines are from one Image, and the White are from the other image.

In gimp, i
1) resized the photo to 800x600 (want it to be fast enough for real time, eventually)
2) converter the image to gray scale
3) ran a Sobel edge detect at 2
4) converter to a binary image

How should i relate x with y? Should i use the surround point's relative/absolute position (e.g, point x also has a point at x+2 and x-4)? Corner Detection? Something else?

If I used relative position, how would i know which points correlate to which points? Should the points be contiguose or separate?.

Any help would be appreciated.