Part 2: Fast Object Tracking in Python using OpenCV

In my previous post I've blabbered about image processing in python using OpenCV libraries. The setup process was discussed and I also went into some basics of reading a image into a variable and showing that image in a window. Now to get into the actual stuff.


The below code is the one from the previous post.
import cv
img = cv.LoadImage("C:\Users\Public\Pictures\Sample Pictures\Koala.jpg")
cv.NamedWindow("hi",0)
while True:
    cv.ShowImage("hi",img)
    k=cv.WaitKey(100)
    if k==27:
        break
cv.DestroyAllWindows()

It loads an image from one of your folders and displays it in a new window. And a correction, you needn't use '\\' instead of '\'. The latter will work fine. We plan on tracking an object based on its color. So we need to isolate it. The typical image data is a combination of three 2D arrays.

The different Layers of an image may be seen above

Now that is how the image data is arranged in rows and columns.As you can see above, there are three layers for each color of the primary color set.  The numerical values in the rows and columns of the array give the shade of Blue,Green or Red to be used. 

Note: The layers are in the order of BGR in OpenCV unlike in Matlab where they are in the order RGB.If it doesn't mean anything to you, forget I said it. 

The HSI color model is just another way of visualizing image data. Below is given the HSI color space

However, this is not the only way the image data may be arranged. It may also be arranged using a HSI color space. Allow me to explain.

As you can see here, the Hue(H) gives the color in specific, the Saturation(S) gives how thick the color has to be and Intensity(I) gives how different it is from black. We are not interested in the S or I, all that we are worried about is the color, so the Hue(H).

When an image is read ,it is read in the RGB format. To perform color based object tracking we need to change t he color space from RGB to HSI

Hue Bar

For that purpose we use a OpenCV function cv.CvtColor(src,dst,conversion_type) to convert the color format to HSI.

The CvtColor function is capable of doing a lot of color conversions. The third parameter defines the type of conversion. 

Before conversion we need to create an another image variable to hold the new HSI image and for that we use cv.CreateImage() function. It takes three parameters-Size of the image,bit depth, channels(one each for H,S,I). The cv.GetSize() function may be used to get the size of an image. The size of the destination image must be same as that of src image.

>>>img_hsi=cv.CreateImage(cv.GetSize(img),8,3)

The above statement simply create a 2D image variable with three layers(one each for H,S,I).

Now for the actual conversion we use CvtColor function as below

>>>cv.CvtColor(img,img_hsi,cv.CV_RGB2HSV) # Intensity(I) is sometimes called Value(V)

the resulting HSI version of 'img' is now copied into 'img_hsi' variable.We may view it using ShowImage function as below. The end code is given below:

import cv
img = cv.LoadImage("C:\Users\Public\Pictures\Sample Pictures\Koala.jpg")
cv.NamedWindow("hi",0)


#lines in addition to previous code
img_hsi=cv.CreateImage(cv.GetSize(img),8,3)
cv.CvtColor(img,img_hsi,cv.CV_RGB2HSV)


while True:
    cv.ShowImage("hi",img_hsi)
    k=cv.WaitKey(100)
    if k==27:
        break
cv.DestroyAllWindows()

The Image appears the same but now it is being read as an HSI image instead like an RGB. To be sure of that let's see all the laye and srs separately. To split the image data by layers we use cv.Split(src,dst0,dst1,dst2,dst3).  So, the layers H,S,I of the image src are split and saved into dst0,ds1,ds2,dst3 in the same order.


Before that we create three images with one layer (dst0,dst1,dst2).Let's do it.
>>>img_h=cv.CreateImage(cv.GetSize(img),8,1) #'1' means it has only one layer
                                                                         # Similarly for others
>>>img_s=cv.CreateImage(cv.GetSize(img),8,1)

>>>img_i=cv.CreateImage(cv.GetSize(img),8,1)


Now to split and store the individual layers and copy them into the respective variables.

>>>cv.Split(img_hsi,img_h,img_s,img_i,None) # it takes in 4 arguments so the fourth       # argument is None as we have just 3 layers 

Viewing the invidual layers in separate windows.
>>>cv.ShowImage("Hue Layer",img_h)
>>>cv.ShowImage("Saturation Layer",img_s)
>>>cv.ShowImage("Intensity Layer",img_i)


Tip: You need not create windows every time to display an image. Instead just use different names for windows in the ShowImage("window_name",img_var) function. The windows are created automatically. Just don't forget to use the DestroyAllWindows() function to destroy all the created windows at the end.

The complete code is given below: 

import cv

img=cv.LoadImage("C:\Users\Public\Pictures\Sample Pictures\Koala.jpg")
img_hsi=cv.CreateImage(cv.GetSize(img),8,3)
cv.CvtColor(img,img_hsi,cv.CV_RGB2HSV);


img_h=cv.CreateImage(cv.GetSize(img),8,1)
img_s=cv.CreateImage(cv.GetSize(img),8,1)
img_i=cv.CreateImage(cv.GetSize(img),8,1)
cv.Split(img_hsi,img_h,img_s,img_i,None)


while True:   
    cv.ShowImage("hue",img_h)
    cv.ShowImage("sat",img_s)
    cv.ShowImage("val",img_i)
    k=cv.WaitKey(100)
    if k==27:break;
    
cv.DestroyAllWindows()

Intensity Layer

The intensity Layer pixel values give the brightness values at various pixel locations. The HSI model is often used to isolate separate this layer which accounts for the lighting conditions.As a thumb rule, we must always try to remove the effect of lighting conditions on the image.

  

Saturation Layer

                    

Hue Layer

The Hue Layer gives the colors at various picture locations. Look at the HSI cone above.

The same code may be implemented for real time images using the second code snippet as follows:

#Under Construction.
The above code is self explanatory,provided you went through the previous post. Thats it for now