Through the "eyes" of my program

I finally have results of my program. It can now detect "major" foregrounds, by learning the background. In the image below, what you see are three images: the backgroud (Top left), the foreground (the colored balls) on the background (top right), and what the program has detected (botton left).

Right now, i am using a single Gaussian distribution for the learning function. The results are not that great on real humans walking in the scene. I might need to use a more complicated Gaussian distribution. Thinking, that i might implement the background subtraction technique by Yamada et al. They use intensity (which is a linear function of R,G and B) of each pixel to plot the single Gaussian distribution, and henceforth, compares the Gaussian model to the intensity of each pixel in the image where it searches for the foreground, to differentiate between the foreground and the background.