Color detection in python : Opencv

 Face tracking device :

Python & Arduino 

Hello and thanks for taking some time to read up my blog today. I recently built a face tracking device which works on a python library called 'OpenCV'. OpenCV stands for 'Computer Vision'. From there, I set up a serial interface between my PC and Arduino UNO.

Could I control a camera using my face? When I first got this idea, I began to think about what kind of hardware is needed to pull off this task. Then I came up with Arduino and motion sensor. These two components will play a key role in making it possible. So, how does it work?

Supplies :

1. Arduino UNO
2. 2 x Servo Motors (Any servo motors will be fine but I used Tower Pro SG90)
3. Installing Python
4. Installing OpenCV
5. Web-Camera

Step 1: Installing Software supplies :

You can download python from : https://www.python.org/downloads/

Once you're done with the installation, open up your command prompt and type the following:

 

pip install opencv-python

The following are the steps which will help you to build a real-time opencv application in python.

Step 2: Understanding how it works:

You see facial recognition everywhere, from the security camera in the front porch of your house to your sensor on your iPhone X. But how exactly does facial recognition work to classify faces, considering the large number of features as input and the striking similarities between humans?

Enter Haar classifiers, classifiers that were used in the first real-time face detector. A Haar classifier, or a Haar cascade classifier, is a machine learning object detection program that identifies objects in an image and video.

A detailed description of Haar classifiers can be seen in Paul Viola and Michael Jones’s paper “Rapid Object Detection using a Boosted Cascade of Simple Features”, linked over here. Note that the article goes into some mathematics, and assumes knowledge of machine learning terminology. If you want a summarized, high-level overview, make sure to keep reading!

Step 3: Coding in Python:

import cv2
import numpy as np
import serial
import time

We import all of the libraries that we need.

ard = serial.Serial("COM3", 9600)

We create a serial object called 'ard'. We also specify the Port Name and the BaudRate as parameters.

face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml') 

We create another object for our Haar Cascade. Make sure the HaarCascade file remains in the same folder as this python program.

vid = cv2.VideoCapture(0)

We create an object to that captures video from the webcam. 0 as the parameter means the first web cam connected to my PC.

https://docs.opencv.org/2.4/modules/objdetect/doc/cascade_classification.html

while True:
	_, frame = vid.read()#reads the current frame to the variable frame
	gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)#converts frame -> grayscaled image
        
        #the following line detects faces.
        #First parameter is the image on which you want to detect on
        #minSize=() specifies the minimum size of the face in terms of pixels
        #Click the above link to know more about the Cascade Classification
	faces = face_cascade.detectMultiScale(gray, minSize=(80, 80), minNeighbors=3)
        
        #A for loop to detect the faces.
	for (x, y, w, h) in faces:
		cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 0, 0), 2)#draws a rectangle around the face
		Xpos = x+(w/2)#calculates the X co-ordinate of the center of the face.
		Ypos = y+(h/2)#calcualtes the Y co-ordinate of the center of the face
		if Xpos > 280:                  #The following code blocks check if the face is    
			ard.write('L'.encode()) #on the left, right, top or bottom with respect to the 
			time.sleep(0.01)        #center of the frame. 
		elif Xpos < 360:                #If any of the conditions are true, it send a command to
			ard.write('R'.encode()) #the arduino through the serial bus.
			time.sleep(0.01)
		else:
			ard.write('S'.encode())
			time.sleep(0.01)
		if Ypos > 280:
			ard.write('D'.encode())
			time.sleep(0.01)
		elif Ypos < 200:
			ard.write('U'.encode())
			time.sleep(0.01)
		else:
			ard.write('S'.encode())
			time.sleep(0.01)
		break

	cv2.imshow('frame', frame)#displays the frame in a seperate window.
	k = cv2.waitKey(1)&0xFF
	if(k == ord('q')): #if 'q' is pressed on the keyboard, it exits the while loop.
		break        

cv2.destroyAllWindows() #closes all windows

ard.close() #closes the serial communication

vid.release() #release the web cam. 

Step 4: Programming the Arduino

Feel free to modify the program as per your hardware setup suiting your needs.

<pre>#include <Servo.h><servo.h><br></servo.h>

Servo servoX;
Servo servoY;

int x = 90;
int y = 90;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
  
  servoX.attach(9);
  servoY.attach(10);
  servoX.write(x);
  servoY.write(y);
  delay(1000);
}

char input = ""; //serial input is stored in this variable

void loop() {
  // put your main code here, to run repeatedly:
 if(Serial.available()){ //checks if any data is in the serial buffer
  input = Serial.read(); //reads the data into a variable
  if(input == 'U'){
   servoY.write(y+1);    //adjusts the servo angle according to the input
   y += 1;               //updates the value of the angle
  }
  else if(input == 'D'){ 
   servoY.write(y-1);
   y -= 1;
  }
  else{
   servoY.write(y);
  } 
  if(input == 'L'){
  servoX.write(x-1);
  x -= 1;
  } else if(input == 'R'){
  servoX.write(x+1);
  x += 1;
  }
  else{
  servoX.write(x);
  }