In the previous article I shared how you can communicate between Arduino and Python using ‘pyserial’ module and control a LED. If you haven’t seen it check it out here: COMMUNICATION BETWEEN ARDUINO & PYTHON !

And how you can detect colour of an object and track it on screen, Check that out here : COLOUR DETECTION USING OPENCV AND PYTHON.

In this tutorial we will see how to detect faces using Python and track the faces using servo, controlled by an Arduino uno. So lets get started…

Requirements:

Setting Up Python Environment:

Installing Python:

So first we need Python 2.7 up and running. To do this first download and Install python 2.7.14. To check if it is installed correctly Goto : Windows Search >> Type “IDLE” >> Hit Enter. A Python Shell should pop up.

Python IDLE

Or in search type ‘CMD’ and hit enter to open Command Prompt. In CMD type >> python and hit enter, Python interface should display .

Python CMD

If you see an error in CMD, Do not panic you probably need to set environment variable. You can follow this tutorial Here to set up Environment Variable.

Installing Modules:

To install the required modules we will use use pip install ,

First open CMD and type the following codes:-

>pip install serial 
>pip install opencv-python
>pip install numpy

these commands will install the necessary modules. Now we can move to the coding part…

Python Script:

Before starting to write code first thing to do is make a new folder as all of the code needs to be stored in same folder. So create a new folder, name it anything you want. and download the ‘Haarcascade’ file and paste it in the folder.

Next open notepad and write the script given below , Save it as ‘face.py’ in the same folder as haarcascade. (You can download the code I have provided here) :

#import all the required modules

import numpy as np
import serial
import time
import sys
import cv2

#Setup Communication path for arduino (In place of 'COM5' put the port to which your arduino is connected)
arduino = serial.Serial('COM5', 9600) 
time.sleep(2)
print("Connected to arduino...")

#importing the Haarcascade for face detection
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')

#To capture the video stream from webcam.
cap = cv2.VideoCapture(0)

#Read the captured image, convert it to Gray image and find faces
while 1:
    ret, img = cap.read()
    cv2.resizeWindow('img', 500,500)
    cv2.line(img,(500,250),(0,250),(0,255,0),1)
    cv2.line(img,(250,0),(250,500),(0,255,0),1)
    cv2.circle(img, (250, 250), 5, (255, 255, 255), -1)
    gray  = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    faces = face_cascade.detectMultiScale(gray, 1.3)

#detect the face and make a rectangle around it.
    for (x,y,w,h) in faces:
        cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),5)
        roi_gray  = gray[y:y+h, x:x+w]
        roi_color = img[y:y+h, x:x+w]

        arr = {y:y+h, x:x+w}
        print (arr)
        
        print ('X :' +str(x))
        print ('Y :'+str(y))
        print ('x+w :' +str(x+w))
        print ('y+h :' +str(y+h))

# Center of roi (Rectangle)
        xx = int(x+(x+h))/2
        yy = int(y+(y+w))/2
        print (xx)
        print (yy)
        center = (xx,yy)

# sending data to arduino
        print("Center of Rectangle is :", center)
        data = "X{0:d}Y{1:d}Z".format(xx, yy)
        print ("output = '" +data+ "'")
        arduino.write(data)

#Display the stream.
    cv2.imshow('img',img)

#Hit 'Esc' to terminate execution 
    k = cv2.waitKey(30) & 0xff
    if k == 27:
       break

Arduino Code:

After the python script is ready we need Arduino sketch to control the servo. Refer the code below , paste it in Arduino IDE and save it as ‘servo.ino‘ in the same folder as face.py and haarcascade . upload the code and move on to the next step to make the connections. You can download the code file from here.

#include<servo.h>

Servo servoVer; //Vertical Servo
Servo servoHor; //Horizontal Servo

int x;
int y;

int prevX;
int prevY;

void setup()
{
  Serial.begin(9600);
  servoVer.attach(5); //Attach Vertical Servo to Pin 5
  servoHor.attach(6); //Attach Horizontal Servo to Pin 6
  servoVer.write(90);
  servoHor.write(90);
}

void Pos()
{
  if(prevX != x || prevY != y)
  {
    int servoX = map(x, 600, 0, 70, 179);
    int servoY = map(y, 450, 0, 179, 95);

    servoX = min(servoX, 179);
    servoX = max(servoX, 70);
    servoY = min(servoY, 179);
    servoY = max(servoY, 95);
    
    servoHor.write(servoX);
    servoVer.write(servoY);
  }
}

void loop()
{
  if(Serial.available() > 0)
  {
    if(Serial.read() == 'X')
    {
      x = Serial.parseInt();
      if(Serial.read() == 'Y')
      {
        y = Serial.parseInt();
       Pos();
      }
    }
    while(Serial.available() > 0)
    {
      Serial.read();
    }
  }
}

Pan-Tilt Mechanism:

I have used a readymade mechanisms available online. It’s cheap and pretty easy to assemble, but you can make one yourself if you want. Now I will not go into details about assembling or making one. But there many good tutorials online you can explore.

Connections:

The Circuit is pretty simple. Just attach two servos to Arduino.

  1. Vertical to Pin 5
  2. Horizontal to Pin 6
  3. Power to +5V
  4. Ground to GND

Check the circuit diagram for reference.

Testing:

Final Setup
  • After everything is done last thing to do is test if it works. To test first make sure that servos are properly connected to Arduino and sketch is uploaded.
  • After sketch is uploaded make sure to close the IDE so the port is free to connect to python.
  • Now open ‘face.py’ with Python IDLE and press ‘F5’ to run the code. It will take a few seconds to connect to Arduino and then you should be able to see a window streaming the web cam. Now the code will detect your face and the servos will track it track it.
  • The Servo should move as you move your face . Now just attach the camera to the servos so it will move along with servos.

That’s all for this tutorial, hope you like it and learnt something new. Keep experimenting, Python has a lot to offer.

If you have any doubts feel free to ask in the comments.