Ball Tracking Robot
The ball tracking robot uses image processing, which is done by a Pi cam and ultrasonic sensors to control motors on a robot. The robot is then able to track a ball of a specific color and moves towards it. In the course of completing this project, I was able to overcome many difficult issues such as having to rewire all my ultrasonic sensors and learned important lessons such as the importance of good communication skills.
| Engineer | School | Area of Interest | Grade |
|---|---|---|---|
| Drisha M | STEM High School | Electrical Engineering | Incoming Freshman |
Final Thoughts and Demo
I am happy with the way my project has come out. I was able to complete the base project as well as add some modifications to it. This is my first step to making many more robots like this in the future.This was one of the best ways to spend my summer. As a class, we had so much fun together while also being very productive. Not only have I gained technical skills such as building robots, making circuits, and programming in Python which will help in future careers, but I have also improved my soft skills such as time management, problem solving, teamwork, and decision making. Words cannot express how grateful I am to have found a community of bright minds like this. I will be in contact with them for many years to come. I would like to think that I had entered BlueStamp with a growth mindset. But, after hearing the stories and philosophies of some of our guest speakers, my growth mindset has gotten even stronger and I want to learn and grow even more than I did before.
Final Milestone
My final milestone was to finish up my project and add in a modification. I put all of my code for my motors, sensors, and camera into one file and added code to allow the robot to move based on where the ball was. For my modification, I added 2 LED lights. The green light turns on when the ball is detected and the red light turns on when the ball is not detected. My two main accomplishments during BlueStamp were understanding new code, and completing the project by myself with minimal help. 2 challenges I faced were meeting deadlines for the project, and making my final file of code work fully. The main things that I learned in the course of making this project are some basics of Raspberry Pi, Python, and building circuits. I was also able to improve my ability to communicate with others. In the end, my project is able to track a ball using image processing and is able to move towards it using motors and ultrasonic sensors. In the future after BlueStamp, I hope to make more modifications to this project such as adding an API and to make more robots similar to this one.
Second Milestone
I have accomplished a lot since my last milestone. My last milestone was getting the robot’s motors to operate properly. Since then, I have also been able to program the ultrasonic sensors to detect how far an object is in front of it. When the project is complete, the sensors will be used to detect how far away the ball is and the motors will move the robot closer to the ball. My second milestone is being able to do image processing on an image taken by the Pi Camera. The main issue I faced during this milestone was my Pi Camera stopped working. I had to complete all my code without having a camera to test it with immediately. When the new camera came, it was able to turn the background black and turn the ball white. This project surprised me because the code is a lot simpler than expected. My next step is to put code from each of my components together and add in a modification.
First Milestone
The main parts of my project are the Raspberry Pi, the Pi camera, the motors, and the ultrasonic sensors. All of these components will be connected to the robot car chassis. The Raspberry Pi will be controlling the robot. So far, I have been able to assemble the car chassis, take images and videos from the Pi camera, and run the motors. I faced challenges when connecting the H bridge, the motors, and the Raspberry Pi. The H bridge is a small circuit that is used to control the motors. I solved this problem by researching on my own and found helpful websites to connect the components. I would like to connect my ultrasonic sensors and do image processing next.
Schematics
Code
from picamera2.encoders import H264Encoder
from picamera2 import Picamera2
import time
import cv2
import numpy as np
from gpiozero import Motor
from gpiozero import DistanceSensor
import RPi.GPIO as GPIO
#Ultrasonic Sensors
def distanceSensors():
ultrasonic1 = DistanceSensor(echo=17, trigger=4, threshold_distance=0.5)
ultrasonic2 = DistanceSensor(echo=6, trigger=5, threshold_distance=0.5)
ultrasonic3 = DistanceSensor(echo=12, trigger=16, threshold_distance=0.5)
return ultrasonic1.distance*100, ultrasonic2.distance*100, ultrasonic3.distance*100
#Motors
motor1 = Motor(21, 20)
motor2 = Motor(15, 14)
def forward():
motor1.forward()
motor2.forward()
def backward():
motor1.backward()
motor2.backward()
def left():
motor1.stop()
motor2.forward()
def right():
motor1.forward()
motor2.stop()
def sharpLeft():
motor1.backward()
motor2.forward()
def sharpRight():
motor1.forward()
motor2.backward()
def stop():
motor1.stop()
motor2.stop()
def find_blob(blob):
largest_contour=0
cont_index=0
contours, hierarchy = cv2.findContours(blob, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
for idx, contour in enumerate(contours):
area=cv2.contourArea(contour)
if (area >largest_contour) :
largest_contour=area
cont_index=idx
r=(0,0,2,2)
if len(contours) > 0:
r = cv2.boundingRect(contours[cont_index])
return r,largest_contour
#LED Modifciation
GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(23, GPIO.OUT)
GPIO.setup(18, GPIO.OUT)
GPIO.output(23, GPIO.LOW)
GPIO.output(18, GPIO.LOW)
def led_green():
GPIO.output(23, GPIO.HIGH)
GPIO.output(18, GPIO.LOW)
def led_red():
GPIO.output(23, GPIO.LOW)
GPIO.output(18, GPIO.HIGH)
global center_x
global center_y
global found
global largeNumberArea
global smallNumberArea
global lowerRange
global upperRange
center_x=0
center_y=0
found = 0
largeNumberArea = 250000
smallNumberArea = 100000
leftRange = 400
rightRange = 1520
picam2 = Picamera2()
video_config = picam2.create_video_configuration(main={"size": (1920, 1080)}, lores={"size": (480, 270)}, display="lores")
picam2.configure(video_config)
encoder = H264Encoder(bitrate=10000000)
output = "myVideo.h264"
picam2.start_recording(encoder, output)
while True:
im = picam2.capture_array()
cv2.imwrite('ballImage.png', im)
found = 1
leftSensorFlag = 0
middleSensorFlag = 0
rightSensorFlag = 0
if found == 1:
#Call ultrasonic sensors
leftSensor, middleSensor, rightSensor = distanceSensors()
print("Sensor 1: ", leftSensor)
print("Sensor 2: ", middleSensor)
print("Sensor 3: ", rightSensor)
if (leftSensor > 5 and leftSensor < 100):
leftSensorFlag = 1
if (middleSensor > 5 and middleSensor < 100):
middleSensorFlag = 1
if (rightSensor > 5 and rightSensor < 100):
rightSensorFlag = 1
if (leftSensorFlag == 1 or middleSensorFlag == 1 or rightSensorFlag == 1):
sensorFlag = 1
print("SensorFlag:", sensorFlag)
#Find the masked image
frame = im[:,:,[0,1,2]]
hsv_img = cv2.cvtColor(frame, cv2.COLOR_RGB2HSV)
mask_1 = cv2.inRange(hsv_img, np.array([155,190,1]), np.array([190,255,255]))
cv2.imwrite('mask1img.png', mask_1)
mask = mask_1
cv2.imwrite('maskimg.png', mask)
kern_dilate = np.ones((4,4), np.uint8)
kern_erode = np.ones((4,4), np.uint8)
mask= cv2.erode(mask,kern_erode)
mask=cv2.dilate(mask,kern_dilate)
(h,w) = mask.shape
#cv2.imshow('maskimg.png', mask)
#Find the area
positions, area = find_blob(mask)
print(area)
x,y,w,h = positions
#print(x)
center_x =x + w/2
center_y =y + h/2
if (area < largeNumberArea and area > smallNumberArea and sensorFlag == 1):
#Green light on
led_green()
#reset found value
print ("Found:", found)
found = 0
print("center X" ,center_x)
if center_x > 0 and center_x < leftRange:
print("centerLeft X" ,center_x)
sharpLeft()
elif center_x < 1920 and center_x > rightRange:
print("centerRight X" ,center_x)
sharpRight()
else:
print("centerForward X" ,center_x)
forward()
else:
stop()
#Red light on
led_red()
found = 0
continue
picam2.stop_recording()
Bill of Materials
| Part | Note | Price | Link |
|---|---|---|---|
| Vemico Raspberry Pi4 Starter Kit | The main computer that controls all other components | $145.99 | Link |
| Elegoo Electronics Fun Kit | Contains various electronic components as well as a breadboard | $9.99 | Link |
| Energizer AA Batteries(4 used) | Were used to power motors | $14.99 | Link |
| Sixthgu Portable Power Bank (1 used) | Used to power Rasperberry Pi | $15.99 | Link |
| 32 in 1 Screwdriver Set | Used to build robot car chassis | $6.99 | Link |
| TT Gearbox Motor(2 used) | Used to make wheels on robot move | $10.99 | Link |
| Emozny Robot Car Chassis Kit | Acts as the base for the ball tracking robot | $12.99 | Link |
| AstroAI Multimeter Tester | Measures voltage and ohms | $12.99 | Link |
| Raspberry Pi Camera Module | When connected to the Raspberry Pi, it is able to take photos and videos | $17.99 | Link |
| DC Moter Drive Controller Board(1 used) | Used to control moters | $5.50 | Link |
| Ultrasonic Sensors with housing | Are able to detect how far away an object is | $9.49 | Link |
| Larger breadboard(Modification) | Allows for more space for circuit wiring | $10.99 | Link |
| Pi T-Cobbler(Modification) | Allows connection between the breadboard and Raspberry Pi. Results in cleaner and easier to manage wiring | $7.95 | Link |
| Arducam(Replacement) | When connected to the Raspberry Pi, it is able to take photos and videos | $6.99 | Link |