2012 TePRA Competition Recap

Here, I'll be publishing my progress on the 2012 TePRA Student competition. I'll be building a robot car similar to the one displayed in the simulation to accomplish the task of being able to autonomously go around the block.

Here's a vid of the demo of my final working project. In the vid, the car is built from a Lego Car Kit

If this page starts getting some hits, I'll post up some of the code I developed to run it.

5/1/12


This project was developed by myself Danny Perez @tchdp and Melvin Cayas @whoaMelvin

Overall Description:


For electronics, the HTC Dream (G1) is connected directly via USB to an Arduino with a USB Host Shield for Android <-> Arduino communication.
The G1 processes frames using OpenCV for Android and calculates a heading and sends commands to the Arduino directly over USB. While the G1 can be hacked to enable USB host mode, the G1 is using ezGingerbread ROM for access to the USB Accessory API available from Android 2.3+. This allows access to the regular Android API without having to rebuild the kernel to allow communication over the USB port.


Also the use of OpenCV with the Android Google API (For USB Accessory mode) requires the latest version of OpenCV, with v. 2.4.0 released a couple of weeks ago. lucky me. this has more compatibility with a wider range of devices and is more stable.


The USB Host Shield is from Sparkfun. I couldn't get it to work with the USB Host Shield 2.0 libraries that the site links to, so I used the 1.0 libraries with better success. The only glitch I encountered was that it wouldn't work right if I had anything plugged into digital pin 7. It would recognize the Android device as soon as anything was disconnected from that pin, so I just made sure not to use it.


The motors are controlled with a Motor Driver Shield for the Arduino which just features a Dual H-Bridge IC capable of handling upto 2A and between 3-30 something V. 


If you have any questions on something that I forgot to cover here, just leave a comment and I'll do my best.


Here's some pictures of the physical setup:

This is the board I used for tracking:

Here's a screencast of the final simulation for lane tracking

The code in the simulation written in C++ using OpenCV was just almost literally copied/pasted to make it compatible with OpenCV for Android.


The biggest issue with this approach to lane tracking is its EXTREME sensitivity to lighting conditions. The design will work as long as the regions of interest in the image as well as some other thresholds are adjusted for your specific situations. This was done on purpose.


The correct way to extract lines from the image is using the Hough Transform. Transforming the image into Hough coordinate space keeps many factors such as lighting and weather conditions (when used in the real-world) from affecting your final calculations.


I decided not to use the Hough transform because I wanted to make the car as cheap as possible and considering the G1 can be obtained for $50-100, I didn't want to have to use a more expensive phone. The G1 features a 528 MHz ARM v5 processor with 192 MB RAM and a 3.15 MP camera. It is the performance capabilities of the phone that prevents it from being able to perform Hough Transforms, the phone can do it, but at a rate of < 1 fps. So I found a crazier way to get it to work.


Using thresholding and high school math, the final rate of processing on the G1 was ~10 fps. Which is actually a lot better than expected. On newer phones, the blend of Android + Arduino is a pretty powerful duo.


For a quick recap of how I identify the lanes:
- Grab a grayscale frame from native OpenCV camera
- obtain a center ROI, equalize the histogram, and threshold it to extract the black blobs on the board.
- In this center ROI, 2 dominant blobs are guaranteed to appear as long as the car is oriented relatively correctly.
- To obtain the locations of these 2 dominant blobs, i use cv::findContours() which runs connected component clustering on the image. this will retain data on ALL blobs in the image provided
- For all contours obtained from the image, calculate the area of the contour and keep the 2 contours with the greatest area - these 2 contours are your lines in the image
- use cv::minAreaRect() to obtain a RotatedRect covering all points in the provided cluster for both largest clusters.
- find the center and orientation of these 2 RotatedRects and calculate the midpoint between them to calculate the heading


This is what is shown in the videos below.


In the end, this project gave me some insight into 3D modeling, game development (an intro at least), C# scripting in Unity, intro to computer vision and data mining, android development, android development with native code, and finally, squeezing every last bit of juice out of the G1.




4/11/12
Haven't had a chance to post my new stuff in a while. Here's some more work on a very simple lane tracking model for the competition

2/26/12
Sorry folks, it appeared the place where I was hosting the Unity3d file went dead since last week. So I switched to a different host, hopefully they're better.

2/22/12 
I got the simulation streaming frames outside of the game engine for processing in OpenCV. Here's the vid:

Check here for more info

Very simple client-server model. C++ server listens for connection, C# client connects to server, grabs frame, encodes to PNG, converts to byte[], sends over socket, server accepts, and processes frame (grayscale -> threshold)

2/17

Finally! After hours and hours of trying to figure out how the hell to make a game, I managed to make a rough simulation of the TePRA student competition. 

Features (for the web player):

Control the car with WSAD or the D-pad

Switch between 1st person camera (inside the phone) and 3rd person Camera. I will be adding some other features over the next couple of weeks. 

Unity Web Player | TePRASimulation

« created with Unity »
Press P to change cameras.