Jan – Apr 2026
Multi-Agent Drone System
For ENPH 353, our simulated robot had to navigate a Gazebo course, find clueboards and read them, fast and without touching the grass or the pedestrians. We flew two drones: one hovering high over the field as a live "satellite", the other flying the course and reading signs with a custom OCR model.
Maximum score, 2nd of 15 teams, over 98% OCR accuracy
Read the full write-up
Strategy
We tried a few ideas before settling: a custom robot through an Onshape-to-URDF pipeline (in Gazebo, cubes are about as good as curves), a robot that jumped between regions (too hard to land, and possibly against the rules), and imitation and reinforcement learning (unstable and thrown off by wind). We ended up with classical control, PID plus state estimation, for all flight, and saved the machine learning for reading signs.
The final design splits global perception from local control. A "mother" drone hovers over the course, stabilized in XY and yaw, and starts the timer. The second drone then uses the mother's downward camera to locate itself and flies a set of waypoints, photographing clueboards as it goes, and stops the timer once the last clue is submitted.
Top drone
A single ROS controller class holds the drone above the field. The course's centre of mass, from the moments of an HSV mask on the down camera, gives the XY error; the IMU gives a world-to-body rotation so world-frame forces can be applied correctly; and the depth camera sets the altitude, averaging the upper percentile of distances so trees don't throw it off. A leaky integrator on the XY error cancels the steady offset from the wind.
Bottom drone
Localization: it subscribes to the top drone's camera, computes a homography from the image to a normalized field frame, and finds its own bright purple top in that frame. That gives a consistent, scale-invariant (u, v) position.
Navigation: a fixed list of waypoints, each with a target position, altitude, yaw and dwell time, run by a small state machine (running, waiting, done). At each waypoint the drone settles, grabs a front-camera frame and reads the sign in-flight, since the model is light enough to run while it moves.
Control: PID on pixel error for XY, PD with gravity compensation on depth for Z, and IMU feedback for yaw and attitude.
Reading the signs
My teammate Derek Wei built the OCR as a library the controller calls with one function, which returns the clue type and value. Blue thresholding and morphological closing find the sign, and its largest contour is warped by homography to a flat 600 × 400 view. The characters are then thresholded, split into the type and clue rows, and cropped to 100 × 100 before classification.
The classifier is a small three-layer CNN trained in Keras on a synthetic set of 36 characters (A–Z, 0–9) rendered in Ubuntu Mono, then noised, augmented, thresholded and eroded to look like the output of the real pipeline, and exported to TensorFlow Lite for fast inference. Validation accuracy was near perfect, but the training data was very close to the real signs, so the real limit was preprocessing: glare and thresholding that thinned, thickened or merged strokes caused far more misreads than the network did.
