Sina's Logo

Media Engineering project

Droneverse

Droneverse was a student project in the Media Engineering degree programme at Ohm (Nuremberg Institute of Technology). Droneverse explored whether small drones, helium balloons and interactive control modes could become a playful Media Engineering experience: a drone course that visitors could navigate without a classic controller.

Goal, path, result

Goal

The project brief started with “interactive drones with balloons”. My focus became the software architecture behind a tangible drone experience: gesture control, phone motion input and a safe way to test commands before sending them to hardware.

Path

Early experiments with C++ in Ubuntu and React/TensorFlow were useful, but not practical enough for the presentation setup. Python, OpenCV and MediaPipe became the strongest base, connected through modular detection and controller components.

Result

The final prototype used a small UI to select input and controller modes, filtered noisy direction signals, checked for blocked movement and then routed commands to print output, a Tello drone or the quadcopter controller.

Software

The prototype connects detection and control as interchangeable modules. Gesture, OSC and keyboard modules all pass directions back to the central application through callbacks. There, the most frequent recent direction is selected, checked against collision data and sent to the chosen controller.

Interaction modes

OSC

OSC used live gyroscope data from the Data OSC iPhone app. After discovering the exact message paths, the software averaged recent X, Y and Z values and translated tilting and rotating the phone into flight directions.

Phone position for the OSC forward command
Forward
Phone position for the OSC backward command
Backward
Phone position for the OSC left command
Left
Phone position for the OSC right command
Right
Phone position for the OSC up command
Up
Phone position for the OSC down command
Down

Gestures

Gesture control used webcam input, OpenCV and MediaPipe hands. Finger regions controlled up and down, thumb-to-index distance controlled forward and backward, and horizontal finger position controlled left and right.

Hand gesture for the forward command
Forward
Hand gesture for the backward command
Backward
Hand gesture for the left command
Left
Hand gesture for the right command
Right
Hand gesture for the up command
Up
Hand gesture for the down command
Down

During the final presentation, gesture control became the audience favorite. It was intuitive enough for visitors to overcome at least one obstacle after a short practice phase, even though forward/backward thresholds still depended on hand size and camera distance.

Project reports

Open either report to read and browse its pages here. You can also open the PDF in a separate tab or download it.

Interim report
Final project report