This project involved developing a low-cost, real-time system to analyze human postural control by correlating physical joint movement with underlying muscle activation.

The system integrates two key data streams: a webcam using OpenCV to track joint motion and a surface Electromyography(EMG) sensor to read muscle activity. As the project lead for the vision and software, I was responsible for developing the PyQt5 application that serves as the central dashboard. This GUI handles camera calibration , real-time ArUco marker detection, 3D pose estimation, and data synchronization. The final application plots both the joint motion data and the live EMG signal on synchronized Matplotlib charts for direct, real-time analysis.

The acquisition layer consists of two parallel systems. For visual data, a standard webcam captures a live video feed, while ArUco markers are placed on the subject's ankle, knee, and hip . For biological data, a 3-electrode EMG sensor is placed on a target muscle (e.g., Tibialis Anterior) following SENIAM guidelines for accurate signal acquisition. The raw EMG signal is processed and filtered by an Arduino Due (12-bit ADC) , which then transmits the clean data to the main PC via a USB serial connection.

I led the development of the main control software in Python. The custom PyQt5 dashboard  I built manages the entire workflow. It guides the user through camera calibration (using an 8x6 chessboard) to remove lens distortion. Using OpenCV, the application detects ArUco markers in real-time and calculates their 3D position and orientation using the Perspective-n-Point (PnP) algorithm.

A key challenge was data synchronization. The application uses a "handshake" protocol and multithreading  to manage the high-frequency data from both Pyserial (EMG) and OpenCV (video) simultaneously. When the user clicks "Start," the GUI aligns both data streams and visualizes them on 7 live Matplotlib plots, providing an immediate, correlated view of muscle firing and joint translation/rotation.