Automating video processing with AWS: PwC

Understanding the challenge

Our client, a global racing organization, faced the challenge of manually watching video footage from races and tagging the timestamps when drivers or cars appeared. This labor-intensive process required the use of media applications and consumed significant time and resources. The client aimed to automate heavy, manual processes to improve operational efficiency and deliver high-quality footage to sponsors or fulfill specific requests promptly.

The PwC solution

To address the client's challenge of manually watching and tagging video footage, PwC took a strategic approach by developing a holistic video processing pipeline. The goal was to automate the identification of drivers or cars in the footage, providing consistent and timely results for sponsors or specific requests. Leveraging the power of AWS technology, our solution streamlined the previously manual process, resulting in significant improvements in operational efficiency, consistency, and overall productivity for the client.

By automating the identification process, the client was able to eliminate the time-consuming task of manually watching and tagging video footage. This not only saved valuable resources but also allowed the client to allocate their workforce more effectively, focusing on higher-value tasks. The automated pipeline, powered by AWS technology, enabled a faster turnaround time for delivering high-quality footage to sponsors or fulfilling specific requests.

Deeper dive into AWS technology used

The video processing pipeline implemented a combination of AWS services and on-premises components to automate the identification of drivers or cars in the video footage. The system had an on-premises component that automatically uploaded files as they were received. However, the trigger for the processing pipeline was not the S3 upload itself. Instead, a separate process wrote a record about the file to DynamoDB, which then kicked off the processing pipeline. To run the machine learning model, EC2 instances were utilized due to specific needs of the machine learning (ML) models. When a record landed in the Simple Queue Service (SQS), it triggered scaling on EC2. A script on the EC2 instances would poll the SQS queue, feed the record to the model, and upload the results to S3.

The S3 upload then triggered further Lambdas that handled post-processing and integration with Avid, a media application. This integration allowed for seamless recording of the machine learning output in Avid, establishing the consistent tagging of drivers or cars in the footage. To support ad-hoc queries, Athena was utilized by pointing it to the files stored in S3. This enabled the client to perform flexible and efficient data analysis by querying the video footage.

The project was a collaborative effort between the client, AWS Proserve (who handled training the ML model), and PwC. The ML model was specifically trained to detect drivers based on their car numbers, ensuring precise identification in the video footage.

Overall, the video processing pipeline leveraged a combination of AWS services, including S3, DynamoDB, EC2, SQS, and Athena, along with an on-premises component. This holistic technology stack enabled the automated identification of drivers or cars in the video footage, specifying the timestamps when they appeared.

Delivering outcomes

The implementation of the automated video processing pipeline resulted in significant time and cost savings for the client. By eliminating the need for manual tagging and watching video footage, the client experienced improved operational efficiency and reduced manual effort.

The enhanced efficiency led to increased productivity and faster turnaround time for delivering high-quality footage to sponsors or fulfilling specific requests. The automated process allowed the client to allocate resources more effectively, focusing on value-added tasks rather than time-consuming manual work.

Moreover, the consistency in identifying drivers and cars in the footage was significantly improved. The machine learning model, trained on a vast dataset, confirmed more reliable and precise results. This enhanced consistency contributed to improved sponsor satisfaction and retention, as sponsors could rely on the footage to showcase their sponsored drivers or cars consistently.

The outcomes achieved through the automated video processing pipeline have the potential to drive new business opportunities and revenue growth for the client. By delivering high-quality footage efficiently and consistently, the client can attract more sponsors and retain existing ones, positioning themselves for continued success in the racing industry.