A major concern, globally, is the safety of our schools. School shootings, knife attacks, and bombings have become far too common. To address this problem, the American Heritage School robotics team developed an autonomous, mobile robot that will travel around a school campus gathering information, and providing this vital information to school security personnel, and, if necessary, first responders. HPCC Systems has partnered with the American Heritage School on autonomous robotics projects for the past few years.
The security robot is designed to be cost-effective and able to be implemented on a universal platform that will benefit a larger population of students. Other considerations include user compatibility with technology that is already in place and user-friendliness based on a traditional skill set.
Jack Fields, a 12th-grade high school student at American Heritage School in Delray, Florida, presented his work, “Using the Generalized Neural Network (GNN) Bundle with TensorFlow to Train a Model to Find Known Faces Leveraging the Robotics API,” during HPCC Systems Tech Talk 35. Jack also presented a poster for the HPCC Systems Community Day 2020 Poster contest, where he was the winner of the “Community Choice Award.”
In this blog, we discuss Jack’s summer intern project, which consisted of using the HPCC System GNN bundle with TensorFlow to train a model to recognize faces, using a previously-developed robotics API (application programming interface. His mentors for this project were David DeHilster and Xiaoming Wang.
The facial recognition portion of the Autonomous Security Robot project builds on the work of a former intern from American Heritage School, Aramis Tanelus, who created an API for ingesting data into HPCC Systems. The software for this project is currently being implemented in the American Heritage security robot.
The code for this API was updated to work with the newest version of ROS, and integrated with the newest robotics firmware. ROS (Robot Operating System) is an open-source, flexible framework for writing robot software. ROS was run on a Linux machine for this project but can be run on any system.
Robot Sensor Features
The security robot has a variety of sensors, from light arc to cameras, and even GPS tracking.
Now, let’s move on to the development process for this project.
Facial Recognition Development Process
With security being a major concern, the project team determined that it would be a good idea to take pictures of faces using the robot, and create a model to recognize known faces. To collect this data, Jack wrote a program that allows a person to scan themselves into the robot. This works by prompting a person for their name and then taking a series of pictures.
OpenCV (Open Source Computer Vision Library) was used to detect the location of faces, as well as to automatically resize and scale the images to a universal format. The software also converted the images to grayscale making the images one dimension and easier to work with. OpenCV is an open-source library for computer vision, machine learning, and image processing
Please note that this method was used for initial data collection, and was not reliable for detecting faces. It was eventually replaced with a trained model.
Now that the images have been collected and the dataset created, it is time to train the model.
TensorFlow is an open-source artificial intelligence library that uses data flow graphs to build models. TensorFlow was the perfect solution for this project because it's meant to be used with large-scale neural networks. Before using the HPCC Systems GNN bundle to train the model, TensorFlow was used to create a baseline test outside the HPCC platform. Jack wrote a program in Python where TensorFlow was used to train the model. It provided more accurate results to compare to those derived from using the GNN bundle.
Processing the Images
A convolutional neural network (CNN) was used to process the collected images. Through a series of convolutions, data was taken from an image and a feature map and pooling were created. Specific regions were selected and the maximum values were extracted. These maximum values were then used as the value for the entire selected area.
So, now, let’s discuss the HPCC Systems GNN bundle and how it was used to train the model.
HPCC Systems GNN bundle
The Generalized Neural Network Bundle (GNN) provides a parallelized interface to Keras / TensorFlow supporting complex Neural Networks for processing multimedia data types such as Image, Video, and Time-series.
For this project, HPCC Systems GNN Bundle was used with TensorFlow to train the model to recognize know faces using the following steps:
1. Ingest/Spray data into HPCC System.
2. Process the data
3. In a future step, ROXIE will be used to query the data, and the trained model will be used to display the results.
For more information about the GNN bundle, see the following:
- HPCC Systems Machine Learning Library
- HPCC Systems GNN on GitHub
- Analyze images, videos, time-series, and more with the Generalized Neural Network bundle (GNN) – This is a fantastic blog by Roger Dev (Sr. Architect, LexisNexis Risk Solutions) about the HPCC Systems GNN bundle.
More information about Jack’s project can be accessed using the following links:
Jack Fields’ Poster - https://wiki.hpccsystems.com/display/hpcc/Jack+Fields+-+2020+Poster+Contest+Resources
Jack Fields’ Community Day Presentation - https://www.youtube.com/watch?v=3XgtlsRwCvg&list=PL-8MJMUpp8IJeErGjzXHdz6wkugkIrXQJ&index=9
Jack Fields’ blog - https://hpccrosgnn.wordpress.com/
About Jack Fields
Jack is a 12th-grade high school student at American Heritage School, who has developed an impressive amount of experience in Java, Python, and C++ from the robotics and computer science courses provided by his school, as well as his involvement with the Stallion Robotics Team 5472. The team is run by Tai Donovan (Robotics Program Director and Instructor). Jack is the Director of Programming for the team. He also presented a poster for the HPCC Systems Community Day 2020 Poster contest, where he was the winner of the “Community Choice Award.”
Special thanks go to Jack Fields for his valuable contributions to HPCC Systems in the area of Machine Learning, and to Xiaoming Wang and David DeHilster for their guidance and as mentors. We also thank Tai Donovan (Robotics Program Director and Instructor at American Heritage School) for his outstanding leadership.