1. Webcam Mouse
This project is a branch-off from my main undergraduate research on Train Tracking and Classification. The original goal I had in mind was to control a mouse cursor on my screen by using object detection. From that, an interest in depth-based actions, ergonomics, and diverse movement became additional goals of the project. the image below, you can see my example application of stem cells being outlined.
Software and Features:
The code is written in MatLab utilizing various computer vision packages. The main functions that the software currently contains are: mouse movement (x, y), mouse left-click, and mouse-right click. The mouse movement is based on the location of a colored tracker in the live webcam feed. The mouse click functions are based on conditional statements dependent on the scale of the colored tracker, e.g. when the tracker is closer to the camera, the software sees it as a larger object, and performs an action. Another method I have tried is using a downward motion with my finger, similar to what a mouse click would actually be like, and found similar results.
As this project is still in progress, I have not yet included a more detailed process and results section. The next step for this project is to ensure that smaller movements can be registered such as the movement of a finger side-to-side.
2. Train Tracking Software - Undergraduate Research
Object detection is a very practical field of programming. There are many physical bodies in this world that are hard to count, track, or categorize with the naked eye alone. Determining the winning car in a head-to-head formula one race, tracking the flight patterns of a flock of geese in the winter, watching stem cells connect to heart cells and become tissue, all of these are examples of physical phenomena that can be simplified greatly with object detection and object tracking software. In the image below, you can see my example application of stem cells being outlined.
For the use of quantitative analysis for moving train cars on a railway, this project results in a piece of software that utilizes three key components, blob Analysis, Kalman filters and probability calculation, to track train cars in motion and compare the sizes of each car to a database of known cars to determine its type and how many similar cars are grouped together. The final result of this software allows the user to input an .mp4 video file of a train passing perpendicularly by on tracks, and immediately display the same video back with data and bounding boxes surrounding each train car. In this particular instance, railroad companies can then begin to analyze the data that this software collects to determine the flow and general grouping of train cars. For example, if a company wanted to know how many uniform cargo trains versus non-uniform trains they have going through a certain area, they can use this software to analyze that. That data can then be used to potentially optimize the flow of trains in and out of the area. This type of information is crucial for railroad companies as one of their biggest costs as a company is the fuel for these trains, coal. If the distances traveled can be reduced by flow optimization, then that amount of coal used can be greatly reduced. There are many other real world applications of this software that range anywhere from biological cell counting to tracking star locations in the sky, in that it is meant for object detection.
The first step to this project is to find a video that is suitable for the task. On YouTube.com, there are many videos of trains going by on a train track; however, there are very few that are recorded perpendicular to the tracks and that stay still for the duration of the video. These two steps are crucial when the background and foreground detection begins. After finding a few of these videos, they were imported into MatLAB, and the operation began. By using the Computer Vision toolbox in MatLAB, a toolbox that allows for video processing in an easy way, and the flowchart below, a smooth tracking program was achieved.
In the figure below, one may notice three different aspects of the frame: The background video, a colored overlay, and a statistics portion at the top left. The background video is a video of a train moving along tracks, which contains multiple types of cargo. It has Double Boxcars, as seen in the figure, Single Boxcars, Oil Tankers, Flat Cars, and a few other obscure classifications. The colored overlay that can be seen is a bounding box that surrounds each car as it travels through the frame. The first time an object gets detected that is greater than a certain size, it gets a yellow bounding box with an index on it. If the object is detected for more than the “minimum frames to be seen” constant, it becomes a “Reliable Car,” and the overlay turns green. Adding in this minimum frames to be seen option bypasses any stray artifacts in the video, such as noise. The statistics in the top left corner of the frame are: The current total count of reliable cars that have been seen since the object detection begin, e.g. since the beginning of the video that was imported. The size in pixels gets displayed on the next line, and finally that size gets compared to a database of approximate sizes and the type of car is estimated.Here is a link to the source code for the project on GitHub
3. Felt Cutter - Practical Arduino Application
There was an issue with a popular rim-striping tool, the Stripe-it-All in which customers were unable to properly cut a 2" x 2" square felt pad into 1/8", 1/4" and 3/8" strips. In order to make the customers happier at a low cost, a small Arduino powered stepper motor circuit was developed to automatically extrude strips of those increments toward the blade of a paper cutter. Manual action is still required to slice and bag the products, but the overall process time is greatly improved, and the customers save time - Introducing, the felt cutter!
The body is made of aluminum, and there is a channel that the two-inch wide felt strip roll is fed into, with a stepper motor with gripping wheels that grabs hold of the felt. One variable of the machining that had to be carefully designed was the height at which the motor would sit where there would be not too much and not too little friction/grip.
During the initial testing, a 12V power source was being used, but there wasn't enough torque to spin the motor, so the motor was skipping a few steps. We decided on a higher voltage supply. The breakout board was designed by Jason Traud at the electronics shop next door, which essentially routed the 24V power, as well as the 5V signal from the Arduino to the stepper motor. There is a potentiometer on the motor controller that allows for control of the current going to the motor. Both the Uno and the breakout board are mounted to the power supply, which is then mounted to the felt cutter.
The stepper motor in this design is a SY42STH47-1206A, which has a 1.8 degree step angle for a total of 200 steps/revolution. When paired with the Bane Bot wheels, which have a diameter of four inches, equating to a circumference of 12.56 inches, this gives a ratio of circumference to the steps-per-rev of 0.0628. Therefore for the 1/8" strips, for example, the stepper motor would have to spin 1.989 steps, assuming no slippage occurs.
The chip on the Arduino Uno is the ATMEGA328P, with a 16MHz external clock. All of the programming is done in the Arduino 1.6.4 IDE. The basic loop of the program is as follows: Detect a button press input, enter state machine to determine current length to cut, e.g. if previous was 1/8", cut a 1/4", then a 3/4" strip, bit bang stepper pulse PWM using digitalWrite and delayMicroseconds, and finally delay.
A few important additions to the code are a button debounce, and a flag to be raised indicating that the program is currently sending the PWM, so as not to duplicate the signal. The stepper motor cannot be driven by a perfect PWM square wave, as it can only deliver a certain amount of torque, so a ramp of stepper pulses eases the motor into each motion. Everything is mounted to the paper cutter with screws, and a lever-button is attached underneath the arm of the slicer so it can feed the felt every time the arm is raised. I used this device to bag hundreds if a few thousand of these felt strips so it definitely saved us some time.
Below is a link to a video of the system in action. Thanks for reading!