I always do a better job learning new things through practical application. Since I am in the journey to transition to a data scientist it left me wondering what I should build to learn the algorithms. It came to me when I was watching my kids play soccer. I figured there must be a better way of doing a post game review with some data augmentation. So here we are.
I want to build an application that can 1) classify objects, 2) track objects and 3) allow for playback with all of this additional information. I also envision having an interface that allows you to select a player and make everything else opaque so you can really focus on that specific player. Then you can take a look at how they are positioning in relation to the ball and to the other players. Then you can take a peek into their game IQ and thought process.
I plan to use the masked region based convolutional neural network (Masked R-CNN) to classify objects such as a player and a ball. Matterport has a GitHub for an implementation that they did, so I am using that as a base implementation. That part is pretty straight forward, because there are already trained data for most of the objects that I am looking for. This is the first level of classification that needs to be done.
Once I have detected players, I need to further classify them as specific players with names. This will be a little more tricky. I think I can freeze the layers of the current model and add layers to do the multi-classification training. One thing that I also want to do it to detect the opponents and maybe classify them if there were seen previously.
I was reading about the challenges with multi-label classifications so this will be a fun challenge to solve for my use case. I am not sure the approach that I will take yet, but I will write more once I get to that point.
I have a GitHub were I am tracking this code.
It's just me 