‘If I have seen further, it is by standing on the shoulders of giants’
Sir Isaak Newton, 1619
Technical disciplines have always progressed by researchers building on past work, but the deep learning research community does this in spades with transfer learning. Transfer learning builds new deep learning systems on top of previously developed ones.
For example, in my recent iMerit blog, I describe a system to detect Alzheimer’s disease from MRI scans. It was built using a very large convolutional neural network (VGG16) that had been previously trained on 14 million visual images. The Alzheimer’s detection system substituted the last few layers in VGG16 with custom, fully-connected layers. 6400 MRI images were used to train the customer layers, while the parameters of the convolutional layers were ‘frozen’ to their previously trained values.
This approach works because VGG16 had already ‘learned’ some general ‘skills’, like how shape, contrast, and texture contribute to recognizing image differences. Applying this ‘knowledge’ allowed the Alzheimer’s detection system to be trained using a relatively small number of MRI images.
Transfer learning is remarkably easy to implement. The deep learning community has many open source repositories, such as the ONYX Model Zoo, which provide downloadable, pre-trained ML systems. In addition, ML system development environments such as TensorFLow make it easy to load previously trained systems and modify and train custom final layers.
To learn more about how transfer learning works, and how new research is extending the ability of previously trained ML systems to tackle new problems, read my iMerit blog.
thinking-teams.com 