Transfer Learning for Deep Learning Applications: Methods, Problems, and a Way to Evaluate Them
Department of Artificial Intelligence and Data Science, Ajeenkya DY Patil School of Engineering, Pune, India · India
Abstract
Deep learning models perform well on image, text and speech problems. The catch is that they normally require a large labelled dataset and powerful hardware, and many small projects lack both. Transfer learning is one way to handle this situation: a model that has already been trained on one task is trained further on a smaller task of one's own. In this paper we review transfer learning for deep learning applications. The work we studied falls into four groups: using a model to extract features, fine-tuning a model, adapting the domain, and changing only a small part of a big model. We also describe the main problems, namely negative transfer, a mismatch between source and target data, overfitting when the dataset is small, and deciding which layers to update. From the review we saw that transfer learning helps a lot. However, the studies are hard to compare, because each uses its own datasets, data sizes and metrics. For this reason we propose a pipeline together with an evaluation plan. The plan looks at four things at once: accuracy, the amount of data needed, training cost, and consistency across repeated runs. This is a pre-implementation paper. The results table is left empty for now and will be filled after the experiments are done.
Keywords
How to Cite This Article
Rights
BODHIVRUKSHA JOURNAL OF DIVERSE DISCIPLINE