Transfer Learning

Representations learned on a large general dataset transfer surprisingly well to narrow tasks. Replace the final layer, fine-tune on a few thousand examples, and you often beat a model trained from scratch on far more data.

This is the single most useful practical technique for teams without a research budget, and it is how most applied computer vision and NLP work actually gets done.