Model selection is the process of choosing between different models built on the same data for a machine learning project, e.g., default k-NN model vs logistic regression model vs SVM model vs k-NN model with k=20, etc.
Model selection is generally a trade-off amongst certain qualities, including:
Simplicity: humans can easily understand how the model works
Transparency: humans can see why the model makes its predictions; this is especially important in regulated industries such as healthcare and banking
Performance metrics
Speed: to train and test
Scalability: ability to be applied to a large dataset
IMPORTANT: Always evaluate models on validation set.
