Some advice on when a deep neural network may or may not outperform Support Vector Machines or Random Forests. By Sebastian Raschka , Michigan State University. comments If we tackle a supervised learning problem, my advice is to start with the simplest hypothesis space first. I.e., try a linear model such as logistic regression. If this doesn't work "well" (i.e., it doesn't meet our expectation or performance criterion that we defined earlier), I would move on to the next experiment.