Support Vector Machines (SVMs) are more advanced classification models that can provide good performance even in high-dimensional spaces and with little data. We discuss how the basic hard-margin SVM works, and why it is stable and not prone to overfitting. Afterward, we move on to the more advanced soft-margin SVM, which can also deal with non-separable problems and outliers. Finally, we discuss how kernels can be used to reduce the amount of computation time and memory needed to build non-linear SVMs.

After this section you should be able to:

• • Explain the concept of margin, support vectors, support vector machine
  • Recognize when feature scaling leads to suboptimal SVM classifiers and explain why
  • Can give two examples that motivate a soft-margin SVM formulation
  • Can compute the hinge loss
  • Can explain the influence of the parameter C on soft-margin SVM training