Classification

<aside> 💡

What is classification?

Turning input features into a category (class). Examples: sentiment (+ / −), image labels, medical diagnosis.

</aside>

Common applications

• Sentiment analysis (positive / negative)
• Image classification (cat / dog / car)
• Multi-class problems (many classes)
• Medical diagnosis, spam detection, etc.

Linear classifier

<aside> 💡

A linear classifier is a broad category of algorithms that make a classification decision based on a linear combination of the features.

</aside>

• Idea: a linear classifier computes a score from features and picks a class based on that score. Training means learning the coefficients (weights).
• In simpler terms, it separates data points into different classes using a straight line or a flat plane (called a hyperplane in higher dimensions).

Model (score and bias):

$$ \begin{align*}

\text{score}(x) &= w_0 + w_1 x_1 + \dots + w_p x_p = \mathbf{w}^\top \mathbf{x}

\end{align*} $$

Decision rule:

$$ \begin{align*} \hat{y} &= \begin{cases} +1 & \text{if } \text{score}(x) \ge 0 \ -1 & \text{if } \text{score}(x) < 0 \end{cases} \end{align*} $$

• The line/plane where $\text{score}(x)=0$ is the decision boundary.