How to perform multi-class SVM in python

Question

I'm able to understand how to code a binary SVM, for example a simple 1, -1 label. However I am going outside my comfort zone to try and perform multi-class and in effect multi-label SVM. However, I can't find anywhere how to do it.

I am going to use the iris data set, which has three classes. So how do you perform three lables? Is it simply -2, -1 , 1 or 2, 1 , -1?

For example, what differs in the way we train a SVM with two classes then having three. I am trying to implement this from scratch to really get a good understanding rather then just use libraries to get me through.

If anyone can find a good example, or possibly explain the process that would be fantastic. Thanks for your time

I recommended looking into the One vs Rest and One vs One approach to multi-class classification. Python has a library called sklearn that has a lot of solid resources and information about such topics, along with the tools to implement (though it sounds like you'll abstain from using the latter). — gallen
– gallen, Commented Jul 9, 2020 at 1:40
You may find useful info in "sklearn multiclass svm function" stackoverflow.com/questions/49848453/… — Roohollah Etemadi
– Roohollah Etemadi, Commented Jul 9, 2020 at 1:49

Ryan S · Accepted Answer · 2020-07-09 16:05:18Z

3

I believe the sklearn library would be helpful here. The sklearn.svm.SVC class is "capable of performing binary and multi-class classification on a dataset.", according to the scikit-learn documentation (https://scikit-learn.org/stable/modules/svm.html).

The labels can actually take any set of integers, as long as they are distinct (e.g. {-1, 1, 2} and {0, 1, 2} and {1, 2, 3} are all valid). Generally, I believe it is best practice to use {0, 1, 2, ..., N} for your label assignments.

Please see the code below for an example:

import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC

N = 1000  # Number of samples

# Create synthetic dataset
X1 = np.random.normal(loc=0, scale=1, size=(N, 2))
Y1 = 0 * np.ones(shape=(1000,))  # LABEL = 0 

X2 = np.random.normal(loc=[-5, 5], scale=1, size=(N, 2))
Y2 = 1 * np.ones(shape=(1000,))  # LABEL = 1

X3 = np.random.normal(loc=[5, -5], scale=1, size=(N, 2))
Y3 = 2 * np.ones(shape=(1000,))  # LABEL = 2

# Create stacked dataset
X = np.vstack((X1, X2, X3))
Y = np.hstack((Y1, Y2, Y3))

# TRAIN SVM LEARNING ALGORITHM
clf = SVC(kernel='linear')
clf = clf.fit(X, Y)

# create decision boundary plot
xx, yy = np.meshgrid(
    np.arange(-10, 10, 0.2),
    np.arange(-10, 10, 0.2))
Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)

# PLOT EVERYTHING
plt.scatter(X1[:,0], X1[:,1], color='r')
plt.scatter(X2[:,0], X2[:,1], color='b')
plt.scatter(X3[:,0], X3[:,1], color='y')
plt.contourf(xx,yy,Z,cmap=plt.cm.coolwarm, alpha=0.8)
plt.title("SVM With Linear Kernel and Three Labels (0, 1, 2)")
plt.show()

Hope this helps!

edited Jul 9, 2020 at 16:05

answered Jul 9, 2020 at 3:43

Ryan S

4793 silver badges7 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Willdomybest18 Over a year ago

Thanks, although I want three labels. Here you have shown two labels which is easy enough to understand, but with three I don't know how the code changes.

Ryan S Over a year ago

@Willdomybest18 my apologies for not clarifying. I've edited my answer above to reflect this, but in short I believe any label assignment works for multi-class SVM, as long as the label assignments are unique (e.g. {-1, 1, 2}, {0, 1, 2}, and {1, 2, 3} would all be appropriate for three-label SVM).

Collectives™ on Stack Overflow

How to perform multi-class SVM in python

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related