**I HAVE MADE A NICER VERSION OF THIS HERE.**

I’ve been playing about with the Perceptron in SciKit Learn but was having trouble getting to to accurately solve a linear separability problem. The problem is clearly solvable and works in Matlab, however I could not get it to work in Python. Anyways whilst writing this post, originally title ‘please help me’ I had an idea, I tested it and it worked.

So now we have a linear separability example using a single perceptron.

### Load the required elements

1 2 3 4 5 6 7 |
import numpy as np import matplotlib.pyplot as plt #import sklearn.linear_model.perceptron as p from sklearn.linear_model import perceptron # Needed to show the plots inline %matplotlib inline |

### Set the data

1 2 3 4 5 6 7 8 |
# Data d = np.array([ [2, 1, 2, 5, 7, 2, 3, 6, 1, 2, 5, 4, 6, 5], [2, 3, 3, 3, 3, 4, 4, 4, 5, 5, 5, 6, 6, 7] ]) # Labels t = np.array([0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 1, 1, 1, 1]) |

### Plot the Data (see what it looks like)

1 2 |
colormap = np.array(['r', 'k']) plt.scatter(d[0], d[1], c=colormap[t], s=40) |

See, the data is clearly separable. Matlab gets it without any problems.

### Work some magic

Here was the original problem, the data rotation was wrong. Originally only had 1 rotation (90 degrees) meaning the data (d or d90) wasn’t matching the labels (t).

1 2 3 4 5 6 7 8 9 10 11 12 13 |
# rotate the data 180 degrees d90 = np.rot90(d) d90 = np.rot90(d90) d90 = np.rot90(d90) # Create the model net = perceptron.Perceptron(n_iter=100, verbose=0, random_state=None, fit_intercept=True, eta0=0.002) net.fit(d90,t) # Print the results print "Prediction " + str(net.predict(d90)) print "Actual " + str(t) print "Accuracy " + str(net.score(d90, t)*100) + "%" |

As we can see, the model has *solved* the problem.

1 2 3 |
Prediction [0 0 0 1 1 0 0 1 0 0 1 1 1 1] Actual [0 0 0 1 1 0 0 1 0 0 1 1 1 1] Accuracy 100.0% |

### Plot the Decision Boundary

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
# Plot the original data plt.scatter(d[0], d[1], c=colormap[t], s=40) # Output the values print "Coefficient 0 " + str(net.coef_[0,0]) print "Coefficient 1 " + str(net.coef_[0,1]) print "Bias " + str(net.intercept_) # Calc the hyperplane (decision boundary) ymin, ymax = plt.ylim() w = net.coef_[0] a = -w[0] / w[1] xx = np.linspace(ymin, ymax) yy = a * xx - (net.intercept_[0]) / w[1] # Plot the line plt.plot(yy,xx, 'k-') |

### Test with more data

First we create some random data in NumPy.

1 2 3 4 5 6 7 8 9 |
nX = np.random.random_integers(10, size=(2,50)) # Have a look at it print nX # Rotate it the same as the previous data nX90 = np.rot90(nX) nX90 = np.rot90(nX90) nX90 = np.rot90(nX90) |

### Predication Time

So now we have the model (called net) we can run the new dummy data through it to make it use a prediction. (At some point I will create a post on the maths behind this).

1 2 3 4 5 |
# Set predication as the predication results prediction = net.predict(nX90) # Print to have a look print prediction |

You should be looking at lots of zeros and ones, that is the predication. The values are either a 0 or a 1, or another way of thinking about it Group A(0) or Group B(1).

Lets plot the new data and the predictions onto a scatter graph.

1 2 3 4 5 6 7 8 9 10 |
# Plot the nX random values AND the prediction as a color plt.scatter(nX[0],nX[1], c=colormap[prediction],s=40) # Now plot the hyperplane ymin, ymax = plt.ylim() # Calc w = net.coef_[0] a = -w[0] / w[1] xx = np.linspace(ymin, ymax) yy = a * xx - (net.intercept_[0]) / w[1] plt.plot(yy,xx, 'k-') |

Lets have a look…

Fantastic. So what are we looking at? Simple, the colour represents the group so anything to the right of the line **should** be black and anything to the left **should** be red. And it is…!

### Summary

This is a starting point for machine learning. A single perceptron based on a neuron which has been trained using a supervised method and dataset.

So what next I hear you cry, well next would be following two topics both leading to the same place. Non-Linear Problems and Multi Layer Perceptrons.

A nice example of a Non-Linear Problem can be seen in the XOR as plotted below, you cannot drawn a single straight line to separate the two colours.

Comments are closed.