A perceptron is the simplest type of artificial neural network and a foundational building block for machine learning and deep learning. It is a supervised learning algorithm designed for binary classification (answering a yes/no or 0/1 question).
How a Perceptron Works
To understand the theory behind it, we can break down its components:
- Perceptron Inputs (nodes)
- Node values (1, 0, 1, 0, 1)
- Node Weights (0.7, 0.6, 0.5, 0.3, 0.4)
- Summation
- Treshold Value
- Activation Function
- Summation (sum > treshold)
A perceptron mimics a single biological nerve cell (neuron). It processes input data through four core components: [1, 2]
- Inputs (\(x_1, x_2, \dots, x_n\)): The measurable features or signals fed into the model. [1]
- Weights (\(w_1, w_2, \dots, w_n\)): Numerical values assigned to each input that determine its importance or strength. Higher weights mean that specific input has a bigger impact on the final decision. [1]
- Bias (b): A constant added to the weighted sum that shifts the decision boundary up or down, allowing the model to fit the data better even when inputs are zero. [1]
- Activation Function: A mathematical rule—traditionally a Heaviside step function—that takes the weighted sum and turns it into a binary output (typically
0or1)
The Math
- Weighted Sum (z):
\(z=(w_{1}\cdot x_{1}+w_{2}\cdot x_{2}+\dots +w_{n}\cdot x_{n})+b\) - Output (y):
- If z ≥ 0, output = 1
- If z < 0, output = 0
How It Learns
The perceptron learns by trial and error through an iterative training process: [1, 2]
- Prediction: It makes a guess on a training example.
- Error Calculation: It compares its prediction to the true label.
- Weight Update: If the prediction is wrong, it adjusts the weights and bias using a learning rate to steer the decision boundary in the right direction.
———————————————————
Example Usage Python: Training the Perceptron on the ‘OR’ gate
———————————————————
import numpy as np
class Perceptron:
def init(self, input_size, learning_rate=0.1, epochs=10):
# Initialize weights and bias to zeros or small random numbers
self.weights = np.zeros(input_size)
self.bias = 0.0
self.lr = learning_rate
self.epochs = epochs
def activation_function(self, x):
# Step function: returns 1 if input >= 0, else 0
return 1 if x >= 0 else 0
def predict(self, inputs):
# Calculate weighted sum + bias
linear_output = np.dot(inputs, self.weights) + self.bias
return self.activation_function(linear_output)
def train(self, training_inputs, labels):
# Training loop over epochs
print(‘START TRAINING – EPOCHS\n’, training_inputs, ‘\n Number of epochs:’, self.epochs)
for _ in range(self.epochs):
for inputs, label in zip(training_inputs, labels):
prediction = self.predict(inputs)
# Update rule: weight = weight + (learning_rate * error * input)
error = label – prediction
self.weights += self.lr * error * inputs
self.bias += self.lr * error
#print(‘inputs: ‘, inputs, ‘prediction: ‘, prediction, ‘weights: ‘, self.weights, ‘bias: ‘, self.bias)
if name == ‘main‘:
# Training data for logic OR: (A, B) -> Output
X_train = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
y_train = np.array([0, 1, 1, 1])
# Create and train the model
perceptron = Perceptron(input_size=2, learning_rate=0.1, epochs=10)
perceptron.train(X_train, y_train)
# Test the trained model
print(‘Testing Perceptron on OR gate:’)
for inputs in X_train:
result = perceptron.predict(inputs)
print(f’Input: {inputs} -> Predicted Output: {result}’)
10 total views, 10 views today
