Categories
28°C, New York
Home Python Python program for backpropagation

Python program for backpropagation

Example of Backpropagation in Machine Learning

Here’s a simple implementation of feedforward neural network with backpropagation in Python:

  1. Neural Network Initialization: The NeuralNetwork class is initialized with parameters for the input size, hidden layer size, and output size. It also initializes the weights and biases with random values.
  2. Sigmoid Activation Function: The sigmoid method implements the sigmoid activation function, which squashes the input to a value between 0 and 1.
  3. Sigmoid Derivative: The sigmoid_derivative method calculates the derivative of the sigmoid function. It computes the gradients of the loss function with respect to weights.
  4. Feedforward Pass: The feedforward method calculates the activations of the hidden and output layers based on the input data and current weights and biases. It uses matrix multiplication to propagate the inputs through the network.
  5. Backpropagation: The backward method performs the backpropagation algorithm. It calculates the error at the output layer and propagates it back through the network to update the weights and biases using gradient descent.
  6. Training the Neural Network: The train method trains the neural network using the specified number of epochs and learning rate. It iterates through the training data, performs the feedforward and backward passes, and updates the weights and biases accordingly.
  7. XOR Dataset: The XOR dataset (X) is defined, which contains input pairs that represent the XOR operation, where the output is 1 if exactly one of the inputs is 1, and 0 otherwise.
  8. Testing the Trained Model: After training, the neural network is tested on the XOR dataset (X) to see how well it has learned the XOR function. The predicted outputs are printed to the console, showing the neural network’s predictions for each input pair.

Python

import numpy as np
  
class NeuralNetwork:
    def __init__(self, input_size, hidden_size, output_size):
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.output_size = output_size
  
        # Initialize weights
        self.weights_input_hidden = np.random.randn(self.input_size, self.hidden_size)
        self.weights_hidden_output = np.random.randn(self.hidden_size, self.output_size)
  
        # Initialize the biases
        self.bias_hidden = np.zeros((1, self.hidden_size))
        self.bias_output = np.zeros((1, self.output_size))
  
    def sigmoid(self, x):
        return 1 / (1 + np.exp(-x))
  
    def sigmoid_derivative(self, x):
        return x * (1 - x)
  
    def feedforward(self, X):
        # Input to hidden
        self.hidden_activation = np.dot(X, self.weights_input_hidden) + self.bias_hidden
        self.hidden_output = self.sigmoid(self.hidden_activation)
  
        # Hidden to output
        self.output_activation = np.dot(self.hidden_output, self.weights_hidden_output) + self.bias_output
        self.predicted_output = self.sigmoid(self.output_activation)
  
        return self.predicted_output
  
    def backward(self, X, y, learning_rate):
        # Compute the output layer error
        output_error = y - self.predicted_output
        output_delta = output_error * self.sigmoid_derivative(self.predicted_output)
  
        # Compute the hidden layer error
        hidden_error = np.dot(output_delta, self.weights_hidden_output.T)
        hidden_delta = hidden_error * self.sigmoid_derivative(self.hidden_output)
  
        # Update weights and biases
        self.weights_hidden_output += np.dot(self.hidden_output.T, output_delta) * learning_rate
        self.bias_output += np.sum(output_delta, axis=0, keepdims=True) * learning_rate
        self.weights_input_hidden += np.dot(X.T, hidden_delta) * learning_rate
        self.bias_hidden += np.sum(hidden_delta, axis=0, keepdims=True) * learning_rate
  
    def train(self, X, y, epochs, learning_rate):
        for epoch in range(epochs):
            output = self.feedforward(X)
            self.backward(X, y, learning_rate)
            if epoch % 4000 == 0:
                loss = np.mean(np.square(y - output))
                print(f"Epoch {epoch}, Loss:{loss}")
  
X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
y = np.array([[0], [1], [1], [0]])
  
nn = NeuralNetwork(input_size=2, hidden_size=4, output_size=1)
nn.train(X, y, epochs=10000, learning_rate=0.1)
  
# Test the trained model
output = nn.feedforward(X)
print("Predictions after training:")
print(output)

Output:

Epoch 0, Loss:0.36270360966344145 Epoch 4000, Loss:0.005546947165311874 Epoch 8000, Loss:0.00202378766386817 Predictions after training: [[0.02477654] [0.95625286] [0.96418129] [0.04729297]]




Backpropagation in Neural Network

Machine learning models learn from data and make predictions. One of the fundamental concepts behind training these models is backpropagation. In this article, we will explore what backpropagation is, why it is crucial in machine learning, and how it works.

Table of Content

  • What is backpropagation?
  • Advantages of Using the Backpropagation Algorithm in Neural Networks
  • Working of Backpropagation Algorithm
  • Example of Backpropagation in Machine Learning
  • Python program for backpropagation

A neural network is a network structure, by the presence of computing units(neurons) the neural network has gained the ability to compute the function. The neurons are connected with the help of edges, and it is said to have an assigned activation function and also contains the adjustable parameters. These adjustable parameters help the neural network to determine the function that needs to be computed by the network. In terms of activation function in neural networks, the higher the activation value is the greater the activation is.

Similar Reads

What is backpropagation?

In machine learning, backpropagation is an effective algorithm used to train artificial neural networks, especially in feed-forward neural networks. Backpropagation is an iterative algorithm, that helps to minimize the cost function by determining which weights and biases should be adjusted. During every epoch, the model learns by adapting the weights and biases to minimize the loss by moving down toward the gradient of the error. Thus, it involves the two most popular optimization algorithms, such as gradient descent or stochastic gradient descent. Computing the gradient in the backpropagation algorithm helps to minimize the cost function and it can be implemented by using the mathematical rule called chain rule from calculus to navigate through complex layers of the neural network....

Advantages of Using the Backpropagation Algorithm in Neural Networks

Backpropagation, a fundamental algorithm in training neural networks, offers several advantages that make it a preferred choice for many machine learning tasks. Here, we discuss some key advantages of using the backpropagation algorithm:...

Working of Backpropagation Algorithm

The Backpropagation algorithm works by two different passes, they are:...

Example of Backpropagation in Machine Learning

Let us now take an example to explain backpropagation in Machine Learning,...

Python program for backpropagation

Here’s a simple implementation of feedforward neural network with backpropagation in Python:...

Tags:
#ML-Statistics #Python-PyTorch #AI-ML-DS #Machine Learning #Machine Learning

Example of Backpropagation in Machine Learning