# How to Implement a Basic Neural Network from Scratch Using Python

## Learn to build a basic neural network in Python without high-level libraries.

Neural networks are the backbone of deep learning and have revolutionized fields such as computer vision, natural language processing, and more. While powerful libraries like TensorFlow and PyTorch make it easy to build complex models, understanding how to implement a basic neural network from scratch can provide valuable insights into how these models work under the hood.

In this tutorial, we will walk through the steps to create a simple feedforward neural network using Python, without relying on any deep learning libraries. We'll implement the forward pass, backpropagation, and training loop manually.

#### Overview of the Neural Network

A basic neural network consists of layers of neurons that are connected by weights. The network we will build will have:

**Input Layer**: Takes the input features.**Hidden Layer**: A single hidden layer with a configurable number of neurons.**Output Layer**: Provides the final output predictions.

We will use the sigmoid activation function for the hidden layer and a binary cross-entropy loss function for training on a binary classification problem.

#### Step 1: Import Libraries

We'll start by importing the necessary libraries. For this implementation, we only need NumPy.

```
import numpy as np
```

#### Step 2: Initialize the Neural Network

Let's define a simple neural network class with an initializer to set up weights and biases.

```
class SimpleNeuralNetwork:
def __init__(self, input_size, hidden_size, output_size):
# Initialize weights with random values
self.weights_input_hidden = np.random.randn(input_size, hidden_size)
self.weights_hidden_output = np.random.randn(hidden_size, output_size)
# Initialize biases with zeros
self.bias_hidden = np.zeros((1, hidden_size))
self.bias_output = np.zeros((1, output_size))
```

Here, `input_size`

is the number of input features, `hidden_size`

is the number of neurons in the hidden layer, and `output_size`

is the number of output classes (1 for binary classification).

#### Step 3: Define Activation and Loss Functions

We'll use the sigmoid function as our activation function and binary cross-entropy for the loss.

```
def sigmoid(x):
return 1 / (1 + np.exp(-x))
def sigmoid_derivative(x):
return x * (1 - x)
def binary_cross_entropy(y_true, y_pred):
return -np.mean(y_true * np.log(y_pred) + (1 - y_true) * np.log(1 - y_pred))
```

#### Step 4: Implement the Forward Pass

The forward pass computes the output of the neural network for a given input.

```
class SimpleNeuralNetwork:
# ... (same __init__ method)
def forward(self, X):
# Compute hidden layer activation
self.hidden_input = np.dot(X, self.weights_input_hidden) + self.bias_hidden
self.hidden_output = sigmoid(self.hidden_input)
# Compute output layer activation
self.output_input = np.dot(self.hidden_output, self.weights_hidden_output) + self.bias_output
self.output = sigmoid(self.output_input)
return self.output
```

#### Step 5: Implement Backpropagation

Backpropagation calculates the gradients of the loss with respect to each weight, which will be used to update the weights.

```
class SimpleNeuralNetwork:
# ... (same __init__ and forward methods)
def backward(self, X, y, learning_rate):
# Calculate the error in the output
output_error = self.output - y
output_delta = output_error * sigmoid_derivative(self.output)
# Calculate the error in the hidden layer
hidden_error = output_delta.dot(self.weights_hidden_output.T)
hidden_delta = hidden_error * sigmoid_derivative(self.hidden_output)
# Update the weights and biases
self.weights_hidden_output -= self.hidden_output.T.dot(output_delta) * learning_rate
self.bias_output -= np.sum(output_delta, axis=0, keepdims=True) * learning_rate
self.weights_input_hidden -= X.T.dot(hidden_delta) * learning_rate
self.bias_hidden -= np.sum(hidden_delta, axis=0, keepdims=True) * learning_rate
```

#### Step 6: Train the Neural Network

Now we need a method to train the network using forward and backward passes.

```
class SimpleNeuralNetwork:
# ... (same __init__, forward, and backward methods)
def train(self, X, y, epochs, learning_rate):
for epoch in range(epochs):
# Forward pass
self.forward(X)
# Backward pass
self.backward(X, y, learning_rate)
# Compute the loss
loss = binary_cross_entropy(y, self.output)
if epoch % 100 == 0:
print(f"Epoch {epoch}, Loss: {loss}")
```

#### Step 7: Test the Neural Network

To test the neural network, we'll create some dummy data. Let's create a simple dataset for binary classification.

```
# Dummy data for XOR problem
X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
y = np.array([[0], [1], [1], [0]])
# Initialize and train the neural network
nn = SimpleNeuralNetwork(input_size=2, hidden_size=2, output_size=1)
nn.train(X, y, epochs=10000, learning_rate=0.1)
# Test the neural network
output = nn.forward(X)
print("Predictions:")
print(output)
```