Deep Learning from Scratch: How to Build Neural Networks Step-by-Step
- Published on
Table of Contents
- Introduction
- Understanding Neural Networks
- Step 1: Implementing Neurons and Activation Functions
- Step 2: Forward Propagation
- Step 3: Implementing Loss Functions
- Step 4: Backpropagation and Gradient Descent
- Step 5: Training the Neural Network
- Step 6: Evaluating the Model
- Conclusion
Introduction
Deep Learning is a subset of Machine Learning that mimics the human brain’s neural networks to process and analyze complex data. Most deep learning models are built using libraries like TensorFlow or PyTorch, but understanding how they work at a fundamental level is essential for mastering AI.
In this guide, we will build a neural network from scratch, without using deep learning libraries, and implement all key components step by step.
Understanding Neural Networks
A neural network consists of multiple layers of neurons, which process inputs and adjust their weights based on errors.
Basic Components of a Neural Network
| Component | Description |
|---|---|
| Neuron | Basic computational unit of the network. |
| Activation Function | Introduces non-linearity (e.g., Sigmoid, ReLU). |
| Forward Propagation | Computes predictions by passing data through layers. |
| Loss Function | Measures prediction errors (e.g., Mean Squared Error, Cross-Entropy). |
| Backpropagation | Adjusts weights using gradient descent. |
We will now implement each of these components from scratch.
Step 1: Implementing Neurons and Activation Functions
A neuron takes multiple inputs, applies weights, and passes the result through an activation function.
Code: Implementing a Neuron
import numpy as np
# Neuron with activation function
def sigmoid(x):
return 1 / (1 + np.exp(-x))
class Neuron:
def __init__(self, inputs):
self.weights = np.random.randn(inputs)
self.bias = np.random.randn()
def forward(self, x):
z = np.dot(self.weights, x) + self.bias
return sigmoid(z)
# Example usage
neuron = Neuron(3)
inputs = np.array([0.5, -0.2, 0.1])
output = neuron.forward(inputs)
print("Neuron Output:", output)Step 2: Forward Propagation
Forward propagation computes the predictions by passing inputs through multiple layers.
Code: Implementing Forward Propagation for a Neural Network
class NeuralNetwork:
def __init__(self, input_size, hidden_size, output_size):
self.hidden_layer = Neuron(input_size)
self.output_layer = Neuron(hidden_size)
def forward(self, x):
hidden_output = self.hidden_layer.forward(x)
return self.output_layer.forward(np.array([hidden_output]))
# Example usage
nn = NeuralNetwork(3, 2, 1)
output = nn.forward(np.array([0.5, -0.2, 0.1]))
print("Neural Network Output:", output)Step 3: Implementing Loss Functions
The loss function measures how far predictions are from actual values. We will implement Mean Squared Error (MSE).
Code: Implementing Mean Squared Error
def mse(y_true, y_pred):
return np.mean((y_true - y_pred) ** 2)
# Example
y_true = np.array([1, 0, 1])
y_pred = np.array([0.9, 0.1, 0.8])
loss = mse(y_true, y_pred)
print("MSE Loss:", loss)Step 4: Backpropagation and Gradient Descent
Backpropagation is used to update weights by computing gradients.
Code: Implementing Backpropagation
class NeuralNetwork:
def __init__(self, input_size, hidden_size, output_size, lr=0.01):
self.weights1 = np.random.randn(hidden_size, input_size)
self.bias1 = np.random.randn(hidden_size)
self.weights2 = np.random.randn(output_size, hidden_size)
self.bias2 = np.random.randn(output_size)
self.lr = lr
def forward(self, x):
self.hidden_input = np.dot(self.weights1, x) + self.bias1
self.hidden_output = sigmoid(self.hidden_input)
self.final_input = np.dot(self.weights2, self.hidden_output) + self.bias2
self.final_output = sigmoid(self.final_input)
return self.final_output
def backward(self, x, y_true):
output_error = self.final_output - y_true
hidden_error = np.dot(self.weights2.T, output_error) * self.hidden_output * (1 - self.hidden_output)
self.weights2 -= self.lr * np.outer(output_error, self.hidden_output)
self.bias2 -= self.lr * output_error
self.weights1 -= self.lr * np.outer(hidden_error, x)
self.bias1 -= self.lr * hidden_error
# Example usage
nn = NeuralNetwork(3, 2, 1)
x = np.array([0.5, -0.2, 0.1])
y_true = np.array([1])
for epoch in range(1000):
nn.forward(x)
nn.backward(x, y_true)
print("Trained Output:", nn.forward(x))Step 5: Training the Neural Network
We will now train the network using a dataset.
Code: Training the Neural Network
# Training on a small dataset
X_train = np.array([[0.5, -0.2, 0.1], [0.1, 0.8, -0.5], [-0.4, 0.2, 0.9]])
y_train = np.array([[1], [0], [1]])
nn = NeuralNetwork(3, 2, 1)
for epoch in range(1000):
for x, y in zip(X_train, y_train):
nn.forward(x)
nn.backward(x, y)
# Testing the trained network
print("Final Predictions:")
for x in X_train:
print(nn.forward(x))Step 6: Evaluating the Model
After training, we evaluate the performance.
Code: Evaluating the Model
def accuracy(y_true, y_pred):
return np.mean((y_pred > 0.5) == y_true)
y_pred = np.array([nn.forward(x) for x in X_train])
acc = accuracy(y_train, y_pred)
print("Model Accuracy:", acc)Conclusion
We have built a fully functional neural network from scratch without using TensorFlow or PyTorch. This step-by-step approach covers:
- Neurons and Activation Functions
- Forward Propagation
- Loss Functions
- Backpropagation and Gradient Descent
- Training and Evaluation
Understanding these concepts will help you grasp deep learning fundamentals, allowing you to build and optimize neural networks from first principles.
🚀 Now, try experimenting with different architectures and datasets to improve accuracy!