Easy explain how a deep learning model is trained step-by-step.

A simple, step-by-step walkthrough of how a deep learning model is trained, from random weights to a fully trained network, with PyTorch code.

5 minutes(715 words)complex

Quick Navigation

Difficulty: Beginner
Estimated Time: 10-15 minutes
Prerequisites: Python basics, Basic linear algebra, PyTorch installed, Calculus fundamentals

Step 1: Start with a model and random weights and biases

A neural network is made of layers of neurons. Each neuron has weights w and bias b that control how strongly it reacts to input.

First we start by setting a random set of w and B

For input x, a neuron calculates w · x + b. Then applies an activation function (like ReLU or sigmoid) to get the output a:

a = f(w · x + b)

Define input, weight, bias with pytorch

import torch
x = torch.tensor([2.0]) # Input
w = torch.randn(1, requires_grad=True) # Random weight
b = torch.randn(1, requires_grad=True) # Random bias

Step 2: Forward pass

We feed an input x into the network and get a prediction ŷ.

For example, if the network predicts a number

Forward pass pytorch

z = w * x + b # Weighted sum
a = torch.sigmoid(z) # Activation function

# Compute weighted sum and apply activation function (sigmoid here)

Step 3: Compute the loss

The loss measures how wrong the prediction is compared to the true label y.

Example: mean squared error (for regression):

L = (ŷ - y)²

Or cross-entropy loss (for classification):

L = -Σ yᵢ log(ŷᵢ)

Loss is just a number telling us how "bad" the prediction is.

Compute loss

y = torch.tensor([4.0]) # Target
loss = (a - y)**2 # MSE loss

Step 4: Backpropagation (compute gradients)

We need to know how to change the weights to reduce the loss.

We compute the gradient, which is the derivative of the loss w.r.t each weight:

∂L/∂w

Intuition: gradient points in the direction that increases the loss. We want to go opposite that direction.

Backpropagation pytorch

loss.backward() # Compute gradients

Step 5: Update the weights (Gradient Descent)

We update weights slightly in the direction that reduces loss:

w w - η · (∂L/∂w)

η is the learning rate, controlling how big a step we take.

Update weights pytorch

learning_rate = 0.1
with torch.no_grad():
    w -= learning_rate * w.grad
    b -= learning_rate * b.grad
    w.grad.zero_()
    b.grad.zero_()

Step 6: Repeat

  • Pick a batch of data x
  • Do forward pass compute y
  • Compute loss
  • Compute gradients
  • Update weights

Repeat thousands of times until loss stops decreasing.

Repeat python

# Repeat steps 2-5 in a loop for many iterations to train the model
for _ in range(1000):
    z = w * x + b
    a = torch.sigmoid(z)
    loss = (a - y)**2
    loss.backward()
    with torch.no_grad():
        w -= learning_rate * w.grad
        b -= learning_rate * b.grad
        w.grad.zero_()
        b.grad.zero_()

Step 7: Model is trained

After many iterations, the weights are adjusted so that the network predicts well on new, unseen data.

Simple example with numbers

Next step, repeat with new w = 1.8 ŷ = 3.6 loss smaller keep updating.

Real PyTorch Example: Multi-Layer, Multi-Neuron Neural Network

import torch
import torch.nn as nn
import torch.optim as optim
# Step 1: Define inputs and targets
# Example: 4 samples, 3 features each
x = torch.tensor([[0.0, 0.0, 1.0],
                  [0.0, 1.0, 1.0],
                  [1.0, 0.0, 1.0],
                  [1.0, 1.0, 1.0]])
y = torch.tensor([[0.0], [1.0], [1.0], [0.0]]) # Example targets
# Step 2: Define a multi-layer neural network
class MultiLayerNN(nn.Module):
    def __init__(self):
        super(MultiLayerNN, self).__init__()
        self.layer1 = nn.Linear(3, 4) # 3 inputs -> 4 neurons
        self.layer2 = nn.Linear(4, 1) # 4 neurons -> 1 output
        self.activation = nn.Sigmoid()

    def forward(self, x):
        x = self.activation(self.layer1(x))
        x = self.activation(self.layer2(x))
        return x
model = MultiLayerNN()
# Step 3: Define loss function and optimizer
criterion = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=0.1)
# Step 4: Training loop
for epoch in range(5000):
    output = model(x)
    loss = criterion(output, y)

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    if epoch % 500 == 0:
        print(f'Epoch {epoch}: Loss = {loss.item():.4f}')
# Step 5: Predictions
predictions = model(x).detach()
print("Predictions:")
print(predictions)

# This example demonstrates a small neural network with 2 layers, multiple neurons, sigmoid activations, and training on a simple dataset.

This example demonstrates a small neural network with 2 layers, multiple neurons, sigmoid activations, and training on a simple dataset.