Neural Network
Connectionist learning model inspired by neurons
A Neural Network is a mathematical model inspired by neuron connections in the human brain. It consists of an Input Layer, Hidden Layer(s), and an Output Layer, where each neuron has weights and biases. It computes predictions via a forward pass, measures errors with a loss function, then updates weights through backpropagation and gradient descent. Repeating this process thousands to millions of times allows the network to learn patterns.
Key Concepts
Input data is passed to neurons in the input layer
Each neuron computes (input x weight + bias) and applies an activation function (ReLU, Sigmoid, etc.)
Final predictions are generated at the output layer through hidden layers (forward pass)
The loss function calculates the difference (error) between predictions and ground truth
Backpropagation computes each weight's contribution to the error (gradients)
Weights are updated via gradient descent โ this process repeats every epoch
Pros
- ✓ Can learn complex nonlinear patterns
- ✓ Highly versatile (images, text, audio, etc.)
- ✓ Performance improves with more data
- ✓ Automates feature engineering
Cons
- ✗ Requires large amounts of data and compute resources
- ✗ Black box โ difficult to interpret why results occur
- ✗ Risk of overfitting
- ✗ Complex hyperparameter tuning