softmax

Softmax takes an N-dimensional vector as input and outputs an N-dimensional vector of probabilities which sums up to 1. It is a generalised version of logistic regression but with multi classes instead of two classes hence it can be used for multi-class classification. Softmax equation: $$A = \frac{e^z}{\Sigma e^z}$$ The ground truth value for a softmax function is a one-hot encoded vector. Cross-entropy loss Cross entropy loss can be derived from the below expression....