Approximating the softmax function by converting to a binary tree. Probabilities are normalised because sum of any path to a leaf is $1$.
Hierarchical Softmax
Max Bartolo
AI Research Scientist at Cohere working on Large Language Models