Grouped-Query Attention (GQA) is an optimization method for transformer attention mechanisms that divides query heads into groups, with each group sharing a single key and value head. - View it on GitHub