# A short primer

This note collects the definitions implemented in `qjax` and their `q → 1`
limits. Throughout, $q \in \mathbf{R}$ is the *entropic index*; `q = 1` recovers ordinary
Boltzmann–Gibbs statistics.

## 1. Deformed logarithm and exponential

The two foundational maps are

$$
\ln_q(x) = \frac{x^{1-q} - 1}{1 - q}, \qquad
\exp_q(x) = \big[1 + (1-q)\,x\big]_+^{\frac{1}{1-q}},
$$

where $[\cdot]_+ = \max(\cdot, 0)$ enforces the **Tsallis cut-off**. They are
mutual inverses and satisfy $\ln_q \to \ln$, $\exp_q \to \exp$ as $q \to 1$.

`qjax` evaluates both with the *double-`where`* trick: the indeterminate
$0/0$ at $q = 1$ is replaced by the analytic limit, and the unused branch is fed
a sanitized argument so that **gradients stay finite everywhere**, including
exactly at $q = 1$.

### q-algebra

The deformed logarithm turns products into a deformed sum, and the deformed
exponential turns sums into a deformed product:

$$
a \oplus_q b = a + b + (1-q)\,ab, \qquad
a \otimes_q b = \big[a^{1-q} + b^{1-q} - 1\big]_+^{\frac{1}{1-q}}.
$$

These give `q_add`/`q_diff` and `q_prod`/`q_div`. They obey
$\ln_q(xy) = \ln_q x \oplus_q \ln_q y$ and
$\exp_q(x+y) = \exp_q x \otimes_q \exp_q y$.

## 2. Tsallis entropy and divergences

The Tsallis entropy of a distribution $p$ is

$$
S_q(p) = \frac{1 - \sum_i p_i^{\,q}}{q - 1}
\;\xrightarrow{q \to 1}\; -\sum_i p_i \ln p_i .
$$

It is concave, non-negative for probability vectors, and maximized by the uniform
distribution. The associated **relative entropy** (q-divergence) is

$$
D_q(p \,\|\, r) = \frac{\sum_i p_i^{\,q} r_i^{1-q} - 1}{q - 1}
\;\xrightarrow{q \to 1}\; \mathrm{KL}(p \,\|\, r),
$$

and the **cross-entropy** used as a classification loss is
$H_q(y, p) = -\sum_i y_i \ln_q p_i$.

## 3. The q-Gaussian

Maximizing $S_q$ under a fixed second moment yields the **q-Gaussian**:

$$
p(x) = \frac{\sqrt{\beta}}{C_q}\,\exp_q\!\big(-\beta x^2\big).
$$

- $q < 1$: compact support.
- $q = 1$: the ordinary Gaussian, $C_1 = \sqrt{\pi}$.
- $1 < q < 3$: heavy (power-law) tails; for $q = 2$ this is the **Cauchy**
  distribution, and in general it is a rescaled **Student-$t$** with
  $\nu = (3-q)/(q-1)$ degrees of freedom.

The variance is finite only for $q < 5/3$, where
$\operatorname{Var} = 1/\big((5 - 3q)\,\beta\big)$. `qjax.sample` exploits the
Student-$t$ relationship, $X = T_\nu / \sqrt{(3-q)\beta}$, which reproduces this
variance exactly (supported for $1 \le q < 3$).

## 4. Tsallis entmax (q-deformed softmax)

Regularizing the maximum-score problem with Tsallis entropy,

$$
\mathrm{entmax}_q(z) = \arg\max_{p \in \Delta}\;
    \langle p, z \rangle + S_q^{T}(p),
$$

gives a probability map with the closed form
$p_i = \big[(q-1) z_i - \tau\big]_+^{1/(q-1)}$, where $\tau$ enforces
$\sum_i p_i = 1$. Then:

- $q = 1$ → **softmax** (dense),
- $q = 2$ → **sparsemax** (sparse; many coordinates are exactly zero),
- intermediate $q$ → a tunable trade-off between smoothness and sparsity.

`qjax` solves for $\tau$ by bisection on the tight bracket
$[(q-1)\max_i z_i - 1,\; (q-1)\max_i z_i]$, which is fully compatible with
`jit`, `grad`, and `vmap`.

## References

- C. Tsallis, *Introduction to Nonextensive Statistical Mechanics*, Springer, 2009.
- W. Thistleton, J. Marsh, K. Nelson, C. Tsallis, "Generalized Box–Müller method
  for generating q-Gaussian random deviates", *IEEE Trans. Inf. Theory*, 2007.
- B. Peters, V. Niculae, A. Martins, "Sparse Sequence-to-Sequence Models"
  (entmax), *ACL*, 2019.