4 7 1 Gibbs Distribution

1. By Jensen's inequality, we have

(1)

\begin{align} KL (q, \pi) =-\sum_{m} q_m \cdot \log(\pi_m/q_m) \geqslant -\log\left(\sum_m q_m \cdot \frac{\pi_m}{q_m}\right) = 0. \end{align}

2.

• Since $\partial^2 [x\log(x)]/{\partial x^2} = 1/x >0$ over $\mathbb{R}^{+}$, $x\mapsto x\log(x)$ is convex on $\mathbb{R}^+$.

• Let $|\mathcal{M}|$ denote the cardinal number of the set $\mathcal{M}$. Since the Hessian matrix

(2)

\begin{align} \nabla^2 \mathcal{L}^{(\lambda)}(q)=\frac{\sigma^2}{\beta}\mathsf{Diag}(1/q_1,1/q_2,\ldots,1/q_{|\mathcal{M}|}) \end{align}

is positive-definite, $q\mapsto \mathcal{L}^{(\lambda)}(q)$ is convex on $(\mathbb{R}^{+})^{\mathcal{M}}$.

3. For any $m\in\mathcal{M}$, solving the equation

(3)

\begin{align} \frac{\mathcal{L}^{(\lambda)}(q)}{\partial q_m} = \hat{r}_m + \frac{\sigma^2}{\beta}\left[ \log(\frac{q_m}{\pi_m})+1\right] + \lambda = 0, \end{align}

yields that $q_m^{(\lambda)}=\pi_m \exp(-\beta \hat{r}_m/\sigma^2)\exp(-1-\beta\lambda/\sigma^2)$.

4. Follows from 3.