Shannon’s differential entropy asymptotic analysis in a Bayesian problem

Kelbert, Mark and Mozgunov, Pavel (2015) Shannon’s differential entropy asymptotic analysis in a Bayesian problem. Mathematical Communications, 20 (2). pp. 219-228. ISSN 1331-0623

Full text not available from this repository.


We consider a Bayesian problem of estimating of probability of success in a series of conditionally independent trials with binary outcomes. We study the asymptotic behaviour of differential entropy for posterior probability density function conditional on $x$ successes after $n$ conditionally independent trials, when $n \to \infty$. Three particular cases are studied: $x$ is a proportion of $n$; $x$ $\sim n^\beta$, where $0<\beta<1$; either $x$ or $n-x$ is a constant. It is shown that after an appropriate normalization in the first and second cases limiting distribution is Gaussian and the differential entropy of standardized RV converges to differential entropy of standard Gaussian random variable. In the third case the limiting distribution in not Gaussian, but still the asymptotic of differential entropy can be found explicitly.

Item Type:
Journal Article
Journal or Publication Title:
Mathematical Communications
Uncontrolled Keywords:
ID Code:
Deposited By:
Deposited On:
19 Aug 2016 16:00
Last Modified:
09 Mar 2021 05:52