Scaling behaviour of entropy estimates

Thomas Schürmann

Published 2002-03-20, updated 2017-04-20Version 2

Entropy estimation of information sources is highly non trivial for symbol sequences with strong long-range correlations. The rabbit sequence, related to the symbolic dynamics of the nonlinear circle map at the critical point as well as the logistic map at the Feigenbaum point have been argued to exhibit long memory tails. For both dynamical systems the scaling behavior of the block entropy of order n has been shown to increase like as log(n). In contrast to probabilistic concepts, we investigate the scaling behavior of certain non-probabilistic entropy estimation schemes suggested by Lempel and Ziv in the context of algorithmic complexity and data compression. These are applied in a sequential manner with the scaling variable being the length N of the sequence. We determine the scaling law for the Lempel-Ziv entropy estimate applied to the case of the critical circle map and the logistic map at the Feigenbaum point in a binary partition.