ИСТИНА

Statistical estimation of the Shannon entropy and various divergences are important for applications, e.g., in machine learning, identification of textures inhomogeneities and feature selection. In this regard one can refer, e.g., to the book by V.Bolon-Canedo and A.Alonso-Betanzos (2018), see also a paper by J.R.Vergara and P.A.Estevez (2014). We develop the quite recent works by A.Bulinski, A.Dimitrov (2018, 2019) and A.Bulinski, A.Kozhevin (2018, 2019) to study statistical estimation of the Shannon entropy, mutual information and other divergences. We investigate the asymptotic properties of proposed estimates constructed by means of i.i.d. (vector-valued) observations. For this purpose we apply the techniques involving the nearest neighbor statistics. Special attention is payed to results of computer simulations in the framework of mixed models (see, e.g. F.Coelho, A.P.Braga, M.Verleysen (2016), W.Gao, S.Kannan, P.Viswanath (2018)) comprising the widely used logistic regression.