A sequential structure of statistical manifolds on deformed exponential family

07/11/2017
Publication GSI2017
OAI : oai:www.see.asso.fr:17410:22624
contenu protégé  Document accessible sous conditions - vous devez vous connecter ou vous enregistrer pour accéder à ou acquérir ce document.
- Accès libre pour les ayants-droit
 

Résumé

Heavily tailed probability distributions are important objects in anomalous statistical physics. For such probability distributions, expectations do not exist in general. Therefore, an escort distribution and an escort expectation have been introduced. In this paper, by generalizing such escort distributions, a sequence of escort distributions is introduced. For a deformed exponential family, we study the fundamental properties of statistical manifold structures derived from the sequence of escort expectations.

A sequential structure of statistical manifolds on deformed exponential family

Collection

application/pdf A sequential structure of statistical manifolds on deformed exponential family (slides)
application/pdf A sequential structure of statistical manifolds on deformed exponential family Hiroshi Matsuzoe, Antonio M. Scarfone, Tatsuaki Wada
Détails de l'article
contenu protégé  Document accessible sous conditions - vous devez vous connecter ou vous enregistrer pour accéder à ou acquérir ce document.
- Accès libre pour les ayants-droit

Heavily tailed probability distributions are important objects in anomalous statistical physics. For such probability distributions, expectations do not exist in general. Therefore, an escort distribution and an escort expectation have been introduced. In this paper, by generalizing such escort distributions, a sequence of escort distributions is introduced. For a deformed exponential family, we study the fundamental properties of statistical manifold structures derived from the sequence of escort expectations.
A sequential structure of statistical manifolds on deformed exponential family

Média

Voir la vidéo

Métriques

0
0
264.18 Ko
 application/pdf
bitcache://6785669c6afc2094716d79432a134cf8777083fc

Licence

Creative Commons Aucune (Tous droits réservés)

Sponsors

Sponsors Platine

alanturinginstitutelogo.png
logothales.jpg

Sponsors Bronze

logo_enac-bleuok.jpg
imag150x185_couleur_rvb.jpg

Sponsors scientifique

logo_smf_cmjn.gif

Sponsors

smai.png
gdrmia_logo.png
gdr_geosto_logo.png
gdr-isis.png
logo-minesparistech.jpg
logo_x.jpeg
springer-logo.png
logo-psl.png

Organisateurs

logo_see.gif
<resource  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                xmlns="http://datacite.org/schema/kernel-4"
                xsi:schemaLocation="http://datacite.org/schema/kernel-4 http://schema.datacite.org/meta/kernel-4/metadata.xsd">
        <identifier identifierType="DOI">10.23723/17410/22624</identifier><creators><creator><creatorName>Hiroshi Matsuzoe</creatorName></creator><creator><creatorName>Antonio M. Scarfone</creatorName></creator><creator><creatorName>Tatsuaki Wada</creatorName></creator></creators><titles>
            <title>A sequential structure of statistical manifolds on deformed exponential family</title></titles>
        <publisher>SEE</publisher>
        <publicationYear>2018</publicationYear>
        <resourceType resourceTypeGeneral="Text">Text</resourceType><subjects><subject>Information geometry</subject><subject>statistical manifold</subject><subject>escort distribution</subject><subject>escort expectation</subject><subject>deformed exponential family</subject></subjects><dates>
	    <date dateType="Created">Fri 9 Mar 2018</date>
	    <date dateType="Updated">Fri 9 Mar 2018</date>
            <date dateType="Submitted">Tue 13 Nov 2018</date>
	</dates>
        <alternateIdentifiers>
	    <alternateIdentifier alternateIdentifierType="bitstream">6785669c6afc2094716d79432a134cf8777083fc</alternateIdentifier>
	</alternateIdentifiers>
        <formats>
	    <format>application/pdf</format>
	</formats>
	<version>37376</version>
        <descriptions>
            <description descriptionType="Abstract">Heavily tailed probability distributions are important objects in anomalous statistical physics. For such probability distributions, expectations do not exist in general. Therefore, an escort distribution and an escort expectation have been introduced. In this paper, by generalizing such escort distributions, a sequence of escort distributions is introduced. For a deformed exponential family, we study the fundamental properties of statistical manifold structures derived from the sequence of escort expectations.
</description>
        </descriptions>
    </resource>
.

A sequential structure of statistical manifolds on deformed exponential family Hiroshi Matsuzoe1 ? , Antonio M. Scarfone2 , and Tatsuaki Wada3 1 Department of Computer Science and Engineering Graduate School of Engineering, Nagoya Institute of Technology Gokiso-cho, Showa-ku, 466-8555 Nagoya, Japan 2 Istituto dei Sistemi Complessi (ISC-CNR) c/o, Politecnico di Torino Corso Duca degli Abruzzi 24, Torino I-10129, Italy 3 Department of Electrical and Electronic Engineering, Ibaraki University, Nakanarusawa-cho, Hitachi, 316-8511, Japan Abstract. Heavily tailed probability distributions are important ob- jects in anomalous statistical physics. For such probability distributions, expectations do not exist in general. Therefore, an escort distribution and an escort expectation have been introduced. In this paper, by gen- eralizing such escort distributions, a sequence of escort distributions is introduced. For a deformed exponential family, we study the fundamen- tal properties of statistical manifold structures derived from the sequence of escort expectations. Keywords: statistical manifold, escort distribution, escort expectation, deformed exponential family, information geometry 1 Introduction Heavily tailed probability distributions are important objects in anomalous sta- tistical physics (cf. [11] and [15]). Such probability distributions do not have expectations in general. Therefore the notion of escort distribution has been in- troduced [4] in order to give a suitable down weight for heavy tail probability. Consequently, there exists a modified expectation for such a probability distri- butions. For a deformed exponential family, an escort distribution is given by the differential of a deformed exponential function. Therefore, the first named author considered further generalizations of escort distributions In q-exponential case, he introduced a sequential structure of escort distributions [7]. In this paper, we consider a sequential structure of escort distributions on a deformed exponential family. It is known that a deformed exponential fam- ily naturally has at least three kinds of different statistical manifold structures ? This research was partially supported by JSPS (Japan Society for the Promo- tion of Science), KAKENHI (Grants-in-Aid for Scientific Research) Grant Numbers JP26108003, JP15K04842 and JP16KT0132. [8]. We elucidate relations between these statistical manifold structures and the structures derived from the sequence of escort expectations. Consequently, we find that dually flat structures and generalized conformal structures for statisti- cal manifolds naturally arise in this framework. 2 Deformed exponential families Throughout this paper, we assume that all the objects are smooth, In this sec- tion, we summarize foundations of deformed exponential functions and deformed exponential families. For further details, see [11]. Let χ be a strictly increasing function from R++ to R++. We call this func- tion χ a deformation function. By use of a deformation function, we define a χ-exponential function expχ t (or a deformed exponential function) by the eigen- function of the following non-linear differential equation d dt expχ t = χ(expχ t). The inverse of a χ-exponential function is called a χ-logarithm function or a deformed logarithm function, and it is given by lnχ s := Z s 1 1 χ(t) dt. If the deformation function is a power function χ(t) = tq (q > 0, q 6= 1), the deformed exponential and the deformed logarithm are given by expq t := (1 + (1 − q)t) 1 1−q , (1 + (1 − q)t > 0), lnq s := s1−q − 1 1 − q , (s > 0), and they are called a q-exponential and a q-logarithm, respectively We suppose that a statistical model Sχ has the following expression Sχ = ( p(x, θ) p(x; θ) = expχ " n X i=1 θi Fi(x) − ψ(θ) # , θ ∈ Θ ⊂ Rn ) , where F1(x), . . . , Fn(x) are functions on the sample space Ω, θ = t (θ1 , . . . , θn ) is a parameter, and ψ(θ) is the normalization defined by R Ω p(x; θ)dx = 1. We call the statistical model Sχ a χ-exponential family or a deformed exponential family. Under suitable conditions, Sχ is regarded as a manifold with coordinate system θ = (θ1 , . . . , θn ). When the deformed exponential function is a q-exponential, we denote the statistical model by Sq and call it a q-exponential family. We remark that the regularity conditions for Sχ is very difficult. To elucidate such conditions is quite an open problem. For example, regularity conditions for a statistical model (see Chapter 2 in [1]) and the well-definedness of a deformed exponential function should be satisfied simultaneously. A few arguments of this problem is given in the first and the third named author’s previous work [9]. 3 A sequential structure of expectations In this section we consider a sequential structure of expectations. As we will see later, statistical manifold structures are defined from this sequence. Let Sχ = {pθ} = {p(x; θ)} be a χ-exponential family. We say that Pχ(x; θ) is an escort distribution of pθ ∈ Sχ if Pχ(x; θ) := Pχ,(1)(x; θ) := χ(pθ). We say that Pesc χ (x; θ) is a normalized escort distribution of pθ if Pesc χ (x; θ) := Pesc χ,(1)(x; θ) := χ(pθ) Zχ(pθ) , where Zχ(pθ) := Zχ,(1)(pθ) := Z Ω χ(pθ)dx. We generalize the escort distribution by use of higher-order differentials. Definition 1. Let Sχ be a χ-exponential family. Denote by exp (n) χ x the n-th differential of the χ-exponential function. For pθ ∈ Sχ, we define the n-th escort distribution Pχ,(n)(x; θ) by Pχ,(n)(x; θ) := exp(n) χ (lnχ pθ) = exp(n) χ n X i=1 θi Fi(x) − ψ(θ) ! , and the normalized n-th escort distribution Pesc χ,(n)(x; θ) by Pesc χ,(n)(x; θ) := Pχ,(n)(x; θ) Zχ,(n)(pθ) , where Zχ,(n)(pθ) = Z Ω Pχ,(n)(x; θ)dx. For a given function f(x) on Ω, we define the n-th escort expectation of f(x) and the normalized n-th escort expectation of f(x) by Eχ,(n),p[f(x)] := Z Ω f(x)Pχ,(n)(x; θ)dx, Eesc χ,(n),p[f(x)] := Z Ω f(x)Pesc χ,(n)(x; θ)dx, respectively. For example, in the case of q-exponential family Sq, the n-th escort distribu- tion of pq(x; θ) is given by Pq,(n)(x; θ) := {q(2q − 1) · · · ((n − 1)q − (n − 2))}{pq(x; θ)}nq−(n−1) . When we consider geometric structure determined from the unbiasedness of generalized score function, that is, Eχ,(1),p[∂i lnχ p(x; θ)] = 0, a sequential structure of expectations naturally arises. This is one of our motiva- tions to study sequential expectations. When we consider correlations of random variables, another kinds of sequence of expectations will be required. 4 Geometry of statistical models Let (M, g) be a Riemannian manifold, and C be a totally symmetric (0, 3)-tensor field on M. We call the triplet (M, g, C) a statistical manifold [6]. In this case, the tensor field C is called a cubic form. For a given statistical manifold (M, g, C), we can define one parameter family of affine connections by g(∇ (α) X Y, Z) := g(∇ (0) X Y, Z) − α 2 C(X, Y, Z), (1) where α ∈ R and ∇(0) is the Levi-Civita connection with respect to g. It is easy to check that ∇(α) and ∇(−α) are mutually dual with respect to g, that is, Xg(Y, Z) = g(∇ (α) X Y, Z) + g(Y, ∇ (−α) X Z). We say that S is a statistical model if S is a set of probability density functions on Ω with parameter ξ ∈ Ξ such that S =  p(x; ξ) Z Ω p(x; ξ)dx = 1, p(x; ξ) > 0, ξ = (ξ1 , . . . , ξn ) ∈ Ξ ⊂ Rn  . Under suitable conditions, we can define a Fisher metric gF on S by gF ij(ξ) = Z Ω  ∂ ∂ξi ln p(x; ξ)   ∂ ∂ξj ln p(x; ξ)  p(x; ξ) dx (2) = Z Ω  ∂ ∂ξi ln p(x; ξ)   ∂ ∂ξj p(x; ξ)  dx (3) = Ep[∂ilξ∂jlξ], where ∂i = ∂/∂ξi , lξ = l(x; ξ) = ln p(x; ξ), and Ep[f] is the standard expectation of f(x) with respect to p(x; ξ). Next, we define a totally symmetric (0, 3)-tensor field CF by CF ijk(ξ) = Ep [(∂ilξ)(∂jlξ)(∂klξ)] . From equation (1), we can define one parameter family of affine connections. In particular, the connection ∇(e) = ∇(1) is called theexponential connection and ∇(m) = ∇(−1) is called the mixture connection. These connections are given by Γ (e) ij,k(ξ) = Z Ω (∂i∂j ln pξ)(∂kpξ)dx, Γ (m) ij,k (ξ) = Z Ω (∂k ln pξ)(∂i∂jpξ)dx. It is known that gF and CF are independent of the choice of reference measure on Ω. Therefore, the triplet (S, gF , CF ) is called an invariant statistical manifold. If a statistical model S is an exponential family, then the invariant statistical manifold (S, gF , CF ) determines a dually flat structure on S. (See [1] and [13].) However, this fact may not be held for a deformed exponential family Sχ and an invariant structure may not be important for Sχ. Therefore, we consider another statistical manifold structures. We summarize statistical manifold structures for Sχ based on [8]. Let Sχ be a χ-exponential family. We define a Riemannian metric gM by gM ij (θ) := Z Ω (∂i lnχ pθ) (∂jpθ) dx, where ∂i = ∂/∂θi . The Riemannian metric gM is a generalization of the repre- sentation of Fisher metric (3). A pair of dual affine connections are given by Γ M(e) ij,k (θ) = Z Ω (∂i∂j lnχ pθ)(∂kpθ)dx, Γ M(m) ij,k (θ) = Z Ω (∂k lnχ pθ)(∂i∂jpθ)dx. The difference of two affine connections CM ijk = Γ M(m) ij,k − Γ M(e) ij,k determines a cubic form. In addition, from the definition of the deformed exponential family Sχ, Γ M(e) ij,k (θ) always vanishes. Therefore, we have the following proposition. Proposition 1. For a χ-exponential family Sχ, the triplet (Sχ, gM , CM ) is a statistical manifold. In particular, (Sχ, gM , ∇M(e) , ∇M(m) ) is a dually flat space. By setting Uχ(s) := Z s 0 (expχ t) dt, we define a U-divergence [10] by Dχ(p||r) = Z Ω {Uχ(lnχ r(x)) − Uχ(lnχ p(x)) − p(x)(lnχ r(x) − lnχ p(x))}dx. It is known that the U-divergence Dχ(p||r) on Sχ coincides with the canonical divergence for (Sχ, gM , ∇M(m) , ∇M(e) ) (See [8] and [10]). Next, we define another statistical manifold structure from the viewpoint of Hessian geometry. For a χ-exponential family Sχ, suppose that the normalization ψ is strictly convex. Then we can define a χ-Fisher metric gχ and a χ-cubic form Cχ [3] by gχ ij(θ) := ∂i∂jψ(θ), Cχ ijk(θ) := ∂i∂j∂kψ(θ). Obviously, the triplet (Sχ, gχ , Cχ ) is a statistical manifold. From equation (1), we can define a torsion-free affine connection ∇χ(α) by gχ (∇ χ(α) X Y, Z) := gχ (∇ χ(0) X Y, Z) − α 2 Cχ (X, Y, Z), where ∇χ(0) is the Levi-Civita connection with respect to gχ . By standard ar- guments in Hessian geometry [13], (Sχ, gχ , ∇χ(1) , ∇χ(−1) ) is a dually flat space. The canonical divergence for (Sχ, gχ , ∇χ(−1) , ∇χ(1) ) is given by Dχ (p||r) = Eesc χ,r [lnχ r(x) − lnχ p(x)]. 5 Statistical manifolds determined from sequential escort expectations In this section, we consider statistical manifold structures determined from se- quential escort expectations. For a χ-exponential family Sχ, we define g(n) and C(n) by g (n) ij (θ) := Z Ω (∂i lnχ pθ)(∂j lnχ pθ)Pχ,(n)(x; θ)dx, C (n) ijk (θ) := Z Ω (∂i lnχ pθ)(∂j lnχ pθ)(∂k lnχ pθ)Pχ,(n+1)(x; θ)dx. We suppose that g(n) is a Riemannian metric on Sχ. Then we obtain a sequence of statistical manifolds: (Sχ, g(1) , C(1) ) → (Sχ, g(2) , C(2) ) → · · · → (Sχ, g(n) , C(n) ) → · · · . The limit of this sequence is not clear at this moment. In the q-Gaussian case, the sequence of normalized escort distributions {Pesc q,(n)(x; θ)} converges to the Dirac’s delta function δ(x − µ) (cf. [14]). Theorem 1. Let Sq = {p(x; θ)} be a χ-exponential family. Then (Sχ, g(1) , C(1) ) coincides with (Sχ, gM , CM ). Proof. From the definition of χ-logarithm and Pχ(x; θ) = Pχ,(1)(x; θ) = χ(pθ), we obtain (∂i lnχ pθ)Pχ,(1)(x; θ) = ∂ipθ χ(pθ) χ(pθ) = ∂ipθ. Therefore, we obtain gM ij (θ) = Z Ω (∂i lnχ pθ)(∂jpθ)dx = Z Ω (∂i lnχ pθ)(∂j lnχ pθ)Pχ,(1)(x; θ)dx = g(1) (θ). Recall that {θi } is a ∇M(e) -affine coordinate system [8]. In addition, the generalized score function ∂i lnχ pθ is unbiased with respect to the escort expec- tation, that is, Eχ,p[∂i lnχ pθ] = Z Ω (∂i lnχ pθ)Pχ,(1)(x; θ)dx = Z Ω ∂ipθdx = 0. Therefore we obtain CM ijk(θ) = Γ M(m) ij,k (θ) = Z Ω (∂k lnχ pθ)(∂i∂jpθ)dx = Z Ω (∂k lnχ pθ)∂i{(∂j lnχ pθ)Pχ,(1)(x; θ)}dx = 0 + Z Ω (∂k lnχ pθ)(∂j lnχ pθ)(∂i lnχ pθ)Pχ,(2)(x; θ)dx = C (1) ijk(θ). From the second escort expectation, we have the following theorem. Theorem 2. Let Sq = {p(x; θ)} be a χ-exponential family. Then (Sχ, g(2) , C(2) ) and (Sχ, gχ , Cχ ) have the following relations: g (2) ij (x; θ) = Zχ(pθ)gχ ij(θ), C (2) ijk(x; θ) = Zχ(pθ)Cχ ij(θ) + gχ ij(θ)∂kZχ(pθ) + gχ jk(θ)∂iZχ(pθ) + gχ ki(θ)∂jZχ(pθ). Proof. Set u(x) = (expq x)0 . Then we have ∂ip(x; θ) = u X θk Fk(x) − ψ(θ)  (Fi(x) − ∂iψ(θ)) ∂i∂jp(x; θ) = u0 X θk Fk(x) − ψ(θ)  (Fi(x) − ∂iψ(θ))(Fj(x) − ∂jψ(θ)) −u X θk Fk(x) − ψ(θ)  ∂i∂jψ(θ) = Pχ,(2)(x; θ)(∂i lnχ pθ)(∂j lnχ pθ) − Pχ,(1)(x; θ)∂i∂jψ(θ). Since R Ω ∂ip(x; θ)dx = R Ω ∂i∂jp(x; θ)dx = 0 and Zχ(p) = R Ω χ(p(x; θ))dx = R Ω Pχ,(1)(x; θ)dx, we obtain g (2) ij (θ) = Zχ(pθ)gχ ij(θ). From a straight forward calculation, we have ∂i∂j∂kp(x; θ) = u00 X θl Fl(x) − ψ(θ)  ×(Fi(x) − ∂iψ(θ))(Fj(x) − ∂jψ(θ))(Fk(x) − ∂kψ(θ)) −u0 X θl Fl(x) − ψ(θ)  (Fk(x) − ∂kψ(θ))∂i∂jψ(θ) −u0 X θl Fl(x) − ψ(θ)  (Fi(x) − ∂iψ(θ))∂j∂kψ(θ) −u0 X θl Fl(x) − ψ(θ)  (Fj(x) − ∂jψ(θ))∂k∂iψ(θ) −u X θl Fl(x) − ψ(θ)  ∂i∂j∂kψ(θ), (4) ∂iZχ(pθ) = Z Ω ∂iPχ,(1)(x; θ)dx = Z Ω u X θl Fl(x) − ψ(θ)  (Fi(x) − ∂iψ(θ))dx. By integrating (4), we obtain the relation C(2) and Cχ . We remark that the statistical manifold (Sχ, g(2) , C(2) ) cannot determine a dually flat structure in general whereas (Sχ, gχ , Cχ ) determines a dually flat structure. The relations in Theorem 2 imply that two statistical manifolds have a generalized conformal equivalence relation in the sense of Kurose [5]. 6 Concluding remarks In this paper, we considered a sequential structure of escort expectations and statistical manifold structures that are defined from the sequence of escort ex- pectations. Further geometric properties of the sequence {(Sχ, g(n) , C(n) )}n∈N are not clear at this moment. However. the sequential structure will be important in the geometric theory of non-exponential type statistical models. Actually, in the case of q-exponential family, (Sq, g(1) , C(1) ) is induced from a β-divergence. In addition, (Sq, g(2) , C(2) ) are essentially equivalent to the invariant statistical manifold structure (Sq, gF .CF ), which are induced from an α-divergence [7]. The authors would like to express their sincere gratitude to the referees for giving helpful comments to improve this paper. References 1. S. Amari and H. Nagaoka, Method of Information Geometry, Amer. Math. Soc., Providence, Oxford University Press, Oxford, 2000. 2. S. Amari, Information Geometry and Its Applications; Springer; Tokyo, 2016. 3. S. Amari, A. Ohara and H. Matsuzoe, Geometry of deformed exponential families: invariant, dually-flat and conformal geometry, Physica A., 391(2012), 4308-4319 4. C. Beck and F. Schlögl, Thermodynamics of Chaotic Systems: An Introduction; Cambridge University Press: Cambridge, UK, 1993. 5. T. Kurose, On the divergences of 1-conformally flat statistical manifolds, Tôhoku Math. J., 46(1994), 427–433. 6. S. L. Lauritzen, Statistical manifolds, Differential geometry in statistical inferences, IMS Lecture Notes Monograph Series 10, Institute of Mathematical Statistics, Hayward California, (1987), 96–163. 7. H. Matsuzoe, A sequence of escort distributions and generalizations of expectations on q-exponential family, Entropy, 19(2017), no. 1, 7 8. H. Matsuzoe and M. Henmi, Hessian Structures and Divergence Functions on Deformed Exponential Families, Geometric Theory of Information, Signals and Communication Technology, Springer, (2014), 57-80. 9. H. Matsuzoe and T. Wada, Deformed algebras and generalizations of independence on deformed exponential families, Entropy, 17(2015), no.8, 5729–5751. 10. N. Murata, T. Takenouchi, T. Kanamori and S. Eguchi, Information geometry of U-boost and Bregman divergence, Neural Comput., 16(2004), 1437-1481. 11. J. Naudts, Generalised Thermostatistics, Springer-Verlag, 2011. 12. M. Sakamoto and H. Matsuzoe, A Generalization of Independence and Multivariate Studentfs t-distributions, Lecture Notes in Comput. Sci., 9389, Springer-Cham, (2015), 740–749. 13. H. Shima, The Geometry of Hessian Structures, World Scientific, 2007. 14. M. Tanaka, Meaning of an escort distribution and τ-transformation, J. Phys. Conf. Ser, 201(2010), 012007. 15. C. Tsallis, Introduction to Nonextensive Statistical Mechanics: Approaching a Com- plex World, Springer, New York, 2009.