Hessian structures on deformed exponential families

28/08/2013
OAI : oai:www.see.asso.fr:2552:5123
DOI : You do not have permission to access embedded form.

Résumé

Hessian structures on deformed exponential families

Média

Voir la vidéo

Métriques

65
7
98.26 Ko
 application/pdf
bitcache://551f16ba6157a2d7fe14b05626c8d7ff53bd991b

Licence

Creative Commons Aucune (Tous droits réservés)

Sponsors

Sponsors scientifique

logo_smf_cmjn.gif

Sponsors financier

logo_gdr-mia.png
logo_inria.png
image010.png
logothales.jpg

Sponsors logistique

logo-minesparistech.jpg
logo-universite-paris-sud.jpg
logo_supelec.png
Séminaire Léon Brillouin Logo
logo_cnrs_2.jpg
logo_ircam.png
logo_imb.png
<resource  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                xmlns="http://datacite.org/schema/kernel-4"
                xsi:schemaLocation="http://datacite.org/schema/kernel-4 http://schema.datacite.org/meta/kernel-4/metadata.xsd">
        <identifier identifierType="DOI">10.23723/2552/5123</identifier><creators><creator><creatorName>Hiroshi Matsuzoe</creatorName></creator><creator><creatorName>Masayuki Henmi</creatorName></creator></creators><titles>
            <title>Hessian structures on deformed exponential families</title></titles>
        <publisher>SEE</publisher>
        <publicationYear>2013</publicationYear>
        <resourceType resourceTypeGeneral="Text">Text</resourceType><dates>
	    <date dateType="Created">Tue 15 Oct 2013</date>
	    <date dateType="Updated">Sun 25 Dec 2016</date>
            <date dateType="Submitted">Sun 18 Feb 2018</date>
	</dates>
        <alternateIdentifiers>
	    <alternateIdentifier alternateIdentifierType="bitstream">551f16ba6157a2d7fe14b05626c8d7ff53bd991b</alternateIdentifier>
	</alternateIdentifiers>
        <formats>
	    <format>application/pdf</format>
	</formats>
	<version>25068</version>
        <descriptions>
            <description descriptionType="Abstract"></description>
        </descriptions>
    </resource>
.

Hessian structures on deformed exponential families MATSUZOE Hiroshi Nagoya Institute of Technology joint works with HENMI Masayuki (The Institute of Statistical Mathematics) 1 Statistical manifolds and statistical models 2 Deformed exponential family 3 Geometry of deformed exponential family (1) 4 Geometry of deformed exponential family (2) 5 Maximum q-likelihood estimator 1 PRELIMINARIES 1 Preliminaries 1.1 Geometry of statistical models   Definition 1.1 S is a statistical model or a parametric model on Ω def ⇐⇒ S is a set of probability densities with parameter ξ ∈ Ξ s.t. S = { p(x; ξ) ∫ Ω p(x; ξ)dx = 1, p(x; ξ) > 0, ξ ∈ Ξ ⊂ Rn } .   We regard S as a manifold with a local coordinate system {Ξ; ξ1 , . . . , ξn }   gF = (gF ij) is the Fisher metric (Fisher information matrix) of S def ⇐⇒ gF ij(ξ) := ∫ Ω ∂ ∂ξi log p(x; ξ) ∂ ∂ξj log p(x; ξ)p(x; ξ)dx = ∫ Ω ∂ipξ ( ∂ ∂ξj log pξ ) dx = Eξ[∂ilξ∂jlξ]   ∂ipξ def ⇐⇒ mixture representation, ∂ilξ = ( ∂ipξ pξ ) def ⇐⇒ exponential representation. (the score function) 2/29 1 PRELIMINARIES A statistical model Se is an exponential family def ⇐⇒ Se = { p(x; θ) p(x; θ) = exp[C(x) + n∑ i=1 θi Fi(x) − ψ(θ)] } , C, F1, · · · , Fn : random variables on Ω ψ : a function on the parameter space Θ The coordinate system [θi ] is called the natural parameters.   Proposition 1.2 For an exponential family Se, (1) ∇(1) is flat (2) [θi ] is an affine coordinate, i.e., Γ (1) k ij ≡ 0   For simplicity, assume that C = 0. gF ij(θ) = E[(∂i log p(x; θ))(∂j log p(x; θ))] = E[−∂i∂j log p(x; θ)] = E[∂i∂jψ(θ)] = ∂i∂jψ(θ) :the Fisher metric CF ijk(θ) = E[(∂i log p(x; θ))(∂j log p(x; θ))(∂k log p(x; θ))] = ∂i∂j∂kψ(θ) :the cubic form The triplets (Se, ∇(e) , gF ) and (Se, ∇(m) , gF ) are Hessian manifolds. Remark: (S, ∇(α) , gF ) is an invariant statistical manifold. 3/29 1 PRELIMINARIES Normal distributions Ω = R, n = 2, ξ = (µ, σ) ∈ R2 + (the upper half plane). S = { p(x; µ, σ) p(x; µ, σ) = 1 √ 2πσ exp [ − (x − u)2 2σ2 ]} The Fisher metric is (gij) = 1 σ2 ( 1 0 0 2 ) ( S is a space of constant negative curvature − 1 2 ) .   ∇(1) and ∇(−1) are flat affine connections. In addition, θ1 = µ σ2 , θ2 = − 1 2σ2 ψ(θ) = − (θ1 )2 4θ2 + 1 2 log ( − π θ2 ) =⇒ p(x; µ, σ) = 1 √ 2πσ exp [ − (x − u)2 2σ2 ] = exp [ xθ1 + x2 θ2 − ψ(θ) ] . {θ1 , θ2 }: natural parameters. (∇(1) -geodesic coordinate system) η1 = E[x] = µ, η2 = E [ x2 ] = σ2 + µ2 . {η1, η2}: moment parameters. (∇(−1) -geodesic coordinate system)   4/29 1 PRELIMINARIES Finite sample space Ω = {x0, x1, · · · , xn}, dim Sn = n p(xi; η) = { ηi (1 ≤ i ≤ n) 1 − ∑n j=1 ηj (i = 0) Ξ = { {η1, · · · , ηn} ηi > 0 (∀ i), ∑n j=1 ηj < 1 } (an n-dimensional simplex) The Fisher metric: (gij) = 1 η0      1 + η0 η1 1 · · · 1 1 1 + η0 η2 ... ... ... ... 1 · · · · · · 1 + η0 ηn      , where η0 = 1 − n∑ j=1 ηj. ( Sn is a space of constant positive curvature 1 4 ) . 5/29 1 PRELIMINARIES Finite sample space Ω = {x0, x1, · · · , xn}, dim Sn = n p(xi; η) = { ηi (1 ≤ i ≤ n) 1 − ∑n j=1 ηj (i = 0) Ξ = { {η1, · · · , ηn} ηi > 0 (∀ i), ∑n j=1 ηj < 1 } (an n-dimensional simplex)   {θ1 , · · · , θn }: natural parameters. (∇(1) -geodesic coordinate system) where θi = log p(xi) − log p(x0) = log ηi 1 − ∑n j=1 ηj ψ(θ) = log  1 + n∑ j=1 eθj   {η1, · · · , ηn}: moment parameters. (∇(−1) -geodesic coordinate sys- tem)   6/29 1 PRELIMINARIES   Proposition 1.3 For Se, the following hold: (1) (Se, gF , ∇(e) , ∇(m) ) is a dually flat space. (2) {θi } is a ∇(e) -affine coordinate system on Se. (3) ψ(θ) is the potential of gF w.r.t. {θi }: gF ij(θ) = ∂i∂jψ(θ). (4) Set the expectations of Fi(x) by ηi =Eθ[Fi(x)] =⇒ {ηi} is the dual coordinate system of {θi } with respect to gM . (5) Set ϕ(η) = Eθ[log pθ]. =⇒ ϕ(η) is the potential of gF w.r.t. {ηi}.   Since (Se, gF , ∇(e) , ∇(m) ) is a dually flat space, the Legendre transfor- mation holds. ∂ψ ∂θi = ηi, ∂ϕ ∂ηi = θi , ψ(p) + ϕ(p) − m∑ i=1 θi (p)ηi(p) = 0 gF ij = ∂2 ψ ∂θi∂θj , CF ijk = ∂3 ψ ∂θi∂θj∂θk 7/29 1 PRELIMINARIES Kullback-Leibler divergence (or relative entropy on S def ⇐⇒ DKL(p, r) = ∫ Ω p(x) log p(x) r(x) dx = Ep[log p(x) − log r(x)] ( = ψ(r) + ϕ(p) − n∑ i=1 θi (r)ηi(p) = D(r, p) ) For Se, DKL coincides with the canonical divergence D on a dually flat space (Se, ∇(m) , gF ). Construction of a divergence from an estimating function  s(x; ξ) =   ∂/∂ξ1 log p(x; ξ) ... ∂/∂ξn log p(x; ξ)  : the score function of p(x; ξ) (estimating function) by Integrating of the score function and by taking an expectation, dKL(p, r) := ∫ Ω p(x; ξ) log r(x; ξ′ )dx the cross entropy on S The KL-divergence is given by the difference of cross entropies. DKL(p, r) = dKL(p, p) − dKL(p, r)   8/29 1 PRELIMINARIES 1 Statistical manifolds and statistical models 2 Deformed exponential family 3 Geometry of deformed exponential family (1) 4 Geometry of deformed exponential family (2) 5 Maximum q-likelihood estimator 9/29 2 DEFORMED EXPONENTIAL FAMILY (χ-EXP. FAMILY) 2 Deformed exponential family (χ-exp. family) χ : (0, ∞) → (0, ∞) : strictly increasing χ-exponential, χ-logarithm  Definition 2.1 logχ x := ∫ x 1 1 χ(t) dt χ-logarithm expχ x := 1 + ∫ x 0 λ(t)dt χ-exponential where λ(logχ t) = χ(t)   Usually, the χ-exponential is called ϕ-exponential in statistical physics. In this talk, ϕ is used as the dual potential on a dually flat space.   Example 2.2 In the case χ(t) = tq , we have ∫ x 1 1 χ(t) dt = ∫ x 1 1 tq dt = x1−q − 1 1 − q = logq x q-logarithm λ(t) = (1 + (1 − q)t) q 1−q 1 + ∫ x 0 λ(t) dt = (1 + (1 − q)x) 1 1−q q-exponential   10/29 2 DEFORMED EXPONENTIAL FAMILY (χ-EXP. FAMILY) χ : (0, ∞) → (0, ∞) : strictly increasing χ-exponential, χ-logarithm  Definition 2.1 logχ x := ∫ x 1 1 χ(t) dt χ-logarithm expχ x := 1 + ∫ x 0 λ(t)dt χ-exponential where λ(logχ t) = χ(t)   F1(x), . . . , Fn(x) : functions on Ω θ = {θ1 , . . . , θn } : parameters S = { p(x, θ) p(x; θ) > 0, ∫ Ω p(x; θ)dx = 1 } : statistical model   Definition 2.3 Sχ = {p(x; θ)} : χ-exponential family, deformed exponential family def ⇐⇒ Sχ := { p(x, θ)p(x; θ) = expχ [ n∑ i=1 θi Fi(x) − ψ(θ) ] , p(x, θ) ∈ S }   11/29 2 DEFORMED EXPONENTIAL FAMILY (χ-EXP. FAMILY)   Proposition 2.4 (discrete distributions) The set of discrete distributions is a χ-exponential family for any χ   (Proof) Ω = {x0, x1, . . . , xn} Sn = { p(x; η) ηi > 0, n∑ i=0 ηi = 1, p(x; η) = n∑ i=0 ηiδi(x) } , η0 = 1 − n∑ i=1 ηi Set θi = logχ p(xi) − logχ p(x0) = logχ ηi − logχ η0 Then logχ p(x) = logχ ( n∑ i=0 ηiδi(x) ) = n∑ i=1 ( logχ ηi − logχ η0 ) δi(x) + logχ(η0) ψ(θ) = − logχ η0 12/29 2 DEFORMED EXPONENTIAL FAMILY (χ-EXP. FAMILY) Finite sample space Ω = {x0, x1, · · · , xn}, dim Sn = n p(xi; η) = { ηi (1 ≤ i ≤ n) 1 − ∑n j=1 ηj (i = 0) Ξ = { {η1, · · · , ηn} ηi > 0 (∀ i), ∑n j=1 ηj < 1 } (an n-dimensional simplex)   {θ1 , · · · , θn }: natural parameters. (∇(1) -geodesic coordinate system) where θi = log p(xi) − log p(x0) = log ηi 1 − ∑n j=1 ηj ψ(θ) = log  1 + n∑ j=1 eθj   {η1, · · · , ηn}: moment parameters. (∇(−1) -geodesic coordinate sys- tem)   13/29 2 DEFORMED EXPONENTIAL FAMILY (χ-EXP. FAMILY) Example 2.5 (Student t-distribution (q-normal distribution)) Ω = R, n = 2, ξ = (µ, σ) ∈ R2 + (the upper half plane), q > 1. p(x; µ, σ) = 1 zq [ 1 − 1 − q 3 − q (x − µ)2 σ2 ] 1 1−q Set θ1 = 2 3 − q zq−1 q · µ σ2 , θ2 = − 1 3 − q zq−1 q · 1 σ2 . Then logq pq(x) = 1 1 − q (p1−q − 1) = 1 1 − q { 1 z1−q q ( 1 − 1 − q 3 − q (x − µ)2 σ2 ) − 1 } = 2µzq−1 q (3 − q)σ2 x − zq−1 q (3 − q)σ2 x2 − zq−1 q 3 − q · µ2 σ2 + zq−1 q − 1 1 − q = θ1 x + θ2 x2 − ψ(θ) ψ(θ) = − (θ1 )2 4θ2 − zq−1 q − 1 1 − q   The set of Student t-distributions is a q-exponential family.   14/29 2 DEFORMED EXPONENTIAL FAMILY (χ-EXP. FAMILY) 1 Statistical manifolds and statistical models 2 Deformed exponential family 3 Geometry of deformed exponential family (1) 4 Geometry of deformed exponential family (2) 5 Maximum q-likelihood estimator 15/29 3 GEOMETRY OF DEFORMED EXPONENTIAL FAMILY (1) 3 Geometry of deformed exponential family (1) Sχ : a deformed exponential family ψ(θ) : strictly convex (normalization for Sχ)   sχ (x; θ) = ( (sχ )1 (x; θ), . . . , (sχ )n (x; θ) )T is the χ-score function def ⇐⇒ (sχ )i (x; θ) = ∂ ∂θi logχ p(x; θ), (i = 1, . . . , n). (1)   Statistical structure for Sχ  Riemannian metric gM : gM ij (θ) = ∫ Ω ∂ip(x; θ)∂j logχ p(x; θ) dx Dual affine connections ∇M(e) , ∇M(m) : Γ M(e) ij,k (θ) = ∫ Ω ∂kp(x; θ)∂i∂j logχ p(x; θ)dx Γ M(m) ij,k (θ) = ∫ Ω ∂i∂jp(x; θ)∂k logχ p(x; θ)dx   (Sχ, ∇M(e) , gM ) and (Sχ, ∇M(m) , gM ) are Hessian manifolds. 16/29 3 GEOMETRY OF DEFORMED EXPONENTIAL FAMILY (1)   Proposition 3.1 For Sχ, the following hold: (1) (Sχ, gM , ∇M(e) , ∇M(m) ) is a dually flat space. (2) {θi } is a ∇M(e) -affine coordinate system on Sχ. (3) Ψ(θ) is the potential of gM with respect to {θi }, that is, gM ij (θ) = ∂i∂jΨ(θ). (4) Set the expectations of Fi(x) by ηi = Eθ[Fi(x)]. =⇒ {ηi} is the dual coordinate system of {θi } with respect to gM . (5) Set Φ(η) = −Iχ(pθ). =⇒ Φ(η) is the potential of gM with respect to {ηi}.   Iχ(pθ) = − ∫ Ω {Uχ(p(x; θ)) + (p(x; θ) − 1)Uχ(0)} dx, where Uχ(t) = ∫ t 1 logχ(s) ds, Uχ(0) = lim t→+0 Uχ(t) < ∞. the generalized entropy functional Ψ(θ) = ∫ Ω p(x; θ) logχ p(x; θ)dx + Iχ(pθ) + ψ(θ), the generalized Massieu potential 17/29 3 GEOMETRY OF DEFORMED EXPONENTIAL FAMILY (1) Construction of β-divergence (β = 1 − q)   uq(x; θ): a weighted score function def ⇐⇒ uq(x; θ) = (u1 q(x; θ), . . . , un q (x; θ))T ui q(x; θ) = p(x; θ)1−q si (x; θ) − Eθ[p(x; θ)1−q si (x; θ)].   From the definition of q-logarithm function, uq(x; θ) is written by ui q(x; θ) = ∂ ∂θi { 1 1 − q p(x; θ)1−q − 1 2 − q ∫ Ω p(x; θ)2−q dx } = ∂ ∂θi logq p(x; θ) − Eθ [ ∂ ∂θi logq p(x; θ) ] Hence, this estimating function is the bias-corrected q-score function. 18/29 3 GEOMETRY OF DEFORMED EXPONENTIAL FAMILY (1) By integrating uq(x; θ), and taking the expectation, we define a cross entropy by d1−q(p, r) = − 1 1 − q ∫ Ω p(x; θ)r(x; θ)1−q + 1 2 − q ∫ Ω r(x; θ)2−q dx Then the β-divergence (β = 1 − q) is given by D1−q(p, r) = −d1−q(p, p) + d1−q(p, r) = 1 (1 − q)(2 − q) ∫ Ω p(x)2−q dx − 1 1 − q ∫ Ω p(x)r(x)1−q dx + 1 2 − q ∫ Ω r(x)2−q dx Remark 3.2 A β-divergence D1−q induces Hessian manifolds (Sq, ∇M(m) , gM ) and (Sq, ∇M(e) , gM ). 19/29 3 GEOMETRY OF DEFORMED EXPONENTIAL FAMILY (1) 1 Statistical manifolds and statistical models 2 Deformed exponential family 3 Geometry of deformed exponential family (1) 4 Geometry of deformed exponential family (2) 5 Maximum q-likelihood estimator 20/29 4 GEOMETRY OF DEFORMED EXPONENTIAL FAMILY (2) 4 Geometry of deformed exponential family (2)   Definition 4.1 Pχ(x) : the escort distribution of p(x; θ), def ⇐⇒ Pχ(x; θ) = 1 Zχ(θ) χ(p(x; θ)), Zχ(θ) = ∫ Ω χ(p(x; θ))dx Eχ,θ[f(x)] : the χ-expectation of p(x) def ⇐⇒ the expectation of f(x) with respect to the escort distribution: Eχ,θ[f(x)] = ∫ f(x)Pχ(x; θ)dx = 1 Zχ(θ) ∫ f(x)χ(p(x; θ))dx     Definition 4.2 Sχ = {p(x; θ)}: a deformed exponential family gχ ij(θ) = ∂i∂jψ(θ) : the χ-Fisher information metric Cχ ijk(θ) = ∂i∂j∂kψ(θ) : the χ-cubic form   Set Γ χ(e) ij,k := Γ χ(0) ij,k − 1 2 Cχ ijk, Γ χ(m) ij,k := Γ χ(0) ij,k + 1 2 Cχ ijk, 21/29 4 GEOMETRY OF DEFORMED EXPONENTIAL FAMILY (2)   Proposition 4.3 For Sχ, the following hold: (1) (Sχ, gχ , ∇χ(e) , ∇χ(m) ) is a dually flat space. (2) {θi } is a ∇χ(e) -affine coordinate system on Sχ. (3) ψ is the potential of gχ with respect to {θi }, that is, gχ ij(θ) = ∂i∂jψ(θ). (4) Set the χ-expectation of Fi(x) by ηi = Eχ,θ[Fi(x)]. =⇒ {ηi} is the dual coordinate system of {θi } with respect to gχ . (5) Set ϕ(η) = Eχ,θ[logχ p(x; θ)] =⇒ ϕ(η) is the potential of gχ with respect to {ηi}.   Proof: Statements 1, 2 and 3 are obtained from the definition of χ-Fisher metric and χ-cubic form. Statements 4 and 5 follow the fact that Eχ,θ[logχ p(x; θ)] = Eχ,θ [ n∑ i=1 θi Fi(x) − ψ(θ) ] = n∑ i=1 θi ηi − ψ(θ) 22/29 4 GEOMETRY OF DEFORMED EXPONENTIAL FAMILY (2) The generalized relative entropy (or χ-relative entropy) of Sχ by Dχ (p, r) = Eχ,p[logχ p(x) − logχ r(x)]. The generalized relative entropy Dχ of Sχ coincides with the canonical divergence D(r, p) on (Sχ, ∇χ(e) , gχ ). In fact, Dχ (pθ, rθ′) = Eχ,p [( n∑ i=1 θi Fi(x) − ψ(θ) ) − ( n∑ i=1 (θ′ )i Fi(x) − ψ(θ′ ) )] = ψ(θ′ ) + ( n∑ i=1 θi ηi − ψ(θ) ) − n∑ i=1 (θ′ )i ηi = D(rθ′, pθ). Tsallis relative entropy (q-exponential case)  Dq (p, r) = Eq,p [ logq p(x) − logq r(x) ] = 1 − ∫ p(x)q r(x)1−q dx (1 − q)Zq(p) = q Zq(p) D(1−2q) (p, r). The Tsallis relative entropy is conformal to α-divergence (α = 1−2q).   23/29 4 GEOMETRY OF DEFORMED EXPONENTIAL FAMILY (2) The generalized relative entropy (or χ-relative entropy) of Sχ by Dχ (p, r) = Eχ,p[logχ p(x) − logχ r(x)]. Construction of χ-relative entropy  sχ (x; θ): the χ-score function def ⇐⇒ (sχ )i (x; θ) = ∂ ∂θi logχ p(x; θ), (i = 1, . . . , n). p The χ-score is unbiased w.r.t. χ-expectation, Eχ,θ[(sχ )i (x; θ)] = 0. =⇒ We regard that sχ (x; θ) is a generalization of estimating function. By integrating the χ-score function, we define the χ-cross entropy by dχ (p, r) = − ∫ Ω P (x) logχ r(x)dx. Then we obtain the generalized relative entropy by Dχ (p, r) = −dχ (p, p) + dχ (p, r) = Eχ,p[logχ p(x) − logχ r(x)].   24/29 5 MAXIMUM Q-LIKELIHOOD ESTIMATORS 5 Maximum q-likelihood estimators 5.1 The q-independence X ∼ p1(x), Y ∼ p2(y) X and Y are independent def ⇐⇒ p(x, y) = p1(x)p2(y). ⇐⇒ p(x, y) = exp [log p1(x) + log p2(x)] (p1(x) > 0, p2(y) > 0)   x > 0, y > 0 and x1−q + y1−q − 1 > 0 (q > 0). x ⊗q y : the q-product of x and y def ⇐⇒ x ⊗q y := [ x1−q + y1−q − 1 ] 1 1−q = expq [ logq x + logq y ]   expq x ⊗q expq y = expq(x + y), logq(x ⊗q y) = logq x + logq y. X and Y : q-independent with m-normalization (mixture normalization) def ⇐⇒ pq(x, y) = p1(x) ⊗ p2(y) Zp1,p2 where Zp1,p2 = ∫ ∫ XY p1(x) ⊗q p2(y)dxdy 25/29 5 MAXIMUM Q-LIKELIHOOD ESTIMATORS 5.2 Geometry for q-likelihood estimators Sq = {p(x; ξ)|ξ ∈ Ξ} : a q-exponential family {x1, . . . , xN} : N-observations from p(x; ξ) ∈ Sq.   Lq(ξ) : q-likelihood function def ⇐⇒ Lq(ξ) = p(x1; ξ) ⊗q p(x2; ξ) ⊗q · · · ⊗q p(xN; ξ) ( ⇐⇒ logq Lq(ξ) = N∑ i=1 logq p(xi; ξ) )   In the case q → 1, Lq is the standard likelihood function on Ξ.   expq(x1 + x2 + · · · + xN) = expq x1 ⊗q expq x2 ⊗q · · · ⊗q expq xN = expq x1 · expq ( x2 1 + (1 − q)x1 ) · · · expq ( xN 1 + (1 − q) ∑N−1 i=1 xi )   Each measurement influences the others.    26/29 5 MAXIMUM Q-LIKELIHOOD ESTIMATORS 5.2 Geometry for q-likelihood estimators Sq = {p(x; ξ)|ξ ∈ Ξ} : a q-exponential family {x1, . . . , xN} : N-observations from p(x; ξ) ∈ Sq.   Lq(ξ) : q-likelihood function def ⇐⇒ Lq(ξ) = p(x1; ξ) ⊗q p(x2; ξ) ⊗q · · · ⊗q p(xN; ξ) ( ⇐⇒ logq Lq(ξ) = N∑ i=1 logq p(xi; ξ) )   In the case q → 1, Lq is the standard likelihood function on Ξ.   ˆξ : the maximum q-likelihood estimator def ⇐⇒ ˆξ = arg max ξ∈Ξ Lq(ξ) ( = arg max ξ∈Ξ logq Lq(ξ) ) .     the q-likelihood is maximum ⇐⇒ the canonical divergence (Tsallis relative entropy) is minimum.   27/29 5 MAXIMUM Q-LIKELIHOOD ESTIMATORS Summary (in the case of q-exponential) β-divergence (Sq, gM , ∇M(e) , ∇M(m) )  estimating function uq(x; θ): ui q(x; θ) = ∂ ∂θi logq p(x; θ) − Eθ [ ∂ ∂θi logq p(x; θ) ] Riemannian metric gM : gM ij (θ) = ∫ Ω ∂ip(x; θ)∂j logq p(x; θ)dx dual coordinates {ηi}: ηi = Ep[Fi(x)]   Tsallis relative entropy (Sq, gq , ∇q(e) , ∇q(m) )  estimating function (sq )(x; θ): (sq )i (x; θ) = ∂ ∂θi logq p(x; θ) (unbiased under q-expectation) Riemannian metric gq : gq ij(θ) = ∂2 ∂θiθj ψ(θ) dual coordinates {ηi}: ηi = Eq,p[Fi(x)]   28/29 5 MAXIMUM Q-LIKELIHOOD ESTIMATORS Summary (in the case of q-exponential) β-divergence (Sq, gM , ∇M(e) , ∇M(m) )  estimating function uq(x; θ): ui q(x; θ) = ∂ ∂θi logq p(x; θ) − Eθ [ ∂ ∂θi logq p(x; θ) ] Riemannian metric gM : gM ij (θ) = ∫ Ω ∂ip(x; θ)∂j logq p(x; θ)dx dual coordinates {ηi}: ηi = Ep[Fi(x)]   Tsallis relative entropy (Sq, gq , ∇q(e) , ∇q(m) )  estimating function (sq )(x; θ): (sq )i (x; θ) = ∂ ∂θi logq p(x; θ) (unbiased under q-expectation) Riemannian metric gq : gq ij(θ) = ∂2 ∂θiθj ψ(θ) dual coordinates {ηi}: ηi = Eq,p[Fi(x)]   The notion of expectations, independence are determined from a geometric structure of the statistical model. 29/29