Computational information geometry in statistics: mixture modelling

28/08/2013
Auteurs : Paul Marriott
OAI : oai:www.see.asso.fr:2552:4898
DOI :

Résumé

Computational information geometry in statistics: mixture modelling

Média

Voir la vidéo

Métriques

72
8
358.22 Ko
 application/pdf
bitcache://d3c771b22a03dd2ad19a1e3b6d7321074fa65dea

Licence

Creative Commons Aucune (Tous droits réservés)

Sponsors

Sponsors scientifique

logo_smf_cmjn.gif

Sponsors financier

logo_gdr-mia.png
logo_inria.png
image010.png
logothales.jpg

Sponsors logistique

logo-minesparistech.jpg
logo-universite-paris-sud.jpg
logo_supelec.png
Séminaire Léon Brillouin Logo
logo_cnrs_2.jpg
logo_ircam.png
logo_imb.png
<resource  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                xmlns="http://datacite.org/schema/kernel-4"
                xsi:schemaLocation="http://datacite.org/schema/kernel-4 http://schema.datacite.org/meta/kernel-4/metadata.xsd">
        <identifier identifierType="DOI">10.23723/2552/4898</identifier><creators><creator><creatorName>Paul Marriott</creatorName></creator></creators><titles>
            <title>Computational information geometry in statistics: mixture modelling</title></titles>
        <publisher>SEE</publisher>
        <publicationYear>2013</publicationYear>
        <resourceType resourceTypeGeneral="Text">Text</resourceType><dates>
	    <date dateType="Created">Tue 17 Sep 2013</date>
	    <date dateType="Updated">Wed 31 Aug 2016</date>
            <date dateType="Submitted">Fri 10 Aug 2018</date>
	</dates>
        <alternateIdentifiers>
	    <alternateIdentifier alternateIdentifierType="bitstream">d3c771b22a03dd2ad19a1e3b6d7321074fa65dea</alternateIdentifier>
	</alternateIdentifiers>
        <formats>
	    <format>application/pdf</format>
	</formats>
	<version>25095</version>
        <descriptions>
            <description descriptionType="Abstract"></description>
        </descriptions>
    </resource>
.

CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Computational information geometry in statistics: mixture modelling Paul Marriott University of Waterloo GSI2013 - Geometric Science of Information CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Overview • Joint work with Karim Anaya-Izquierdo, Frank Critchley and Paul Vos • This paper applies the tools of computation information geometry, see Frank’s talk • High dimensional extended multinomial families as proxies for the ‘space of all distributions’ • Look in the inferentially demanding area of statistical mixture modelling. CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Overview • We look at, and show the relationship between, different geometrically approaches to mixture modelling • Lindsay’s data dependent, finite dimensional affine space • Our, extended multinomial embedding space • Show a new algorithm which uses the full Information Geometry of the problem to its advantage • Exploit the idea of polytope approximation in the ‘correct’ geometry CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixture models • Mixture models form an extremely flexible class of models • Used when some data not observed, hidden dependence structures or when there is unexplained heterogeneity • They are of the form ρifX (x; θi) or fX (x; θ)dQ(θ) • Consider ρ0N(µ0, σ2 0) + ρ1N(µ1, σ2 1) + ρ2N(µ2, σ2 2) CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixtures 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Mixing distribution ρρ1 ρρ2 q −3 −2 −1 0 1 2 3 0.00.51.01.5 Mixed density x Density CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Convex Geometry • Inference for mixture models can be problematic • They can be ‘too flexible’ hence can overfit • The likelihood function can have multiple modes, singularities and be unbounded • The underlying structure is not a manifold so have to be careful using calculus • Inference questions where Z ∼ f(z; θ)dQ(θ) 1 what can we learn about E(Z)? 2 what can we learn about Q? 3 Can we predict the next value of Z? CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Mixture of binomial distributions • First example comes from Kupper and Haseman (1978) • Concerns frequency of death of implanted foetuses in laboratory animals • It could be expected that there is underlying clustering - hence mixture modelling is appropriate • Paper states: ‘simple one-parameter binomial and Poisson models generally provide poor fits to this type of binary data’ • It is of interest to look in a ‘neighbourhood’ of these models. • The extended multinomial space is a natural place to define such a ‘neighbourhood’ • Our new computational algorithm is used for inference. CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Tripod model • Second example is the tripod, discussed in Zwiernik and Smith (2011) • Graphical model given by 2 1 H 3 • Binary variables Xi, i = 1, 2, 3, on each of the terminal nodes, these being assumed independent given the binary variable at the internal node H • H is unobserved • Get very complex likelihood structure - problematic for inference CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Extended Multinomial • Look at discrete models • Space of distributions simplicial • Boundaries where probabilities are zero • Information geometry of extended multinomial models • Applications to graphical models and elsewhere • Proxy for space of all models • IG explicit: computable? CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Convex Geometry • Lindsay’s (1995) fundamental result characterises the maximum likelihood estimate in the class of all mixtures of fX (x; θ) • Finds Q which maximises the likelihood of f(x, θ)dQ(θ) over all possible Q when f(x, θ) is exponential family • This is called the Non-parametric maximum likelihood estimate of Q. • Uses results from finite dimensional convex geometry • Tangent spaces replaced by tangent cones • Asymptotic limits are mixtures of χ2 distributions. CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Convex Geometry • Lindsay’s geometry lies in an affine space which is determined by the observed data. • In particular, it is always finite dimensional, and the dimension is determined by the number of distinct observation • Define Lθ = (L1(θ), . . . , LN∗ (θ)) represent the N∗ distinct likelihood values. • The likelihood on the space of mixtures is defined on the convex hull of the image of the map θ → (L1(θ), . . . , LN∗ (θ)) ⊂ RN∗ . • Find the non-parametric likelihood estimate, f(y; Q), maximising a concave function over this convex set. CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Embedding in Extended Multinomial • In our examples can embed families in different affine space • Assume a discrete sample space and data is of the form (n0, n1, . . . , nk ) • Define ∆k := π = (π0, π1, . . . , πk) : πi ≥ 0 , k i=0 πi = 1 • Embed unmixed model in ∆k and look convex hull • Define the observed face P to be determined by index set of the strictly positive observed counts. • The affine structure of Lindsay is determined by the vertices of P (Theorem 1) CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Embedding in Extended Multinomial • Definition: Define ΠL to be the Euclidean orthogonal projection from the simplex ∆k to the smallest vector space containing the vertices indexed by P. • Theorem: (a) The likelihood on the simplex is completely determined by the likelihood on the image of ΠL. In particular, all elements of the pre-image of ΠL have the same likelihood value. (b) ΠL maps −1 convex hulls in the −1-simplex to the convex hull of Lindsay’s geometry. CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Embedding in Extended Multinomial (a) (1,0,0) (0,1,0) (0,0,1) (0,0,0) Observed face (b) (1,0,0) (0,0,1) (0,0,0) Observed face CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Embedding in Extended Multinomial • There are some definite advantages to working in larger space • Can define a new search algorithm which exploits the information geometry of the full simplex. • Enables finessing the label-switching problem encountered by many other methods. • Lindsay’s geometry captures the −1- and likelihood structure, it does not capture the full information geometry. • For example, the expected Fisher information cannot be represented CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Total positivity and local mixing • Two seemingly contradictory results: • Theorem: The −1-convex hull of an open subset of a generic one dimensional exponential family π(θ) is of full dimension. • Anaya and Marriott (2007) show, under regularity but for many applications, mixtures of exponential families have accurate low dimensional representations: local mixtures • Curve π(θ) for θ ∈ U ⊂ Θ lies ‘close’ to a low dimensional −1-affine subspace, then all mixtures over U ⊂ Θ also lie ‘close’ to this space. • Such subspaces are determined by −1-curvature • Can get good approximations using polygonal approximations CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Polygonal approximations • Given a norm · , the curve π(θ) and the polygonal path ∪Si, define the distance function by, for each θ, d(π(θ)) := inf π∈∪Si π(θ) − π . • Which norm? • Define the inner product v, w π := k i=0 viwi πi for v, w ∈ Vmix and π such that πi > 0 for all i. • This defines a preferred point metric as discussed in Critchley et al (1993) . Further, let · π be the corresponding norm. CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Polygonal approximations • As motivation for using such a metric, consider the Taylor expansion for the likelihood around ˆπ (π) − (ˆπ) ≈ − N 2 π − ˆπ 2 ˆπ . • Theorem: Let π(θ) be an exponential family, and {θi} a finite and fixed set of support points such that d(π(θ)) ≤ for all θ. Further, denote by ˆπNP and ˆπ the maximum likelihood estimates in the convex hulls of π(θ) and {π(θi)|i = 1, . . . , M} respectively, and by ˆπG i := ni N the global maximiser in the simplex. Then, (ˆπNP ) − (ˆπ) ≤ N||(ˆπG − ˆπNP )||ˆπ + o( ) (1) CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions 0 1 2 3 4 5 6 7 0.00.10.20.30.4 Data and Fit counts Probability/Proportion X X X X X X X X 0 1 2 3 4 5 6 7 0.00.10.20.30.40.50.60.7 Mixing proportions Support points Probability 0 1 2 3 4 5 6 7 −500−400−300−200−1000 Directional Derivative mu DirectionalDerivative Figure : The mixture fit using polygonal approximation CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Polygonal approximation tripod example Figure : The bipod model: space of unmixed independent distributions showing the ruled-surface structure. CIG in statistics: mixture modelling Paul Marriott Introduction Examples Extended Multinomial Models Inference on Mixtures Total positivity and local mixing New algorithm Examples Conclusions Conclusions • We look at, and show the relationship between, different geometrically approaches to mixture modelling • Lindsay’s data dependent, finite dimensional affine space • Our, extended multinomial embedding space • Show a new algorithm which uses the full Information Geometry of the problem to its advantage • Exploit the idea of polytope approximation in the correct geometry