Stochastic PDE projection on manifolds Assumed-Density and Galerkin Filters

28/10/2015
Publication GSI2015
OAI : oai:www.see.asso.fr:11784:14269

Résumé

We review the manifold projection method for stochastic nonlinear filtering in a more general setting than in our previous paper in Geometric Science of Information 2013. We still use a Hilbert space structure on a space of probability densities to project the infinite dimensional stochastic partial differential equation for the optimal filter onto a finite dimensional exponential or mixture family, respectively, with two different metrics, the Hellinger distance and the L2 direct metric. This reduces the problem to finite dimensional stochastic differential equations. In this paper we summarize a previous equivalence result between Assumed Density Filters (ADF) and Hellinger/Exponential projection filters, and introduce a new equivalence between Galerkin method based filters and Direct metric/Mixture projection filters. This result allows us to give a rigorous geometric interpretation to ADF and Galerkin filters. We also discuss the different finite-dimensional filters obtained when projecting the stochastic partial differential equation for either the normalized (Kushner-Stratonovich) or a specific unnormalized (Zakai) density of the optimal filter.

Stochastic PDE projection on manifolds Assumed-Density and Galerkin Filters

Collection

application/pdf Stochastic PDE projection on manifolds Assumed-Density and Galerkin Filters Damiano Brigo, John Armstrong

Média

Voir la vidéo

Métriques

119
6
772.35 Ko
 application/pdf
bitcache://4b7a125c540f9971cedd7f4947f451066b3b7d00

Licence

Creative Commons Attribution-ShareAlike 4.0 International

Sponsors

Organisateurs

logo_see.gif
logocampusparissaclay.png

Sponsors

entropy1-01.png
springer-logo.png
lncs_logo.png
Séminaire Léon Brillouin Logo
logothales.jpg
smai.png
logo_cnrs_2.jpg
gdr-isis.png
logo_gdr-mia.png
logo_x.jpeg
logo-lix.png
logorioniledefrance.jpg
isc-pif_logo.png
logo_telecom_paristech.png
csdcunitwinlogo.jpg
<resource  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                xmlns="http://datacite.org/schema/kernel-4"
                xsi:schemaLocation="http://datacite.org/schema/kernel-4 http://schema.datacite.org/meta/kernel-4/metadata.xsd">
        <identifier identifierType="DOI">10.23723/11784/14269</identifier><creators><creator><creatorName>John Armstrong</creatorName></creator><creator><creatorName>Damiano Brigo</creatorName></creator></creators><titles>
            <title>Stochastic PDE projection on manifolds Assumed-Density and Galerkin Filters</title></titles>
        <publisher>SEE</publisher>
        <publicationYear>2015</publicationYear>
        <resourceType resourceTypeGeneral="Text">Text</resourceType><dates>
	    <date dateType="Created">Sat 7 Nov 2015</date>
	    <date dateType="Updated">Wed 31 Aug 2016</date>
            <date dateType="Submitted">Fri 10 Aug 2018</date>
	</dates>
        <alternateIdentifiers>
	    <alternateIdentifier alternateIdentifierType="bitstream">4b7a125c540f9971cedd7f4947f451066b3b7d00</alternateIdentifier>
	</alternateIdentifiers>
        <formats>
	    <format>application/pdf</format>
	</formats>
	<version>24663</version>
        <descriptions>
            <description descriptionType="Abstract">We review the manifold projection method for stochastic nonlinear filtering in a more general setting than in our previous paper in Geometric Science of Information 2013. We still use a Hilbert space structure on a space of probability densities to project the infinite dimensional stochastic partial differential equation for the optimal filter onto a finite dimensional exponential or mixture family, respectively, with two different metrics, the Hellinger distance and the L2 direct metric. This reduces the problem to finite dimensional stochastic differential equations. In this paper we summarize a previous equivalence result between Assumed Density Filters (ADF) and Hellinger/Exponential projection filters, and introduce a new equivalence between Galerkin method based filters and Direct metric/Mixture projection filters. This result allows us to give a rigorous geometric interpretation to ADF and Galerkin filters. We also discuss the different finite-dimensional filters obtained when projecting the stochastic partial differential equation for either the normalized (Kushner-Stratonovich) or a specific unnormalized (Zakai) density of the optimal filter.
</description>
        </descriptions>
    </resource>
.

Stochastic PDE projection on manifolds: Assumed-Density and Galerkin Filters GSI 2015, Oct 28, 2015, Paris Damiano Brigo Dept. of Mathematics, Imperial College, London www.damianobrigo.it — Joint work with John Armstrong Dept. of Mathematics, King’s College, London — Full paper to appear in MCSS, see also arXiv.org D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 1 / 37 Inner Products, Metrics and Projections Spaces of densities Spaces of probability densities Consider a parametric family of probability densities S = {p(·, θ), θ ∈ Θ ⊂ Rm }, S1/2 = { p(·, θ), θ ∈ Θ ⊂ Rm }. If S (or S1/2) is a subset of a function space having an L2 structure (⇒ inner product, norm & metric), then we may ask whether p(·, θ) → θ Rm , ( p(·, θ) → θ respectively) is a Chart of a m-dim manifold (?) S (S1/2). D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 2 / 37 Inner Products, Metrics and Projections Spaces of densities Spaces of probability densities Consider a parametric family of probability densities S = {p(·, θ), θ ∈ Θ ⊂ Rm }, S1/2 = { p(·, θ), θ ∈ Θ ⊂ Rm }. If S (or S1/2) is a subset of a function space having an L2 structure (⇒ inner product, norm & metric), then we may ask whether p(·, θ) → θ Rm , ( p(·, θ) → θ respectively) is a Chart of a m-dim manifold (?) S (S1/2). The topology & differential structure in the chart is the L2 structure, but two possibilities: S : d2(p1, p2) = p1 − p2 (L2 direct distance), p1,2 ∈ L2 S1/2 : dH( √ p1, √ p2) = √ p1 − √ p2 (Hellinger distance), p1,2 ∈ L1 where · is the norm of Hilbert space L2. D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 2 / 37 Inner Products, Metrics and Projections Manifolds, Charts and Tangent Vectors Tangent vectors, metrics and projection If ϕ : θ → p(·, θ) (θ → p(·, θ) resp.) is the inverse of a chart then { ∂ϕ(·, θ) ∂θ1 , · · · , ∂ϕ(·, θ) ∂θm } are linearly independent L2(λ) vector that span Tangent Space at θ. D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 3 / 37 Inner Products, Metrics and Projections Manifolds, Charts and Tangent Vectors Tangent vectors, metrics and projection If ϕ : θ → p(·, θ) (θ → p(·, θ) resp.) is the inverse of a chart then { ∂ϕ(·, θ) ∂θ1 , · · · , ∂ϕ(·, θ) ∂θm } are linearly independent L2(λ) vector that span Tangent Space at θ. The inner product of 2 basis elements is defined (L2 structure) ∂p(·, θ) ∂θi ∂p(·, θ) ∂θj = 1 4 ∂p(x, θ) ∂θi ∂p(x, θ) ∂θj dx = 1 4 γij(θ) . ∂ √ p ∂θi ∂ √ p ∂θj = 1 4 1 p(x, θ) ∂p(x, θ) ∂θi ∂p(x, θ) ∂θj dx = 1 4 gij(θ) . γ(θ): direct L2 matrix (d2); g(θ): famous Fisher-Rao matrix (dH) D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 3 / 37 Inner Products, Metrics and Projections Manifolds, Charts and Tangent Vectors Tangent vectors, metrics and projection If ϕ : θ → p(·, θ) (θ → p(·, θ) resp.) is the inverse of a chart then { ∂ϕ(·, θ) ∂θ1 , · · · , ∂ϕ(·, θ) ∂θm } are linearly independent L2(λ) vector that span Tangent Space at θ. The inner product of 2 basis elements is defined (L2 structure) ∂p(·, θ) ∂θi ∂p(·, θ) ∂θj = 1 4 ∂p(x, θ) ∂θi ∂p(x, θ) ∂θj dx = 1 4 γij(θ) . ∂ √ p ∂θi ∂ √ p ∂θj = 1 4 1 p(x, θ) ∂p(x, θ) ∂θi ∂p(x, θ) ∂θj dx = 1 4 gij(θ) . γ(θ): direct L2 matrix (d2); g(θ): famous Fisher-Rao matrix (dH) d2 ort. projection: Πγ θ [v] = m i=1 [ m j=1 γij (θ) v, ∂p(·, θ) ∂θj ] ∂p(·, θ) ∂θi (dH proj. analogous inserting √ · and replacing γ with g) D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 3 / 37 Nonlinear Projection Filtering Nonlinear filtering problem The nonlinear filtering problem for diffusion signals dXt = ft (Xt ) dt + σt (Xt ) dWt , X0, (signal) dYt = bt (Xt ) dt + dVt , Y0 = 0 (noisy observation) (1) These are Itˆo SDE’s. We use both Itˆo and Stratonovich (Str) SDE’s. Str SDE’s are necessary to deal with manifolds, since second order Itˆo terms not clear in terms of manifolds [16], although we are working on a direct projection of Ito equations with good optimality properties (John Armstrong) D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 4 / 37 Nonlinear Projection Filtering Nonlinear filtering problem The nonlinear filtering problem for diffusion signals dXt = ft (Xt ) dt + σt (Xt ) dWt , X0, (signal) dYt = bt (Xt ) dt + dVt , Y0 = 0 (noisy observation) (1) These are Itˆo SDE’s. We use both Itˆo and Stratonovich (Str) SDE’s. Str SDE’s are necessary to deal with manifolds, since second order Itˆo terms not clear in terms of manifolds [16], although we are working on a direct projection of Ito equations with good optimality properties (John Armstrong) The nonlinear filtering problem consists in finding the conditional probability distribution πt of the state Xt given the observations up to time t, i.e. πt (dx) := P[Xt ∈ dx | Yt ], where Yt := σ(Ys , 0 ≤ s ≤ t). Assume πt has a density pt : then pt satisfies the Str SPDE: D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 4 / 37 Nonlinear Projection Filtering Nonlinear filtering problem The nonlinear filtering problem for diffusion signals dpt = L∗ t pt dt − 1 2 pt [|bt |2 − Ept {|bt |2 }] dt + d k=1 pt [bk t − Ept {bk t }] ◦ dYk t . with the forward operator L∗ t φ = − n i=1 ∂ ∂xi [fi t φ] + 1 2 n i,j=1 ∂2 ∂xi ∂xj [aij t φ] D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 5 / 37 Nonlinear Projection Filtering Nonlinear filtering problem The nonlinear filtering problem for diffusion signals dpt = L∗ t pt dt − 1 2 pt [|bt |2 − Ept {|bt |2 }] dt + d k=1 pt [bk t − Ept {bk t }] ◦ dYk t . with the forward operator L∗ t φ = − n i=1 ∂ ∂xi [fi t φ] + 1 2 n i,j=1 ∂2 ∂xi ∂xj [aij t φ] ∞-dimensional SPDE. Solutions for even toy systems the like cubic sensor, f = 0, σ = 1, b = x3, do not belong in any finite dim p(·, θ) [19]. D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 5 / 37 Nonlinear Projection Filtering Nonlinear filtering problem The nonlinear filtering problem for diffusion signals dpt = L∗ t pt dt − 1 2 pt [|bt |2 − Ept {|bt |2 }] dt + d k=1 pt [bk t − Ept {bk t }] ◦ dYk t . with the forward operator L∗ t φ = − n i=1 ∂ ∂xi [fi t φ] + 1 2 n i,j=1 ∂2 ∂xi ∂xj [aij t φ] ∞-dimensional SPDE. Solutions for even toy systems the like cubic sensor, f = 0, σ = 1, b = x3, do not belong in any finite dim p(·, θ) [19]. We need finite dimensional approximations. We can project SPDE according to either the L2 direct metric (γ(θ)) or, by deriving the analogous equation for √ pt , according to the Hellinger metric (g(θ)). D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 5 / 37 Nonlinear Projection Filtering Nonlinear filtering problem The nonlinear filtering problem for diffusion signals dpt = L∗ t pt dt − 1 2 pt [|bt |2 − Ept {|bt |2 }] dt + d k=1 pt [bk t − Ept {bk t }] ◦ dYk t . with the forward operator L∗ t φ = − n i=1 ∂ ∂xi [fi t φ] + 1 2 n i,j=1 ∂2 ∂xi ∂xj [aij t φ] ∞-dimensional SPDE. Solutions for even toy systems the like cubic sensor, f = 0, σ = 1, b = x3, do not belong in any finite dim p(·, θ) [19]. We need finite dimensional approximations. We can project SPDE according to either the L2 direct metric (γ(θ)) or, by deriving the analogous equation for √ pt , according to the Hellinger metric (g(θ)). Projection transforms the SPDE to a finite dimensional SDE for θ via the chain rule (hence Str calculus): dp(·, θt ) = m j=1 ∂p(·,θ) ∂θj ◦ dθj(t). D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 5 / 37 Nonlinear Projection Filtering Nonlinear filtering problem The nonlinear filtering problem for diffusion signals dpt = L∗ t pt dt − 1 2 pt [|bt |2 − Ept {|bt |2 }] dt + d k=1 pt [bk t − Ept {bk t }] ◦ dYk t . with the forward operator L∗ t φ = − n i=1 ∂ ∂xi [fi t φ] + 1 2 n i,j=1 ∂2 ∂xi ∂xj [aij t φ] ∞-dimensional SPDE. Solutions for even toy systems the like cubic sensor, f = 0, σ = 1, b = x3, do not belong in any finite dim p(·, θ) [19]. We need finite dimensional approximations. We can project SPDE according to either the L2 direct metric (γ(θ)) or, by deriving the analogous equation for √ pt , according to the Hellinger metric (g(θ)). Projection transforms the SPDE to a finite dimensional SDE for θ via the chain rule (hence Str calculus): dp(·, θt ) = m j=1 ∂p(·,θ) ∂θj ◦ dθj(t). With Ito calculus we would have terms ∂2p(·,θ) ∂θi ∂θj d θi, θj (not tang vec) D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 5 / 37 Nonlinear Projection Filtering Projection Filters Projection filter in the metrics h (L2) and g (Fisher) dθi t =   m j=1 γij (θt ) L∗ t p(x, θt ) ∂p(x, θt ) ∂θj dx − m j=1 γij (θt ) 1 2 |bt (x)|2 ∂p ∂θj dx   dt + d k=1 [ m j=1 γij (θt ) bk t (x) ∂p(x, θt ) ∂θj dx] ◦ dYk t , θi 0 . The above is the projected equation in d2 metric and Πγ . D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 6 / 37 Nonlinear Projection Filtering Projection Filters Projection filter in the metrics h (L2) and g (Fisher) dθi t =   m j=1 γij (θt ) L∗ t p(x, θt ) ∂p(x, θt ) ∂θj dx − m j=1 γij (θt ) 1 2 |bt (x)|2 ∂p ∂θj dx   dt + d k=1 [ m j=1 γij (θt ) bk t (x) ∂p(x, θt ) ∂θj dx] ◦ dYk t , θi 0 . The above is the projected equation in d2 metric and Πγ . Instead, using the Hellinger distance & the Fisher metric with projection Πg dθi t =   m j=1 gij (θt ) L∗ t p(x, θt ) p(x, θt ) ∂p(x, θt ) ∂θj dx − m j=1 gij (θt ) 1 2 |bt (x)|2 ∂p ∂θj dx   dt + d k=1 [ m j=1 gij (θt ) bk t (x) ∂p(x, θt ) ∂θj dx] ◦ dYk t , θi 0 . D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 6 / 37 Choice of the family Exponential Families Choosing the family/manifold: Exponential In past literature and in several papers in Bernoulli, IEEE Automatic Control etc, B. Hanzon and LeGland have developed a theory for the projection filter using the Fisher metric g and exponential families p(x, θ) := exp[θT c(x) − ψ(θ)]. Good combination: D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 7 / 37 Choice of the family Exponential Families Choosing the family/manifold: Exponential In past literature and in several papers in Bernoulli, IEEE Automatic Control etc, B. Hanzon and LeGland have developed a theory for the projection filter using the Fisher metric g and exponential families p(x, θ) := exp[θT c(x) − ψ(θ)]. Good combination: The tangent space has a simple structure: square roots do not complicate issues thanks to the exponential structure. D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 7 / 37 Choice of the family Exponential Families Choosing the family/manifold: Exponential In past literature and in several papers in Bernoulli, IEEE Automatic Control etc, B. Hanzon and LeGland have developed a theory for the projection filter using the Fisher metric g and exponential families p(x, θ) := exp[θT c(x) − ψ(θ)]. Good combination: The tangent space has a simple structure: square roots do not complicate issues thanks to the exponential structure. The Fisher matrix has a simple structure: ∂2 θi ,θj ψ(θ) = gij(θ) D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 7 / 37 Choice of the family Exponential Families Choosing the family/manifold: Exponential In past literature and in several papers in Bernoulli, IEEE Automatic Control etc, B. Hanzon and LeGland have developed a theory for the projection filter using the Fisher metric g and exponential families p(x, θ) := exp[θT c(x) − ψ(θ)]. Good combination: The tangent space has a simple structure: square roots do not complicate issues thanks to the exponential structure. The Fisher matrix has a simple structure: ∂2 θi ,θj ψ(θ) = gij(θ) The structure of the projection Πg is simple for exp families D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 7 / 37 Choice of the family Exponential Families Choosing the family/manifold: Exponential In past literature and in several papers in Bernoulli, IEEE Automatic Control etc, B. Hanzon and LeGland have developed a theory for the projection filter using the Fisher metric g and exponential families p(x, θ) := exp[θT c(x) − ψ(θ)]. Good combination: The tangent space has a simple structure: square roots do not complicate issues thanks to the exponential structure. The Fisher matrix has a simple structure: ∂2 θi ,θj ψ(θ) = gij(θ) The structure of the projection Πg is simple for exp families Special exp family with Y-function b among c(x) exponents makes filter correction step (projection of dY term) exact D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 7 / 37 Choice of the family Exponential Families Choosing the family/manifold: Exponential In past literature and in several papers in Bernoulli, IEEE Automatic Control etc, B. Hanzon and LeGland have developed a theory for the projection filter using the Fisher metric g and exponential families p(x, θ) := exp[θT c(x) − ψ(θ)]. Good combination: The tangent space has a simple structure: square roots do not complicate issues thanks to the exponential structure. The Fisher matrix has a simple structure: ∂2 θi ,θj ψ(θ) = gij(θ) The structure of the projection Πg is simple for exp families Special exp family with Y-function b among c(x) exponents makes filter correction step (projection of dY term) exact One can define both a local and global filtering error through dH D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 7 / 37 Choice of the family Exponential Families Choosing the family/manifold: Exponential In past literature and in several papers in Bernoulli, IEEE Automatic Control etc, B. Hanzon and LeGland have developed a theory for the projection filter using the Fisher metric g and exponential families p(x, θ) := exp[θT c(x) − ψ(θ)]. Good combination: The tangent space has a simple structure: square roots do not complicate issues thanks to the exponential structure. The Fisher matrix has a simple structure: ∂2 θi ,θj ψ(θ) = gij(θ) The structure of the projection Πg is simple for exp families Special exp family with Y-function b among c(x) exponents makes filter correction step (projection of dY term) exact One can define both a local and global filtering error through dH Alternative coordinates, expectation param., η = Eθ[c] = ∂θψ(θ). D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 7 / 37 Choice of the family Exponential Families Choosing the family/manifold: Exponential In past literature and in several papers in Bernoulli, IEEE Automatic Control etc, B. Hanzon and LeGland have developed a theory for the projection filter using the Fisher metric g and exponential families p(x, θ) := exp[θT c(x) − ψ(θ)]. Good combination: The tangent space has a simple structure: square roots do not complicate issues thanks to the exponential structure. The Fisher matrix has a simple structure: ∂2 θi ,θj ψ(θ) = gij(θ) The structure of the projection Πg is simple for exp families Special exp family with Y-function b among c(x) exponents makes filter correction step (projection of dY term) exact One can define both a local and global filtering error through dH Alternative coordinates, expectation param., η = Eθ[c] = ∂θψ(θ). Projection filter in η coincides with classical approx filter: assumed density filter (based on generalized “moment matching”) D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 7 / 37 Choice of the family Mixture Families Mixture families However, exponential families do not couple as well with the metric γ(θ). Is there some important family for which the metric γ(θ) is preferable to the classical Fisher metric g(θ), in that the metric, the tangent space and the filter equations are simpler? D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 8 / 37 Choice of the family Mixture Families Mixture families However, exponential families do not couple as well with the metric γ(θ). Is there some important family for which the metric γ(θ) is preferable to the classical Fisher metric g(θ), in that the metric, the tangent space and the filter equations are simpler? The answer is affirmative, and this is the mixture family. D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 8 / 37 Choice of the family Mixture Families Mixture families However, exponential families do not couple as well with the metric γ(θ). Is there some important family for which the metric γ(θ) is preferable to the classical Fisher metric g(θ), in that the metric, the tangent space and the filter equations are simpler? The answer is affirmative, and this is the mixture family. We define a simple mixture family as follows. Given m + 1 fixed squared integrable probability densities q = [q1, q2, . . . , qm+1]T , define ˆθ(θ) := [θ1, θ2, . . . , θm, 1 − θ1 − θ2 − . . . − θm]T for all θ ∈ Rm. We write ˆθ instead of ˆθ(θ). Mixture family (simplex): SM (q) = {ˆθ(θ)T q, θi ≥ 0 for all i, θ1 + · · · + θm < 1} D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 8 / 37 Choice of the family Mixture Families Mixture families If we consider the L2 / γ(θ) distance, the metric γ(θ) itself and the related projection become very simple. Indeed, ∂p(·, θ) ∂θi = qi −qm+1 and γij(θ) = (qi(x)−qm(x))(qj(x)−qm(x))dx (NO inline numeric integr). D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 9 / 37 Choice of the family Mixture Families Mixture families If we consider the L2 / γ(θ) distance, the metric γ(θ) itself and the related projection become very simple. Indeed, ∂p(·, θ) ∂θi = qi −qm+1 and γij(θ) = (qi(x)−qm(x))(qj(x)−qm(x))dx (NO inline numeric integr). The L2 metric does not depend on the specific point θ of the manifold. The same holds for the tangent space at p(·, θ), which is given by span{q1 − qm+1, q2 − qm+1, · · · , qm − qm+1} Also the L2 projection becomes particularly simple. D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 9 / 37 Mixture Projection Filter Mixture Projection Filter Armstrong and B. (MCSS 2016 [3]) show that the mixture family + metric γ(θ) lead to a Projection filter that is the same as approximate filtering via Galerkin [5] methods. D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 10 / 37 Mixture Projection Filter Mixture Projection Filter Armstrong and B. (MCSS 2016 [3]) show that the mixture family + metric γ(θ) lead to a Projection filter that is the same as approximate filtering via Galerkin [5] methods. See the full paper for the details. Summing up: Family → Exponential Basic Mixture Metric ↓ Hellinger dH Good Nothing special Fisher g(θ) ∼ADF ≈ local moment matching Direct L2 d2 Nothing special Good matrix γ(θ) (∼Galerkin) D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 10 / 37 Mixture Projection Filter Mixture Projection Filter However, despite the simplicity above, the mixture family has an important drawback: for all θ, filter mean is constrained min i mean of qi ≤ mean of p(·, θ) ≤ max i mean of qi D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 11 / 37 Mixture Projection Filter Mixture Projection Filter However, despite the simplicity above, the mixture family has an important drawback: for all θ, filter mean is constrained min i mean of qi ≤ mean of p(·, θ) ≤ max i mean of qi As a consequence, we are going to enrich our family to a mixture where some of the parameters are also in the core densities q. D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 11 / 37 Mixture Projection Filter Mixture Projection Filter However, despite the simplicity above, the mixture family has an important drawback: for all θ, filter mean is constrained min i mean of qi ≤ mean of p(·, θ) ≤ max i mean of qi As a consequence, we are going to enrich our family to a mixture where some of the parameters are also in the core densities q. Specifically, we consider a mixture of GAUSSIAN DENSITIES with MEANS AND VARIANCES in each component not fixed. For example for a mixture of two Gaussians we have 5 parameters. θpN(µ1,v1)(x) + (1 − θ)pN(µ2,v2)(x), param. θ, µ1, v1, µ2, v2 D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 11 / 37 Mixture Projection Filter Mixture Projection Filter However, despite the simplicity above, the mixture family has an important drawback: for all θ, filter mean is constrained min i mean of qi ≤ mean of p(·, θ) ≤ max i mean of qi As a consequence, we are going to enrich our family to a mixture where some of the parameters are also in the core densities q. Specifically, we consider a mixture of GAUSSIAN DENSITIES with MEANS AND VARIANCES in each component not fixed. For example for a mixture of two Gaussians we have 5 parameters. θpN(µ1,v1)(x) + (1 − θ)pN(µ2,v2)(x), param. θ, µ1, v1, µ2, v2 We are now going to illustrate the Gaussian mixture projection filter (GMPF) in a fundamental example. D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 11 / 37 Mixture Projection Filter The quadratic sensor The quadratic sensor Consider the quadratic sensor dXt = σdWt dYt = X2 dt + σdVt . D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 12 / 37 Mixture Projection Filter The quadratic sensor The quadratic sensor Consider the quadratic sensor dXt = σdWt dYt = X2 dt + σdVt . The measurements tell us nothing about the sign of X D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 12 / 37 Mixture Projection Filter The quadratic sensor The quadratic sensor Consider the quadratic sensor dXt = σdWt dYt = X2 dt + σdVt . The measurements tell us nothing about the sign of X Once it seems likely that the state has moved past the origin, the distribution will become nearly symmetrical D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 12 / 37 Mixture Projection Filter The quadratic sensor The quadratic sensor Consider the quadratic sensor dXt = σdWt dYt = X2 dt + σdVt . The measurements tell us nothing about the sign of X Once it seems likely that the state has moved past the origin, the distribution will become nearly symmetrical We expect a bimodal distribution D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 12 / 37 Mixture Projection Filter The quadratic sensor The quadratic sensor Consider the quadratic sensor dXt = σdWt dYt = X2 dt + σdVt . The measurements tell us nothing about the sign of X Once it seems likely that the state has moved past the origin, the distribution will become nearly symmetrical We expect a bimodal distribution θpN(µ1,v1)(x) + (1 − θ)pN(µ2,v2)(x) (red) vs eθ1x+θ2x2+θ3x3+θ4x4−ψ(θ) (pink) vs EKF (N) (blue) vs exact (green, finite diff. method, grid 1000 state & 5000 time) D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 12 / 37 Mixture Projection Filter The quadratic sensor Simulation for the Quadratic Sensor 0 0.2 0.4 0.6 0.8 1 -8 -6 -4 -2 0 2 4 6 8 X Distribution at time 0 Projection Exact Extended Kalman Exponential D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 13 / 37 Mixture Projection Filter The quadratic sensor Simulation for the Quadratic Sensor 0 0.2 0.4 0.6 0.8 1 -8 -6 -4 -2 0 2 4 6 8 X Distribution at time 1 Projection Exact Extended Kalman Exponential D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 14 / 37 Mixture Projection Filter The quadratic sensor Simulation for the Quadratic Sensor 0 0.2 0.4 0.6 0.8 1 -8 -6 -4 -2 0 2 4 6 8 X Distribution at time 2 Projection Exact Extended Kalman Exponential D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 15 / 37 Mixture Projection Filter The quadratic sensor Simulation for the Quadratic Sensor 0 0.2 0.4 0.6 0.8 1 -8 -6 -4 -2 0 2 4 6 8 X Distribution at time 3 Projection Exact Extended Kalman Exponential D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 16 / 37 Mixture Projection Filter The quadratic sensor Simulation for the Quadratic Sensor 0 0.2 0.4 0.6 0.8 1 -8 -6 -4 -2 0 2 4 6 8 X Distribution at time 4 Projection Exact Extended Kalman Exponential D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 17 / 37 Mixture Projection Filter The quadratic sensor Simulation for the Quadratic Sensor 0 0.2 0.4 0.6 0.8 1 -8 -6 -4 -2 0 2 4 6 8 X Distribution at time 5 Projection Exact Extended Kalman Exponential D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 18 / 37 Mixture Projection Filter The quadratic sensor Simulation for the Quadratic Sensor 0 0.2 0.4 0.6 0.8 1 -8 -6 -4 -2 0 2 4 6 8 X Distribution at time 6 Projection Exact Extended Kalman Exponential D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 19 / 37 Mixture Projection Filter The quadratic sensor Simulation for the Quadratic Sensor 0 0.2 0.4 0.6 0.8 1 -8 -6 -4 -2 0 2 4 6 8 X Distribution at time 7 Projection Exact Extended Kalman Exponential D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 20 / 37 Mixture Projection Filter The quadratic sensor Simulation for the Quadratic Sensor 0 0.2 0.4 0.6 0.8 1 -8 -6 -4 -2 0 2 4 6 8 X Distribution at time 8 Projection Exact Extended Kalman Exponential D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 21 / 37 Mixture Projection Filter The quadratic sensor Simulation for the Quadratic Sensor 0 0.2 0.4 0.6 0.8 1 -8 -6 -4 -2 0 2 4 6 8 X Distribution at time 9 Projection Exact Extended Kalman Exponential D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 22 / 37 Mixture Projection Filter The quadratic sensor Simulation for the Quadratic Sensor 0 0.2 0.4 0.6 0.8 1 -8 -6 -4 -2 0 2 4 6 8 X Distribution at time 10 Projection Exact Extended Kalman Exponential D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 23 / 37 Mixture Projection Filter The quadratic sensor Comparing local approximation errors (L2 residuals) εt ε2 t = (pexact,t (x) − papprox,t (x))2 dx papprox,t (x): three possible choices. θpN(µ1,v1)(x) + (1 − θ)pN(µ2,v2)(x) (red) vs eθ1x+θ2x2+θ3x3+θ4x4−ψ(θ) (blue) vs EKF (N) (green) D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 24 / 37 Mixture Projection Filter The quadratic sensor L2 residuals for the quadratic sensor 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0 2 4 6 8 10 Time Residuals Projection Residual (L2 norm) Extended Kalman Residual (L2 norm) Hellinger Projection Residual (L2 norm) D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 25 / 37 Mixture Projection Filter The quadratic sensor Comparing local approx errors (Prokhorov residuals) εt εt = inf{ : Fexact,t (x − ) − ≤ Fapprox,t (x) ≤ Fexact,t (x + ) + ∀x} with F the CDF of p’s. Levy-Prokhorov metric works well with singular densities like particles where L2 metric not ideal. θpN(µ1,v1)(x) + (1 − θ)pN(µ2,v2)(x) (red) vs eθ1x+θ2x2+θ3x3+θ4x4−ψ(θ) (green) vs best three particles (blue) D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 26 / 37 Mixture Projection Filter The quadratic sensor L´evy residuals for the quadratic sensor 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0 1 2 3 4 5 6 7 8 9 10 Time ProkhorovResiduals Prokhorov Residual (L2NM) Prokhorov Residual (HE) Best possible residual (3Deltas) D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 27 / 37 Mixture Projection Filter Cubic sensors Cubic sensors 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 0 2 4 6 8 10 Time Residuals Projection Residual (L2 norm) Extended Kalman Residual (L2 norm) Hellinger Projection Residual (L2 norm) Qualitatively similar results up to a stopping time D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 28 / 37 Mixture Projection Filter Cubic sensors Cubic sensors 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 0 2 4 6 8 10 Time Residuals Projection Residual (L2 norm) Extended Kalman Residual (L2 norm) Hellinger Projection Residual (L2 norm) Qualitatively similar results up to a stopping time As one approaches the boundary γij becomes singular D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 28 / 37 Mixture Projection Filter Cubic sensors Cubic sensors 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 0 2 4 6 8 10 Time Residuals Projection Residual (L2 norm) Extended Kalman Residual (L2 norm) Hellinger Projection Residual (L2 norm) Qualitatively similar results up to a stopping time As one approaches the boundary γij becomes singular The solution is to dynamically change the parameterization and even the dimension of the manifold. D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 28 / 37 Conclusions and References Conclusions Approximate finite-dimensional filtering by rigorous projection on a chosen manifold of densities D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 29 / 37 Conclusions and References Conclusions Approximate finite-dimensional filtering by rigorous projection on a chosen manifold of densities Projection uses overarching L2 structure D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 29 / 37 Conclusions and References Conclusions Approximate finite-dimensional filtering by rigorous projection on a chosen manifold of densities Projection uses overarching L2 structure Two different metrics: direct L2 and Hellinger/Fisher (L2 on √ .) D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 29 / 37 Conclusions and References Conclusions Approximate finite-dimensional filtering by rigorous projection on a chosen manifold of densities Projection uses overarching L2 structure Two different metrics: direct L2 and Hellinger/Fisher (L2 on √ .) Fisher works well with exponential families: multimodality, correction step exact, simplicity of implementation equivalence with Assumed Density Filters “moment matching” D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 29 / 37 Conclusions and References Conclusions Approximate finite-dimensional filtering by rigorous projection on a chosen manifold of densities Projection uses overarching L2 structure Two different metrics: direct L2 and Hellinger/Fisher (L2 on √ .) Fisher works well with exponential families: multimodality, correction step exact, simplicity of implementation equivalence with Assumed Density Filters “moment matching” Direct L2 works well with mixture families even simpler filter equations, no inline numerical integration basic version equivalent to Galerkin methods suited also for multimodality (quadratic sensor tests, L2 global error) comparable with particle methods D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 29 / 37 Conclusions and References Conclusions Approximate finite-dimensional filtering by rigorous projection on a chosen manifold of densities Projection uses overarching L2 structure Two different metrics: direct L2 and Hellinger/Fisher (L2 on √ .) Fisher works well with exponential families: multimodality, correction step exact, simplicity of implementation equivalence with Assumed Density Filters “moment matching” Direct L2 works well with mixture families even simpler filter equations, no inline numerical integration basic version equivalent to Galerkin methods suited also for multimodality (quadratic sensor tests, L2 global error) comparable with particle methods Further investigation: convergence, more on optimality? D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 29 / 37 Conclusions and References Conclusions Approximate finite-dimensional filtering by rigorous projection on a chosen manifold of densities Projection uses overarching L2 structure Two different metrics: direct L2 and Hellinger/Fisher (L2 on √ .) Fisher works well with exponential families: multimodality, correction step exact, simplicity of implementation equivalence with Assumed Density Filters “moment matching” Direct L2 works well with mixture families even simpler filter equations, no inline numerical integration basic version equivalent to Galerkin methods suited also for multimodality (quadratic sensor tests, L2 global error) comparable with particle methods Further investigation: convergence, more on optimality? Optimality: introducing new projections (forthcoming J. Armstrong) D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 29 / 37 Conclusions and References Thanks With thanks to the organizing committee. Thank you for your attention. Questions and comments welcome D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 30 / 37 Conclusions and References References I [1] J. Aggrawal: Sur l’information de Fisher. In: Theories de l’Information (J. Kampe de Feriet, ed.), Springer-Verlag, Berlin–New York 1974, pp. 111-117. [2] Amari, S. Differential-geometrical methods in statistics, Lecture notes in statistics, Springer-Verlag, Berlin, 1985 [3] Armstrong, J., and Brigo, D. (2016). Nonlinear filtering via stochastic PDE projection on mixture manifolds in L2 direct metric, Mathematics of Control, Signals and Systems, 2016, accepted. [4] Beard, R., Kenney, J., Gunther, J., Lawton, J., and Stirling, W. (1999). Nonlinear Projection Filter based on Galerkin approximation. AIAA Journal of Guidance Control and Dynamics, 22 (2): 258-266. D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 31 / 37 Conclusions and References References II [5] Beard, R. and Gunther, J. (1997). Galerkin Approximations of the Kushner Equation in Nonlinear Estimation. Working Paper, Brigham Young University. [6] Barndorff-Nielsen, O.E. (1978). Information and Exponential Families. John Wiley and Sons, New York. [7] Brigo, D. Diffusion Processes, Manifolds of Exponential Densities, and Nonlinear Filtering, In: Ole E. Barndorff-Nielsen and Eva B. Vedel Jensen, editor, Geometry in Present Day Science, World Scientific, 1999 [8] Brigo, D, On SDEs with marginal laws evolving in finite-dimensional exponential families, STAT PROBABIL LETT, 2000, Vol: 49, Pages: 127 – 134 D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 32 / 37 Conclusions and References References III [9] Brigo, D. (2011). The direct L2 geometric structure on a manifold of probability densities with applications to Filtering. Available on arXiv.org and damianobrigo.it [10] Brigo, D, Hanzon, B, LeGland, F, A differential geometric approach to nonlinear filtering: The projection filter, IEEE T AUTOMAT CONTR, 1998, Vol: 43, Pages: 247 – 252 [11] Brigo, D, Hanzon, B, Le Gland, F, Approximate nonlinear filtering by projection on exponential manifolds of densities, BERNOULLI, 1999, Vol: 5, Pages: 495 – 534 [12] D. Brigo, Filtering by Projection on the Manifold of Exponential Densities, PhD Thesis, Free University of Amsterdam, 1996. D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 33 / 37 Conclusions and References References IV [13] Brigo, D., and Pistone, G. (1996). Projecting the Fokker-Planck Equation onto a finite dimensional exponential family. Available at arXiv.org [14] Crisan, D., and Rozovskii, B. (Eds) (2011). The Oxford Handbook of Nonlinear Filtering, Oxford University Press. [15] M. H. A. Davis, S. I. Marcus, An introduction to nonlinear filtering, in: M. Hazewinkel, J. C. Willems, Eds., Stochastic Systems: The Mathematics of Filtering and Identification and Applications (Reidel, Dordrecht, 1981) 53–75. [16] Elworthy, D. (1982). Stochastic Differential Equations on Manifolds. LMS Lecture Notes. D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 34 / 37 Conclusions and References References V [17] Hanzon, B. A differential-geometric approach to approximate nonlinear filtering. In C.T.J. Dodson, Geometrization of Statistical Theory, pages 219 – 223,ULMD Publications, University of Lancaster, 1987. [18] B. Hanzon, Identifiability, recursive identification and spaces of linear dynamical systems, CWI Tracts 63 and 64, CWI, Amsterdam, 1989 [19] M. Hazewinkel, S.I.Marcus, and H.J. Sussmann, Nonexistence of finite dimensional filters for conditional statistics of the cubic sensor problem, Systems and Control Letters 3 (1983) 331–340. [20] J. Jacod, A. N. Shiryaev, Limit theorems for stochastic processes. Grundlehren der Mathematischen Wissenschaften, vol. 288 (1987), Springer-Verlag, Berlin, D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 35 / 37 Conclusions and References References VI [21] A. H. Jazwinski, Stochastic Processes and Filtering Theory, Academic Press, New York, 1970. [22] M. Fujisaki, G. Kallianpur, and H. Kunita (1972). Stochastic differential equations for the non linear filtering problem. Osaka J. Math. Volume 9, Number 1 (1972), 19-40. [23] Kenney, J., Stirling, W. Nonlinear Filtering of Convex Sets of Probability Distributions. Presented at the 1st International Symposium on Imprecise Probabilities and Their Applications, Ghent, Belgium, 29 June - 2 July 1999 [24] R. Z. Khasminskii (1980). Stochastic Stability of Differential Equations. Alphen aan den Reijn [25] R.S. Liptser, A.N. Shiryayev, Statistics of Random Processes I, General Theory (Springer Verlag, Berlin, 1978). D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 36 / 37 Conclusions and References References VII [26] M. Murray and J. Rice - Differential geometry and statistics, Monographs on Statistics and Applied Probability 48, Chapman and Hall, 1993. [27] D. Ocone, E. Pardoux, A Lie algebraic criterion for non-existence of finite dimensionally computable filters, Lecture notes in mathematics 1390, 197–204 (Springer Verlag, 1989) [28] Pistone, G., and Sempi, C. (1995). An Infinite Dimensional Geometric Structure On the space of All the Probability Measures Equivalent to a Given one. The Annals of Statistics 23(5), 1995 D. Brigo and J. Armstrong (ICL and KCL) SPDE Projection Filters GSI 2015 37 / 37