Image processing in the semidiscrete group of rototranslations

28/10/2015
Publication GSI2015
OAI : oai:www.see.asso.fr:11784:14333

Résumé

It is well-known, since [12], that cells in the primary visual cortex V1 do much more than merely signaling position in the visual field: most cortical cells signal the local orientation of a contrast edge or bar . they are tuned to a particular local orientation. This orientation tuning has been given a mathematical interpretation in a sub-Riemannian model by Petitot, Citti, and Sarti [6,14]. According to this model, the primary visual cortex V1 lifts grey-scale images, given as functions f : ℝ2 → [0, 1], to functions Lf defined on the projectivized tangent bundle of the plane PTℝ2 = ℝ2 ×ℙ1. Recently, in [1], the authors presented a promising semidiscrete variant of this model where the Euclidean group of rototranslations SE(2), which is the double covering of PTℝ2, is replaced by SE(2,N), the group of translations and discrete rotations. In particular, in [15], an implementation of this model allowed for state-of-the-art image inpaintings. In this work, we review the inpainting results and introduce an application of the semidiscrete model to image recognition. We remark that both these applications deeply exploit the Moore structure of SE(2,N) that guarantees that its unitary representations behaves similarly to those of a compact group. This allows for nice properties of the Fourier transform on SE(2,N) exploiting which one obtains numerical advantages.

Image processing in the semidiscrete group of rototranslations

Collection

application/pdf Image processing in the semidiscrete group of rototranslations Dario Prandi, Ugo Boscain, Jean-Paul Gauthier

Média

Voir la vidéo

Métriques

184
4
5.5 Mo
 application/pdf
bitcache://e54a9de113f0cb0d67c7e16d539bc8222dc4b937

Licence

Creative Commons Attribution-ShareAlike 4.0 International

Sponsors

Organisateurs

logo_see.gif
logocampusparissaclay.png

Sponsors

entropy1-01.png
springer-logo.png
lncs_logo.png
Séminaire Léon Brillouin Logo
logothales.jpg
smai.png
logo_cnrs_2.jpg
gdr-isis.png
logo_gdr-mia.png
logo_x.jpeg
logo-lix.png
logorioniledefrance.jpg
isc-pif_logo.png
logo_telecom_paristech.png
csdcunitwinlogo.jpg
<resource  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                xmlns="http://datacite.org/schema/kernel-4"
                xsi:schemaLocation="http://datacite.org/schema/kernel-4 http://schema.datacite.org/meta/kernel-4/metadata.xsd">
        <identifier identifierType="DOI">10.23723/11784/14333</identifier><creators><creator><creatorName>Dario Prandi</creatorName></creator><creator><creatorName>Ugo Boscain</creatorName></creator><creator><creatorName>Jean-Paul Gauthier</creatorName></creator></creators><titles>
            <title>Image processing in the semidiscrete group of rototranslations</title></titles>
        <publisher>SEE</publisher>
        <publicationYear>2015</publicationYear>
        <resourceType resourceTypeGeneral="Text">Text</resourceType><dates>
	    <date dateType="Created">Sun 8 Nov 2015</date>
	    <date dateType="Updated">Wed 31 Aug 2016</date>
            <date dateType="Submitted">Fri 20 Apr 2018</date>
	</dates>
        <alternateIdentifiers>
	    <alternateIdentifier alternateIdentifierType="bitstream">e54a9de113f0cb0d67c7e16d539bc8222dc4b937</alternateIdentifier>
	</alternateIdentifiers>
        <formats>
	    <format>application/pdf</format>
	</formats>
	<version>24728</version>
        <descriptions>
            <description descriptionType="Abstract">
It is well-known, since [12], that cells in the primary visual cortex V1 do much more than merely signaling position in the visual field: most cortical cells signal the local orientation of a contrast edge or bar . they are tuned to a particular local orientation. This orientation tuning has been given a mathematical interpretation in a sub-Riemannian model by Petitot, Citti, and Sarti [6,14]. According to this model, the primary visual cortex V1 lifts grey-scale images, given as functions f : ℝ2 → [0, 1], to functions Lf defined on the projectivized tangent bundle of the plane PTℝ2 = ℝ2 ×ℙ1. Recently, in [1], the authors presented a promising semidiscrete variant of this model where the Euclidean group of rototranslations SE(2), which is the double covering of PTℝ2, is replaced by SE(2,N), the group of translations and discrete rotations. In particular, in [15], an implementation of this model allowed for state-of-the-art image inpaintings. In this work, we review the inpainting results and introduce an application of the semidiscrete model to image recognition. We remark that both these applications deeply exploit the Moore structure of SE(2,N) that guarantees that its unitary representations behaves similarly to those of a compact group. This allows for nice properties of the Fourier transform on SE(2,N) exploiting which one obtains numerical advantages.

</description>
        </descriptions>
    </resource>
.

IMAGE PROCESSING IN THE SEMIDISCRETE GROUP OF ROTOTRANSLATIONS DARIO PRANDI, J.P. GAUTHIER, U. BOSCAIN  UNIV. PARIS-DAUPHINE, UNIV. DE TOULON, ÉCOLE POLYTECHNIQUE GEOMETRIC SCIENCE OF INFORMATION - SECOND INTERNATIONAL CONFERENCE  30 OCTOBER 2015 OUTLINE OF THE TALK: 1. 2. 3. A mathematical model of the primary visual cortex V1 Image inpainting Image recognition A MATHEMATICAL MODEL  OF THE PRIMARY VISUAL CORTEX  NEUROPHYSIOLOGICAL FACT STRUCTURE OF THE PRIMARY VISUAL CORTEX Hubel and Wiesel (Nobel prize 1981) observed that in the primary visual cortex V1, groups of neurons are sensitive to both positions and directions. Patterns of orientation columns and long-range horizontal connections in V1 of a tree shrew. Taken from Kaschube et al. (2008). THE SEMI-DISCRETE MODEL Conjecture: The visual cortex V1 can detect a finite (small) number of directions only ( 30),≈ ⟹ V 1 ≅ ×ℝ2 ℤN/2 GROUP THEORETICAL APPROACH Consider with lawSE(2, N) = ⋊ℝ2 ℤN (x, k)(y, h) = (x + y, k + h)Rk is a locally compact non-commutative group is the double covering of , so we can work here.V 1 Left regular representation : action of on viaΛ SE(2, N) (SE(2, N))L 2 Λ(x, k)φ(y, h) = φ((x, k (y, h)) = φ( (y − x), h − k)) −1 R−k Quasi-regular representation action of on viaπ SE(2, N) ( )L 2 ℝ2 π(x, α)f (y) = f ( (y − x))R−α RECEPTIVE FIELDS Problem: how are visual stimuli lifted to cortical ones? A neuron in is described by a function(x, k) V 1 ∈ ( )Ψ (x,k) L 2 ℝ2 A greyscale visual stimulus feeds with an extracellular voltage given by A good fit for is the Gabor filter (sinusoidal wave multiplied by Gaussian function) centered at of orientation . f : → [0, 1]ℝ2 (x, k) f ↦ Lf (x, k) Lf (x, k) = ⟨f , .Ψ (x,k) ⟩ ( )L 2 ℝ2 Ψ (x,k) (x, y) θ ⟹ = π(x, k)ΨΨ (x,k) Definition: A lift operator is left-invariant ifL : ( ) → (SE(2))L 2 ℝ2 L 2 Λ(a)Lf = L(π(a)f ) ∀f ∈ ( ), a ∈ SE(2).L 2 ℝ2 Theorem: (J.-P. Gauthier, D.P.) Let be a left-invariant lift such that is linear, is densely defined and bounded. Then, there exists such that L L f ↦ Lf (0) Ψ ∈ ( )L 2 ℝ2 Lf (x, θ) = ⟨f , π(x, θ)Ψ (= f ⋆ (− ))(x).⟩ ( )L 2 ℝ2 Rθ Ψ ¯ REMARKS V1 receptive fields through Gabor filters define a left-invariant lift. The function is known as the wavelet transform of w.r.t. . (x, k) ↦ ⟨f , π(x, k)Ψ ⟩ (ℍ)L 2 f Ψ ADVANTAGES OF THE APPROACH Solves crossing problems: Natural formulation of problems invariant under translations and rotations Fourier analysis: all irreducible representations of are finite dimensional SE(2, N) IMAGE INPAINTING  ILLUSORY CONTOURS Two illusory contours Neurophysiological assumption: We reconstruct images through the natural diffusion in V1, which minimizes the energy required to activate unexcitated neurons. SPONTANEOUS EVOLUTION IN V1 Two types of connections between neurons: Lateral connections: Between iso-oriented neurons, in the direction of their orientation. Represented by the integral lines of the vector field Local connections: Between neurons in the same hypercolumn. X(x, y, r) = cos + sin for  = .θr ∂x θr ∂y θr πr N Connections made by V1 cells of a tree shew. Picture from Bosking et al. (1997) SPONTANEOUS EVOLUTION IN V1 Given a Wiener process on , neuron excitation evolve according to the SDE Here, is a Poisson Markov process on . The infinitesimal generator of is the semi-discrete evolution operator, , where is the generator of . It yields the hypoelliptic equation for the evolution of a stimulus : W ℝ2 d = Xd + .At Wt Θ t Θ t ℤN/2 A = +N X 2 Λ N Λ N Θ t Ψ ∈ ( × )L 2 ℝ2 ℤN/2 Ψ = Ψ. d dt N Remarks Highly anisotropic evolution Invariant under the action of SE(2) ANTHROPOMORPHIC IMAGE RECONSTRUCTION Let with on be an image corrupted on .f ∈ ( )L 2 ℝ2 f ≡ 0 Ω ⊂ ℝ2 Ω 0. Smooth by a Gaussian filter 1. Lift to on 2. Evolve through the spontaneous semi-discrete evolution 3. Project the evolved function back to as f f Lf V 1 Lf Ψ ℝ2 ψ(x) = Ψ(x, r).max r∈ℤN/2 REMARKS Fast and parallelizable thanks to Fourier transform techniques Reasonable results for small corruptions Adding some heuristic procedures yield very good reconstructions RESULTS Inpainting results from Boscain et al. (2014) HIGHLY CORRUPTED IMAGES RECONSTRUCTION Based upon this diffusion and certain heuristic complements, we get nice results on images with more than 85% of pixels missing. Image from Boscain, Gauthier, P., Remizov (preprint) IMAGE RECOGNITION THE PROBLEM Given two images , how can we tell iff , g ∈ ( )L 2 ℝ2 g = π(x, k)f for some (x, k) ∈ SE(2, N)? CLASSICAL FOURIER DESCRIPTORS ON ℝ Given , how can we tell if for some ?f , g ∈ (ℝ)L 2 g = fτa a ∈ ℝ f (x) = f (x − a) ⟺ (λ) = (λ) ∀λ ∈τa fτa ˆ e iaλ f ̂ ℝˆ Theorem: If are supported on a compact interval and on an open and dense set, then Here, are the bispectral invariants: Bispectral invariants are used in several areas of signal processing (e.g. to identify music timbre and texture, Dubnov et al. (1997)) f , g K (λ), (λ) ≠ 0f ̂ ĝ ≡ ⟺ g = f  for some a ∈ ℝ.Bf Bg τa : × → ℝBf ℝˆ ℝˆ ( , ) = ( + ) ∀ , ∈ .Bf λ1 λ2 f ̂ λ1 λ2 ( )f ̂ λ1 ⎯ ⎯⎯⎯⎯⎯⎯⎯ ( )f ̂ λ2 ⎯ ⎯⎯⎯⎯⎯⎯⎯ λ1 λ2 ℝˆ NON-COMMUTATIVE FOURIER TRANSFORM The dual of is the set of equivalence classes of unitary irreducible representations of . The Fourier transform of is the matrix W.r.t. the Plancherel measure on , the Fourier transform extends to an isometry . SE(2, N)ˆ SE(2, N) SE(2, N) φ ∈ ∩ (SE(2, N))L 1 L 2 (R) = φ(a) R(a dμ(a) ∀R ∈ .φ̂ ∫ SE(2,N) ) −1 SE(2, N)ˆ dμ̂ SE(2, N)ˆ  : (SE(2, N)) → ( )L 2 L 2 SE(2, N)ˆ Fundamental property: For any ,(x, k) ∈ SE(2, N) φ = Λ(x, k)ψ ⟺ (R) = (R) ∘ R(a)φ̂ ψ̂ GENERALIZED BISPECTRAL INVARIANTS The support of the Plancherel measure on are the - dimensional representations parametrized by , where Bispectral invariants are generalized to as the following functions of : SE(2, N)ˆ N T λ λ ∈  ⊂ ℝ2  = { (ρ, θ) ∣ ρ > 0, 0 ≤ θ < } . 2π N φ ∈ (SE(2, N))L 2 , ∈ λ1 λ2 ( , ) = ( ⊗ ) ∘ ( ⊗ ( .Bφ λ1 λ2 φ̂ T λ1 T λ2 φ̂ T λ1 ) ∗ φ̂ T λ2 ) ∗ COMPLETENESS Theorem: If have compact support, and are invertible for in an open and dense set, then φ, ψ ∈ (SE(2, N))L 2 ( )φ̂ T λ ( )ψ̂ T λ λ ≡ ⟺ φ = Λ(x, k)ψ for some (x, k) ∈ SE(2, N).Bφ Bψ Fixed an injective left-invariant lift we define the bispectral invariants as . Corollary: If are compactly supported, and are invertible for in an open and dense set, then L : ( ) → (SE(2, N))L 2 ℝ2 L 2 Bf BLf f , g ∈ ( )L 2 ℝ2 ( )Lf ^ T λ ( )Lgˆ T λ λ ≡ ⟺ g = π(x, k)f  for some (x, k) ∈ SE(2, N).Bf Bg Unfortunately, this is never true: Lf (x, k) = ⟨f , π(x, k)Ψ ⟹ rk ( ) ≤ 1.⟩ ( )L 2 ℝ2 Lfˆ T λ ROTATIONAL BISPECTRAL INVARIANTS The rotational bispectral invariants for are, for any and any , the rotational-invariant quantities f ∈ ( )L 2 ℝ2 , ∈ λ1 λ2 k ∈ ℤN R ( , , k) = ( ⊗ ) ∘ ( ⊗ ( .Bf λ1 λ2 Lfˆ T λ1 T λ2 Lfˆ T λ1 ) ∗ Lfˆ T Rk λ2 ) ∗ Theorem: There exists a residual set such that for any compactly supported it holds  ⊂ ( )L 2 ℝ2 f , g ∈  R ≡ R ⟺ g = f  for some k ∈ .Bf Bg Rk ℤN The semi-discretization is essential for this result In practice they can be efficiently computed as: R ( , , k) ⟺ ∑ ( ( + )) .Bf λ1 λ2 f ̂ R−ℓ λ1 λ2 ( )f ̂ R−ℓλ1 ⎯ ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ ( )f ̂ Rk−ℓλ2 ⎯ ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ EXPERIMENTAL RESULTS THANK YOU FOR YOUR ATTENTION