[DL輪読会]Disentangling by Factorising

DEEP LEARNING JP [DL Papers] Disentangling by Factorising Hirono Okamoto, Matsuo Lab http://deeplearning.jp/ 1

• http://deeplearning.jp/

: Disentangling by Factorising n 2018 ICML accepted n : Hyunjik Kim, Andriy Mnih β-VAE disentanglement metric n n disentangle n rotation position x scale position y Shape ( ) https://github.com/1Konny/FactorVAE gif

• https://github.com/1Konny/FactorVAE

: beta-VAE n : β-VAE: LEARNING BASIC VISUAL CONCEPTS WITH A CONSTRAINED VARIATIONAL FRAMEWORK n ICLR 2017(poster) n : n VAE disentangle n disentanglement VAE … ?? azimuth entangle

β-VAE n : : disentangle n ⇒ n p(z) n β N(0, I) 1 βVAE VAE n β z disentangle n β trade off …

β-VAE : disentangle Disentangle Metric n accuracy n 1. disentangle k ( scale) n 2. L n n encode n encode n 3. 2 z ( (y=Wz y n disentangle ) ) z y n n K-1 100% = metric score disentangle … Scale Scale 0 z

FactorVAE β-VAE n β-VAE ( ) old metric disentanglement n disentanglement n ⇒ Total Correlation Penalty n β-VAE disentanglement metric n ( n n ( (K K-1 100% ) n ⇒ a new metric for disentanglement :L … disentangle iteration ) ) new metric

FactorVAE : Total Correlation Penalty n FactorVAE VAE objective Total Correlation Penalty q(z) = <latexit sha1_base64="PEMDYUO5OQ+UOtLO81MQvQtmmIU=">AAAChHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIF2pUaSrYKsRk5pUoFMRXpySWJNZqVGgChWsqNFMq4gWUDfQMwEABk2EIZSgzQEFAvsByhhiGFIZ8hmSGUoZchlSGPIYSIDuHIZGhGAijGQwZDBgKgGKxDNVAsSIgKxMsn8pQy8AF1FsKVJUKVJEIFM0GkulAXjRUNA/IB5lZDNadDLQlB4iLgDoVGFQNrhqsNPhscMJgtcFLgz84zaoGmwFySyWQToLoTS2I5++SCP5OUFcukC5hyEDowuvmEoY0BguwWzOBbi8Ai4B8kQzRX1Y1/XOwVZBqtZrBIoPXQPcvNLhpcBjog7yyL8lLA1ODZjNwASPAED24MRlhRnqGBnqGgSbKDk7QqOBgkGZQYtAAhrc5gwODB0MAQyjQ3laG1QxbGLYysTHpMBkzmUKUMjFC9QgzoAAmOwDSDpSU</latexit> Z pdata (x)q(z|x)dx n Total Correlation (TC): density-ratio trick density-ratio trick n GAN z q(z) p(z|y = 1) p(y = 1|z) D(z) = = ⇡ q̄(z) p(z|y = 0) p(y = 0|z) 1 D(z)

FactorVAE : Total Correlation Penalty z n FactorVAE q(z) = <latexit sha1_base64="PEMDYUO5OQ+UOtLO81MQvQtmmIU=">AAAChHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIF2pUaSrYKsRk5pUoFMRXpySWJNZqVGgChWsqNFMq4gWUDfQMwEABk2EIZSgzQEFAvsByhhiGFIZ8hmSGUoZchlSGPIYSIDuHIZGhGAijGQwZDBgKgGKxDNVAsSIgKxMsn8pQy8AF1FsKVJUKVJEIFM0GkulAXjRUNA/IB5lZDNadDLQlB4iLgDoVGFQNrhqsNPhscMJgtcFLgz84zaoGmwFySyWQToLoTS2I5++SCP5OUFcukC5hyEDowuvmEoY0BguwWzOBbi8Ai4B8kQzRX1Y1/XOwVZBqtZrBIoPXQPcvNLhpcBjog7yyL8lLA1ODZjNwASPAED24MRlhRnqGBnqGgSbKDk7QqOBgkGZQYtAAhrc5gwODB0MAQyjQ3laG1QxbGLYysTHpMBkzmUKUMjFC9QgzoAAmOwDSDpSU</latexit> Z pdata (x)q(z|x)dx n Total Correlation (TC): GAN density-ratio trick n GAN z z

FactorVAE n 1. n 2. x n 3. : A New Metric for Disentanglement k encode x L z d k Old metric 0 New metric z

: β-VAE vs factorVAE, 2D Shapes n β-VAE FactorVAE ( n disentanglement metric ) ( ) ( n y x size shape shape entangle … )

: InfoWGAN-GP, 2D Shapes n Info-GAN + WGAN-GP n n infoGAN ( ) n (infoGAN …( )) better InfoWGAN-GP β-VAE Factor-VAE

: β-VAE vs factorVAE, 3D Shapes n disentangle n factorVAE shape scale disentangle ( ( ) )

: β-VAE vs factorVAE, 3D Chairs βVAE …? leg style ??

: β-VAE vs factorVAE, 3D Faces βVAE …? azimuth ??

: β-VAE vs factorVAE, CelebA FactorVAE

n contribution n 2D Shapes 3D Shapes disentanglement scores n β-VAE disentanglement metric n n GAN n limitation n Total Correlation n n future work n n q(z|x) disentangling p(z)=N(z|0,I) TC=0 betaVAE metric failure mode VAE n ex) factorVAE x

n Adversarial Autoencoder n infoGAN

: adversarial autoencoder n n AAE VAE n ICLR 2016 workshop

: InfoGAN n InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets n NIPS 2016 n disentangle n disentangle n ex) mnist

: n : n I(X; Y) = H(X) - H(X|Y) n H(X) H(X|Y) X n x, y n KL n p(x) p(x|y) ( n wikipedia ) Y

: n : n n c : : n I(c; G(c, z)) GAN c c n z n disentangle n …? L

: n p(c|x) Q(c|x) n n lemma A.1 P(c|x) Q D

: MNIST n condition n n c ~ unif(-1, 1) 10 y ~ cat(10) n D c c’ n https://github.com/znxlwm/pytorch-generative-model- collections/blob/master/infoGAN.py n n c c’ n disentangle c c’

• https://github.com/znxlwm/pytorch-generative-model-collections/blob/master/infoGAN.py

: yz n s c ( ) yz c ( )