[DL輪読会]Semi-convolutional Operators for Instance Segmentation

DEEP LEARNING JP [DL Papers] “Semi-convolutional Operators for Instance Segmentation” , http://deeplearning.jp/

• http://deeplearning.jp/

• – • -0 )2 - 2182 – – O 21 ( - O npeE 2 O a iP O t E r iP • –s –V o luE rCmv S 2

• • https://engineering.matterport.com/splash-of-color-instancesegmentation-with-mask-r-cnn-and-tensorflow-7c761e238b46 3

• https://engineering.matterport.com/splash-of-color-instance-

• • 2) &) – – • V 2 – – & P () C I 4

Propose & Verify • & – ⭕ ❌ SM P N P VR SM C [2014 R. Girshick] Rich feature hierarchies for accurate object detection and semantic segmentation [2017, K. He] Mask R-CNN 5

P&V (RCNN) [2014 R. Girshick] Rich feature hierarchies for accurate object detection and semantic segmentation M VR Ø C 232 . Ø N & CVR a e C , ,. & . VR 1 F S P 6

P&V (Mask RCNN) [2017 K. He] Mask R-CNN . Ø 2 3 1 7

Instance Coloring • – I ⭕ ⭕ C ( ) D( [2017 A. Kendall] Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics 8

IC [2017 A. Fathi] Semantic Instance Segmentation via Deep Metric Learning • • 9

IC [2018 S. Kong] Recurrent Pixel Embedding for Instance Grouping • • N N – • R - - E (参考) arXiveTimes (https://github.com/arXivTimes/arXivTimes/issues/628) 10

• https://github.com/arXivTimes/arXivTimes/issues/628

IC • • I – – C & V N P& I CNNの演算の位置不変性により同 じ埋め込み表現となってしまいイ ンスタンスを区別できない 位置不変でない演算方法を提案 (Semi-Convolutional Operation) 2つの羊はほ ぼ同じ見た目 11

• P –& – V S ( ) • o g - vm t I rp I ceO i ) P n Os I C i au • au Ø f l O O y ) O C 12

Semi-Convolutional Operation for IC (1) • • • <latexit sha1_base64="EonYQ5G0ZK07MkJ3/smomCeY4Dg=">AAACu3ichVFNTxRBEH0MKDCorHox8TJxA8EL6SUkGI0JkYsHE5fFXTZhYNPTNLsNPR+Z6d0EJvMH+AMePGFijPEvePPCH/DA1ZvBGyRePFAzOwkqUavT3a9e1auu7vYirRLD2MmINTp27fr4xKQ9dePmrenK7TutJOzHQjZFqMO47fFEahXIplFGy3YUS+57Wq57eyt5fH0g40SFwSuzH8lNn3cDtaMEN0R1Ki233lOOmxcKHNfnpie4TtuZ45rw0n+RbaXuS192eWa7T/Jhz/0SdFxVij0vbWRb2w87lSqbZ4U5V0GtBFWUVg8r7+FiGyEE+vAhEcAQ1uBIaGygBoaIuE2kxMWEVBGXyGCTtk9ZkjI4sXu0dsnbKNmA/LxmUqgFnaJpxqR0MMO+sA/sjB2zj+wb+/nXWmlRI+9ln3ZvqJVRZ/rw3tqP/6p82g16l6p/9mywg0dFr4p6jwomv4UY6gcHr8/WHjdm0ln2lp1S/0fshH2mGwSDc/FuVTbewKYPqP353FdBa2G+Rnh1sbr8rPyKCdzHA8zRey9hGc9RR5PO/YSvOMV366klrF1LD1OtkVJzF7+Z1b8ApxWrPw==</latexit> • :X !L ⌦ d (L 2 R ) C 13

Semi-Convolutional Operation for IC (2) • a nvxC –r • c – – a r ( - - a (- u n n • r – – l kN lt ) a u u C - - i N a C u C l muoCpNu l mupNe S 14

Semi-Convolutional Operation for IC (3) • • () 15

Semi-Convolutional Operation for IC (4) • – – • 2 – 2 16

Steered bilateral filter • 2 2 ) – – 2 – 2 –) ( ) 17

1 .2 3 18

• • • • • O i 5 8 10 a S e c 10 1 5-10 2 e 51 C R - 10 1 5-10 2 51 N 19

Mask RCNN [2017 K. He] Mask R-CNN • • M + N • C a + R S a 20

• • • • C C M R N ) ( 21

• • 2 A2 10 22

23

( • a –& – • ( – – • v o ) - p g • g ) V c y C eil I p p um r ) n V f cg s OP ) S fv c t 24

• a • 21 –. e v P h R w /& R 0 1 0 0 A 8 0 2 2 – – – • a dk R ] C c v t ML C C L h wR f g[ v – 0 8 P –s nl i Vr[ R N C ML uo 25

• • • • • • K KO S 7AIE K KHPOEK H 4LAM OKMN KM NO A 7ACIA O OEK -EMNDE G E D A OPMA DEAM M DEAN KM PM OA K FA O AOA OEK NAI OE NACIA O OEK 0 .A 2 NG 0A HH 2PHOE 8 NG 1A M E C NE C AMO E OS OK AECD 1KNNAN KM 7 A A -AKIAOMS 7AI OE N , ODE 7AI OE NO A 7ACIA O OEK E AAL 2AOME 1A M E C 7 0K C A PMMA O ERAH I A E C KM NO A -MKPLE C 26

Appendix: Hough Voting • dn o u – – u 1 1 H . l . a 3 i m s 参考) 和歌山大学講義資料 http://www.wakayama-u.ac.jp/~wuhy/CV/CV2010/CV06.pdf • 2 u o o g u u h s et V p 27

• http://www.wakayama-u.ac.jp/~wuhy/CV/CV2010/CV06.pdf