[DL輪読会]Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference

DEEP LEARNING JP [DL Papers] - - - - - http://deeplearning.jp/

• http://deeplearning.jp/

• LQH QHML LC O HLHLF ME :D O J :DQTMOI EMO 1EEHBHDLQ 5LQDFDO -OHQGKDQHB LJ 5LEDODLBD • .DLMHQ BMA IHOK LQ 7JHF .M /GDL 9DLFJMLF G 9 QQGDT LF -LCODT MT OC OQTHF -C K 0KHQO 7 JDLHBGDLIM MMFJD • AKHQQQDC ML ( 0DB ) • GQQN , O HS MOF A ) ( )) • HLQ don • DL MO2JMT 8HQD rZb a pif h g c b lm ke

• • a eki q X c • // g N ORS M Pw bf •o • . // 8/8D 3 • e • yt • zh • • r • ) e o un 88H8/8D 3 h t B /8 B 1B 99 8/8D ,8 l /8D B 8/8D s /02 8D 8D (

• • • 0 -0 A Ge ,8/4 0-0 M e h bg • • 042 k /4 sl k 41 0 V w 89 h A Nn MA xs f M A • i •4 N • k ( ) 8 Go r VMA t

r3 = r1 r2 k • 5 0(2 e /2 (i,k) S3 (q3 Z3 ) = N X (i,j) S1 (q1 (j,k) Z1 )S2 (q2 j=1 r = S(q Z) 8b (i,k) q3 = Z3 + M N X (i,j) (q1 (j,k) Z1 )(q2 Z2 ) j=1 0(2 /2 M := 00 1 2() k a 9 i 5 S1 S2 S3 M 2 (0, 1) M =2 n M0 (M0 2 [0.5, 1)) 231 M0 /2 f Z2 )

(i,k) q3 = Z3 + M N X (i,j) (q1 (j,k) Z1 )(q2 Z2 ) j=1 (i,k) q3 0 = Z 3 + M @N Z 1 Z 2 (k) a2 := (k) Z 1 a2 (i) Z2 ā1 + N X N X j=1 (j,k) (i) q2 , ā1 := N X j=1 (i,j) (j,k) A q2 q1 j=1 (i,j) q1 1

1 o ru in in a F fh 8 7 23 8 2 8 N X ( , 8 * IMin (i,j) (j,k) q2 j=1 a l 9 /8 os t 8 * q1 0 v g M L 8 *T c l = e fh 0 e 8 * n w Sbias = S1 S2 , Zbias = 0 * + )

• s • • • p b g ixh 9,0 /8 ( c b a e • 9,0 /8 )0( 2 (8 tf • )( 1 ( (80 l • 9( 0 g e , n wor d k

l^ 56=l^ 1 9: ec q(r; a, b, n) := round ✓ b s(a, b, n) := n clamp(r; a, b) s(a, b, n) a ◆ s(a, b, n) + a a 1 2 1 = 1= ;:l^ [m .1,2/ .1,2/ l^ 91 b h 12: clamp(r; a, b) := min(max(x, a), b) ag .1,2/ l^ i : 1:= 1= ;: 8 8 )2 = ) : 0) ( S = s(a, b, n), Z = z(a, b, n) ]

• • ) 0 1 9 3: 8 C 3: 8 C 1 : 0 /: 3 • 1 3/ 3. / : : . / 1 3/ 3.3 /8. / • 3: 8 C, 3 • : 3 3 • 3 ( ) 1 F 33 9/ 3 3: 8 C L /: 3 0 1 9 3: 8 C 3: 8 C 08 0 9/ 3 2 1 1928 :3. 3 3 3:13 92 3: 8 C 1 : 0 8

-

• D6 A A 6A8 A 9D9A 9 I F 19FI D /4 ( • AF ( ) A 99C 19 D6 RN • 0 98 D9 A D6 A A A AF9 9D 2C9D6F A • AF AF9 9D T PO -0 T A F A6 19 D6 19FI D /4 ( LM 98 CD9 A AF9 9D FD6 A A