[DL輪読会]Deep Learning 第6章深層順伝播型ネットワーク

306 Views

June 01, 18

#日本語 #英語 #スライド #エンコード #不明確

スライド概要

2017/11/13
Deep Learning JP:
http://deeplearning.jp/seminar-2/

Deep Learning JP

@DeepLearning2023

スライド一覧

DL輪読会資料

またはPlayer版

埋め込む »CMSなどでJSが使えない場合

（ダウンロード不可）

関連スライド

【DL輪読会】KAN: Kolmogorov–Arnold Networks

Deep Learning JP 73.3K

【DL輪読会】Evolutionary Optimization of Model Merging Recipes モデルマージの進化的最適化

Deep Learning JP 53K

【拡散モデル勉強会】拡散モデルの数理

Deep Learning JP 36.7K

【拡散モデル勉強会】Introduction to Diffusion Models

Deep Learning JP 28K

【拡散モデル勉強会】拡散モデルのサンプラーまとめ

Deep Learning JP 24K

【DL輪読会】Generative Agents: Interactive Simulacra of Human Behavior

Deep Learning JP 22.9K

各ページのテキスト

,5 "6 04318. )-+/2 *'+/ #9 ! #%&+/2 ($ 7

$- . (,+)/& " !%#'* #' 0

[beta]

.*

 1

)5

 </-)5
 8$
 



#+,

 9062;(4:'&%"
 73!

[beta]


• BeGV
– _MJgIb

JgIbQf

,KO

:'9*6`H@CS

• >e GV
– Z

AT.

]dc^h\2-/<

Ib

• ]dc^h\2-/<
– &)/DN
• UY#=/;3 a
– ?E81-/
• W[
• (%7"0
• +4/5-$)

$7.:LP

TRiKO

$

".!

XF

�� • �� • �� • �� • ��Lp�� • ��L��r �Ld�� – ��m�� – �� d

[beta]

������������
• ����������������
• ������ )����� �
• ) ������*

����)��������

�����������*���
%!(
>

/2:
y

'#

y

"

G:
D@!*/
2:+85:
/2<9
 
6

h1

f(n)In:?)&*

w

h2

h
W

x1

x2

WIxh., A1
wIhy., $"(

x

-2:
%!(CH3 ;70  4
7=F0 BE

[beta]

NT'

"7U$

• PJV94'E
–

M(

-/A(

-/.*0E&%$HICB

– V8=DS
• 25>@
– L

,+)13#

O5'F!

• ;?26':Q
– W='KR$

,+)>@$G<$

�� 1�� • ��1.�� Original x space x2 1 0 0 1 x1

�� 0�,-�� 1�� 1�� 1�� /��2�� /.��2 ��

10.

�� g(z) = max{0, z} • �� 0 0 z • ��

11.

�� )�� • �� • ��

12.

•

13.

[beta]

Y

\JC

8U

• (+,-)&'."SHM<RK2?!
#$'2?O
3

SI5

OD



\J

S[

6FL19T

0

• GX<4E

@:O

7A

5

A;VQZN(+,-)&'."

,/%*O@:3

• PW(+,-)&'."=BA;>










A;

14.

�� . �� .��

15.

[beta]

!$.:THI57
• 3N)

("&<LD,*

=OEA


• Q9

;RB

+0C62.:Q9
• F-:S1
– 8:*



>

?/



>

FK!$.:Q94M<:.:J


– @G



P'%#$+0'%#$

16.

�� • ��;��6 – �� – ��a��6�� • ��;6��y��oe�� • �� • ��6��;�� t;��6��

17.

[beta]

 5=1.,• *+-;D@X5=1.,-@X

BH!%

• 5=AC8?@X$JP(4-20;DR<"%
• 5=1.,-G%

E>F

Y&1.,-5=;

#7TK%

• 9IVUZS/,-3)    :N&%Y&LWOM6'I
Q%

18.

[beta]

 ).*I
• /L?D%"

?D%"

!

• G?D-7J#(=F9
• <KC

• M6



A



!



,

NB9

1'&$;82>

%"

!

).

4H1+035

@E:4H

19.

�� • ��,�� • �� • ��,�� 0�� • �� • ��

20.

[beta]

37,1

•

&8?;e+2.3-'*50-.

YE!M"^f]>"RV;e%DG

<:

X+2.3-

'*>IKB"
• O]TI>"RV;e%DG"KB#)(4&/>I%9
bAP

#"

• ;`\8?FQ$#"

Ya"'6*="RV;

e%DG"
• O]TI

_Z"

• W[cHUA#

C$!


%Ld"

=IRV%J@SN@Ld"

21.

[beta]

26*0


"9B:_)1-2+$(4/+-

)1-2+$(AN

W7'(,!+$&%3".I8=N`D

QE>;]

?Y
<BUK^F;(#5


P)1-2+$(9BHT

L[

E



N\RS@J?)1-2+$(OCG

9BMZ



4/+-\8

VX

22.

Output Type Binary Discrete Output Distribution Output Layer Cost Function Bernoulli Binary crossSigmoid entropy Multinoulli Continuous Gaussian Continuous Mixture of Gaussian Continuous Arbitrary Discrete Softmax cross-entropy Gaussian Linear cross-entropy (MSE) Mixture Density See part III: GAN, VAE, FVBN Crossentropy Various

23.

�� 2��

24.

# * !" +$(%- )& ' ,

25.

[beta]

f)1/-.

KC>9D

• #+!f)1/-.  !X" KC>9D!X

 $"  $ ! # 
3g'" c\mZ0-.2 , bL!BE W<

•

 $   % 
• ?Q8Aa HM=FU"$P Y[]%K6
• R4lh =G 0-.2 ,*id CJ_I(
• H38k^ H R1/-.*:7
–

`O8;e  N)(

• KC>9Dj5@SJ(" =G TV&BE

26.

[beta]

 



"4O; 

! " = $%&{(, "}

• HL8CP(">N;&
• 20./F@;(KJ#! "Q53%%
• RI#347
• 1,+-"D?M#"<B"M A7('=E(
– $*"69>D#F@;RI):G(



h"ps://towardsdatascience.com/ac3va3on-func3ons-neuralnetworks-1cbd9f8d91d6

27.

�� • ��n ! " = max 0, " + *min(0, ") • �� ! " = |"| • ��3�� * = 0.01�� • �� *�� 3�� • �� https://towardsdatascience.com/activation-functions-neuralnetworks-1cbd9f8d91d6 • ��3�� • �� h"ps://www.slideshare.net/Takayosi/miru2014-tutorial-deeplearning-37219713

https://towardsdatascience.com/activation-functions-neural-

28.

�� • �� – ��! " = $(") – ��! " = tanh(") • ��tanh " = 2$ 2" − 1 �� • ��2��2�� – �� • ��2�� – �� • �� – ��2�� – ��

29.

[beta]

!'`'d0@<9;
  
• *.OH0&
• fCVR$0)]QZ_'FN[+#-LR /
• 1SMGFN%#

:869;1Ia3AU

• e\%Eb','("2/DK0&
 
• SMGFN& gbFN
• WYT^
– AB41X /
• 7=;?945@<9;
• JhPcFN
• 7=;>A5
•

30.

31.

[beta]

+.
     



•  )&45

– ,
– $30!
• =:?9 +. -*

– 
– 
• 




(7% /12!>8<; 

'6%#"

32.

[beta]

 S][nU$e 

• B&

#1*(KW2QV#0

• mk3@5?6<(KW2Pi#0)N\1"&

– Oa[n

KW'Hb0;>=7(d2GF#&,

1&
– Oa`ohC'-/Ic!

KW2Ml0,1&

• :89A4_DB1*^T(jL#KW2[n#0%(.(B'&
0()g+.1"&
• Rf(ECYWKWJ'ZX

33.

�� • ��( �� • �� • ��/4(��

34.

[beta]

"

#*$(0AG[JS8`





• bK,1 34-)+5 $

a

%6/1 '.&26E

• <ZL9T,1 34-)+5 $ a

X]P?E


• :=M\ dRU 

ResNet

^L9


• N>T-)+5 $ DYa I VBCO
• VBC!H;Q @F7_W

cL9

a

35.

[beta]

vkslyM($"p"xB/:2948
��� ��������������������
• ZXEJu)<Fu.Kl-*>Ju%ly,+d,
������ ���������������������������
• zg"Y_.@ qw136"h CM
• ni)"ZX.756;0=sW!ly
• oNH!#^U" ! ! zg "# !(#, &) .Y_+CM

– # #"xB.[&

bV"`A

– & #IV"EJf

– # ! ">J

),+xB"Y_aT"
+S'?tH

bV"`A

DA!eGjL

vkslyM#Rr/:2948QN.UmzgY_!\P,+'"
Y_zg.P Rr.O@+"#]cHzgM "/:2948

36.

��ae�� • �e �� • �� – �� • �� e�� ! = #$ �ae $% = &(() * + ,) ��e ��

37.

�� )�� • �� • ��6�� • �� (6.44) • �� (6.45) • �� (6.46) • �� (6.47) &=' # $ =' ! !=# $ , '6#�� ( ∈ ℝ+ , , ∈ ℝ- , #�ℝ+ ��ℝ- �� '�ℝ- ��ℝ��

38.

[beta]

  A8=:C,B@1;*<.  


•  
– 



– '03?7 5-()>
•  

4

– ! " ?703   45-()#
–

!"/6 &$29%+

39.

��

40.

�� • ��

41.

42.

43.

�� &' • ��s��h�� c� • ��nu�� y�� %."$ (-#/! • �� • �� *+,) • �� • ��

44.

�� • ��

45.

[beta]

  9`>$dZ^[gB 




• dZ^[g,_f%C]+#?;V&b*Y%)*C:<=MIA
I>%OJ
• F>NUB    ,E+#"&HK,@a
– QNT,a+(%8=>$PR,D\!SE
– G&ec(-6/504
– W32'&32&hX,D\+.715,G8%L

46.

"$! % • & #

47.

��3��

48.

�� .��

49.

��

50.

��/�� !"($) &3�� '3(()) /�� !"($) & = +, ' + 2/"())

51.

�� !"($) &�� )�� !' & = )" * + , (-) !"($) & = .+ )/ + 22"(-)

52.

�� • �� • �� ! " , "�� – �� • �� ! " – �� • �� %&' ��%�� &' �� – ��(��

53.

�� '%"(& ! #$ • ��6)�� • �� – ��)�� /�� • �� – ��/5)��/��/�� • ��

54.

[beta]

  >@/%0= 
 

•   


 #"((%,+;$5 9?40

 

• -*&03( A9<.8")6:27'!10

55.

[beta]

  ms]q;ILC@7a_,0yV 


"  

• U^yV  0VX

– ND

<K

Bzl -"#fi9M<

• *0S0e{.WZ-(+0vjrn}Z
  !
• k!=GAH?9<Y`09P<8RAPF oh! = − ∑% &% log *% 0yV6c$5

– \`d6pw[)5-+,- ! = *% − &%

– vjrn|/1'03#.pw[%,&."
•

 2  03#.GO JQ 91xb0E> R6uT(t4g(:M
G6pw[(+"5

56.

�� • �� • �� • �� – �� • �� • �� – ��

57.

�� • �� – ��9�� • �� – ��36�� – ��9�36��3�� • ��5��3�� – �� • ��9�36��35��

58.

[beta]

 /8C.  
 
•

 

  /8C.09

– $,)+'#C.!C.?2
• ;A3@%"*157>C.!6
<

D('&-B4:=

59.

�� • ��1�� – ��! ∶ ℝ$ → ℝ ��1&×& – ��&1�� • �� – �� – ��(��)�� – ��

60.

�� "$ # • �� ! • �� 4�� 4�� • �� – �� • �� 4�� – �� – �� • �� • �� 4��

61.

�� )��)�� • ��2�2�� – �� • ��2�� – �� • �� – �� • ��8�� – ��

62.

�� • ��7��7�� • �� – �� • �� – �� • �7��7�� – �� 3��7��7�� • �7��7�� • �� – ��3�� 7�� • �� • 1702�� 1702 ��7��7��

63.

�� • �� • �� – �� – �� • �� • ��

64.

�� • ��9�D�D� – 69D��9 ��D��9��D�� – �� 9�9��D�� 69D��

[DL輪読会]Deep Learning 第6章 深層順伝播型ネットワーク

Deep Learning JP

関連スライド

【DL輪読会】KAN: Kolmogorov–Arnold Networks

【DL輪読会】Evolutionary Optimization of Model Merging Recipes モデルマージの進化的最適化

【拡散モデル勉強会】拡散モデルの数理

【拡散モデル勉強会】Introduction to Diffusion Models

【拡散モデル勉強会】拡散モデルのサンプラーまとめ

【DL輪読会】Generative Agents: Interactive Simulacra of Human Behavior

各ページのテキスト

[DL輪読会]Deep Learning 第6章深層順伝播型ネットワーク