[DL輪読会]最新の深層強化学習

%-
IBDL
࠷৽ͷਂ૚‫ڧ‬Խֶश ૑଄৘ใֶઐ߈.
ॳ୩ྯ࣊

#BDL
HSPVOE w ਂ૚‫ڧ‬ԽֶशͰৗউ'9USBEFSΛҭ͍ͯͨ

0VUMJOF w ਂ૚‫ڧ‬Խֶश͓͞Β͍
w "$&3
w 6/3&"-
w 21SPQ
w 1(2

ਂ૚‫ڧ‬Խֶश͓͞Β͍

%2/ w WBMVF
CBTFE ํࡦΛཅʹද‫
͍ͳ͠ݱ‬ w ߦಈՁ஋ؔ਺ 2ؔ਺ Λ//Ͱද‫
ݱ‬ w P⒎QPMJDZͰFYQFSJFODF
SFQMBZΛ࢖ͬͯߋ৽
w ޯ഑ʹۙࣅ2Λ࢖͍ͬͯΔͨΊCJBTFE 1MBZJOH
"UBSJ
XJUI
%FFQ
3FJOGPSDFNFOU
-FBSOJOH

.OJI
FU
BM

"$ w QPMJDZͱঢ়ଶՁ஋ؔ਺7Λ//Ͱද‫
ݱ‬ w POQPMJDZ
ෳ਺"HFOUͰඇಉ‫ֶظ‬श
w POQPMJDZͷͨΊલͷQPMJDZͷαϯϓϧ͸࢖͑ͳ͍
w ਤதLͰCJBT
WBSJBODF
USBEF
P⒎ΛDPOUSPM " "TZODISPOPVT
.FUIPET
GPS
%FFQ
3FJOGPSDFNFOU
-FBSOJOH

.OJI
FU
BM

%%1( w QPMJDZΛֶश
w QPMJDZΛ֬཰෼෍Ͱͳܾ͘ఆతͳؔ਺ͱͯ͠ද‫
ݱ‬ w P⒎QPMJDZͰTBNQMF
F⒏DJFOU
w ޯ഑ʹۙࣅ2Λ࢖͍ͬͯΔͨΊCJBTFE $POUJOVPVT
DPOUSPM
XJUI
EFFQ
SFJOGPSDFNFOU
MFBSOJOH

-JMMJDSBQ
FU
BM

େମͷτϨϯυ ओ‫؍‬ w w w WBMVF
CBTFEΑΓQPMJDZ
CBTFE
w ࿈ଓ஋Λද‫͖Ͱݱ‬Δ
w ํࡦΛ֬཰తʹද‫͖Ͱݱ‬Δ
POQPMJDZΑΓP⒎QPMJDZ
w ‫ੈ࣮ݱ‬ք΁ͷԠ༻࣌ʹPOQPMJDZͩͱίετߴ TBNQMF
F⒏DJFOU
w ํࡦ͕‫ॴہ‬ղʹؕΓͮΒ͍
CJBT
WBSJBODF
USBEF
P⒎Λվળͯ͠ऩଋΛ଎͘΋͘͠͸҆ఆͤ͞Δ
w "$͸POQPMJDZ͔͠࢖͑ͳ͍
w %%1(͸P⒎QPMJDZ࢖͑Δ͚Ͳ2ͷޯ഑࢖͏͔ΒCJBTߴͰ҆ఆ͠ͳ͍

7BMVF
CBTFE 1PMJDZ
CBTFE POQPMJDZ WBSJBODFߴ "$ %2/ P⒎QPMJDZ CJBTߴ %%1(

21SPQ 21SPQ
4BNQMF&⒏DJFOU
1PMJDZ
(SBEJFOU
XJUI
"O
0⒎1PMJDZ
$SJUJD

(V
FU
BM

21SPQᶃ w ςʔϥʔల։ͷ߲̍࣍Λ࢖ͬͯMJLFMJIPPE
SBUJPʹ %%1(ͷޯ഑Λ௥Ճ
w ௥Ճ߲ʹ܎਺Λಋೖͯ͠CJBTҰఆͷ··WBSJBODFΛ Լ͛Δ
w %%1(ͷ߲Ͱ࢖͏2͸P⒎QPMJDZͰߋ৽Մೳ

21SPQᶄ ೚ҙͷؔ਺Gʹ͍ͭͯ੒ཱ ৄࡉ͸࿦จ"QQFOEJY Gʹ"Λ࢖͍
ςʔϥʔల։ΛЖपΓʹ͢Δ MJLFMJIPPE
SBUJP %%1( ΠʔλΛಋೖͯ͠όΠΞεΛ૿Ճͤ͞ͳ͍··
όϦΞϯεΛ‫ݮ‬গͤ͞Δ

21SPQᶅ w w //Ͱද͢ͷ͸
w QPMJDZɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹ ЖͱЄͰΨ΢γΞϯQPMJDZ
Ж͸%%1(߲Ͱ΋࢖͏
w 7 "EWBOUBHF
GVODUJPOͷͨΊ
w 2 %%1(ͷͨΊ
͜ͷ಺2͸P⒎QPMJDZͰֶशՄೳ

21SPQᶆ

21SPQ·ͱΊ w όΠΞε্͛ͣʹόϦΞϯεΛԼ͛ͯ҆ఆੑͷ޲্
w 2ΛP⒎QPMJDZͰֶश
w ΠʔλΛ্ख͘બͿͱ2ͷֶश೗ԿʹґΒֶͣश͕ ҆ఆ͢Δ

7BMVF
CBTFE 1PMJDZ
CBTFE POQPMJDZ WBSJBODFߴ "$ 21SPQ %2/ P⒎QPMJDZ CJBTߴ %%1(

6/3&"-

6/TVQFSWJTFE
3&JOGPSDFNFOU
BOE
"VYJMJBSZ
-FBSOJOH 3FJOGPSDFNFOU
-FBSOJOH
XJUI
6OTVQFSWJTFE
"VYJMJBSZ
5BTLT

+BEFSCBSH
FU
BM

6/3&"-ᶃ w ಉ͡ωοτϫʔΫΛ࢖ͬͯิॿతͳλεΫΛղ͔ͤ Δ͜ͱͰऩଋΛ଎͘͢Δ
w ิॿλεΫʹ͸FYQFSJFODF
SFQMBZ͕࢖͑Δ
w ਓ͕ເΛ‫ݟ‬ΔͷΛ‫͑ߟʹݩ‬ΒΕ͍ͯΔ

6/3&"-ᶄ

6/3&"-ᶅ w SFXBSE
QSFEJDUJPOͷ࣌͸SFXBSE͕ͳ͍֬཰͕ ʹͳΔΑ͏ʹ&3͔Βαϯϓϧ͢Δ
w QJYFM
DPOUSPMΑ͘Θ͔Βͳ͍
w ऩଋ͸"$ΑΓ଎͍͠ੑೳྑ͍
w ϩόετੑ΋͋Δ

7BMVF
CBTFE 1PMJDZ
CBTFE POQPMJDZ WBSJBODFߴ "$ 21SPQ %2/ P⒎QPMJDZ CJBTߴ %%1( 6/3&"-

"$&3
"DUPS$SJUJD
&YQFSJFODF
3FQMBZ 4BNQMF
&⒏DJFOU
"DUPS$SJUJD
XJUI
&YQFSJFODF
3FQMBZ

8BOH
FU
BM

"$&3 w ॏཁ౓αϯϓϦϯάΛߦ͏͜ͱͰP⒎QPMJDZͷσʔ λΛ࢖ͬͯ‫ࡏݱ‬ͷQPMJDZΛֶश
w 3FUSBDFΛ࢖ͬͯॏཁ౓ॏΈΛ੍‫
ݶ‬ w 5SVTU
3FHJPO
1PMJDZ
0QUJNJ[BUJPOΛޮ཰Խ 4BGF
BOE
F⒏DJFOU
P⒎QPMJDZ
SFJOGPSDFNFOU
MFBSOJOH

.VOPT
FU
BM

7BMVF
CBTFE 1PMJDZ
CBTFE POQPMJDZ WBSJBODFߴ "$ 6/3&"- 21SPQ %2/ P⒎QPMJDZ CJBTߴ %%1( "$&3

1(2 1(2
$PNCJOJOH
QPMJDZ
HSBEJFOU
BOE
2MFBSOJOH

0`%POPHIVF
FU
BM

1(2 w ऩଋ‫ݻ‬ఆ఺ʹ͓͍ͯ੒Γཱͭ౳ࣜΛ࢖ͬͯ2ΛQPMJDZ Λ࢖ͬͯද‫
ݱ‬ w ͦͷ2ʹରͯ͠2MFBSOJOHΛߦ͏͜ͱͰQPMJDZΛ VQEBUF

1PMJDZ
CBTFE POQPMJDZ WBSJBODFߴ 7BMVF
CBTFE "$ 1(2 6/3&"- 21SPQ %2/ P⒎QPMJDZ CJBTߴ %%1( "$&3

·ͱΊ w "$΍%%1(͕࣋ͭ՝୊͕໌Β͔ʹͳ͖ͬͯͯTBNQMF
F⒏DJFODZ΍CJBT
WBSJBODF
USBEF
P⒎ͷվળ͕ॏཁ
w "$&3ͷΑ͏ͳૉ๿ʹP⒎QPMJDZΛద༻͢Δͷ΋͋Ε͹ 6/3&"-ͷΑ͏ʹ‫ڧ‬Խֶशͱ͍͏ΑΓ//ͷಛੑΛ ࢖ͬͨ΋ͷ΋͋Γ‫ڵ‬ຯਂ͍
w ‫ݸ‬ਓతʹ͸21SPQ͕Ұ൪Ͳ͏͍͏ൃ૝Ͱͦ͏ͨ͠ͷ͔ Θ͔Βͳ͔ͬͨ
(V͞Μͷ.V1SPQͱ͔ͪΌΜͱಡΊ ͹Θ͔Δ͔΋͠Εͳ͍