A comparison of eligibility trace and momentum on SARSA in continuous state-and action-space

Barry D. Nichols

doi:10.1109/CEEC.2017.8101599

A comparison of eligibility trace and momentum on SARSA in continuous state-and action-space

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Here the Newton's Method direct action selection approach to continuous action-space reinforcement learning is extended to use an eligibility trace. This is then compared to the momentum term approach from the literature in terms of the update equations and also the success rate and number of trials required to train on two variants of the simulated Cart-Pole benchmark problem. The eligibility trace approach achieves a higher success rate with a far wider range of parameter values than the momentum approach and also trains in fewer trials on the Cart-Pole problem.

Original language	English
Title of host publication	2017 9th Computer Science and Electronic Engineering (CEEC)
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	55-59
Number of pages	5
ISBN (Print)	978-1-5386-3008-2
DOIs	https://doi.org/10.1109/CEEC.2017.8101599
Publication status	Published - 29 Sept 2017
Event	2017 9th Computer Science and Electronic Engineering (CEEC) - Colchester, UK Duration: 27 Sept 2017 → 29 Sept 2017

Conference

Conference	2017 9th Computer Science and Electronic Engineering (CEEC)
Period	27/09/17 → 29/09/17

Keywords

reinforcement learning
eligibility trace
momentum
continuous state- and action-space
artificial neural networks

Access to Document

10.1109/CEEC.2017.8101599

https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8101599

Cite this

@inproceedings{f972ad25cf224412841f4ad0eb9ca532,

title = "A comparison of eligibility trace and momentum on SARSA in continuous state-and action-space",

abstract = "Here the Newton's Method direct action selection approach to continuous action-space reinforcement learning is extended to use an eligibility trace. This is then compared to the momentum term approach from the literature in terms of the update equations and also the success rate and number of trials required to train on two variants of the simulated Cart-Pole benchmark problem. The eligibility trace approach achieves a higher success rate with a far wider range of parameter values than the momentum approach and also trains in fewer trials on the Cart-Pole problem.",

keywords = "reinforcement learning, eligibility trace, momentum, continuous state- and action-space, artificial neural networks",

author = "Nichols, {Barry D.}",

year = "2017",

month = sep,

day = "29",

doi = "10.1109/CEEC.2017.8101599",

language = "English",

isbn = "978-1-5386-3008-2",

pages = "55--59",

booktitle = "2017 9th Computer Science and Electronic Engineering (CEEC)",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

address = "United States",

note = "2017 9th Computer Science and Electronic Engineering (CEEC) ; Conference date: 27-09-2017 Through 29-09-2017",

}

TY - GEN

T1 - A comparison of eligibility trace and momentum on SARSA in continuous state-and action-space

AU - Nichols, Barry D.

PY - 2017/9/29

Y1 - 2017/9/29

N2 - Here the Newton's Method direct action selection approach to continuous action-space reinforcement learning is extended to use an eligibility trace. This is then compared to the momentum term approach from the literature in terms of the update equations and also the success rate and number of trials required to train on two variants of the simulated Cart-Pole benchmark problem. The eligibility trace approach achieves a higher success rate with a far wider range of parameter values than the momentum approach and also trains in fewer trials on the Cart-Pole problem.

AB - Here the Newton's Method direct action selection approach to continuous action-space reinforcement learning is extended to use an eligibility trace. This is then compared to the momentum term approach from the literature in terms of the update equations and also the success rate and number of trials required to train on two variants of the simulated Cart-Pole benchmark problem. The eligibility trace approach achieves a higher success rate with a far wider range of parameter values than the momentum approach and also trains in fewer trials on the Cart-Pole problem.

KW - reinforcement learning

KW - eligibility trace

KW - momentum

KW - continuous state- and action-space

KW - artificial neural networks

UR - https://ieeexplore.ieee.org/document/8101599/

U2 - 10.1109/CEEC.2017.8101599

DO - 10.1109/CEEC.2017.8101599

M3 - Conference contribution

SN - 978-1-5386-3008-2

SP - 55

EP - 59

BT - 2017 9th Computer Science and Electronic Engineering (CEEC)

PB - Institute of Electrical and Electronics Engineers (IEEE)

T2 - 2017 9th Computer Science and Electronic Engineering (CEEC)

Y2 - 27 September 2017 through 29 September 2017

ER -

A comparison of eligibility trace and momentum on SARSA in continuous state-and action-space

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this