|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectedu.iastate.jrelm.rl.AbstractStatlessLearner<REParameters,I,A,Feedback<java.lang.Double>,REPolicy<I,A>>
edu.iastate.jrelm.rl.rotherev.RELearner<I,A>
public class RELearner<I,A extends Action<I>>
Roth-Erev Learner
The original Roth-Erev reinforcement learning algorithm was presented by A. Roth and
I. Erev in
"Learning in Extensive-Form Games: Experimental Data and Simple Dynamic
Models in the Intermediate Term," Games and Economic Behavior,
Special Issue: Nobel Symposium, vol. 8, January 1995, 164-212.
and
"Predicting How People Play Games with Unique Mixed-Strategy Equilibria,"
American Economics Review, Volume 88, 1998, 848-881.
This ReinforcementLearner implements a this later version of the algorithm Implementation adapted, in part, from the RothErevLearner in the Java Auction Simulator API (JASA) by Steve Phelps, Department of Computer Science, University of Liverpool.
Field Summary | |
---|---|
protected java.util.ArrayList<I> |
actionIDList
|
protected int |
domainSize
|
Constructor Summary | |
---|---|
RELearner(REParameters learningParams,
ActionDomain<I,A> aDomain)
Construct a RothErev learning component with parameters specified in a AREParameters object. |
|
RELearner(REParameters learningParams,
REPolicy<I,A> aPolicy)
Construct a RothErev learning component with parameters specified in a AREParameters object and the given policy. |
Method Summary | |
---|---|
protected double |
experience(int actionIndex,
double reward)
|
protected void |
generateBoltzmanProbs()
|
double |
getInitialPropensity()
Retreive the initial propensity value. |
java.lang.String |
getName()
Retrieves the name of the learning algorithm this learner implements. |
REParameters |
getParameters()
Returns the parameters currently being used by this RELearner. |
REPolicy<I,A> |
getPolicy()
Retrieve the StatelessPolicy being used to represent learned knowledge. |
protected void |
init()
|
REParameters |
makeParameters()
Create a default set of parameters that can be used with this learner. |
void |
reset()
Clear all learned knowledge. |
void |
setInitialPropensityValue(double initProp)
Set the initial propensity value. |
void |
setPolicy(REPolicy<I,A> newPolicy)
Set the StatelessPolicy to be used to represent learned knowledge. |
void |
update(double feedback)
Convenience version of the update(FeedbackDoubleValue) method |
void |
update(Feedback<java.lang.Double> feedback)
This activates the learning process according to the modified Roth-Erev learning algorithm. |
protected void |
updateProbabilities()
Updates the probability for each action to be chosen in the policy. |
protected void |
updatePropensities(double reward)
|
Methods inherited from class edu.iastate.jrelm.rl.AbstractStatlessLearner |
---|
chooseAction, getLastRandSeed, getLastSelectedAction, getUpdateCount, incrementUpdateCount, resetUpdateCount, setLastRandSeed, setLastSelectedAction, setParameters, setUpdateCount |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
protected int domainSize
protected java.util.ArrayList<I> actionIDList
Constructor Detail |
---|
public RELearner(REParameters learningParams, ActionDomain<I,A> aDomain)
learningParams
- - the collection of parameter settings for the
RELearneraDomain
- - the ActionDomain to learn overpublic RELearner(REParameters learningParams, REPolicy<I,A> aPolicy)
learningParams
- - the collection of parameter settings for the
RELearneraPolicy
- - a Roth-Erev specific policy type (REPolicy).Method Detail |
---|
protected void init()
init
in class AbstractStatlessLearner<REParameters,I,A extends Action<I>,Feedback<java.lang.Double>,REPolicy<I,A extends Action<I>>>
protected void updatePropensities(double reward)
protected double experience(int actionIndex, double reward)
protected void updateProbabilities()
protected void generateBoltzmanProbs()
public void update(Feedback<java.lang.Double> feedback)
This algorithm expects feedback to be given as a Double.
feedback
- - reward for the specified actionReinforcementLearner
public void update(double feedback)
feedback
- - reward for the specified action, given as a primitive
double.ReinforcementLearner
public void reset()
public double getInitialPropensity()
public void setInitialPropensityValue(double initProp)
initProp
- public java.lang.String getName()
ReinforcementLearner
public REParameters getParameters()
getParameters
in interface ReinforcementLearner<REParameters,I,A extends Action<I>,Feedback<java.lang.Double>,REPolicy<I,A extends Action<I>>>
getParameters
in class AbstractStatlessLearner<REParameters,I,A extends Action<I>,Feedback<java.lang.Double>,REPolicy<I,A extends Action<I>>>
ReinforcementLearner.getParameters();
public REParameters makeParameters()
ReinforcementLearner
ReinforcementLearner.makeParameters()
public REPolicy<I,A> getPolicy()
ReinforcementLearner
getPolicy
in interface ReinforcementLearner<REParameters,I,A extends Action<I>,Feedback<java.lang.Double>,REPolicy<I,A extends Action<I>>>
getPolicy
in class AbstractStatlessLearner<REParameters,I,A extends Action<I>,Feedback<java.lang.Double>,REPolicy<I,A extends Action<I>>>
ReinforcementLearner.getPolicy()
public void setPolicy(REPolicy<I,A> newPolicy)
ReinforcementLearner
setPolicy
in interface ReinforcementLearner<REParameters,I,A extends Action<I>,Feedback<java.lang.Double>,REPolicy<I,A extends Action<I>>>
setPolicy
in class AbstractStatlessLearner<REParameters,I,A extends Action<I>,Feedback<java.lang.Double>,REPolicy<I,A extends Action<I>>>
edu.iastate.jrelm.rl.ReinforcementLearner#setPolicy(edu.iastate.jrelm.rl.StatelessPolicy)
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |