|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectedu.iastate.jrelm.rl.AbstractStatlessLearner<REParameters,I,A,Feedback<java.lang.Double>,REPolicy<I,A>>
edu.iastate.jrelm.rl.rotherev.RELearner<I,A>
edu.iastate.jrelm.rl.rotherev.variant.VRELearner<I,A>
public class VRELearner<I,A extends Action<I>>
Variant Roth-Erev Learner
This ReinforcementLearner implements a variation of the Roth-Ere'ev algorithm
as presented in
James Nicolaisen, Valentin Petrov, and Leigh Tesfatsian, "Market Power and
Efficiency in a Computational Electricity Market with Discriminatory
Double-Auction Pricing," IEEE Transactions on Evolutionary Computation,
Volume 5, Number 5, 2001, 504-523.
See VRELearner.experience(int, double) for more details.
In addition, this variation allows updating with given reward-action pairs. See VRELearner.update(FeedbackDoubleValue, Action).
See RELearner for details on the original Roth-Erev algorithm
Field Summary |
---|
Fields inherited from class edu.iastate.jrelm.rl.rotherev.RELearner |
---|
actionIDList, domainSize |
Constructor Summary | |
---|---|
VRELearner(VREParameters learningParams,
ActionDomain<I,A> aDomain)
Construct a learner using Variant Roth-Erev with parameters specified in a VREParameters object. |
|
VRELearner(VREParameters learningParams,
REPolicy<I,A> aPolicy)
Construct a RothErev learning component with parameters specified in a AMREParameters object and the given policy. |
Method Summary | |
---|---|
protected double |
experience(int actionIndex,
double reward)
|
protected void |
generateBoltzmanProbs()
|
java.lang.String |
getName()
Retrieves the name of the learning algorithm this learner implements. |
VREParameters |
getParameters()
Returns the parameters currently being used by this RELearner. |
VREParameters |
makeParameters()
Create a default set of parameters that can be used with this learner. |
void |
update(double feedback,
A actionToReinforce)
Convenience version of the update(Feedback, Action) method. |
void |
update(Feedback<java.lang.Double> feedback,
A actionToReinforce)
Update the Policy according to the Variant Roth-Erev algorithm, but associate the given feedback with the given Action. |
protected void |
updateProbabilities()
Updates the probability for each action to be chosen in the policy. |
Methods inherited from class edu.iastate.jrelm.rl.rotherev.RELearner |
---|
getInitialPropensity, getPolicy, init, reset, setInitialPropensityValue, setPolicy, update, update, updatePropensities |
Methods inherited from class edu.iastate.jrelm.rl.AbstractStatlessLearner |
---|
chooseAction, getLastRandSeed, getLastSelectedAction, getUpdateCount, incrementUpdateCount, resetUpdateCount, setLastRandSeed, setLastSelectedAction, setParameters, setUpdateCount |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public VRELearner(VREParameters learningParams, ActionDomain<I,A> aDomain)
learningParams
- - the collection of parameter settings for the
VRELearneraDomain
- - the ActionDomain to learn overpublic VRELearner(VREParameters learningParams, REPolicy<I,A> aPolicy)
learningParams
- - the collection of parameter settings for the
VRELearneraPolicy
- - a SimpleStatelessPolicyMethod Detail |
---|
public void update(Feedback<java.lang.Double> feedback, A actionToReinforce) throws java.lang.Exception
Note, the actual record of the last Action chosen will not be changed. Thus a call to getLastSelectedAction() will yield the same Action before and after a call to this update method.
This update method will throw an Exception if this learner does not know about the given Action. That is, an Exception will be thrown if the given Action is not in the ActionDomain that this learner is using.
feedback
- - reward for the specified action, given as a primitive
double.actionToReinforce
- - update will proceed associating the given
feedback with this Action
java.lang.Exception
public void update(double feedback, A actionToReinforce) throws java.lang.Exception
feedback
- - reward for the specified action, given as a primitive
double.actionToReinforce
- - update will proceed associating the given
feedback with this Action
java.lang.Exception
update(Feedback, Action)
protected void updateProbabilities()
RELearner
updateProbabilities
in class RELearner<I,A extends Action<I>>
protected void generateBoltzmanProbs()
generateBoltzmanProbs
in class RELearner<I,A extends Action<I>>
protected double experience(int actionIndex, double reward)
experience
in class RELearner<I,A extends Action<I>>
public java.lang.String getName()
ReinforcementLearner
getName
in interface ReinforcementLearner<REParameters,I,A extends Action<I>,Feedback<java.lang.Double>,REPolicy<I,A extends Action<I>>>
getName
in class RELearner<I,A extends Action<I>>
public VREParameters getParameters()
RELearner
getParameters
in interface ReinforcementLearner<REParameters,I,A extends Action<I>,Feedback<java.lang.Double>,REPolicy<I,A extends Action<I>>>
getParameters
in class RELearner<I,A extends Action<I>>
ReinforcementLearner.getParameters();
public VREParameters makeParameters()
ReinforcementLearner
makeParameters
in interface ReinforcementLearner<REParameters,I,A extends Action<I>,Feedback<java.lang.Double>,REPolicy<I,A extends Action<I>>>
makeParameters
in class RELearner<I,A extends Action<I>>
ReinforcementLearner.makeParameters()
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |