edu.iastate.jrelm.rl
Class AbstractStatlessLearner<PA extends RLParameters,I,A extends Action<I>,F extends Feedback,PO extends StatelessPolicy<I,A>>

java.lang.Object
  extended by edu.iastate.jrelm.rl.AbstractStatlessLearner<PA,I,A,F,PO>
Type Parameters:
PA - - the type of ReinforcementLearner parameters this learner accepts
I - - the type of identifier being used to distiguish Actions
A - - the type of Actions this learner is working with
F - - the type of Feedback that this learner accepts
PO - - the type of Policy that this learner updates and uses make new Action selections
All Implemented Interfaces:
ReinforcementLearner<PA,I,A,F,PO>
Direct Known Subclasses:
GBMLearner, RELearner

public abstract class AbstractStatlessLearner<PA extends RLParameters,I,A extends Action<I>,F extends Feedback,PO extends StatelessPolicy<I,A>>
extends java.lang.Object
implements ReinforcementLearner<PA,I,A,F,PO>

Base abstract class for ReinforcementLearner classes that implement algorithms that do not make use of external environmental states. This class takes care of most of the ho-hum functionality that will likely be common to most statless ReinforcementLearner classes. Child classes will be free to focus on implementing the specifics of thier particular learning algorithms.

Author:
Charles Gieseler

Constructor Summary
AbstractStatlessLearner(PA learningParams, PO aPolicy)
          Construct an learning component with parameters specified in a PA and the given SimpleStatelessPolicy.
 
Method Summary
 A chooseAction()
          This is a basic implementation of ReinforcementLearner.chooseAction().
 int getLastRandSeed()
           
 A getLastSelectedAction()
          Retrieve the Action chosen from the last call to chooseAction().
 PA getParameters()
          Retrieve the RLParameters that contain settings for this learning algorithm.
 PO getPolicy()
          Retrieve the StatelessPolicy being used to represent learned knowledge.
 int getUpdateCount()
           
protected  void incrementUpdateCount()
          Allow child classes to mark each time update(Feedback) is called.
protected  void init()
           
protected  void resetUpdateCount()
          Sets the number of times update(Feedback) has been called back to zero.
 void setLastRandSeed(int lastRandSeed)
           
 void setLastSelectedAction(A lastSelectedAction)
           
 void setParameters(PA newParams)
          Note, this method will check that these parameters are valid before accepting them (PA.validate()).
 void setPolicy(PO newPolicy)
          Set the StatelessPolicy to be used to represent learned knowledge.
 void setUpdateCount(int updateCount)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface edu.iastate.jrelm.rl.ReinforcementLearner
getName, makeParameters, update
 

Constructor Detail

AbstractStatlessLearner

public AbstractStatlessLearner(PA learningParams,
                               PO aPolicy)
Construct an learning component with parameters specified in a PA and the given SimpleStatelessPolicy. Note: any random seed that is already set in the given policy will be overwritten with the seed in the given parameters.

Parameters:
learningParams - - the collection of parameter settings for the this AbstractStatlessLearner
aPolicy - - a SimpleStatelessPolicy
Method Detail

init

protected void init()

chooseAction

public A chooseAction()
This is a basic implementation of ReinforcementLearner.chooseAction(). Essentially, this method simply asks the learner's StatlessPolicy for a new Action choice, updates the last Action chosen and then forwards the choice on. IMPORTANT: It should be noted that this method also checks to see if this learner's parameters have changed and takes appropriate action. Specifically, it checks to see if the random seed has changed and, if so, updates the learner's policy with the new setting. Child classes overriding this method should keep this in mind. If we do not poll to see if the random seed has changed here, it will not get updated in the policy and any new random seed settings will not be used. NOTE: This will change in the future. Ideally we would like to have some kind of ActionListener structure that would allow the learner to take appropriate action whenever ANY parameter setting is changed.

Specified by:
chooseAction in interface ReinforcementLearner<PA extends RLParameters,I,A extends Action<I>,F extends Feedback,PO extends StatelessPolicy<I,A>>
Returns:
the next action chosen. Selected actions can be any object implementing the Action interface.
See Also:
edu.iastate.jrelm.ReinforcementLearner#chooseAction()

getUpdateCount

public int getUpdateCount()
Returns:
the number of times this ReinforcementLearner has been updated (i.e. the number of times update(Feedback) has been called).

incrementUpdateCount

protected void incrementUpdateCount()
Allow child classes to mark each time update(Feedback) is called.


resetUpdateCount

protected void resetUpdateCount()
Sets the number of times update(Feedback) has been called back to zero.


getLastSelectedAction

public A getLastSelectedAction()
Retrieve the Action chosen from the last call to chooseAction().

Returns:
the last Action choice selected.

getParameters

public PA getParameters()
Description copied from interface: ReinforcementLearner
Retrieve the RLParameters that contain settings for this learning algorithm.

Specified by:
getParameters in interface ReinforcementLearner<PA extends RLParameters,I,A extends Action<I>,F extends Feedback,PO extends StatelessPolicy<I,A>>
Returns:
learning algorithm settings as RLParameters
See Also:
ReinforcementLearner.getParameters()

setParameters

public void setParameters(PA newParams)
Note, this method will check that these parameters are valid before accepting them (PA.validate()). If the parameters fail validation, the old settings will be used and the new settings will be ignored.

Specified by:
setParameters in interface ReinforcementLearner<PA extends RLParameters,I,A extends Action<I>,F extends Feedback,PO extends StatelessPolicy<I,A>>
See Also:
edu.iastate.jrelm.rl.PA#validateParameters(), edu.iastate.jrelm.rl.ReinforcementLearner#setParameters(PA)

getPolicy

public PO getPolicy()
Description copied from interface: ReinforcementLearner
Retrieve the StatelessPolicy being used to represent learned knowledge.

Specified by:
getPolicy in interface ReinforcementLearner<PA extends RLParameters,I,A extends Action<I>,F extends Feedback,PO extends StatelessPolicy<I,A>>
Returns:
the Policy being used by this ReinforcementLearner. The policy can be any object implementing the StatelessPolicy interface.
See Also:
StatelessPolicy

setPolicy

public void setPolicy(PO newPolicy)
Description copied from interface: ReinforcementLearner
Set the StatelessPolicy to be used to represent learned knowledge.

Specified by:
setPolicy in interface ReinforcementLearner<PA extends RLParameters,I,A extends Action<I>,F extends Feedback,PO extends StatelessPolicy<I,A>>
See Also:
StatelessPolicy

getLastRandSeed

public int getLastRandSeed()

setLastRandSeed

public void setLastRandSeed(int lastRandSeed)

setLastSelectedAction

public void setLastSelectedAction(A lastSelectedAction)

setUpdateCount

public void setUpdateCount(int updateCount)