edu.iastate.jrelm.rl.rotherev
Class REParameters

java.lang.Object
  extended by edu.iastate.jrelm.rl.rotherev.REParameters
All Implemented Interfaces:
RLParameters, uchicago.src.reflector.DescriptorContainer
Direct Known Subclasses:
VREParameters

public class REParameters
extends java.lang.Object
implements RLParameters, uchicago.src.reflector.DescriptorContainer

Collects and manages parameters settings required for the modified Roth-Erev reinforcement learning algorithm. Roth-Erev learning parameters: Experiementation: --explaination pending-- Note: Be careful not to choose value e where (1-e) == e / (N - 1), where N is the size of the action domain (i.e. e == 0.75 and N == 4). This will result in all action propensities receiving the same experience update value, regardless of the last action chosen. Action choice probabilities will then remain uniform and no learning will occur. Initial Propensity: --explaination pending-- Recency: --explaination pending-- Additional parameters:

Author:
Charles Gieseler

Field Summary
static double DEFAULT_BOLTZMANN
           
static double DEFAULT_EXPERIMENTATION
           
static double DEFAULT_INIT_PROPENSITY
           
static double DEFAULT_RECENCY
           
 
Constructor Summary
REParameters()
          This is meant solely as a convenience constructor.
REParameters(double experimentation, double initialPropensity, double recency)
          Build a VREParameters object with the given learning parameter values.
REParameters(double boltzmannTemp, double experimentation, double initialPropensity, double recency)
          Build a VREParameters object with the given learning parameter values.
REParameters(double boltzmannTemp, double experimentation, double initialPropensity, double recency, int randSeed)
          Additional parameters.
 
Method Summary
 double getBoltzmannTemp()
           
 double getExperimentation()
           
 double getInitialPropensity()
           
 java.lang.String getName()
          Get the name of the algorithm these parameters are for.
 java.util.Hashtable getParameterDescriptors()
           
 java.lang.String[] getParameterNames()
          Get the names of the parameters
 int getRandomSeed()
          Get the seed for the RandomEngine used in the REPolicy.
 double getRecency()
           
 void setBoltzmannTemp(double bTemp)
           
 void setExperimentation(double e)
           
 void setInitialPropensity(double initProp)
           
 void setRandomSeed(int randSeed)
          Set the seed to be used with the RandomEngine used in MREPolicy
 void setRecency(double r)
           
 boolean useBoltzmann()
          Indicate whether to use the Gibbs-Boltzmann probability distribution or default to the proportional distribution.
 boolean validateParameters()
          Checks to make sure the current values for all parameters are valid.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

DEFAULT_BOLTZMANN

public static final double DEFAULT_BOLTZMANN
See Also:
Constant Field Values

DEFAULT_EXPERIMENTATION

public static final double DEFAULT_EXPERIMENTATION
See Also:
Constant Field Values

DEFAULT_INIT_PROPENSITY

public static final double DEFAULT_INIT_PROPENSITY
See Also:
Constant Field Values

DEFAULT_RECENCY

public static final double DEFAULT_RECENCY
See Also:
Constant Field Values
Constructor Detail

REParameters

public REParameters()
This is meant solely as a convenience constructor. Parameters are set to the


REParameters

public REParameters(double experimentation,
                    double initialPropensity,
                    double recency)
Build a VREParameters object with the given learning parameter values.

Parameters:
experimentation - -
recency - -
initialPropensity -

REParameters

public REParameters(double boltzmannTemp,
                    double experimentation,
                    double initialPropensity,
                    double recency)
Build a VREParameters object with the given learning parameter values. Includes a temperture value for use in a Gibb-Boltzmann probabilty distribution. The learner will use this distribution in selecting new action choices, rather than the default proportional distribution.

Parameters:
boltzmannCooling -
experimentation - -
initialPropensity -
recency - -

REParameters

public REParameters(double boltzmannTemp,
                    double experimentation,
                    double initialPropensity,
                    double recency,
                    int randSeed)
Additional parameters. Specify a seed for the psuedo-random number generator used by REPolicy and temperture value for use in a Gibb-Boltzmann probabilty distribution. The learner will use this distribution in selecting new action choices, rather than the default proportional distribution.

Parameters:
boltzmannCooling -
experimentation -
initialPropensity -
numberOfActions -
recency -
randSeed -
Method Detail

validateParameters

public boolean validateParameters()
Description copied from interface: RLParameters
Checks to make sure the current values for all parameters are valid.

Specified by:
validateParameters in interface RLParameters
Returns:
true if all parameter values are valid, false otherwise.
See Also:
edu.iastate.jrelm.core.ai.learning.rl.RLParameters#validateParameters()

useBoltzmann

public boolean useBoltzmann()
Indicate whether to use the Gibbs-Boltzmann probability distribution or default to the proportional distribution.


getBoltzmannTemp

public double getBoltzmannTemp()
Returns:
Returns the value of the boltzmann temperature parameter.

setBoltzmannTemp

public void setBoltzmannTemp(double bTemp)
Parameters:
bTemp - Sets the value of the scaling paremeter.

getExperimentation

public double getExperimentation()
Returns:
Returns the experimentation value.

setExperimentation

public void setExperimentation(double e)

getInitialPropensity

public double getInitialPropensity()

setInitialPropensity

public void setInitialPropensity(double initProp)

getName

public java.lang.String getName()
Description copied from interface: RLParameters
Get the name of the algorithm these parameters are for.

Specified by:
getName in interface RLParameters
Returns:
String name of the algorithm.

getRecency

public double getRecency()
Returns:
Returns the value of the recency parameter.

setRecency

public void setRecency(double r)
Parameters:
recency - Sets the value of the recency parameter.

getRandomSeed

public int getRandomSeed()
Get the seed for the RandomEngine used in the REPolicy. If the seed has not been set, this will return (int)System.currentTimeMillis().

Specified by:
getRandomSeed in interface RLParameters
Returns:
- seed value for use in the policy
See Also:
REPolicy, RandomEngine

setRandomSeed

public void setRandomSeed(int randSeed)
Set the seed to be used with the RandomEngine used in MREPolicy

Parameters:
randSeed - - seed value

getParameterNames

public java.lang.String[] getParameterNames()
Get the names of the parameters

Specified by:
getParameterNames in interface RLParameters
Returns:
Parameter names as an array of Strings

getParameterDescriptors

public java.util.Hashtable getParameterDescriptors()
Specified by:
getParameterDescriptors in interface uchicago.src.reflector.DescriptorContainer