edu.iastate.jrelm.rl.bushmosteller
Class LinearGBMParameters

java.lang.Object
  extended by edu.iastate.jrelm.rl.bushmosteller.GBMParameters
      extended by edu.iastate.jrelm.rl.bushmosteller.LinearGBMParameters
All Implemented Interfaces:
RLParameters

public class LinearGBMParameters
extends GBMParameters

An implementation of AbstractGBMParmaters that specifies a linear function for the reinforcement strength modifier used in Generalised Bush-Mosteller learning. Specifically, v(r) = v * r. In this class, v is designated as the 'multiplier value' and is accessed via getMultiplierValue() and setMultiplierValue(double).

Note, in Generalised Bush-Mosteller learning, the feedback modifier function is constrained so that v(0) = 0 and 0 <= v(r) <= 1. This means the multiplier value v in v(r) = v * r must be chosen such that (v * r_min) >= 0 and (v * r_max) <= 1, where r_min and r_max are the minimum and maximum expected values for reinforcement strengths.

If you wish to use LinearGMBParameters to check that the current setting for the multiplier value v is valid, you must first set the min and max expected reinforcement strengths through setFeedbackRange(min, max).

Author:
Charles Gieseler

Field Summary
static double DEFAULT_LINEAR_MULTIPLIER
           
static double DEFAULT_MAX_EXPECTED_STRENGH
           
static double DEFAULT_MIN_EXPECTED_STRENGH
           
 
Constructor Summary
LinearGBMParameters()
          Default constructor.
LinearGBMParameters(double multiplierValue, double minExpectedStrength, double maxExpectedStrength)
          Construct a new LinearGBMParameters with the given linear multiplier, minimum expected strength and maximum expected stregth.
 
Method Summary
 double feedbackModifier(double reinforcementStrength)
          Bush-Mosteller learning uses a function v(r) in updating action choice selection probabilities.
 double getMaxExpectedFeedback()
           
 double getMinExpectedFeedback()
           
 double getMultiplierValue()
          Retrieve the current setting for the multiplier value v used in the feedback modifier function v(r) = v * r.
 java.lang.String getName()
          Get the name of the algorithm these parameters are for.
 java.lang.String[] getParameterNames()
          Get a the names of the parameters
 void setFeedbackRange(double minimum, double maximum)
          Set the range of expected reinforcement strength values.
 void setMultiplierValue(double newValue)
          Set the current multiplier value v used in the feedback modifier function v(r) = v * r.
 boolean validateParameters()
          Other methods
 
Methods inherited from class edu.iastate.jrelm.rl.bushmosteller.GBMParameters
getRandomSeed, setRandomSeed
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

DEFAULT_LINEAR_MULTIPLIER

public static final double DEFAULT_LINEAR_MULTIPLIER
See Also:
Constant Field Values

DEFAULT_MIN_EXPECTED_STRENGH

public static final double DEFAULT_MIN_EXPECTED_STRENGH
See Also:
Constant Field Values

DEFAULT_MAX_EXPECTED_STRENGH

public static final double DEFAULT_MAX_EXPECTED_STRENGH
See Also:
Constant Field Values
Constructor Detail

LinearGBMParameters

public LinearGBMParameters()
Default constructor. Note, the defualt settings for the feedback multiplier value v is v = 1.0. The default settings for the minimum and maximum expected reinforcement strength are 0.0 and 1.0 respectively. Note, the default settings are only appropriate in situations where feedback values are garaunteed to be between 0 and 1 and you want to use the values directly (without modification).


LinearGBMParameters

public LinearGBMParameters(double multiplierValue,
                           double minExpectedStrength,
                           double maxExpectedStrength)
Construct a new LinearGBMParameters with the given linear multiplier, minimum expected strength and maximum expected stregth. Note, completion of construction does not garauntee that the given parameter values are valid. Please use validateParameters() after construction if you wish to confirm that the settings are appropriate for Linear Generalized Bush-Mosteller learning.

Parameters:
multiplierValue -
minExpectedStrength -
maxExpectedStrength -
Method Detail

feedbackModifier

public double feedbackModifier(double reinforcementStrength)
Description copied from class: GBMParameters
Bush-Mosteller learning uses a function v(r) in updating action choice selection probabilities. Here r is a reinforcement strength value resulting from the last action chosen. Essentially v(r) yields a modified feedback value, associated with the last selected action choice, that is then used in updating a Policy according to generalize Bush-Mosteller learning. Note, v(r) must be monotonically increasing in r, for r > 0, where v(0) = 0 and 0 <= v(r) <= 1.

Specified by:
feedbackModifier in class GBMParameters
Returns:
modified reinforcement strength

validateParameters

public boolean validateParameters()
Other methods

Specified by:
validateParameters in interface RLParameters
Specified by:
validateParameters in class GBMParameters
Returns:
true if all parameter values are valid, false otherwise.

getMultiplierValue

public double getMultiplierValue()
Retrieve the current setting for the multiplier value v used in the feedback modifier function v(r) = v * r.


setMultiplierValue

public void setMultiplierValue(double newValue)
Set the current multiplier value v used in the feedback modifier function v(r) = v * r. Note, in Generalised Bush-Mosteller learning, the feedback modifier function is constrained so that v(0) = 0 and 0 <= v(r) <= 1. This means the multiplier value v in v(r) = v * r must be chosen such that (v * r_min) >= 0 and (v * r_max) <= 1, where r_min and r_max are the minimum and maximum expected values for reinforcement strengths.


setFeedbackRange

public void setFeedbackRange(double minimum,
                             double maximum)
Set the range of expected reinforcement strength values. This range will be used in validating the current feedback multiplier setting.


getMinExpectedFeedback

public double getMinExpectedFeedback()

getMaxExpectedFeedback

public double getMaxExpectedFeedback()

getName

public java.lang.String getName()
Description copied from interface: RLParameters
Get the name of the algorithm these parameters are for.

Specified by:
getName in interface RLParameters
Specified by:
getName in class GBMParameters
Returns:
String name of the algorithm.

getParameterNames

public java.lang.String[] getParameterNames()
Description copied from interface: RLParameters
Get a the names of the parameters

Specified by:
getParameterNames in interface RLParameters
Overrides:
getParameterNames in class GBMParameters
Returns:
Array of parameter names as String