Single Variable Optimization

Contents

1. Max-Min Theorem for Continuous Functions		6. Extreme Points Examples			
	2. Absolute (global) and Local Extreme Values		7. Increasing and Decreasing Functions	
	3. Mathematica Command					8. Tests for Local Extreme Values	 
	4. Extreme Points and Derivatives				9. The First Derivative Test for Local Extreme Values	
	5. Using First Derivative Test to Find Extreme Points	10. The Second Derivative Test for Local Extreme Values

Department of Economics
Iowa State University

1. Max-Min Theorem for Continuous Functions

Note on open and closed intervals:

If a and b are two numbers with a < b, then the open interval from a to b is the collection of all numbers which are both larger than a and smaller than b.  The open interval consists of all numbers between a and b.  A compact way of writing this is a < x < b.  We denote an open interval with parentheses as (a, b).

If a and b are two numbers with a < b, then the closed interval from a to b is the collection of all numbers which are both greater than or equal to a and less than or equal to b.  The closed interval consists of all points between a and b including a and b.  A compact way of writing this is a # x # b.  We denote a closed interval with brackets as [a, b].

Intervals that are closed on one end and open on the other are called half-open intervals.  We can denote these as half open on the right or the left and use a mixture of brackets and parentheses as appropriate.

Max-Min theorem for continuous functions:

2. Absolute (Global) and Local Extreme Values

Absolute maximum:

Let f be a function with domain D.  Then f has an absolute maximum value on D at a point c if
		f(x) ⩽ f(c)    for all x ϵ D

Absolute minimum:

Let f be a function with domain D.  Then f has an absolute minimum value on D at a point d if 
		f(x) ⩾ f(c)    for all x ϵ D

Local maximum:

Consider a real valued function f defined with domain D.   Then f is said to have a local maximum at an interior point \(TraditionalForm\`\(x\^*\)\) ϵ D if there exists a real number δ > 0 such that
		f(x) ⩽ f(\(TraditionalForm\`\(x\^*\)\))    for all x satisfying ||x-\(TraditionalForm\`\(x\^*\)\) || < δ

Local minimum:

Consider a real valued function f defined with domain D.   Then f is said to have a local minimum at an interior point \(TraditionalForm\`\(x\^*\)\) ϵ D if there exists a real number δ > 0 such that
		f(x) ⩾ f(\(TraditionalForm\`\(x\^*\)\))    for all x satisfying ||x-\(TraditionalForm\`\(x\^*\)\) || < δ

Strict maxima and minima, optimal points, and extreme points:

If the value of f at \(TraditionalForm\`\(x\^*\)\) is strictly larger than at any other point in the interval, the x* is a strict maximum point.  Similarly \(TraditionalForm\`x\&~\) is a strict minimum point if f(x) > f(\(TraditionalForm\`x\&~\)) for all x in the interval, x ≠ \(TraditionalForm\`\(\(x\&~\)\(\ \)\)\). We often refer to maxima and minima as optimal points or extreme points.

3. Mathematica Command

In Mathematica, the commands for local maximun and local minimum are:
	■ FindMaximum[f, {x, (Subscript["x", 0])}] searches for a local maximum in f, starting from the point x = (Subscript["x", 0]).
	■ FindMinimum[f, {x, (Subscript["x", 0])}] searches for a local minimum in f, starting from the point x=(Subscript["x", 0]).
Notice that there is no command in Mathematica for absolute maximum or absolute minimum.

Here is an example for finding local maximum

Clear[f, x,  sol];
f[x_] := (8  x - x^4);
Print["The graph for the function f[x] = ",  
        f[x]];
Plot[f[x], {x, -3,  3}, AxesLabel -> {"x", "f(x)"},  
        PlotStyle -> {RGBColor[ .1,  .3,  .8], 
            Thickness[0.011]}];
sol  =  FindMaximum[f[x], {x, 1}]
Print["The maximum point is at x = ",  
        sol[[2, 1, 2]] ];
\(\(Print[\*""The maximum value of f[x] is f[x]_max = "", \ sol[\([1]\)]\ ];\)\)

Here is an example for finding local minimum

Clear[f, x,  sol];
f[x_] := x^2 +  2  x;
Print["The graph for the function f[x] = ",  
        f[x]];
Plot[f[x], {x, -3,  3}, AxesLabel -> {"x", "f(x)"},  
        PlotStyle -> {RGBColor[ .1,  .3,  .8], 
            Thickness[0.011]}];
sol  =  FindMinimum[f[x], {x, 1}]
Print["The minimum point is at x = ",  
        sol[[2, 1, 2]] ];
\(\(Print[\*""The minimum value of f[x] is f[x]_min = "", \ sol[\([1]\)]\ ];\)\)

The algorithm that Mathematica uses to find a maximum or minimum of a function is based on somewhat sophisticated serarch techniques.  If the function to be optimized has no local or global or is badly behaved near the starting values (or elsewhere), FindMinimum and FindMaximum may be unable to find a solution in a finite number of iterations.

4. Extreme Points and Derivatives

A first derivative test for an extreme point:

Proof: To show that f´(c) is zero at a local extremum, we show first that f´(c) cannot be positive and second that it cannot be negative.  The only number that is neither positive or negative is zero, so that is what f´(c) must be.  Suppose that f has a local maximum value at x = c so that f(x) - f(c) # 0 for all values of x sufficiently close to c. Since c is an interior point of f=s domain, f´(c) is defined by the two-sided limit
		\(\(\(\(lim\)\(\ \)\)\+\(x\  -> \ \ c\)\) \(f \((x)\)\  - \ f \((c)\)\)\/\(x\  - \ c\)\)
This means that the right-hand side and left-hand side limits both exist at c and equal f´(c).  When we examine these limits separately, we find that
			\(\(f\^′\) \((c)\)\  = \ 
        lim\+\(x\  -> \ \(c\^+\)\)\(f \((x)\)\  - \ f \((c)\)\)\/\(x\  - \ c\) ≤ \ 0\ \ \ \ because\ \ \ \((x - c)\) > 
          0\ \ and\ \ f \((x)\)\  ≤ \ f \((c)\)\)
Similarly,		\(\(f\^′\) \((c)\)\  = \ 
        lim\+\(x\  -> \ \(c\^-\)\)\(f \((x)\)\  - \ f \((c)\)\)\/\(x\  - \ c\) ≥ \ 0\ \ \ \ because\ \ \ \((x - c)\) ≺ 
          0\ \ and\ \ f \((x)\)\  ≤ \ f \((c)\)\)
Together the above inequations imply that (f^′) (c)=0. 
The proof for minimum values is similar.

Implication of Theorem 2:

The only places where a function f can possibly have an extreme value (local or global) are
	1. interior points where (f^′)(▪) = 0
	2. interior points where f´(▪) is undefined
	3. endpoints of the domain of f.

Critical (stationary) points:

An interior point of the domain of a function f where f´ is zero or undefined is a critical (stationary) point of f.  
The only domain points where a function f can assume extreme values are critical points and endpoints.

Graphical illustration of extreme points:

Relative maximum:

Clear[f,  x,  p1,  p2,  p3];
f[x_]  :=  
        20  +  10 x  -  0.5 x^2;
d1 = D[f[x], x];
d1
xs = Solve[d1 == 0, x]
xstar = xs[[1, 1, 2]]
\(\(p1\  = \ 
        Plot[f[x], \ {x, \ \(-5\), \ 25}, \ AspectRatio\  -> \ 0.6, 
          AxesLabel\  -> \ {"x", \ ""}, \ \ AxesOrigin\  -> \ {0, 0}, \ PlotRange\  -> \ {{\(-5\), \ 25}, \ {\(-40\), \ 80}}, 
          PlotStyle -> {RGBColor[ .1,  .3,  .8], Thickness[0.011]}, \ 
          DisplayFunction\  -> \ Identity];\)\)
p2  =  
        Plot[f[xstar],  {x,  -5,  25},  
          PlotStyle -> {RGBColor[ .8,  .3,  .1], Thickness[0.008]},  
          DisplayFunction  ->  Identity];
p3  =  
        Graphics[{Text["f(x)",  {22,  20}] ,  
            Text["f'(x)",  {22,  
                65}]  }];
Show[p1,  p2, p3,  
        DisplayFunction  ->  $DisplayFunction ];

f[x_]  :=  
        20  +  10 x  -  0.5 x^2;
Print["The solution from FindMaximum: "]
FindMaximum[f[x],  {x,  5}]

Relative minimum:

Clear[f,  x,  p1,  p2,  p3];
f[x_]  :=   x^2;
d1 = D[f[x], x];
d1
xs = Solve[d1 == 0, x]
xstar = xs[[1, 1, 2]]
\(\(p1\  = \ 
        Plot[f[x], \ {x, \ \(-10\), \ 20}, \ AspectRatio\  -> \ 0.6, 
          AxesLabel\  -> \ {"x", \ ""}, \ \ AxesOrigin\  -> \ {0, 0}, \ PlotRange\  -> \ {{\(-15\), \ 25}, \ {\(-50\), \ 450}}, 
          PlotStyle -> {RGBColor[ .1,  .3,  .8], Thickness[0.011]}, \ 
          DisplayFunction\  -> \ Identity];\)\)
p2  =  
        Plot[f[xstar],  {x,  -10,  20},  
          PlotStyle -> {RGBColor[ .8,  .3,  .1], Thickness[0.008]},  
          DisplayFunction  ->  Identity];
p3  =  
        Graphics[{Text["f(x)",  {22,  300}] ,  
            Text["f'(x)",  {22,  
                25}]  }];
Show[p1,  p2, p3,  
        DisplayFunction  ->  $DisplayFunction ];

f[x_]  :=  x^2;
Print["The solution from FindMinimum: "]
FindMinimum[f[x],  {x,  5}]

5. Using First Derivative Test to Find Extreme Points or Critical Values

The only places where a function f can possibly have an extreme value (local or global) are
	1. interior points where (f^′)(▪) = 0
	2. interior points where f´(▪) is undefined
	3. endpoints of the domain of f.
We use the first derivative test to check if condition 1 is satisfied.  Specifically, if f has a local maximum or a local minimum at an interior point c ϵ D, and if f´ is defined at c, then  (f^′)(
          c)  =   ((d f(▪))/(d x)) (Subscript[| , c])  = 0. We find the first derivative of the function, set it equal to zero and then solve the equation for x, often denoted by x*.  We obtain the value of the function at x* by substituting in f( ).

Consider the example below.

Clear[f, x];
f[x_] := (8  x - x^2);
Print["The graph for the function f[x] = ",  
        f[x]];
Plot[f[x], {x, 0,  8}, AxesLabel -> {"x", "f(x)"},  
        PlotStyle -> {RGBColor[ .1,  .3,  .8], 
            Thickness[0.011]}];
d1 = D[f[x], x];
Print["This is the first derivative of the function f'(x) = ",  
      d1]
xs  = Solve[d1 == 0, x];
xstar = xs[[1, 1, 2]];
Print["This is the value of x that solves the equation setting the \
first derivative equal to zero, xstar = ", 
        xstar];
Print["This is the value of f(x) evaluated at xstar, f[xstar] = \
", f[xstar] ]

       
Print["This is the replacement rule for the value of x that solves \
the equation setting the first derivative equal to zero"];
c = Solve[{D[f[x], x] == 0}, {x}]
Print["This is the value of f(x) evaluated at xstar" ]
f[ c[[1, 1, 2]] ]

6. Examples of Extreme Points

(1). Find the absolute maximum and minimum values of f(x) = (x^2), x ϵ [-2, 1] :

The only critical value of f(x) is at x = 0 because (f^′)(x) = 2x = 0  =>  x = 0.  At x = 0, f(x) = 0.  The endpoint values are f(-2) = 4 and f(1) = 1.  So the function has an absolute maximum value of 4 at x = -2 and an absolute minimum value of 0 at x = 0.

f[x_]  :=   x^2;
\(\(Plot[f[x], \ {x, \ \(-2\), \ 1}, \ AspectRatio\  -> \ 0.6, 
        AxesLabel\  -> \ {"x", \ "f(x)"}, \ \ PlotRange\  -> \ All, AxesOrigin\  -> \ {0, 0}, \ 
        PlotStyle -> {RGBColor[ .1,  .3,  .8], 
            Thickness[0.011]}];\)\)

f[x_]  :=  x^2;
Print["The solution from FindMinimum: "]
FindMinimum[f[x],  {x,  5}]
Print["The solution from FindMaximum: "]
FindMaximum[f[x],  {x,  5}]
Print["The solution from first derivative test: "]
c = Solve[{D[f[x], x] == 0}, {x}]
f[ c[[1, 1, 2]] ]

(2). Find the absolute maximum and minimum values of f(x) = 8x - (x^4), x ϵ [-2, 1] :

Setting the derivative equal to zero we obtain
		8 - 4 (x^3) = 0
		=>	 4 (x^3) = 8
		=>	    (x^3) = 2
		=>	      x = (2^(1/3))=(2^(1/3))
This point is not in the given domain.  So we check the endpoints.  This yields f(-2) = -32 and f(1) =7.  So the absolute maximum is 7 at x = 1 and the absolute minimum is -32 at x = -2.

Clear[f];
f[x_]  :=   8 x  -  x^4;
\(\(Plot[f[x], \ {x, \ \(-2\), \ 1}, \ AspectRatio\  -> \ 0.6, 
        AxesLabel\  -> \ {"x", \ "f(x)"}, \ \ PlotRange\  -> \ All, AxesOrigin\  -> \ {0, 0}, \ 
        PlotStyle -> {RGBColor[ .1,  .3,  .8], 
            Thickness[0.011]}];\)\)

f[x_]  :=  8 x  -  x^4;
Print["The solution from FindMaximum: "]
FindMaximum[f[x],  {x,  0}]
Print["The solution from FindMinimum: "]
FindMinimum[f[x],  {x,  0}]
Print["The solution from first derivative test: "]
c = Solve[{D[f[x], x] == 0}, {x}]  // N
f[ c[[2, 1, 2]] ]

Notice that the real root of first derivative test is x = 1.2599, which is beyong the region of x ϵ [-2, 1]. Thus the maximum and minimun is at the two endpoints.

Clear[f];
f[x_]  :=   8 x  -  x^4;
\(\(Plot[f[x], \ {x, \ \(-2\), \ 2}, \ AspectRatio\  -> \ 0.6, 
        AxesLabel\  -> \ {"x", \ "f(x)"}, \ \ PlotRange\  -> \ All, AxesOrigin\  -> \ {0, 0}, \ 
        PlotStyle -> {RGBColor[ .1,  .3,  .8], 
            Thickness[0.011]}];\)\)

7. Increasing and Decreasing Functions

Definition:

First derivative test for increasing and decreasing:

Suppose that f is continuous on [a, b] and differentiable on (a, b).
	a. If f´ > 0 at each point of (a, b), then f increases on [a, b].
	b. If f´ < 0 at each point of (a, b), then f decreases on [a, b].

8. Tests for Local Extreme Values

While the zero derivative condition (critical value) is necessary for a maximum or minimum it is clear that it is not sufficient.  In other words, while all local extreme points are critical values, not all critical values are extreme points.  Consider the four cases or types of critical points below.

(1). Unique global minimum:

First consider a function where there is a unique global minimum so that a relative minimum is also the global minimum.

f[x_]  :=   x^2;
d1 = D[f[x], x];
d1
xs = Solve[d1 == 0, x]
xstar = xs[[1, 1, 2]]
\(\(p1\  = \ 
        Plot[f[x], \ {x, \ \(-10\), \ 20}, \ AspectRatio\  -> \ 0.6, 
          AxesLabel\  -> \ {"x", \ ""}, \ \ AxesOrigin\  -> \ {0, 0}, \ PlotRange\  -> \ {{\(-15\), \ 25}, \ {\(-50\), \ 450}}, 
          PlotLabel -> \ "Unique Global Mimimum", 
          PlotStyle -> {RGBColor[ .1,  .3,  .8], Thickness[0.011]}, \ 
          DisplayFunction\  -> \ Identity];\)\)
p2  =  
        Plot[f[xstar],  {x,  -10,  20},  
          PlotStyle -> {RGBColor[ .8,  .3,  .1], Thickness[0.008]},  
          DisplayFunction  ->  Identity];
p3  =  
        Graphics[{Text["f(x)",  {22,  300}] ,  
            Text["f'(x)",  {22,  
                25}]  }];
Show[p1,  p2, p3,  
        DisplayFunction  ->  $DisplayFunction ];

f[x_]  :=  x^2;
Print["The solution from FindMinimum: "]
FindMinimum[f[x],  {x,  5}]
Print["The solution from first derivative test: "]
c = Solve[{D[f[x], x] == 0}, {x}]
f[ c[[1, 1, 2]] ]

(2). Unique global maximum:

Now consider a case where the relative maximum is a global maximum and the derivative of f(x) is a positive constant.

f[x_]  :=  6 x  -   x^2;
d1 = D[f[x], x];
d1
xs = Solve[d1 == 0, x]
xstar = xs[[1, 1, 2]]
\(\(p1\  = \ 
        Plot[f[x], \ {x, \ \(-10\), \ 20}, \ AspectRatio\  -> \ 0.6, 
          AxesLabel\  -> \ {"x", \ ""}, \ \ AxesOrigin\  -> \ {0, 0}, \ PlotRange\  -> \ {{\(-15\), \ 25}, \ {\(-300\), \ 50}}, 
          PlotStyle -> {RGBColor[ .1,  .3,  .8], Thickness[0.011]}, 
          PlotLabel -> \ "Unique Global Maximum", \ 
          DisplayFunction\  -> \ Identity];\)\)
p2  =  
        Plot[f[xstar],  {x,  -10,  20},  
          PlotStyle -> {RGBColor[ .8,  .3,  .1], Thickness[0.008]},  
          DisplayFunction  ->  Identity];
p3  =  
        Graphics[{Text["f(x)",  {22,  -250}] ,  
            Text["f'(x)",  {22,  
                25}]  }];
Show[p1,  p2, 
        p3,   DisplayFunction  ->  $DisplayFunction ];

f[x_]  := 6 x  -   x^2;
Print["The solution from FindMaximum: "]
FindMaximum[f[x],  {x,  5}]
Print["The solution from first derivative test: "]
c = Solve[{D[f[x], x] == 0}, {x}]
f[ c[[1, 1, 2]] ]

(3). Local maxima and minima but not global extreme values over the entire domain (in this case the real numbers)

Now consider a case where the function has local maxima and minima and there are no global maximums or minimums.

f[x_]  :=  50  - 5 x  + 0.05 x^3;
d1 = D[f[x], x];
d1
xs = Solve[d1 == 0, x]
x1star = xs[[1, 1, 2]]
x2star = xs[[2, 1, 2]]
\(\(p1\  = \ 
        Plot[f[x], \ {x, \ \(-10\), \ 15}, \ AspectRatio\  -> \ 0.6, 
          AxesLabel\  -> \ {"x", \ ""}, \ \ AxesOrigin\  -> \ {0, 0}, \ PlotRange\  -> \ {{\(-15\), \ 20}, \ {\(-70\), \ 170}}, 
          PlotStyle -> {RGBColor[ .1,  .3,  .8], Thickness[0.011]}, 
          PlotLabel -> \ "Relative Max & Min", \ 
          DisplayFunction\  -> \ Identity];\)\)
p2  =  
        Plot[f[x1star],  {x,  -10,  15},  
          PlotStyle -> {RGBColor[ .8,  .3,  .1], Thickness[0.008]},  
          DisplayFunction  ->  Identity];
p3  =  
        Plot[f[x2star],  {x,  -10,  15},  
          PlotStyle -> {RGBColor[ .5,  .3,  .5], Thickness[0.008]},  
          DisplayFunction  ->  Identity];
p4  =  
        Graphics[{Text["f(x)",  {17,  150}] ,  
            Text["f'(x) 1",  {18,  75}]  ,  
            Text["f'(x) 2",  {18,  35}]}];
Show[p1,  p2, p3,  
        p4,   DisplayFunction  ->  $DisplayFunction,  
        ImageSize  ->  400 ];

f[x_]  :=  50  - 5 x  + 0.05 x^3;
Print["The solution from FindMinimum: "]
FindMinimum[f[x],  {x,  5}]
Print["The solution from FindMaximum: "]
FindMaximum[f[x],  {x,  5}]
Print["The solution from first derivative test: "]
c = Solve[{D[f[x], x] == 0}, {x}]
f[ c[[1, 1, 2]] ]
f[ c[[2, 1, 2]] ]

(4). No relative maxima or minima at critical points

Consider the case where the first derivative is equal to zero but there is no relative maximum or minimum.  Such a point is called an inflection point..  The second derivative is zero at this point.

f[x_]  :=   x^3;
\(\(p1\  = \ 
        Plot[f[x], \ {x, \ \(-15\), \ 15}, \ AspectRatio\  -> \ 0.6, 
          AxesLabel\  -> \ {"x", \ ""}, \ \ AxesOrigin\  -> \ {0, 0}, \ PlotRange\  -> \ {{\(-20\), \ 20}, \ {\(-4000\), \ 4000}}, 
          PlotStyle -> {RGBColor[ .1,  .3,  .8], Thickness[0.011]}, 
          PlotLabel -> \ "Inflection Point", \ 
          DisplayFunction\  -> \ Identity];\)\)
p2  =  
        Plot[0,  {x,  -15,  15},  
          PlotStyle -> {RGBColor[ .8,  .3,  .1], Thickness[0.008]},  
          DisplayFunction  ->  Identity];
p3  =  
        Graphics[{Text["f(x)",  {17,  3000}] ,  
            Text["f'(x)",  {17,  
                400}]  }];
Show[p1,  p2, 
        p3,   DisplayFunction  ->  $DisplayFunction,  
        ImageSize  ->  400 ];

f[x_]  :=  x^3;
Print["The solution from FindMinimum: "]
FindMinimum[f[x],  {x,  0}]
Print["The solution from FindMaximum: "]
FindMaximum[f[x],  {x,  5}]
Print["The solution from first derivative test: "]
c = Solve[{D[f[x], x] == 0}, {x}]
f[ c[[1, 1, 2]] ]
f[ c[[2, 1, 2]] ]

9. The First Derivative Test for Local Extreme Values

At critical points

a. If f´ changes from positive to negative at c (f´ > 0 for x < c and f´ < 0 for x > c), then f has a local maximum value at c.

Clear[f, g, p1,  p2,  p3,  p4,  ptext1,  
        ptext2];
f[x_]  := 100  -  x^2;
g[x_]  :=  (x - 40)^2  /;  x > 30
g[x_]  :=  (x - 20)^2  /;  x < 30
p1  =  
        Plot[f[x],  {x,  -10,  10},  AspectRatio  ->  0.3,  
          Axes  ->  None,  
          PlotRange  ->  {{-15,  50},  {-30,  120}}, 
          PlotStyle -> {RGBColor[ .1,  .3,  .8], Thickness[0.005]},  
          DisplayFunction  ->  Identity];
p2  =  
        Plot[100,  {x,  -10,  10},  
          PlotStyle -> {RGBColor[ .8,  .3,  .1], Thickness[0.005]},  
          DisplayFunction  ->  Identity];
\(\(ptext1\  = \ 
        Graphics[{\ 
Thickness[0.004], \ 
            Line[{{\(-15\), \ \(-20\)}, \ {14, \ \(-20\)}}], 
RGBColor[ .9, 0, 0], Thickness[0.004], \ Dashing[{0.01}], 
            Line[{{0, \(-20\)}, {0, 100}}], 
RGBColor[0, 0, 
              0], 
Text["Local Max ", \ {8, \ 
                90}], 
Text["f'(c) = 0", \ {0, \ 
                110}], 
Text["f' > 0", \ {\(-4\), \ 
                50}], \ 
Text["f' < 0", \ {4, \ 
                50}], 
Text["c", \ {0, \ \(-25\)}]
}];\)\)
p3  =  
        Plot[g[x],  {x,  20,  40},  AspectRatio  ->  0.6,  
          Axes  ->  None,  
          PlotRange  ->  {{-15,  50},  {-30,  120}}, 
          PlotStyle -> {RGBColor[ .1,  .3,  .8], Thickness[0.005]},  
          DisplayFunction  ->  Identity];
\(\(ptext2\  = \ 
        Graphics[{\ 
Thickness[0.004], \ 
            Line[{{16, \ \(-20\)}, \ {45, \ \(-20\)}}], 
RGBColor[ .9, 0, 0], Thickness[0.004], \ Dashing[{0.01}], 
            Line[{{30, \(-20\)}, {30, 100}}], 
RGBColor[0, 
              0, 0], 
Text["Local Max ", \ {38, \ 
                90}], 
Text["f'(c) is undefined ", \ {30, \ 110}], 
Text["f' > 0", \ {23, \ 
                50}], \ 
Text["f' < 0", \ {37, \ 
                50}], 
Text["c", \ {30, \ \(-25\)}]
}];\)\)
$TextStyle = {FontSize -> 12, 
          FontWeight -> "Bold"};
Show[p1,  p2, ptext1,   p3,  ptext2,  
        DisplayFunction  ->  $DisplayFunction,  
        ImageSize  ->  700 ];
$TextStyle = {};

b. If f´ changes from negative to positive at c (f´ < 0 for x < c and f´ > 0 for x > c), then f has a local minimum value at c.

Clear[f, g, p1,  p2,  p3,  p4,  ptext1,  
        ptext2];
f[x_]  :=  40 + x^2;
g[x_]  :=  
      4 (x - 30)  +  40  /;  x > 30
g[x_]  :=  -7 (x - 30) + 40  /;  
        x < 30
p1  =  
        Plot[f[x],  {x,  -10,  10},  AspectRatio  ->  0.3,  
          Axes  ->  None,  
          PlotRange  ->  {{-15,  50},  {-30,  120}}, 
          PlotStyle -> {RGBColor[ .1,  .3,  .8], Thickness[0.005]},  
          DisplayFunction  ->  Identity];
p2  =  
        Plot[40,  {x,  -10,  10},  
          PlotStyle -> {RGBColor[ .8,  .3,  .1], Thickness[0.005]},  
          DisplayFunction  ->  Identity];
\(\(ptext1\  = \ 
        Graphics[{\ 
Thickness[0.004], \ 
            Line[{{\(-15\), \ \(-20\)}, \ {14, \ \(-20\)}}], 
RGBColor[ .9, 0, 0], Thickness[0.004], \ Dashing[{0.01}], 
            Line[{{0, \(-20\)}, {0, 40}}], 
RGBColor[0, 0, 
              0], 
Text["Local Min ", \ {0, \ 
                70}], 
Text["f'(c) = 0", \ {0, \ 
                110}], 
Text["f' < 0", \ {\(-4\), \ 
                30}], \ 
Text["f' > 0", \ {4, \ 
                30}], 
Text["c", \ {0, \ \(-25\)}]
}];\)\)
p3  =  
        Plot[g[x],  {x,  20,  40},  AspectRatio  ->  0.6,  
          Axes  ->  None,  
          PlotRange  ->  {{-15,  50},  {-30,  120}}, 
          PlotStyle -> {RGBColor[ .1,  .3,  .8], Thickness[0.005]},  
          DisplayFunction  ->  Identity];
\(\(ptext2\  = \ 
        Graphics[{\ 
Thickness[0.004], \ 
            Line[{{16, \ \(-20\)}, \ {45, \ \(-20\)}}], 
RGBColor[ .9, 0, 0], Thickness[0.004], \ Dashing[{0.01}], 
            Line[{{30, \(-20\)}, {30, 40}}], 
RGBColor[0, 
              0, 0], 
Text["Local Min ", \ {30, \ 
                70}], 
Text["f'(c) is undefined ", \ {30, \ 110}], 
Text["f' < 0", \ {23, \ 
                30}], \ 
Text["f' > 0", \ {37, \ 
                30}], 
Text["c", \ {30, \ \(-25\)}]
}];\)\)
$TextStyle = {FontSize -> 12, 
          FontWeight -> "Bold"};
Show[p1,  p2, ptext1,   p3,  ptext2,  
        DisplayFunction  ->  $DisplayFunction,  
        ImageSize  ->  700 ];
$TextStyle = {};

c. If f´ does not change sign at c  (f´ has the same sign on both sides of c), then f has no local extreme value at c.

Clear[f, g, p1,  p2,  p3,  p4,  ptext1,  
        ptext2];
f[x_]  :=  40 +  x^3/8;
g[x_]  :=  -7 (x - 30) + 40  /;  
        x > 30
g[x_]  :=  
      25 (30 - x)^(1/2)  +  40  /;  
        x < 30
p1  =  
        Plot[f[x],  {x,  -10,  10},  AspectRatio  ->  0.3,  
          Axes  ->  None,  
          PlotRange  ->  {{-15,  50},  {-30,  120}}, 
          PlotStyle -> {RGBColor[ .1,  .3,  .8], Thickness[0.005]},  
          DisplayFunction  ->  Identity];
p2  =  
        Plot[40,  {x,  -10,  10},  
          PlotStyle -> {RGBColor[ .8,  .3,  .1], Thickness[0.005]},  
          DisplayFunction  ->  Identity];
\(\(ptext1\  = \ 
        Graphics[{\ 
Thickness[0.004], \ 
            Line[{{\(-15\), \ \(-20\)}, \ {14, \ \(-20\)}}], 
RGBColor[ .9, 0, 0], Thickness[0.004], \ Dashing[{0.01}], 
            Line[{{0, \(-20\)}, {0, 40}}], 
RGBColor[0, 0, 
              0], 
Text["No Extreme", \ {0, \ 
                70}], 
Text["f'(c) = 0", \ {0, \ 
                110}], 
Text["f' > 0", \ {\(-4\), \ 
                30}], \ 
Text["f' > 0", \ {4, \ 
                30}], 
Text["c", \ {0, \ \(-25\)}]
}];\)\)
p3  =  
        Plot[g[x],  {x,  20,  40},  AspectRatio  ->  0.6,  
          Axes  ->  None,  
          PlotRange  ->  {{-15,  50},  {-30,  120}}, 
          PlotStyle -> {RGBColor[ .1,  .3,  .8], Thickness[0.005]},  
          DisplayFunction  ->  Identity];
\(\(ptext2\  = \ 
        Graphics[{\ 
Thickness[0.004], \ 
            Line[{{16, \ \(-20\)}, \ {45, \ \(-20\)}}], 
RGBColor[ .9, 0, 0], Thickness[0.004], \ Dashing[{0.01}], 
            Line[{{30, \(-20\)}, {30, 40}}], 
RGBColor[0, 
              0, 0], 
Text["No Extreme", \ {30, \ 
                70}], 
Text["f'(c) is undefined ", \ {30, \ 110}], 
Text["f' < 0", \ {23, \ 
                30}], \ 
Text["f' < 0", \ {37, \ 
                30}], 
Text["c", \ {30, \ \(-25\)}]
}];\)\)
$TextStyle = {FontSize -> 12, 
          FontWeight -> "Bold"};
Show[p1,  p2, ptext1,   p3,  ptext2,  
        DisplayFunction  ->  $DisplayFunction,  
        ImageSize  ->  700 ];
$TextStyle = {};

At endpoints

a. At the left endpoint a, if f´ < 0 (f´ > 0) for x > a, then f has a local maximum (minimum) at a.

Clear[f, g, p1,  p2,  p3,  p4,  ptext1,  
        ptext2];
f[x_]  :=  
        100 -  1/2 (x + 10)^2;
g[x_]  :=  1/2 (x - 20)^2 +  30;
p1  =  
        Plot[f[x],  {x,  -10,  3},  AspectRatio  ->  0.3,  
          Axes  ->  None,  
          PlotRange  ->  {{-15,  50},  {-30,  120}}, 
          PlotStyle -> {RGBColor[ .1,  .3,  .8], Thickness[0.005]},  
          DisplayFunction  ->  Identity];
\(\(ptext1\  = \ 
        Graphics[{\ 
Thickness[0.004], \ 
            Line[{{\(-15\), \ \(-20\)}, \ {14, \ \(-20\)}}], 
RGBColor[ .9, 0, 0], Thickness[0.004], 
            Line[{{\(-10\), \(-20\)}, {\(-10\), 
                  120}}], 
RGBColor[0, 0, 
              0], 
Text["Local Max ", \ {0, \ 
                110}], 
Text["f' < 0", \ {\(-4\), \ 
                30}], \ 
Text["a", \ {\(-10\), \ \(-25\)}]
}];\)\)
p3  =  
        Plot[g[x],  {x,  20,  32},  AspectRatio  ->  0.6,  
          Axes  ->  None,  
          PlotRange  ->  {{-15,  50},  {-30,  120}}, 
          PlotStyle -> {RGBColor[ .1,  .3,  .8], Thickness[0.005]},  
          DisplayFunction  ->  Identity];
\(\(ptext2\  = \ 
        Graphics[{\ 
Thickness[0.004], \ 
            Line[{{16, \ \(-20\)}, \ {45, \ \(-20\)}}], 
RGBColor[ .9, 0, 0], Thickness[0.004], \ 
            Line[{{20, \(-20\)}, {20, 120}}], 
RGBColor[0, 
              0, 0], 
Text["Local Min ", \ {30, \ 
                110}], 
Text["f' > 0", \ {30, \ 
                30}], \ 
Text["a", \ {20, \ \(-25\)}]
}];\)\)
$TextStyle = {FontSize -> 12, 
          FontWeight -> "Bold"};
Show[p1,  ptext1,   p3,  ptext2,  
        DisplayFunction  ->  $DisplayFunction,  
        ImageSize  ->  700 ];
$TextStyle = {};

b. At the right endpoint b, if f´ < 0 (f´ > 0) for x < b, then f has a local minimum (maximum)  at b.

Clear[f, g, p1,  p2,  p3,  p4,  ptext1,  
        ptext2];
f[x_]  :=  100 -  1/2 (x + 3)^2;
g[x_]  :=   2/3 (x - 30)^2 +  
          30;
p1  =  
        Plot[f[x],  {x,  -3,  10},  AspectRatio  ->  0.3,  
          Axes  ->  None,  
          PlotRange  ->  {{-15,  50},  {-30,  120}}, 
          PlotStyle -> {RGBColor[ .1,  .3,  .8], Thickness[0.005]},  
          DisplayFunction  ->  Identity];
\(\(ptext1\  = \ 
        Graphics[{\ 
Thickness[0.004], \ 
            Line[{{\(-15\), \ \(-20\)}, \ {14, \ \(-20\)}}], 
RGBColor[ .9, 0, 0], Thickness[0.004], 
            Line[{{10, \(-20\)}, {10, 120}}], 
RGBColor[0, 
              0, 0], 
Text["Local Min", \ {0, \ 
                110}], 
Text["f' < 0", \ {4, \ 
                30}], \ 
Text["b", \ {10, \ \(-25\)}]
}];\)\)
p3  =  
        Plot[g[x],  {x,  28,  40},  AspectRatio  ->  0.6,  
          Axes  ->  None,  
          PlotRange  ->  {{-15,  50},  {-30,  120}}, 
          PlotStyle -> {RGBColor[ .1,  .3,  .8], Thickness[0.005]},  
          DisplayFunction  ->  Identity];
\(\(ptext2\  = \ 
        Graphics[{\ 
Thickness[0.004], \ 
            Line[{{16, \ \(-20\)}, \ {45, \ \(-20\)}}], 
RGBColor[ .9, 0, 0], Thickness[0.004], \ 
            Line[{{40, \(-20\)}, {40, 120}}], 
RGBColor[0, 
              0, 0], 
Text["Local Max", \ {30, \ 
                110}], 
Text["f' > 0", \ {36, \ 
                30}], \ 
Text["b", \ {40, \ \(-25\)}]
}];\)\)
$TextStyle = {FontSize -> 12, 
          FontWeight -> "Bold"};
Show[p1,  ptext1,   p3,  ptext2,  
        DisplayFunction  ->  $DisplayFunction,  
        ImageSize  ->  700 ];
$TextStyle = {};

10. The Second Derivative Test for Local Extreme Values

The second derivative test

If f´(c) = 0 and f´´(c) < 0, then f has a local maximum at x = c.
	If f´(c) = 0 and f´´(c) > 0, then f has a local minimum at x = c.

Sufficient condition for a maximum or minimum

For some integer n $ 1, let f have a continuous nth derivative in the open interval (a,b).  Suppose also that for some interior point c in (a,b) we have
	(f^′)(c) = (f^″)(c) = ... = (f^(n - 1))  =  0, but (f^n)(c)≠ 0
	Then for n even, f has a local minimum at c if fn(c) >0, and a local maximum if fn(c) < 0.  If n is odd, there is neither a local maximum nor a local minimum at c.

Examples

a. y = f(x) = x2
	f'(x) = 2x = 0 
	=>	x = 0 is an extreme point 
	f''(x) = 2 which is not zero (n is even) and positive	
	=>	x = 0 is a local minimum

Plot[x^2,  {x,  -1,  1},  AspectRatio  ->  0.6,  
      PlotRange  ->  All,  
      PlotStyle -> {RGBColor[ .1,  .3,  .8], 
          Thickness[0.011]}]

f[x_]  :=   x^2;

    Print["The solution from FindMinimum: "]

    FindMinimum[f[x],  {x,  5}]

    Print["The solution from first derivative test: "]

    xstar = Solve[{D[f[x], x] == 0}, {x}]

    f[ xstar[[1, 1, 2]] ]

    Print["The second derivative test:"]

      

    D [ D[f[x],  x] ,   x] /.  xstar

The first derivative of f at 0 is equal to zero, so 0 is a critical point.  The second derivative of f at zero is positive, so 0 yields a relative minimum.

b.  example 2
	y = f(x) = 6x -x2
	f'(x) = 6 -  2x = 0 
	=>	x = 3 is an extreme point 
	f''(x) = -2 which is not zero (n is even) and negative
	=>	x = 3 is a local maximum

Plot[6 x  -  x^2,  {x,  1,  5},  AspectRatio  ->  0.6,  
      PlotRange  ->  All,  GridLines ->  {{3}, {9}},  
      PlotStyle -> {RGBColor[ .1,  .3,  .8], 
          Thickness[0.011]}]

f[x_]  :=   6 x  -  x^2;
Print["The solution from FindMaximum: "]
FindMaximum[f[x],  {x,  5}]
xstar = Solve[{D[f[x], x] == 0}, {x}]
f[ xstar[[1, 1, 2]] ]
Print["The second derivative test:"]
D [ D[f[x],  x] ,   x] /.  xstar

The first derivative of f at 3 is equal to zero, so 3 is a critical point.  The second derivative of f at 3 is negative, so 3 yields  a relative maximum.

c.  example 3
	y = f(x) = x3
	f'(x) = 3x2 = 0 
	=>	x = 0 is an extreme point 
	f''(x) = 6x which is zero at zero (n is even) 
	f'''(x) = 6 which is not zero but positive (n is odd)
	=>	x = 0 is a neither a local maximum nor a local minimum

Plot[x^3,  {x,  -1,  1},  AspectRatio  ->  0.6,  
      PlotRange  ->  All,  
      PlotStyle -> {RGBColor[ .1,  .3,  .8], 
          Thickness[0.011]}]

f[x_]  :=   x^3;
Print["The solution from FindMaximum: "]
FindMaximum[f[x],  {x,  0}]
Print["The solution from FindMinimum: "]
FindMinimum[f[x],  {x,  0}]
xstar = Solve[{D[f[x], x] == 0}, {x}]
f[ xstar[[1, 1, 2]] ]
Print["The second derivative test:"]
D [ D[f[x],  x] ,   x] /.  xstar
Print["The third derivative test:"]
D[D [ D[f[x],  x] ,   x], x] /.  xstar

The first derivative of f at 0 is equal to zero, so 0 is a critical point.  In fact the equation has two roots at zero.  The second derivative of f at 0 is zero, and the third derivative is positive, so we cannot say whether 0 is a relative maximum, a relative minimum or something else.

Partial converse of second derivative test

Suppose f´´(a) exists.  If f has a local minimum at a, then  f´´(a)  $ 0; if f has a local maximum at a, then  f´´(a)  # 0.