Constrained Optimization

Contents

1. Method of direct substitution                                          	2. Hessian of the Lagrangian Function 
                                                 	3. Mathematica Commands	                                             	4. Method of Lagrange  
                                                   	5. Interpretation of the Lagrangian multiplier			         	6. Excercises

Department of Economics
Iowa State University

1. Method of direct substitution.

Many economics problems involve optimization with certain side constraints. For example, a consumer might want to maximize a utility function subject to a budget constraint. In this section, we discuss this type of optimization.
Example 1.
Consider the maximization of the utility function f(x,y)=2xy, subject to the budget constraint, g(x,y)=x+2y-60=0. Graphically we want to pick points on the surface that also lie on the plane above the line. The solution of this constrained maximization problem is obtained by searching for extreme points along the constraint x = 60 - 2y on the surface z = f(x,y)=2xy.

Clear[f, x, y];
f[x_, y_] := 2*x*y
utility = 
        Plot3D[f[x, y], {x, 24, 36}, {y, 12, 18}, 
          AxesLabel -> {"x", "y", "f(x,y)"}, 
          ViewPoint -> {-1, -2, 0.5}, 
          DisplayFunction -> Identity];
budget = 
        ParametricPlot3D[{60 - 2*y, y, z}, {y, 12, 18}, {z, 500, 1300}, 
          DisplayFunction -> Identity];
Show[utility, budget, 
        DisplayFunction -> $DisplayFunction];
Print["This is the 3D graph of the utility function and the \
constraint function.The utility function slopes upwards toward the back of \
the diagram while the constraint is a vertical wall."]

Since the constraint implies an explicit restriction, x=60-2y, a direct substitution of the restriction in the utility function f(x,y) = 2(60-2y)y = F(y) converts a constrained maximization to an unconstrained one. Then, it is easy for us to find the extreme point.

Clear[F,y];
F[y_]=2*(60-2*y)*y;
Plot[F[y],{y,0,30}, AxesLabel->{"y", "F(y)"}]
FindMaximum[F[y],{y,10}]

Mathenatica estimated that a relative maximum occurs at y = 15, with a value of utility equal to 900. At that point, x=60-2y=15.

However, sometimes it is impossible for us to solve this kind of problem by direct substitution, if the constraint is more complicated. So, we will introduce another more powerful method called method of Lagrange.

2. Definition of Hessian of the Lagrangian Function (Bordered Hessian)

Consider the problem defined by 

                                                                 (Subscript[maximize, (Subscript[x, 1,]) 
                (Subscript[x, 2,]) ...  (Subscript[x, n,])]) f(x)
                                                               subject to (Subscript[g, 1])(x)=0 , (Subscript[g, 2])(x)=0, ..., (Subscript[g, m])(x)=0.  Where  m < n.
                                                               
 The solution can be obtained by using the Lagrangian function:
                       
                                                               L(x; λ)= f(x) - λ'g(x) Where λ' = \(TraditionalForm\`\((λ\_1, λ\_2, \  ... \ , \ λ\_m\)\))
                                                                          =  f((Subscript[x, 1]),(Subscript[x, 2]), ... , (Subscript[x, n])) - (Subscript[λ, 1])(Subscript[g, 1])(x) - (Subscript[λ, 2])(Subscript[g, 2])(x) - ... - (Subscript[λ, m])(Subscript[g, m])(x)
         
 Notice that the gradient of L will involve a set of derivatives as follows.                                 
               
Gradient of L:

(Subscript[∇, x]) L(x) = (Subscript[∇, x]) f(x) - (∂(g/∂)x)λ
Where 



                               (∂(g/∂)x)= (Subscript[J, g]) = (({{\(\(\(∂\)\(\ \)\)\^\(\(g\_1\) \((\(x\^*\))\)\)\/∂x\_1\), \(\(\(∂\)\(\ \)\)\^\(\(g\_2\) \((\(x\^*\))\)\)\/∂x\_1\), ,...,, \(\(\(∂\)\(\ \)\)\^\(\(g\_m\) \((\(x\^*\))\)\)\/∂x\_1\)}, {\(\(\(∂\)\(\ \)\)\^\(\(g\_1\) \((\(x\^*\))\)\)\/∂x\_2\), \(\(\(∂\)\(\ \)\)\^\(\(g\_2\) \((\(x\^*\))\)\)\/∂x\_2\), ,...,, \(\(\(∂\)\(\ \)\)\^\(\(g\_m\) \((\(x\^*\))\)\)\/∂x\_2\)}, {⋮, ⋮, ⋮, ⋮}, {\(\(\(∂\)\(\ \)\)\^\(\(g\_1\) \((\(x\^*\))\)\)\/∂x\_n\), \(\(\(∂\)\(\ \)\)\^\(\(g\_2\) \((\(x\^*\))\)\)\/∂x\_n\), ,...,, \(\(\(∂\)\(\ \)\)\^\(\(g\_m\) \((\(x\^*\))\)\)\/∂x\_n\)}}))



There will be one equation for each x .  There will also be equations involving the derivatives of L
with respect to each λ.

Checking the Sufficient Conditions and Bordered Hessian matrix of f:

\(The\ conditions\ for\ a\ maximum\ or\ minimum\ can\ be\ stated\ in\ terms\ of\ the\ Hessian\ of\ Lagrangian\ function\) 
(or bordered Hessian) .  Let f ,  (Subscript[g, 1]), 
      (Subscript[g, 2]),   ...  , 
      (Subscript[g, m]) be twice continuously differentialable real valued 
\(functions . \ 
\((1)\) 
            If\ there\ exist\ vectors\ \(x\^*\)\  ∈ \ R\^n\) ,(SuperStar[λ])∈ (R^m), such that
                      
                                          		∇ L(\(x\^*\), (SuperStar[λ])) = 0

and if 
             
                                    
                                           (((-1))^m)det(({{((∂^(2L(\(x\^*\), (SuperStar[λ]))))/(\(∂x\_1 ∂x\_1\)))...
     \(\(.\)\(\ \ \ \ \ \ \ \ \ \ \)\(
\)\(\ \ \ \)\(.\)\), ((∂^(2L(\(x\^*\), (SuperStar[λ]))))/(∂(Subscript[x, 1]) ∂(Subscript[x, p])))
  .
., ((\(∂\^\(\(g\_1\) \((\(x\^*\))\)\)\))/(∂(Subscript[x, 1])
  .   ))...
      ., \(∂\^\(\(g\_m\) \((\(x\^*\))\)\)\/∂x\_1

                    \ \ \ \ \(\(.\)\(
\)\(\ \ \)\(.\)\)\)}, {((∂^(2L(\(x\^*\), (SuperStar[λ]))))/(∂(Subscript[x, p]) ∂(Subscript[x, 1])))..., ((∂^(2L(\(x\^*\), (SuperStar[λ]))))/(∂(Subscript[x, p]) ∂(Subscript[x, p]))), \(∂\^\(\(g\_1\) \((\(x\^*\))\)\)\/∂x\_p ... \), \(∂\^\(\(g\_m\) \((\(x\^*\))\)\)\/∂x\_p\)}, {\(∂\^\(\(g\_1\) \((\(x\^*\))\)\)\/∂x\_1 ... 

                    \ \ \ \ \ \(\(.\)\(
\)\(\ \ \ \)\(.\)\)\), \(∂\^\(\(g\_1\) \((\(x\^*\))\)\)\/∂x\_p

                    \ \ \ \ \(\(.\)\(\ \)\(
\)\(\ \ \)\(.\)\)\), \(0.  .. 

                    \ \ \ \ \ \(\(.\)\(
\)\(\ \ \ \)\(.\)\)\), 0

                      .
.}, {\(∂\^\(\(g\_m\) \((\(x\^*\))\)\)\/∂x\_1 ... \), \(∂\^\(\(g\_m\) \((\(x\^*\))\)\)\/∂x\_p\), 0.  .. , 0}})) > 0
\(For\ p\  = \ m + 1, \  ... \ , \ n, \ 
      then\ f\ has\ a\ strict\ local\ minimum\ at\ \ \(x\^*\), \ such\ that\) 
                                                     
                                       (Subscript[g, i])\((\(x\^*\))\)=0,   i  =  1, ..., m . 

(2)If there exist vectors \(x\^*\) ∈ (R^n) ,(SuperStar[λ]) ∈ (R^m) , such that
                  
	                                  ∇ L(\(x\^*\), (SuperStar[λ])) = 0

and if 
                                         (((-1))^p)det(({{((∂^(2L(\(x\^*\), (SuperStar[λ]))))/(∂(Subscript[x, 1]) ∂(Subscript[x, 1])))...
     \(\(.\)\(\ \ \ \ \ \ \ \ \ \ \)\(
\)\(\ \ \ \)\(.\)\), ((∂^(2L(\(x\^*\), (SuperStar[λ]))))/(∂(Subscript[x, 1]) ∂(Subscript[x, p])))
  .
., \(∂\^\(\(g\_1\) \((\(x\^*\))\)\)\/\(∂x\_1

                      \(\(.\)\(\ \ \ \)\)\) ... 

                  \ \ \  . \), \(∂\^\(\(g\_m\) \((\(x\^*\))\)\)\/∂x\_1

                  \ \ \ \(\(.\)\(
\)\(\ \)\(.\)\)\)}, {((∂^(2L(\(x\^*\), (SuperStar[λ]))))/(∂(Subscript[x, p]) ∂(Subscript[x, 1])))..., ((∂^(2L(\(x\^*\), (SuperStar[λ]))))/(∂(Subscript[x, p]) ∂(Subscript[x, p]))), \(∂\^\(\(g\_1\) \((\(x\^*\))\)\)\/∂x\_p ... \), \(∂\^\(\(g\_m\) \((\(x\^*\))\)\)\/∂x\_p\)}, {\(∂\^\(\(g\_1\) \((\(x\^*\))\)\)\/∂x\_1\  ... 

                  \ \ \ \ \ \(\(.\)\(
\)\(\ \ \ \)\(.\)\)\), \(∂\^\(\(g\_1\) \((\(x\^*\))\)\)\/∂x\_p

                  \ \ \ \ \(\(.\)\(\ \)\(
\)\(\ \ \)\(.\)\)\), \(0\ \ \ \ \  ... 

                  \ \ \ \ \ \(\(.\)\(
\)\(\ \ \ \)\(.\)\)\), 0

                      .
  .}, {\(∂\^\(\(g\_m\) \((\(x\^*\))\)\)\/∂x\_1 ... \), \(∂\^\(\(g\_m\) \((\(x\^*\))\)\)\/∂x\_p\), \(0\ \ \ \  ... \), 0}})) > 0
\(For\ p\  = \ m + 1, \  ... \ , \ n, \ 
      then\ f\ has\ a\ strict\ local\ maximum\ at\ \ \(x\^*\), \ such\ that\) 
                                                     
                                       (Subscript[g, i])\((\(x\^*\))\)=0,   i  =  1, ..., \(\(m\)\(.\)\(
\)\(
\)\(
\)\(
\)\(
\)\(
\)\)

3. Mathematica Commands

In this section, we will introduce some Mathematica Commands that are useful to solve constrained optimization problems as outlined above.
(1) Caculating Derivatives

The function D and ' are used to differentiate functions. Assuming that y = f(x) is differemtiable,
          1. D[f(x) , x] computes and returns \(TraditionalForm\`f\^\(\(\ \)\('\)\)\ \((x)\)\  = \ \(\(\(d\ f\)\/\(d\ x\)\)\(.\)\)\)
          2.  D[f(x) , {x, n}] computes and returns (f^((n))) (x)  = (( (d^n) 
                f)/((d^n) x)).
          Mathematica knows the numerous differemtiation rules, including the product, quotient, and chain rules.

Clear[f,g,x]
Print["(d/dx) (f(x) g(
                x)) = "]
D[f[x]g[x], x]
Print["(d/dx) (f(
              g(x))) = "]
D[f[g[x]], x]
Print["(d/dx) (f(x)/g(
                x)) = "]
Simplify[D[f[x]/g[x], x]]

So, if we have a constrained optimization problem with associated Lagrangian: 
                                                     L=30  x +12  y - (x^2)+ xy - (y^2) -λ (10  x  + 4  y  -  260)
\(we\ can\ use\ two\ Mathematica\ commands\ to\ obtain\ the\ critical\ points . \ \ The\ first\ defines\ the\ Lagrangian\ function, \ 
      the\ second\ takes\ the\ derivatives\ of\ the\ Lagrangian\ function, \ 
      sets\ them\ equal\ to\ zero, \ 
      and\ then\ solves\ the\ three\ equations\ for\ x, \ y, \ 
      and\ λ . \ \ For\ \(\(example\)\(:\)\)\)

Clear[L,x,y,lambda];
L[x_,y_,lambda_]:= 30  x +12  y - (x^2)+ xy - (y^2) -lambda (10  x  + 4  y  -  260)
Solve[{D[L[x,y,lambda],x]==0,D[L[x,y,lambda],y]==0,
      D[L[x,y,lambda],lambda]==0},{x,y,lambda}]

(2) Define a Matrix

In Mathematica, a matrix is a list of lists where each list represent a row of the matrix. Therefore, the 3 × 3 matrix      
                                                                          
                        A=(({{(Subscript[a, 11]), (Subscript[a, 12]), (Subscript[a, 13])}, {(Subscript[a, 21]), (Subscript[a, 22]), (Subscript[a, 23])}, {(Subscript[a, 31]), (Subscript[a, 32]), (Subscript[a, 33])}}))
 is entered with

A={{a11,a12,a13},{a21,a22,a23},{a31,a32,a33}}

One can also enter a matrix using the Basic Input Palette and the  (({{□, □}, {□, □}})) symbol.  Once a matrix A has been entered, it can be viewed in the traditional row-and-column form using the command MatrixForm[A], or A // MatrixForm. (Mathematica is capital censitive.)

A={{a11,a12,a13},{a21,a22,a23},{a31,a32,a33}};
Print["Using command  'MatrixForm[A]'" ]
MatrixForm[A]
Print["Or Using command 'A // MatrixForm'"]
A // MatrixForm
B=(({{1, -2}, {3, 4}}));
Print["This is the matrix B printed using the list form", B ];
Print["This is the matrix B printed using the matrix form", B//MatrixForm ];

Using the previous example, we have objective function f(x,y) = 30  x +12  y - (x^2)+ xy - (y^2)  and constraint g(x, y) = 10x +4y - 260 = 0.  The Hessian matrix for this problem will be a constant matrix because the second derivatives of the Lagrangian function will not depend on x, y, or λ.

Clear[L, x, y, lambda];
f[x_, y_]:=30  x +12  y - (x^2)+ xy - (y^2);
g[x_, y_]  :=  
        10  x  +  4  y  - 260 ;
L[x_, y_, lambda_]:=f[x, y] -lambda*g[x, y];
Print["This is the objective \
function  ",  
        f[x, y]];
Print["This is the constraint \
",  g[x, 
          y]];
Print["This is the Lagrangian \
function  ",  L[x, y, lambda]];
h11 = 
        D[D[L[x, y, lambda], x], x];
h12 = 
        D[D[L[x, y, lambda], x], y];
g1 = 
        D[g[x, y], x];
h21 = 
        D[D[L[x, y, lambda], y], x];
h22 = 
        D[D[L[x, y, lambda], y], y];
g2 = 
        D[g[x, y], 
          y];
Print["This is the Bordered Hessian matrix for \
this problem:"]
HB=(({{h11, h12, g1}, {h21, h22, g2}, {g1, g2, 0}}));
HB  //  
      MatrixForm

(3) Extracting Elements of Matrices

For the 3 × 3 matrix A = {{a11,a12,a13},{a21,a22,a23},{a31,a32,a33}} defined earlier, A[[1]] yields the first element of matrix A which is the list {a11,a12,a13} or the first row of A; A[[2,1]] yields the first element of the second element of matrix A which is a21, and A[[{1,2},{1,2}]] yields a new matrix {{a11,a12},{a21,a22}}. 
For example, from the result of the previous example we have HB={{-2,0,10},{0,-2,4},{10,4,0}}. We can use this Mathematica command to get any submatrix of A.

HB={{-2,0,10},{0,-2,4},{10,4,0}};
Print["This is the matrix HB"];
MatrixForm[HB]
A1=HB[[1,1]];
A2=HB[[{1,2},{1,2}]];
Print["This is (1,1) element of HB "];
A1//MatrixForm
Print["This is 2x2 matrix in the upper left hand corner of HB "];
A2//MatrixForm

This is useful when we need to caculate the n minors in principle positions of the n × n Bordered Hessian matrix in order to verfiy if critical values repreent maximum or minimum or saddle points.

4. Method of Lagrangian

In this section, we will use some examples to illustrate how to use Mathematica to solve constrained optimization problems.

Example 1: Maximizing Output Subject to a Cost Constraint. Consider a production function given by f(x,y) =30x + 12y - (x^2)+ xy - (y^2). Let the price of x and y be 10 and 4 respectively with a cost constraint of $260. Then to maximize output with a cost of $260 given these prices we set up the following Lagrangian:
                      
               L= 30x + 12y - (x^2)+ xy - (y^2) -λ(10x +4y - 260)



Step 1.  First Derivative Test for Local Extreme Values:

Clear[f,g,L,x,y,lambda];
f[x_,y_]:=30*x +12*y - x^2+ x*y - y^2;
g[x_,y_]:=10*x + 4*y - 260;
L[x_,y_,lambda_]:=f[x,y] - lambda*g[x,y];
Solve[{D[L[x,y,lambda],x]==0,D[L[x,y,lambda],y]==0,
      D[L[x,y,lambda],lambda]==0},{x,y,lambda}]

Step 2. Construct the Hessian matrix

Clear[f, g, L, x, y, lambda];
f[x_, y_]:=30*x +12*y - x^2+x*y-y^2;
g[x_, y_] := 
        10*x + 4*y - 260;
L[x_, y_, lambda_] := 
        f[x, y] + lambda*g[x, y];
sol = 
        Solve[{D[L[x, y, lambda], x] == 0, 
            D[L[x, y, lambda], y] == 0, 
            D[L[x, y, lambda], lambda] == 0}, {x, y, 
            lambda}];
sol
h11 = 
        D[D[L[x, y, lambda], x], x];
h12 = 
        D[D[L[x, y, lambda], x], y];
g1 = 
        D[g[x, y], x];
h21 = 
        D[D[L[x, y, lambda], y], x];
h22 = 
        D[D[L[x, y, lambda], y], y];
g2 = 
        D[g[x, y], y];
xstar = 
        sol[[1, 1, 2]];
ystar = 
        sol[[1, 2, 2]];
λstar = 
        sol[[1, 3, 
            2]];
Print["At critical point x= \
", xstar, "  y= ", 
      ystar,  "  lambda= ", λstar]
Print["This is the Bordered Hessian matrix for this problem is:"]
HB=(({{h11, h12, g1}, {h21, h22, g2}, {g1, g2, 0}}))/.{x -> xstar,  y -> ystar, 
            lambda -> λstar};
HB // 
      MatrixForm
A1 = 
        HB[[1, 1]];
A2 = 
        HB[[{1, 2}, {1, 2}]];
A2 // 
        MatrixForm;
A3 = 
        HB[[{1, 2, 3}, {1, 2, 3}]];

Step 3. Determine the properties of the critical value.

Print["The determinant of the bordered Hessian is:"]
C1 = Det[HB]
If[C1==0,Print["The Lagrangian method is inconclusive in this case. Please try other method."]];
If[C1>0, Print["This is a relative maximum point! The maximum production is:"], Print["This is a relative minimum point!The minimum production is:"]]
f[xstar,ystar]

Example 2. Minimizing cost subject to output constraint. Consider a production function given by  f(x,y) = 20x - (x^2)+ 15y - (y^2). Let the price of x and y be 10 and 5 respecticely with an output constraint of 55. Then to minimize the cost of producing 55 units of output given these prices.  We set up the following Lagrangian:
                        
            L= 10X + 5y -λ(20X - (x^2)+ 15y - (y^2)-55)
                          
 
Step 1.  First Derivative Test for Local Extreme Values:

Clear[f,g,L,x,y,lambda];
f[x_,y_]:=10*x +5*y;
g[x_,y_]:= 20*x -x^2 + 15*y - y^2 - 55;
L[x_,y_,lambda_]:=f[x,y] + lambda*g[x,y];
Solve[{D[L[x,y,lambda],x]==0,D[L[x,y,lambda],y]==0,
      D[L[x,y,lambda],lambda]==0},{x,y,lambda}]

Notice that in this case we have two sets of roots to the equation.

Step 2. Construct the Hessian matrix

Clear[f,g,L,x,y,lambda,sol];
f[x_,y_]:=10*x +5*y;
g[x_,y_]:= 20*x -x^2 + 15*y - y^2 - 55;
L[x_,y_,lambda_]:=f[x,y]+lambda*g[x,y];
sol=Solve[{D[L[x,y,lambda],x]==0,D[L[x,y,lambda],y]==0,
      D[L[x,y,lambda],lambda]==0},{x,y,lambda}];
sol
h11=D[D[L[x,y,lambda],x],x];
h12=D[D[L[x,y,lambda],x],y];
g1=D[g[x,y],x];
h21=D[D[L[x,y,lambda],y],x];
h22=D[D[L[x,y,lambda],y],y];
g2=D[g[x,y],y];
h31=D[g[x,y],x];
h32=D[g[x,y],y];
HB={{h11,h12,g1},{h21,h22,g2},{h31,h32,0}};
A1=HB[[1,1]];
A2=HB[[{1,2},{1,2}]];
Print["This is the Bordered Hessian matrix for this problem:"]
HB // MatrixForm

Step 3. Determine the property of the critical value
Given that here are two roots, we will need to evaluate the second order or sufficient conditions at each root.

sd dsλ

Clear[f,g,L,x,y,lambda,sol];
f[x_,y_]:=10*x +5*y;
g[x_,y_]:= 20*x -x^2 + 15*y - y^2 - 55;
L[x_,y_,lambda_]:=f[x,y]+lambda*g[x,y];
sol=Solve[{D[L[x,y,lambda],x]==0,D[L[x,y,lambda],y]==0,
      D[L[x,y,lambda],lambda]==0},{x,y,lambda}];
lambdastar=sol[[1,1,2]];
xstar=sol[[1,2,2]];
ystar=sol[[1,3,2]];
Print["(1) At point x = ",  xstar  , ", y= ",  ystar  ,", lambda = ",   lambdastar ," the determinant of this Hessian matrix is "]
h11=D[D[L[x,y,lambda],x],x];
h12=D[D[L[x,y,lambda],x],y];
g1=D[g[x,y],x];
h21=D[D[L[x,y,lambda],y],x];
h22=D[D[L[x,y,lambda],y],y];
g2=D[g[x,y],y];
h31=D[g[x,y],x];
h32=D[g[x,y],y];
HB=(({{h11, h12, g1}, {h21, h22, g2}, {g1, g2, 0}}))/.{x -> xstar,  y -> ystar, 
              lambda -> lambdastar};
C1 = Det[HB]
If[C1==0,Print["The Lagrangian method is inconclusive in this case. Please try other methods."]];
If[C1>0, Print["This is a relative maximum point."]]
If[C1<0, Print["This is a relative minimum point."]]
Print["-----------------------------------------------------------------------------"]
lambdastar=sol[[2,1,2]];
xstar=sol[[2,2,2]];
ystar=sol[[2,3,2]];
Print["(2) At point x = ",  xstar  , ", y= ",  ystar  ,", lambda = ",   lambdastar ," the determinant of this Hessian matrix is "]
HB=(({{h11, h12, g1}, {h21, h22, g2}, {g1, g2, 0}}))/.{x -> xstar,  y -> ystar, 
              lambda -> lambdastar};
C1 = Det[HB]
If[C1==0,Print["The Lagrangian method is inconclusive in this case. Please try other methods."]];
If[C1>0, Print["This is a relative maximum point."]];
If[C1<0, Print["This is a relative minimum point."]];

5.Interpretation of the Lagrangian multiplier

The Lagrangian multiplier λ was treated as an additional variable in the Lagrangian function for solving the constrained optimization problem. No significant importance or interpretation was given to λ. However, one of the reason for the use of the Lagrangian multiplier method in economics is that λ often has an interesting interpretation, and its value often yields useful information.

Consider the problem of maximizing f(x,y) subject to a constraint g(x,y) = k. Previously we set k=0. For example, in the case of utility maximization, the budget constraint is written as g(x,y) =  ax + by - m = 0, where a and b are prices, and m is the income. Hence g(x,y) = ax + by -m = k for any arbitrary value k and indicates the change of  budget constraint, i.e., the income increase from m to m+k. It is intuitively clear that as the constraint varies, the critical point ((Subscript[x, 0]),  (Subscript[y, 0, ]) (Subscript[λ, 0])) will change accordingly. The critical points depend upon k. We are interested in the effect of k on ((Subscript[x, 0]),  (Subscript[y, 0, ]) (Subscript[λ, 0])) and on the maximum value of the objective function, f((Subscript[x, 0]),  (Subscript[y, 0])).
Consider the Lagrangian function of the problem L(x,y,λ) = f(x,y) + λ[g(x,y) - k].
A necessary condition for maximization is:
(Subscript[L, x]) = (Subscript[f, x]) + λ(Subscript[g, x]) = 0
(Subscript[L, y])= (Subscript[f, y]) + λ(Subscript[g, y]) = 0
(Subscript[L, λ])=  g(x,y) - k = 0
Solving these three equations for x, y, and λ, we obtain the critical point, ((Subscript[x, 0]),  (Subscript[y, 0, ]) (Subscript[λ, 0])), which is a function of k. Namely, we make this functional relationship explicit by writing:
(Subscript[x, 0])  =  (Subscript[x, 0])(k), (Subscript[y, 0]) = (Subscript[y, 0]) (k), (Subscript[λ, 0])=(Subscript[λ, 0])(k).
Substituting these values in the Lagrangian function, we have

              L((Subscript[x, 0]),  (Subscript[y, 0, ]) (Subscript[λ, 0])) = f((Subscript[x, 0]),  (Subscript[y, 0])) + (Subscript[λ, 0])[g((Subscript[x, 0]),  (Subscript[y, 0])) - k] =  f((Subscript[x, 0]),  (Subscript[y, 0]))

The last equation is due to the fact that g((Subscript[x, 0]),  (Subscript[y, 0])) - k = 0, since (Subscript[x, 0]) and  (Subscript[y, 0]) are both functions of k, the maximum value of the objective function must be a function of k. That is, Let

                  ϕ(k) = L((Subscript[x, 0])(k), (Subscript[y, 0]) (k),  (Subscript[λ, 0])(k))

The function ϕ(k) is called the indirective objective function. We are interested in the effect of the k on ϕ(k). To examine it, we take the total derivative of  L((Subscript[x, 0]),  (Subscript[y, 0, ]) (Subscript[λ, 0]))  with respect to k 

((d L((Subscript[x, 0])(k), (Subscript[y, 0]) (k),  (Subscript[λ, 0])(k)))/(d k)) = (Subscript[f, x]) ((d (Subscript[x, 0]))/(d k)) + (Subscript[f, y])((d (Subscript[y, 0]))/(d k)) + ((d (Subscript[λ, 0]))/(d k))[g((Subscript[x, 0]),  (Subscript[y, 0])) - k] +  (Subscript[λ, 0])(\(TraditionalForm\`\(\(g\_\(\(x\)\(\ \)\)\ \(d\ x\_0\)\/\(d\ k\)\)\(\ \ \)\(+\)\(\ \)\(g\_\(\(y\)\(\ \)\)\ \(d\ y\_0\)\/\(d\ k\)\)\(\ \)\(-\)\(\ \)\(1\)\(\ \)\)\)) 
                    = ((Subscript[f, x]) + (Subscript[λ, 0]) (Subscript[g, x ]) ) ((d (Subscript[x, 0]))/(d k)) + ((Subscript[f, y]) + (Subscript[λ, 0]) (Subscript[g, y ])) ((d (Subscript[y, 0]))/(d k)) + ((d (Subscript[λ, 0]))/(d k))[g((Subscript[x, 0]),  (Subscript[y, 0])) - k] - (Subscript[λ, 0])
                    = - (Subscript[λ, 0])
                                 
All derivatives are evaluated at  ((Subscript[x, 0]),  (Subscript[y, 0, ]) (Subscript[λ, 0])). Since L  ((Subscript[x, 0]),  (Subscript[y, 0, ]) (Subscript[λ, 0])) =  f((Subscript[x, 0]),  (Subscript[y, 0])), The derivative of   L  ((Subscript[x, 0]),  (Subscript[y, 0, ]) (Subscript[λ, 0])) with respect to k must be identical to the derivative of f((Subscript[x, 0]),  (Subscript[y, 0])) with respect to k. Thus,
 
                                ((d f((Subscript[x, 0]),  (Subscript[y, 0])))/(d k)) = - (Subscript[λ, 0])
 
If the objective function of the maximum value of the utility function with respect to the change in income, that is , - (Subscript[λ, 0]) is called the marginal utility of income. In other cases, if the objective function f(x,y) is the profit of a firm, and k is regarded as the amount of some resourses that are limited according to the constraints, then - (Subscript[λ, 0]) may be interpreted as dollars per unit of the resource. The firm would be willing to pay - (Subscript[λ, 0]) dollars for an increment in the resource. The Lagrangian multiplier is often called the Shadow Price or Opportunity Cost of the resource.

6.  Excercises

1. Find the critical points, if any, of the following functions:

(a) f(x)  =  (x^2)(e^x),  -2 < x <  2
(b) f(x) = -5x + 6 (x^2) - 3 , 
(c)  f(x, y) = (x^2) - 3xy +10 (y^2) - 2y + 1 
 (d) f(x, y) = (x + y)(e^(-x)), x, y > 0
 (e) f(x, y) = (x^2) + (y^2) , x, y > 0
 
 2. Use the direct substitute method to find the relative extreme (if any) for the following functions subject to their constraints:
 
 (a) f(x, y) = xy, subject to g(x, y) = (x^2) + (y^2)- 1 = 0.
 (b) f(x, y) = (x^2)+ 2(y^2),  subject to g(x, y) = x + 2y - 5 = 0.
 (c)  f(x, y, z)  = (x^2)+ 2(y^2) + 3(z^2),  subject to  g(x, y, z) = x + y + z - 5 = 0.
 
 3.Use method of Lagrangian to find the relative minimum points ( if any ) of the following functions:
 
 (a)  f(x, y) = (x^2)- 2xy + (y^2),  subject to xy = -1.
 (b)  f(x, y, z) = x + y + z - 5 sunject to x +  (y^2) + z = 4, and x + y + (z^2) = 1.
 (c)   f(x, y, z) = x + y + z subject to logx + logy  +  logz  =  3.