Convexity and concavity for functions of many variables

Any stationary point of a concave function of a single variable is a global maximizer; any stationary point of a convex function of a single variable is a global minimizer (see an earlier page). Convexity and concavity are key also in the case of optima of functions of many variables.

Convex sets

To extend the notions of concavity and convexity to functions of many variables we first need to define the notion of a convex set.

Definition
A set S of n-vectors is convex if

(1-l)x + lx¢ Î S whenever x Î S, x¢ Î S, and l Î [0,1].

We call (1 - l)x + lx¢ a convex combination of x and x¢.

For example, the two-dimensional set on the left of the following figure is convex, since the line segment joining every pair of points in the set lies entirely in the set. The set on the right is not convex, because the line segment joining the points x and x¢ does not lie entirely in the set.

The following property of convex sets is sometimes useful.

Proposition
The intersection of convex sets is convex.

Note that the union of convex sets is not necessarily convex.

Exercises

Convex and concave functions

Let f be a function of many variables, defined on a convex set S. We say that f is concave if the line segment joining any two points on the graph of f is never above the graph; f is convex if the line segment joining any two points on the graph is never below the graph. (That is the definitions are the same as the definitions for functions of a single variable.)

More precisely, we can make the following definition (which is again the same as the corresponding definition for a function of a single variable).

Definition
A function f of many variables defined on the convex set S is

concave if for all x Î S, all x¢ Î S, and all l Î (0,1) we have
f ((1-l)x + lx¢) ³ (1-l) f (x) + l f (x¢)

convex if for all x Î S, all x¢ Î S, and all l Î (0,1) we have
f ((1-l)x + lx¢) £ (1-l) f (x) + l f (x¢).

Once again, a strictly concave function is one that satisfies the definition for concavity with a strict inequality (> rather than ³) for all x ¹ x¢, and a strictly convex function is one that satisfies the definition for concavity with a strict inequality (< rather than £) for all x ¹ x¢.

Example
Let f be a linear function, defined by f (x) = a₁x₁ + ... + a_nx_n = a·x, where a_i is a constant for each i. Then f is both concave and convex:

f ((1 - l)x + lx¢) = a·[(1-l)x + lx¢]

= (1-l)a·x + la·x¢

= (1-l) f (x) + l f (x¢)

Example
Suppose the function g of a single variable is concave on [a,b], and the function f of two variables is defined by f (x,y) = g(x) on [a, b] ´ [c, d]. Is f concave?
First note that the domain of f is a convex set, so the definition of concavity can apply.
Now,

f ((1-l)(x, y) + l(x¢, y¢)) = f ((1-l)x + lx¢, (1-l)y + ly¢)

= g((1-l)x + lx¢)

³ (1-l)g(x) + lg(x¢)

= (1-l) f (x, y) + l f (x¢, y¢)

so f is concave.

Example
Let f and g be defined as in the previous example. Assume now that g is strictly concave. Is f strictly concave?
The strict concavity of f implies that

f ((1-l)(x, y) + l(x¢, y¢)) > (1-l) f (x, y) + l f (x¢, y¢)

for all x ¹ x¢. But to show that f is strictly concave we need to show that the inequality is strict whenever (x, y) ¹ (x¢, y¢)---in particular, for cases in which x = x¢ and y ¹ y¢. In such a case, we have

f ((1-l)(x, y) + l(x¢, y¢)) = f (x, (1-l)y + ly¢)

= g(x)

= (1-l) f (x, y) + l f (x, y¢).

Thus f is not strictly concave.

Another definition of convex and concave functions

An definition of a concave function equivalent to the one given previously is sometimes useful.

Definition
A function f of many variables defined on the convex set S is

concave if the set of points below its graph is convex:

{(x, y): x Î S and y £ f (x)} is convex

convex if the set of points above its graph is convex:

{(x, y): x Î S and y ³ f (x)} is convex.

How can we tell if a function is concave or convex?

A twice-differentiable function of a single variable is concave if and only if its second derivative is nonpositive everywhere. (Indeed, that is how we originally defined a concave function of a single variable.) Plausibly, to determine whether a twice-differentiable function of many variables is concave or convex, we need to examine all its second partial derivatives. Its turns out that what we need to look at is the Hessian of the function.

Proposition
Let f be a function of many variables with continuous partial derivatives of first and second order on the convex open set S and denote the Hessian of f at the point x by H(x). Then

f is concave if and only if H(x) is negative semidefinite for all x Î S
if H(x) is negative definite for all x Î S then f is strictly concave
f is convex if and only if H(x) is positive semidefinite for all x Î S.
if H(x) is positive definite for all x Î S then f is strictly convex.

Note that the result does not claim that if f is strictly concave then H(x) is negative definite for all x Î S. Indeed, consider the function f (x,y) = -x⁴ - y⁴. This function is strictly concave, but H(0, 0) is not negative definite.

Example
Consider the function f (x, y) = 2x - y - x² + 2xy - y². Its Hessian is

-2 2

2 -2

which is negative semidefinite. (In this case the Hessian does not depend on (x, y); in general it does.) Thus f is concave.

Example
Consider the function f (x₁, x₂, x₃) = x₁² + 2x₂² + 3x₃² + 2x₁x₂ + 2x₁x₃. Its first partials are

f ¢₁(x₁, x₂, x₃) = 2x₁ + 2x₂ + 2x₃

f ¢₂(x₁, x₂, x₃) = 4x₂ + 2x₁

f ¢₃(x₁, x₂, x₃) = 6x₃ + 2x₁.

So its Hessian is

  f ¢¢₁₁ f ¢¢₁₂ f ¢¢₁₃

  f ¢¢₂₁ f ¢¢₂₂ f ¢¢₂₃

  f ¢¢₃₁ f ¢¢₃₂   f ¢¢₃₃

=

2 2 2

2 4 0

2 0 6

.

(Note that is this example the Hessian is independent of (x₁, x₂, x₃); in general, this is not the case.) The leading principal minors of the Hessian are 2 > 0, 4 > 0, and 8 > 0. So the Hessian is positive definite, and f is strictly convex.

Example
Consider the Cobb-Douglas production function, defined by f (K, L) = AK^aL^b. The Hessian of this function is

a(a-1)AK^a-2L^b abAK^a-1L^b-1

abAK^a-1L^b-1 b(b-1)AK^aL^b-2

Thus in order that f be concave we need a(a-1)AK^a-2L^b £ 0, b(b-1)AK^aL^b-2 £ 0, and abA²K^2a-2L^2b-2(1 - (a + b)) ³ 0. Thus f is concave if A ³ 0, a ³ 0, b ³ 0, and a + b £ 1, and is strictly concave if A > 0, a > 0, b > 0, and a + b < 1.

The following result is sometimes useful when we wish to determine if a function is concave or convex.

Proposition

If the functions f and g are concave and a ³ 0 and b ³ 0 then the function a f + bg is concave.
If the functions f and g are convex and a ³ 0 and b ³ 0 then the function a f + bg is convex.
If the function f is concave and the function F is concave and increasing then the function U defined by U(x) = F ( f (x)) is concave.
If the function f is convex and the function F is convex and increasing then the function U defined by U(x) = F ( f (x)) is convex.

Exercises