《高级微观经济学》动态最优化基础之一

Mathematical Methods: Dynamic Optimization Michaelmas Term 2003 Godfrey Keller, Department of Economics, University of Oxford Godfrey.Keller@economics.ox.ac.uk Reading: Barro & Sala-i-Martin (1995) Economic Growth, McGraw-Hill. (Chapters 1, 2 and Appendix 1.2 on Mathematical Methods.) Chiang (1992) Elements of Dynamic Optimization, McGraw-Hill. (Part 3.) Dixit (1990)Optimization in Economic Theory (2E), Oxford University Press. (Chap- ters 10 and 11.) 1 Preliminaries 1.1 Relationship between discrete- and continuous-time growth In dynamic optimization we have a choice between modelling time t discretely (t = 0, 1, 2, . . .) or continuously (where t takes any real value). In economic terms one hopes this decision makes little difference, and we usually use the method that is most con- venient for getting solutions. Often we decide to use continuous time, mainly because differential equations are usually easier to solve than difference equations. To illustrate the relationship between the two frameworks, consider the case of interest rates. Suppose a bank pays the interest rate r annually, i.e. if you put £1 in the bank now, you have £(1+ r) in the bank a year later. Therefore, in t years you have £(1+ r)t in the bank. Next, suppose the bank keeps the same annual rate but pays interest quarterly, so that after three months you are paid interest 1 4 r on your money in the bank over the period. Therefore, after a year you have( 1 + r 4 )4 in the bank. After t years you have ( 1 + r 4 )4t in the bank. More generally, if the banks pays interest n times a year, after t years you have ( 1 + r n )nt in the bank. If the bank pays interest continuously, so that n becomes large, then after t years you have lim n→∞ ( 1 + r n )nt = ert. in the bank. Therefore, growth in continuous-time models is the natural limit of growth in discrete-time models where the time interval shrinks to zero. 1 1.2 The advantage of geometric discounting We almost invariably model an agent’s inter-temporal utility using a constant discount factor δ with 0 < δ ≤ 1, so that from some point in time T , say, her utility derived from the consumption stream xT , xT+1, . . . is given by u(xT ) + δu(xT+1) + δ 2u(xT+2) + . . . = ∞∑ t=0 δtu(xT+t). However, this geometric form of discounting is often violated in empirical studies and experiments, where an increasing discount factor is often a better fit. � ??? A simple way to model an increasing discount factor is to suppose that the above utility is modified by introducing an extra parameter α, so that utility is given by u(xT ) + α [ δu(xT+1) + δ 2u(xT+2) + . . . ] = u(xT ) + α ∞∑ t=1 δtu(xT+t). If α = 1 then we have geometric discounting, but if α < 1 then we have an increasing discount factor. (The agent is more willing to substitute consumption from period T +1 to T+2 than she is from period T to T+1.) The major problem with such non-geometric discounting is that the agent might decide on some consumption plan today, but when she gets to tomorrow she will want to change her plan from then on, i.e. there is a ‘time-inconsistency’ problem. To see this, consider the following simple example. An agent lives for three periods, and she has some total quantity s to consume, so that x1+x2+x3 = s. (This is a special case of the ‘cake-eating’ problem discussed below.) At the start of time her utility is given by 2x 1/2 1 + [ x 1/2 2 + x 1/2 3 ] whereas from the next period her utility is 2x 1/2 2 + [ x 1/2 3 ] . It is easy to show that, at time T = 1, the agent would like to consume according to the plan x1 = 2 3 s ; x2 = x3 = 1 6 s . However, at time T = 2, when the agent has 1 3 s left according to this initial plan, the agent would like to change to the plan x2 = 4 15 s ; x3 = 1 15 s . If the agent knows that she will change her mind like this in period 2, then she will alter her initial plan. (In fact she foresees that she will choose x2 = 4x3 in period 2 and so will choose to consume x1 = 20 29 s in the first period.) This kind of time-consistency problem makes life very difficult when solving dynamic optimization problems, so we stick to the geometric approach in the following (so that agents have no desire to change their minds over time).1 1If you are interested in non-geometric discounting and wish to read more, see the various papers by David Laibson on his webpage http://post.economics.harvard.edu/faculty/laibson/papers.html 2 2 Examples of dynamic optimization problems 2.1 The ‘cake-eating’ problem A person has a cake (which never goes stale) of initial size s and wishes to consume it in such a way as to maximize her total utility over time. In a discrete-time framework, in each period t = 0, 1, 2, . . . she obtains utility u(xt) if she eats quantity xt in that period. However, she discounts future utility using the factor δ < 1 per period, i.e. if she consumes quantity x next period that is worth δu(x) to her now, and if she consumes it t periods from now it is worth δtu(x) to her now. Therefore, if she consumes quantities x0, x1, . . . her total utility is U(s) = ∞∑ t=0 δtu(xt) (1) subject to the resource constraint ∞∑ t=0 xt ≤ s. Her (fairly easy) problem is to find a sequence {x∗t}∞t=0 that maximizes U(s) subject to the constraint. In a continuous-time framework, u(x) is the consumer’s instantaneous utility if she consumes at rate x, i.e. if she consumes at rate x for some short interval of time [0, dt) then by the end of the interval she will have consumed x dt and got utility u(x) dt from this. (Thus u(x) is like a density function in probability.) It is conventional to suppose that if she consumes at rate x for some short time dt at some point t in the future that is worth exp(−ρt)u(x) dt to her now. Here, ρ is the discount rate, with ρ ≥ 0. (Take care not to confuse this with the discount factor δ : a person who values the future highly has a discount factor δ close to 1, but a discount rate ρ close to 0; someone who is very impatient has a discount factor δ close to 0, but a very high discount rate ρ.) Therefore, her total utility if she consumes according to the time-path x(t) is U(s) = ∫ ∞ 0 e−ρtu(x(t)) dt (2) subject to the resource constraint ∫ ∞ 0 x(t) dt ≤ s. Although it is not necessary for this problem, for more complex problems it is useful to re-write the resource constraint. In the discrete-time problem, let st be the amount of cake remaining at time t, so that s0 = s and st+1 = st − xt. If we write ∆st = st+1 − st then the problem becomes that of maximizing (1) subject to ∆st = −xt s0 = s st ≥ 0 for t = 1, 2, . . . . 3 In the continuous-time problem, if we write s(t) for the stock of cake left at time t then ds(t)/dt = −x(t) and the problem is to maximize (2) subject to s˙(t) = −x(t) s(0) = s s(t) ≥ 0 for t ≥ 0, and the solution is the optimal time-path x∗(t), t ≥ 0. A serious economic application of this problem is that of mineral/oil extraction, where a firm has rights to a stock s of some mineral. In each period, if it extracts and sells a quantity x it makes a profit pi(x); it discounts future profits by some factor δ and wishes to maximize ∑∞ t=0 δ tpi(xt) subject to the usual constraints. 2.2 Optimal consumption with growth Consider a similar problem except that the stock grows at some rate r if left untouched. (For instance, s could be wealth which yields interest at the rate r, or s could be the stock of fish in a lake which grows at the rate r.) In the discrete-time problem, if st is the stock at the beginning of period t, this stock is worth (1 + r)st at the end of that period, and if xt is then consumed then the stock at the beginning of period t + 1 is st+1 = (1+ r)st−xt, i.e. ∆st = rst−xt. The problem is then to maximize (1) subject to ∆st = rst − xt s0 = s st ≥ 0 for t = 1, 2, . . . . In the continuous case we have s˙(t) = rs(t)−x(t) and so we wish to maximize (2) subject to s˙(t) = rs(t)− x(t) s(0) = s s(t) ≥ 0 for t ≥ 0. Note that for a given discount factor or rate, if a solution is to exist at all then we cannot have r too large or else the consumer postpones consumption indefinitely. 2.3 A simple search model Suppose a worker is currently unemployed and in each period of unemployment is offered a single job with a wage w per period, drawn from some distribution F (·), i.e. the probability that he is offered a job paying less than w is F (w). (This distribution is unchanged over time and the draws are uncorrelated.) If he accepts the job he keeps it for life with the same wage each period (and cannot search for another job whilst in work). If he does not accept the job he receives unemployment compensation of a known amount c and in the next period receives another job offer, and the process begins again. He aims to maximize total discounted income (using a discount factor δ). The problem is: which wage offers should he accept and which should he reject? The solution is a rule that tells him what to do for each possible wage offer w. 4 2.4 The Ramsey growth model A population grows at the exogenous rate n, has an initial population L0 and an initial capital stock K0. Therefore, the population at time t is L(t) = L0e nt. There is an instantaneous concave production function F (L,K) which exhibits constant returns to scale, i.e. it is homogeneous of degree 1 in (L,K). At any point in time let s = K/L be the capital-labour ratio, and let f(s) = F (1, s); the initial capital-labour ratio is s0 = K0/L0. Output is then just Lf(s), and output per worker is f(s). Since F is concave, so is f . Capital, if left to itself, depreciates at the rate γ. If there is gross investment I(t) at time t, then the capital stock K(t) satisfies K˙(t) = I(t)− γK(t). (3) Alternatively, since s˙(t) = d dt ( K(t) L(t) ) = LK˙ − L˙K L2 = K˙(t) L(t) − ns(t), equation (3) implies that s˙(t) = i(t)− (n+ γ)s(t) where i = I/L is investment per worker. As F (L(t), K(t)) is output at time t, and X(t) is total consumption, then investment is I(t) = F (L(t), K(t))−X(t) and so i(t) = f(s(t))− x(t) where x = X/L is consumption per worker. Putting this together implies that the relationship between capital growth and consumption is s˙(t) = f(s(t))− x(t)− (n+ γ)s(t) (4) where everything is expressed in per-worker terms. The instantaneous utility of a representative worker consuming x is u(x), and workers discount the future at the rate ρ. The aim of the economy is assumed to be to maximize the representative worker’s total discounted utility∫ ∞ 0 e−ρtu(x(t)) dt subject to s(0) = s0 and the relationship (4). Again, the solution is the optimal time- path x∗(t), t ≥ 0, but in general, unlike the previous examples, it is hard to obtain closed-form solutions to this problem. Note that if, for some reason, workers have a constant savings rate β < 1, then x(t) = (1− β)f(s(t)) and (4) just becomes s˙(t) = βf(s(t))− (n+ γ)s(t). However, this is no longer a dynamic optimization problem. (In fact, it is the ‘Solow- Swan’ growth model.) Reading: Barro & Sala-i-Martin chapter 1 for the Solow-Swan model and chapter 2 for the full Ramsey optimization problem. 5 2.5 Human capital accumulation In each period t = 0, 1, . . . , T an individual allocates a fraction 0 ≤ xt ≤ 1 of her available time to work and earns st xt where st is her level of skill or human capital. The constant interest rate is r > 0 and so the worker’s discount factor is δ = 1/(1+r). The worker’s objective is to find a sequence {x∗t}Tt=0 that maximizes her total discounted life-time earnings, ∑T t=0 δ t st xt. Human capital depletes at the rate 0 < β < 1 if all time is devoted to work, but accumulates at the rate λ > 0 if all time is devoted to improving skills. More specifically, assume that st+1 = ψ(xt) st where ψ : [0, 1]→ [1−β, 1+λ] is strictly decreasing, strictly concave and continuously differentiable with ψ(0) = 1 + λ and ψ(1) = 1− β. To solve this problem, we proceed along the following lines. If we let vt(s) denote the value of having skill level s with t periods remaining then it can be shown (by induction) that vt(s) = at s for some sequence of constants with at+1 > at. Moreover, pure depletion is optimal in the later periods of her working life, and, under certain conditions on (r, λ) and ψ′, pure accumulation is optimal in the early periods of her working life if T is sufficiently large. Note that when pure depletion or pure accumulation is optimal, the choice is not char- acterized by the FOC – we have a corner solution. 2.6 Learning-by-doing A monopolist receives revenue R(x) in any instant of time from selling a quantity x to consumers in that instant. At an instant in time it has a constant marginal cost of production c (i.e. independent of x); however this marginal cost falls as the firm produces more output over time. Specifically, if s(t) is the total quantity produced by the firm by time t, so that s(t) = ∫ t 0 x(t) dt , then the firm’s marginal cost at time t is c(s(t)), where c′(s) ≤ 0. Let c(0) = c0 be the firm’s initial cost level. The firm discounts future profits at the rate ρ and wishes to choose the output time-path x(t) so as to maximize total discounted profits:∫ ∞ 0 e−ρt [R(x(t))− c(s(t))x(t)] dt subject to s˙(t) = x(t) s(0) = 0. 6 3 The general problem As is clear from the preceding examples, many dynamic problems take the following form: maximize ∫ T 0 e−ρtu(x(t), s(t)) dt (5) subject to s˙(t) = φ(x(t), s(t)) (6) and s(0) = A; (7) in addition there may be a constraint s(T ) = B. (8) Here, T is the ‘terminal point’ (which might be infinite if the problem has an infinite horizon). The variable x(t), which is what the problem-solver chooses, is the ‘control variable’, and s(t) is the ‘state variable’. Equation (6) is the ‘equation of motion’ (or the ‘state transition’), condition (7) is the ‘initial condition’ and, where it is imposed, condition (8) is the ‘terminal condition’. If no terminal condition is imposed the problem has a ‘free endpoint’, whereas if it is imposed the problem has a ‘fixed endpoint’. Finally, there may be additional constraints, such as s(t), x(t) ≥ 0 which are easily allowed for. For discrete-time settings, the problem is to maximize T∑ t=0 δtu(xt, st) (9) subject to ∆st = φ(xt, st) and s0 = A; in addition there may be a constraint sT = B. The crucial feature of these kinds of problem is that the objective function is additively separable across time; for instance, in (9) the objective function does not take the general form u(x1, x2, . . . , s1, s2, . . .). This feature enables us to use either optimal control theory or dynamic programming to solve the problem. We next discuss the optimal control theory and dynamic programming methods of solving such problems. The former provides the most useful way to find explicit solutions for some of the simple problems discussed here. However, dynamic programming is in some ways more elegant and often gives useful economic results easily (even though the full solution is often not obtainable). In addition, it is much the better method when there is uncertainty in the environment (as in the Job Search example). 7