Home > EE364a Homework 5 solutions

# EE364a Homework 5 solutions

 Page 1
EE364a, Winter 2007-08 Prof. S. Boyd
EE364a Homework 5 solutions
4.15 Relaxation of Boolean LP. In a Boolean linear program, the variable x is constrained to have components equal to zero or one: minimize cT x subject to Ax ≼ b xi �� {0,1}, i = 1,...,n. (1) In general, such problems are very difficult to solve, even though the feasible set is finite (containing at most 2n points). In a general method called relaxation, the constraint that xi be zero or one is replaced with the linear inequalities 0 �� xi �� 1: minimize cT x subject to Ax ≼ b 0 �� xi �� 1, i = 1,...,n. (2) We refer to this problem as the LP relaxation of the Boolean LP (4.67). The LP relaxation is far easier to solve than the original Boolean LP. (a) Show that the optimal value of the LP relaxation (4.68) is a lower bound on the optimal value of the Boolean LP (4.67). What can you say about the Boolean LP if the LP relaxation is infeasible? (b) It sometimes happens that the LP relaxation has a solution with xi �� {0,1}. What can you say in this case? Solution. (a) The feasible set of the relaxation includes the feasible set of the Boolean LP. It follows that the Boolean LP is infeasible if the relaxation is infeasible, and that the optimal value of the relaxation is less than or equal to the optimal value of the Boolean LP. (b) The optimal solution of the relaxation is also optimal for the Boolean LP. 4.60 Log-optimal investment strategy. We consider a portfolio problem with n assets held over N periods. At the beginning of each period, we re-invest our total wealth, redis- tributing it over the n assets using a fixed, constant, allocation strategy x �� Rn, where x ≽ 0, 1T x = 1. In other words, if W(t − 1) is our wealth at the beginning of period t, then during period t we invest xiW(t − 1) in asset i. We denote by ��(t) the total 1

 Page 2
return during period t, i.e., ��(t) = W(t)/W(t − 1). At the end of the N periods our wealth has been multiplied by the factor ��N
t=1 ��(t). We call
1 N
N
��
t=1
log ��(t) the growth rate of the investment over the N periods. We are interested in determining an allocation strategy x that maximizes growth of our total wealth for large N. We use a discrete stochastic model to account for the uncertainty in the returns. We assume that during each period there are m possible scenarios, with probabilities ��j, j = 1,...,m. In scenario j, the return for asset i over one period is given by pij. Therefore, the return ��(t) of our portfolio during period t is a random variable, with m possible values pT
1 x,...,pT mx, and distribution
��j = prob(��(t) = pT
j x), j = 1,...,m.
We assume the same scenarios for each period, with (identical) independent distribu- tions. Using the law of large numbers, we have lim
N����
1 N log (W(N) W(0) ) = lim
N����
1 N
N
��
t=1
log ��(t) = Elog ��(t) =
m
��
j=1
��j log(pT
j x).
In other words, with investment strategy x, the long term growth rate is given by Rlt =
m
��
j=1
��j log(pT
j x).
The investment strategy x that maximizes this quantity is called the log-optimal in- vestment strategy, and can be found by solving the optimization problem maximize ��m
j=1 ��j log(pT j x)
subject to x ≽ 0, 1T x = 1, with variable x �� Rn. Show that this is a convex optimization problem. Solution. Actually, there��s not much to do in this problem. The constraints, x ≽ 0, 1T x = 1, are clearly convex, so we just need to show that the objective is concave (since it is to be maximized). We can do that in just a few steps: First, note that log is concave, so log(pT
j x) is concave in x (on the domain, which is the open halfspace
{x | pT
j x > 0}). Since ��j �� 0, we conclude that the sum of concave functions m
��
j=1
��j log(pT
j x)
is concave. 2

 Page 3
5.13 Lagrangian relaxation of Boolean LP. A Boolean linear program is an optimization problem of the form minimize cT x subject to Ax ≼ b xi �� {0,1}, i = 1,...,n, and is, in general, very difficult to solve. In exercise 4.15 we studied the LP relaxation of this problem, minimize cT x subject to Ax ≼ b 0 �� xi �� 1, i = 1,...,n, (3) which is far easier to solve, and gives a lower bound on the optimal value of the Boolean LP. In this problem we derive another lower bound for the Boolean LP, and work out the relation between the two lower bounds. (a) Lagrangian relaxation. The Boolean LP can be reformulated as the problem minimize cT x subject to Ax ≼ b xi(1 − xi)=0, i = 1,...,n, which has quadratic equality constraints. Find the Lagrange dual of this problem. The optimal value of the dual problem (which is convex) gives a lower bound on the optimal value of the Boolean LP. This method of finding a lower bound on the optimal value is called Lagrangian relaxation. (b) Show that the lower bound obtained via Lagrangian relaxation, and via the LP re- laxation (5.107), are the same. Hint. Derive the dual of the LP relaxation (5.107). Solution. (a) The Lagrangian is L(x, µ, ��) = cT x + µT (Ax − b) − ��T x + xT diag(��)x = xT diag(��)x + (c + AT µ − ��)T x − bT µ. Minimizing over x gives the dual function g(µ, ��) = { −bT µ − (1/4)��n
i=1(ci + aT i µ − ��i)2/��i
�� ≽ 0 −�� otherwise where ai is the ith column of A, and we adopt the convention that a2/0 = �� if a = 0, and a2/0=0if a = 0. 3

 Page 4
The resulting dual problem is maximize −bT µ − (1/4)��n
i=1(ci + aT i µ − ��i)2/��i
subject to �� ≽ 0, µ ≽ 0. In order to simplify this dual, we optimize analytically over ��, by noting that sup
��i��0 (−
(ci + aT
i µ − ��i)2
��i
) = { 4(ci + aT
i µ) ci + aT i µ �� 0
0 ci + aT
i µ �� 0
= min{0,4(ci + aT
i µ)}.
This allows us to eliminate �� from the dual problem, and simplify it as maximize −bT µ + ��n
i=1 min{0,ci + aT i µ}
subject to µ ≽ 0. (b) We follow the hint. The Lagrangian and dual function of the LP relaxation are L(x, u, v, w) = cT x + uT (Ax − b) − vT x + wT (x − 1) = (c + AT u − v + w)T x − bT u − 1T w g(u, v, w) = { −bT u − 1T w AT u − v + w + c = 0 −�� otherwise. The dual problem is maximize −bT u − 1T w subject to AT u − v + w + c = 0 u ≽ 0,v ≽ 0,w ≽ 0, which is equivalent to the Lagrange relaxation problem derived above. We con- clude that the two relaxations give the same value. 6.2 ℓ1-, ℓ2-, and ℓ��-norm approximation by a constant vector. What is the solution of the norm approximation problem with one scalar variable x �� R, minimize x1 − b, for the ℓ1-, ℓ2-, and ℓ��-norms? Solution. (a) ℓ2-norm: the average 1T b/m. (b) ℓ1-norm: the (or a) median of the coefficients of b. (c) ℓ��-norm: the midrange point (maxbi − minbi)/2. 4

 Page 5
1. Schur complements. Consider a matrix X = XT �� Rn��n partitioned as X = [ A B BT C ] , where A �� Rk��k. If detA = 0, the matrix S = C − BT A−1B is called the Schur complement of A in X. Schur complements arise in many situations and appear in many important formulas and theorems. For example, we have detX = detAdetS. (You don��t have to prove this.) (a) The Schur complement arises when you minimize a quadratic form over some of the variables. Let f(u, v)=[uT vT ]X[uT vT ]T , where u �� Rk. Let g(v) be the minimum value of f over u, i.e., g(v) = infu f(u, v). Of course g(v) can be −��. Show that if A ≻ 0, we have g(v) = vT Sv. (b) The Schur complement arises in several characterizations of positive definiteness or semidefiniteness of a block matrix. As examples we have the following three theorems: • X ≻ 0 if and only if A ≻ 0 and S ≻ 0. • If A ≻ 0, then X ≽ 0 if and only if S ≽ 0. • X ≽ 0 if and only if A ≽ 0, BT (I − AA†) = 0 and C − BT A†B ≽ 0, where A† is the pseudo-inverse of A. (C − BT A†B serves as a generalization of the Schur complement in the case where A is positive semidefinite but singular.) Prove one of these theorems. (You can choose which one.) (c) Recognizing Schur complements often helps to represent nonlinear convex con- straints as linear matrix inequalities (LMIs). Consider the function f(x)=(Ax + b)T (P0 + x1P1 + ··· + xnPn)−1(Ax + b) where A �� Rm��n, b �� Rm, and Pi = PT
i �� Rm��m, with domain
domf = {x �� Rn | P0 + x1P1 + ··· + xnPn ≻ 0}. This is the composition of the matrix fractional function and an affine mapping, and so is convex. Give an LMI representation of epif. That is, find a symmetric matrix F(x, t), affine in (x, t), for which x �� domf, f(x) �� t ⇐⇒ F(x, t) ≽ 0. Solution 5

 Page 6
(a) If A ≻ 0, then g(v) = vT Sv. We have f(u, v) = uT Au+2vT Bu+vT Cv. If A ≻ 0, we can minimize f over u by setting the gradient with respect to u equal to zero. We obtain u⋆(v) = −A−1Bv, and g(v) = f(u⋆(v),v) = vT (C − BT A−1B)v = vT Sv. (b) Positive definite and semidefinite block matrices. • X ≻ 0 if and only if A ≻ 0 and S ≻ 0. Suppose X ≻ 0. Then f(u, v) > 0 for all non-zero (u, v), and in particular, f(u,0) = uT Au > 0 for all non-zero u (hence, A ≻ 0), and f(−A−1Bv,v) = vT (C −BT A−1B)v > 0 (hence, S = C −BT A−1B ≻ 0). This proves the ��only if�� part. To prove the ��if�� part, we have to show that if A ≻ 0 and S ≻ 0, then f(u, v) > 0 for all nonzero (u, v) (that is, for all u, v that are not both zero). If v = 0, then it follows from (a) that f(u, v) �� inf
u
f(u, v) = vT Sv > 0. If v = 0 and u = 0, f(u,0) = uT Au > 0. • If A ≻ 0, then X ≽ 0 if and only if S ≽ 0. From part (a) we know that if A ≻ 0, then infu f(u, v) = vT Sv. If S ≽ 0, then f(u, v) �� inf
u
f(u, v) = vT Sv �� 0 for all u, v, and hence X ≽ 0. This proves the ��if��-part. To prove the ��only if��-part we note that f(u, v) �� 0 for all (u, v) implies that infu f(u, v) �� 0 for all v, i.e., S ≽ 0. • X ≽ 0 if and only if A ≽ 0, BT (I − AA†)=0, C − BT A†B ≽ 0. Suppose A �� Rk��k with rank(A) = r. Then there exist matrices Q1 �� Rk��r, Q2 �� Rk��(k−r) and an invertible diagonal matrix �� �� Rr��r such that A = [ Q1 Q2 ]
[ �� 0
0 0 ]
[ Q1
Q2 ]T , and [Q1 Q2]T [Q1 Q2] = I. The matrix
[ Q1
Q2 0 0 0 I ] �� Rn��n is nonsingular, and therefore
[ A B
BT C ] ≽ 0 ⇐⇒ [ Q1 Q2 0 0 0 I ]T [ A B BT C ] [ Q1 Q2 0 0 0 I ] ≽ 0 6

 Page 7
⇐⇒ 
 
�� 0 QT
1 B
0 0 QT
2 B
BT Q1 BT Q2 C
   ≽ 0
⇐⇒ QT
2 B = 0, [
�� QT
1 B
BT Q1 C ] ≽ 0. We have �� ≻ 0 if and only if A ≽ 0. It can be verified that A† = Q1��−1QT
1 , I − AA† = Q2QT 2 .
Therefore QT
2 B = 0 ⇐⇒ Q2QT 2 B = (I − A†A)B = 0.
Moreover, since �� is invertible,
[
�� QT
1 B
BT Q1 C ] ≽ 0 ⇐⇒ �� ≻ 0, C − BT Q1��−1QT
1 B = C − BT A†B ≽ 0.
(c) The epigraph of f is the set of points (x, t) that satisfy P0 +x1P1 +···+xnPn ≻ 0 and (Ax + b)T (P0 + x1P1 + ··· + xnPn)−1(Ax + b) �� t. Using the second result of part (b), we can write the second inequality as
[
t (Ax + b)T (Ax + b) P0 + x1P1 + ··· + xnPn ] ≽ 0. This a linear matrix inequality in the variables x, t, i.e., a convex constraint. 2. Formulate the following optimization problems as semidefinite programs. The variable is x �� Rn; F(x) is defined as F(x) = F0 + x1F1 + x2F2 + ··· + xnFn with Fi �� Sm. The domain of f in each subproblem is domf = {x �� Rn | F(x) ≻ 0}. (a) Minimize f(x) = cT F(x)−1c where c �� Rm. (b) Minimize f(x) = maxi=1,...,K cT
i F(x)−1ci where ci �� Rm, i = 1,...,K.
(c) Minimize f(x) = sup
c2��1
cT F(x)−1c. (d) Minimize f(x) = E(cT F(x)−1c) where c is a random vector with mean E c = ¯c and covariance E(c − ¯c)(c − ¯c)T = S. Solution. 7

 Page 8
(a) minimize t subject to [ F(x) c cT t ] ≽ 0. (b) minimize t subject to [ F(x) ci cT
i
t ] ≽ 0, i = 1,...,K. (c) f(x) = ��max(F(x)−1), so f(x) �� t if and only if F(x)−1 ≼ tI. Using a Schur complement we get minimize t subject to [ F(x) I I tI ] ≽ 0. (d) f(x)=¯cT F(x)−1¯c+ tr(F(x)−1S). If we factor S as S = ��m
k=1 ckcT k the problem
is equivalent to minimize ¯cT F(x)−1¯c+ ��m
k=1 cT k F(x)−1ck,
which we can write as an SDP minimize t0 + ��k tk subject to [ F(x) ¯c ¯cT t0 ] ≽ 0
[ F(x) ck
cT
k
tk ] ≽ 0, k = 1,...,m. 3. Optimality conditions and dual for log-optimal investment problem. (a) Show that the optimality conditions for the log-optimal investment problem de- scribed in exercise 4.60 can be expressed as: 1T x = 1, x ≽ 0, and for each i, xi > 0 ⇒
m
��
j=1
��j pij pT
j x
= 1, xi = 0 ⇒
m
��
j=1
��j pij pT
j x
�� 1. We can interpret this as follows. pij/pT
j x is a random variable, which gives the
ratio of the investment gain with asset i only, to the investment gain with our mixed portfolio x. The optimality condition is that, for each asset we invest in, the expected value of this ratio is one, and for each asset we do not invest in, the expected value cannot exceed one. Very roughly speaking, this means our portfolio does as well as any of the assets that we choose to invest in, and cannot do worse than any assets that we do not invest in. Hint. You can start from the simple criterion given in ��4.2.3, or the KKT condi- tions, or additional exercise 1 from homework 4. 8

 Page 9
(b) In this part we will derive the dual of the log-optimal investment problem. We start by writing the problem as, minimize −��m
j=1 ��j log yj
subject to y = PT x, x ≽ 0, 1T x = 1. Here, P has columns p1,...,pm, and we have the introduced new variables y1,...,ym, with the implicit constraint y ≻ 0. We will associate dual variables ��, �� and ��0 with the constraints y = PT x, x ≽ 0, and 1T x = 1, respectively. Defining ˜��j = ��j/��0 for j = 1,...,m, show that the dual problem can be written as maximize ��m
j=1 ��j log(˜��j/��j)
subject to P ˜�� ≼ 1, with variable ˜��. The objective here is the (negative) Kullback-Leibler divergence between the given distribution �� and the dual variable ˜��. Solution. (a) The problem is the same as minimizing f(x) = −��m
j=1 ��j log(pT j x) over the prob-
ability simplex. If x is feasible the optimality condition is that ∇f(x)T (z−x) �� 0 for all z with z ≽ 0, 1T z = 1. This holds if and only if for each k, xk > 0 ⇒ ∂f ∂xk = min
i=1,...,n
∂f ∂xi = ∇f(x)T x. From f(x) = −��m
j=1 ��j log(pT j x), we get
∇f(x)i = −
m
��
j=1
��j(1/pT
j x)pij,
so ∇f(x)T x = −
m
��
j=1
��j(1/pT
j x)pT j x = −1.
The optimality conditions are, 1T x = 1, x ≽ 0, and for each i, xi > 0 ⇒
m
��
j=1
��j pij pT
j x
= 1, xi = 0 ⇒
m
��
j=1
��j pij pT
j x
�� 1. (b) The Lagrangian is L(x, ��, ��, ��0) = −
m
��
j=1
��j log yj + ��T (y − PT x) − ��T x + ��0(1T x − 1). 9

 Page 10
This is unbounded below unless −��T PT − ��T + ��01T = 0. Since L is separable in y we can minimize it over y by minimizing over yj. We find the minimum is obtained for yj = ��j/��j, so we have g(��, ��, ��0)=1 − ��0 +
m
��
j=1
��j log(��j/��j), provided P�� ≼ ��01. Thus we can write the dual problem as maximize 1 − ��0 + ��m
j=1 ��j log(��j/��j)
subject to P�� ≼ ��01 with variables �� �� Rm, ��0 �� R. This has implicit constraint �� ≻ 0. We can further simplify this, by analytically optimizing over ��0. From the con- straint inequality we see that ��0 > 0. Defining ˜�� = ��/��0, we get the problem in variables ˜��, ��0 maximize 1 − ��0 + ��m
j=1 ��j log(˜��j��0/��j)
subject to ��0P ˜�� ≼ ��01. Cancelling ��0 from the constraint we get maximize 1 − ��0 + log ��0 + ��m
j=1 ��j log(˜��j/��j)
subject to P ˜�� ≼ 1. The optimal value of ��0 is evidently ��0 = 1, so we end up with the dual problem maximize ��m
j=1 ��j log(˜��j/��j)
subject to P ˜�� ≼ 1. 4. Log-optimal investment strategy. In this problem you will solve a specific instance of the log-optimal investment problem described in exercise 4.60, with n = 5 assets and m = 10 possible outcomes in each period. The problem data are defined in log_opt_invest.m, with the rows of the matrix P giving the asset return vectors pT
j .
The outcomes are equiprobable, i.e., we have ��j = 1/m. Each column of the matrix P gives the return of the associated asset in the different posible outcomes. You can examine the columns to get an idea of the types of assets. For example, the last asset gives a fixed and certain return of 1%; the first asset is a very risky one, with occasional large return, and (more often) substantial loss. Find the log-optimal investment strategy x⋆, and its associated long term growth rate R⋆
lt. Compare this to the long term growth rate obtained with a uniform allocation
strategy, i.e., x = (1/n)1, and also with a pure investment in each asset. For the optimal investment strategy, and also the uniform investment strategy, plot 10 sample trajectories of the accumulated wealth, i.e., W(T) = W(0)��T
t=1 ��(t), for
T = 0,...,200, with initial wealth W(0) = 1. 10

 Page 11
To save you the trouble of figuring out how to simulate the wealth trajectories or plot them nicely, we��ve included the simulation and plotting code in log_opt_invest.m; you just have to add the code needed to find x⋆. Hint: The current version of cvx doesn��t handle the logarithm, but you can use geomean() to solve the problem. Solution. (a) The following code was used to solve this problem: P = [3.5000 1.1100 1.1100 1.0400 1.0100; 0.5000 0.9700 0.9800 1.0500 1.0100; 0.5000 0.9900 0.9900 0.9900 1.0100; 0.5000 1.0500 1.0600 0.9900 1.0100; 0.5000 1.1600 0.9900 1.0700 1.0100; 0.5000 0.9900 0.9900 1.0600 1.0100; 0.5000 0.9200 1.0800 0.9900 1.0100; 0.5000 1.1300 1.1000 0.9900 1.0100; 0.5000 0.9300 0.9500 1.0400 1.0100; 3.5000 0.9900 0.9700 0.9800 1.0100]; [m,n] = size(P); % Find log-optimal investment policy cvx_begin variable x_opt(n) maximize(geomean(P*x_opt)) sum(x_opt) == 1 x_opt >= 0 cvx_end x_opt x_unif = ones(n,1)/n; R_opt = sum(log(P*x_opt))/m R_unif = sum(log(P*x_unif))/m It was found that the log-optimal investment strategy is: xopt = (0.0580, 0.4000, 0.2923, 0.2497, 0.0000). This strategy achieves a long term growth rate R∗
lt = 0.0231. In contrast, the
uniform allocation strategy achieves a growth rate of Runif = 0.0114. Clearly asset 1 is a high-risk asset. The amount that we invest in this asset will grow by a factor of 3.50 with probability 20% and will be halved with probability 11

 Page 12
80%. On the other hand, asset 5 is an asset with a certain return of 1% per time period. Finally, assets 2, 3 and 4 are low-risk assets. It turns out that the log-optimal policy in this case is to invest very little wealth in the high-risk asset and no wealth on the sure asset and to invest most of the wealth in asset 2. (b) The following code was used to generate the random event sequences and the trajectory plots: % Generate random event sequences rand(��state��,0); N = 10; T = 200; w_opt = []; w_unif = []; for i = 1:N events = ceil(rand(1,T)*m); P_event = P(events,:); w_opt = [w_opt [1; cumprod(P_event*x_opt)]]; w_unif = [w_unif [1; cumprod(P_event*x_unif)]]; end % Plot wealth versus time figure semilogy(w_opt,��g��) hold on semilogy(w_unif,��r--��) grid axis tight xlabel(��time��) ylabel(��wealth��) This generates the following plot: 12

 Page 13
20 40 60 80 100 120 140 160 180 200 10−1 100 101 102 103
t W The log-optimal investment policy consistently increases the wealth. On the other hand the uniform allocation policy generates quite random trajectories, a few with very large increases in wealth, and many with poor performance. This is due to the fact that with this policy 20% of the wealth is invested in the high-risk asset. 5. Maximizing house profit in a gamble and imputed probabilities. A set of n participants bet on which one of m outcomes, labeled 1,...,m, will occur. Participant i offers to purchase up to qi > 0 gambling contracts, at price pi > 0, that the true outcome will be in the set Si ⊂ {1,...,m}. The house then sells her xi contracts, with 0 �� xi �� qi. If the true outcome j is in Si, then participant i receives \$1 per contract, i.e., xi. Otherwise, she loses, and receives nothing. The house collects a total of x1p1+···+xnpn, and pays out an amount that depends on the outcome j,
��
j��Si
xi. The difference is the house profit. (a) Optimal house strategy. How should the house decide on x so that its worst-case profit (over the possible outcomes) is maximized? (The house determines x after examining all the participant offers.) (b) Imputed probabilities. Suppose x⋆ maximizes the worst-case house profit. Show that there exists a probability distribution �� on the possible outcomes (i.e., �� �� Rm
+ , 1T �� = 1) for which x⋆ also maximizes the expected house profit. Explain
how to find ��. 13

 Page 14
Hint. Formulate the problem in part (a) as an LP; you can construct �� from optimal dual variables for this LP. Remark. Given ��, the ��fair�� price for offer i is pfair
i
= ��j��Si ��j. All offers with pi > pfair
i
will be completely filled (i.e., xi = qi); all offers with pi < pfair
i
will be rejected (i.e., xi = 0). Remark. This exercise shows how the probabilities of outcomes (e.g., elections) can be guessed from the offers of a set of gamblers. (c) Numerical example. Carry out your method on the simple example below with n = 5 participants, m = 5 possible outcomes, and participant offers Participant i pi qi Si 1 0.50 10 {1,2} 2 0.60 5 {4} 3 0.60 5 {1,4,5} 4 0.60 20 {2,5} 5 0.20 10 {3} Compare the optimal worst-case house profit with the worst-case house profit, if all offers were accepted (i.e., xi = qi). Find the imputed probabilities. Solution. (a) The worst-case house profit is pT x − max
j=1,...,m �� j��Si
xi, which is a piecewise-linear concave function of x. To find the x that maximizes the worst-case profit, we solve the problem, maximize pT x − maxj=1,...,m aT
j x
subject to 0 ≼ x ≼ q, with variable x. aT
j are the rows of the subset matrix A, with
Aji = { 1 j �� Si 0 otherwise. (b) The problem from part (a) can be expressed as maximize pT x − t subject to t1 ≽ Ax 0 ≼ x ≼ q, (4) where t is a new scalar variable. The Lagrangian is L(x, t, ��1,��2,��3) = t − pT x + ��T
1 (Ax − t1) − ��T 2 x + ��T 3 (x − q).
14

 Page 15
This is bounded below if and only if 1T ��1 = 1, and AT ��1 −��2 +��3 = p. The dual can be written as maximize −qT ��3 subject to 1T ��1 = 1 AT ��1 − ��2 + ��3 = p ��1 ≽ 0, ��2 ≽ 0, ��3 ≽ 0, (5) with variables ��1, ��2, and ��3. Notice that ��1 must satisfy 1T ��1 = 1, and ��1 ≽ 0, hence it is a probability distribution. Suppose x⋆
wc, t⋆, ��⋆ 1, ��⋆ 2, and ��⋆ 3 are primal and dual optimal for problem (4), and
let us set �� = ��⋆
1. To maximize the expected house profit we solve the problem,
maximize pT x − ��T Ax subject to 0 ≼ x ≼ q. (6) Let bT
1 ,...,bT n be the rows of AT . We know that a point x⋆ e is optimal for problem
(6) if and only if x⋆
ei = qi when pi − bT i �� > 0, x⋆ ei = 0 when pi − bT i �� < 0, and
0 �� x⋆
ei �� qi when pi − bT i �� = 0.
To see why x⋆
e = x⋆ wc, let us take a look at one of the KKT conditions for problem
(4). This can be written as p − AT �� = ��⋆
3 − ��⋆ 2,
with �� = ��⋆
1. If pi − bT i �� > 0, then we must have ��⋆ 3i − ��⋆ 2i > 0, which means that
��⋆
2i = 0 and ��⋆ 3i > 0 (by complementary slackness), and so x⋆ wci = qi. Similarly, if
pi − bT
i �� < 0, then ��⋆ 3i − ��⋆ 2i < 0, which means that ��⋆ 2i > 0 and ��⋆ 3i = 0, and so
x⋆
wci = 0. Finally, when pi − bT i �� = 0, we must have ��⋆ 2i = 0 and ��⋆ 3i = 0, and so
0 �� x⋆
wci �� qi.
In summary, in order to find a probability distribution on the possible outcomes for which the same x⋆ maximizes both the worst-case as well as the expected house profit, we solve the dual LP (5), and set �� = ��⋆
1.
(c) The following cvx code solves the problem. % solution for gambling problem A = [1 0 1 0 0; 1 0 0 1 0; 0 0 0 0 1; 0 1 1 0 0; 0 0 1 1 0]; p = [0.5; 0.6; 0.6; 0.6; 0.2]; q = [10; 5; 5; 20; 10]; n = 5; m = 5; 15

 Page 16
cvx_begin variables x(n) t dual variable lambda1 maximize (p��*x-t) subject to lambda1: A*x <= t x >= 0 x <= q cvx_end % optimal worst case house profit pwc = cvx_optval % optimal worst case profit if all offer are accepted pwc_accept = p��*q-max(A*q) % imputed probabilities pi = lambda1 % fair prices pfair = A��*pi % optimal purchase quantities xopt = x Our results are summarized in the following table. We find that the optimal worst case house profit is 3.5, and the worst case house profit, if all offers are accepted, is −5. The imputed probabilities are �� = (0.1145, 0.3855, 0.0945, 0.1910, 0.2145). The associated fair prices and optimal contract numbers are shown below. Participant i pi pfair
i
qi xi Si 1 0.50 0.5000 10 5 {1,2} 2 0.60 0.1910 5 5 {4} 3 0.60 0.5200 5 5 {1,4,5} 4 0.60 0.6000 20 5 {2,5} 5 0.20 0.0945 10 10 {3} 6. Heuristic suboptimal solution for Boolean LP. This exercise builds on exercises 4.15 and 5.13, which involve the Boolean LP minimize cT x subject to Ax ≼ b xi �� {0,1}, i = 1,...,n, 16

 Page 17
with optimal value p⋆. Let xrlx be a solution of the LP relaxation minimize cT x subject to Ax ≼ b 0 ≼ x ≼ 1, so L = cT xrlx is a lower bound on p⋆. The relaxed solution xrlx can also be used to guess a Boolean point x, by rounding its entries, based on a threshold t �� [0,1]: xi = { 1 xrlx
i
�� t 0 otherwise, for i = 1,...,n. Evidently x is Boolean (i.e., has entries in {0,1}). If it is feasible for the Boolean LP, i.e., if Ax ≼ b, then it can be considered a guess at a good, if not optimal, point for the Boolean LP. Its objective value, U = cT x, is an upper bound on p⋆. If U and L are close, then x is nearly optimal; specifically, x cannot be more than (U − L)-suboptimal for the Boolean LP. This rounding need not work; indeed, it can happen that for all threshold values, x is infeasible. But for some problem instances, it can work well. Of course, there are many variations on this simple scheme for (possibly) constructing a feasible, good point from xrlx. Finally, we get to the problem. Generate problem data using rand(��state��,0); n=100; m=300; A=rand(m,n); b=A*ones(n,1)/2; c=-rand(n,1); You can think of xi as a job we either accept or decline, and −ci as the (positive) revenue we generate if we accept job i. We can think of Ax ≼ b as a set of limits on m resources. Aij, which is positive, is the amount of resource i consumed if we accept job j; bi, which is positive, is the amount of resource i available. Find a solution of the relaxed LP and examine its entries. Note the associated lower bound L. Carry out threshold rounding for (say) 100 values of t, uniformly spaced over [0,1]. For each value of t, note the objective value cT x and the maximum constraint violation maxi(Ax − b)i. Plot the objective value and the maximum violation versus t. Be sure to indicate on the plot the values of t for which x is feasible, and those for which it is not. Find a value of t for which x is feasible, and gives minimum objective value, and note the associated upper bound U. Give the gap U − L between the upper bound on p⋆ 17

 Page 18
and the lower bound on p⋆. If you define vectors obj and maxviol, you can find the upper bound as U=min(obj(find(maxviol<=0))). Solution. The following Matlab code finds the solution % generate data for boolean LP relaxation & heuristic rand(��state��,0); n=100; m=300; A=rand(m,n); b=A*ones(n,1)/2; c=-rand(n,1); % solve LP relaxation cvx_begin variable x(n) minimize (c��*x) subject to A*x <= b x>=0 x<=1 cvx_end xrlx = x; L=cvx_optval; % sweep over threshold & round thres=0:0.01:1; maxviol = zeros(length(thres),1); obj = zeros(length(thres),1); for i=1:length(thres) xhat = (xrlx>=thres(i)); maxviol(i) = max(A*xhat-b); obj(i) = c��*xhat; end % find least upper bound and associated threshold i_feas=find(maxviol<=0); U=min(obj(i_feas)) t=min(i_feas); min_thresh=thres(t) % plot objective and max violation versus threshold 18

 Page 19
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −30 −20 −10 0 10 20 30 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −50 −40 −30 −20 −10 0
threshold threshold m ax v iolation ob jective
Figure 1 Plots of violation and objective vs threshold rule.
subplot(2,1,1) plot(thres(1:t-1),maxviol(1:t-1),��r��,thres(t:end),maxviol(t:end),��b��,��linewidth��,2) xlabel(��threshold��); ylabel(��max violation��); subplot(2,1,2) hold on; plot(thres,L*ones(size(thres)),��k��,��linewidth��,2); plot(thres(1:t-1),obj(1:t-1),��r��,thres(t:end),obj(t:end),��b��,��linewidth��,2); xlabel(��threshold��); ylabel(��objective��); The lower bound found from the relaxed LP is L = −33.1672. We find that the threshold value t = 0.6006 gives the best (smallest) objective value for feasible x: U = −32.4450. The difference is 0.7222. So x, with t = 0.6006, can be no more than 0.7222 suboptimal. In figure 1, the red lines indicate values for thresholding values which give infeasible x, and the blue lines correspond to feasible x. We see that the maximum violation decreases as the threshold is increased. This occurs because the constraint matrix A only has nonnegative entries. At a threshold of 0, all jobs are selected, which is an infeasible solution. As we increase the threshold, projects are removed in sequence (without adding new projects), which monotonically decreases the maximum violation. For a general boolean LP, the corresponding plots need not exhibit monotonic behavior. 19
Search more related documents:EE364a Homework 5 solutions