Home > 5. Conjugate functions

Page 1 |

• closed functions • conjugate function • duality

5.1

Page 2 |

a set is

�� , �� ¯ =⇒ ¯ ��

• the intersection of (finitely or infinitely many) closed sets is closed • the union of a finite number of closed sets is closed • inverse under linear mapping: { | �� } is closed if

is closed

Conjugate functions 5.2

Page 3 |

the image of a closed set under a linear mapping is not necessarily closed

= {(1,2) ��

2 + | 1 2 �� 1},

= [ 1 0 ] , =

•

is closed and convex

• and

does not have a recession direction in the nullspace of ,

= 0, ˆ �� , ˆ + �� for all �� 0 =⇒ = 0

in particular, this holds for any matrix if is bounded

Conjugate functions 5.3

Page 4 |

• () = −log(1 −

2

) with dom = { | || < 1} • () = log with dom =

• () = log with dom =

if is not closed

Conjugate functions 5.4

Page 5 |

•

is closed in each case, we assume dom

∅

Conjugate functions 5.5

Page 6 |

• closed functions •

Page 7 |

the

∗

() = sup

��dom

( − ())

∗ is closed and convex (even when is not)

() +

∗

() ��

for all , this is an extension to non-quadratic convex of the inequality

1 2 + 1 2 ��

Conjugate functions 5.6

Page 8 |

() = 1 2 + +

∗

() = 1 2( − )

−1

( − ) −

∗

() = 1 2( − )

†

( − ) − , dom ∗ = range() +

Conjugate functions 5.7

Page 9 |

() = ��

=1

log

∗

() = ��

=1 −1

() = −��

=1

log

∗

() = −��

=1

log(− ) −

() = −log det (dom =

++) ∗

() = −log det(−) −

Conjugate functions 5.8

Page 10 |

() = { 0 �� +��

∗

() = sup

��

∗

() = −∗ () = ∗ (−) = { 0 �� 0 ∀ �� +�� otherwise

() =

∗

() = { 0

∗ �� 1

+�� ∗ > 1

(see next page)

Conjugate functions 5.9

Page 11 |

∗ = sup ��1

to evaluate ∗

() = sup ( − ) we distinguish two cases • if

∗ �� 1, then (by definition of dual norm)

�� for all

and equality holds if = 0; therefore sup ( − ) = 0

• if

∗ > 1, there exists an with

�� 1, > 1; then

∗

() �� ( )− = ( − )

and right-hand side goes to infinity if �� ��

Conjugate functions 5.10

Page 12 |

(1,2) = (1) + (2)

∗

(1, 2) =

∗

(1) +

∗

(2)

() = ()

∗

() =

∗

(/) () = (/)

∗

() =

∗

() • the operation () = (/) is sometimes called ��right scalar multiplication�� • a convenient notation is =

for the function ( )() =

(/) • conjugates can be written concisely as ( )

∗ = ∗ and ( ) ∗ = ∗

Conjugate functions 5.11

Page 13 |

() = () + +

∗

() =

∗

( − ) −

() = ( − )

∗

() = +

∗

()

() = ( )

∗

() =

∗

(

−

)

() = inf

+=

(() + ())

∗

() =

∗

() +

∗

()

Conjugate functions 5.12

Page 14 |

∗∗

() = sup

��dom ∗

( −

∗

()) •

∗∗ is closed and convex

• from Fenchel��s inequality, −

∗

() �� () for all and ; therefore

∗∗

() �� () for all

equivalently, epi ⊆ epi ∗∗ (for any )

• if is closed and convex, then

∗∗

() = () for all

equivalently, epi = epi ∗∗ (if is closed and convex); proof on next page

Conjugate functions 5.13

Page 15 |

epi

suppose ( ,

∗∗

()) epi ; then there is a strict separating hyperplane: [ ] [ − −

∗∗

() ] �� < 0

for all ( , ) �� epi holds for some , , with �� 0 ( > 0 gives a contradiction as �� ��)

• if < 0, define = /(−) and maximize left-hand side over ( , ) �� epi :

∗

() − +

∗∗

() �� /(−) < 0

this contradicts Fenchel��s inequality

• if = 0, choose ˆ �� dom ∗ and add small multiple of (ˆ,−1) to ( , ): [ + ˆ − ] [ − −

∗∗

() ] �� + ( ∗ (ˆ) − ˆ + ∗∗ () ) < 0

now apply the argument for < 0

Conjugate functions 5.14

Page 16 |

if is closed and convex, then

�� () ⇐⇒ ��

∗

() ⇐⇒ = () +

∗

()

(), then ∗ () = sup ( − ()) = − (); hence

∗

() = sup ( − ()) �� − () = ( − ) − () + =

∗

() + ( − )

this holds for all ; therefore, ��

∗

()

reverse implication ��

∗

() =⇒ �� () follows from ∗∗ =

Conjugate functions 5.15

Page 17 |

−1 1

() = [−1,1]()

∗() = ||

1 −1

()

1 −1

∗()

Conjugate functions 5.16

Page 18 |

( + (1 − )) �� ()+(1 − ) () − 2 (1 − ) −

2

for all , �� dom and �� [0,1]

• if is -strongly convex, then () �� () + ( − ) + 2 −

2

for all , �� dom , ��

() • for differentiable this is the inequality (4) on page 1.19

Conjugate functions 5.17

Page 19 |

• recall the definition of directional derivative (page 2.28 and 2.29): ( , − ) = inf>0 ( + ( − )) − ()

and the infimum is approached as �� 0

• if is -strongly convex and subdifferentiable at , then for all �� dom , ( , − ) �� inf

��(0,1]

(1 − ) () + ()−(/2)(1 − ) −

2

− () = () − () − 2 −

2

• from page 2.31, the directional derivative is the support function of (): ( − ) �� sup

˜�� ()

˜ ( − ) = (; − ) �� () − () − 2 −

2

Conjugate functions 5.18

Page 20 |

assume is closed and strongly convex with parameter > 0 for the norm ·

•

∗ is defined for all (

•

∗ is differentiable everywhere, with gradient

∇

∗

() = argmax ( − ()) • ∇

∗ is Lipschitz continuous with constant 1/ for the dual norm · ∗:

∇

∗

()−∇

∗

( ) �� 1 −

∗ for all and

Conjugate functions 5.19

Page 21 |

• − () has a unique maximizer for every • maximizes − () if and only if �� (); from page 5.15 �� () ⇐⇒ ��

∗

() = {∇

∗

()}

hence ∇

∗

() = argmax ( − ()) • from first-order condition on page 5.17: if �� (), �� ( ): ( ) �� () + ( − ) + 2 −

2

() �� ( )+( ) ( − ) + 2 −

2

combining these inequalities shows

−

2

�� ( − ) ( − )�� −

∗

− • now substitute = ∇

∗

() and = ∇

∗

( )

Conjugate functions 5.20

Page 22 |

• closed functions • conjugate function •

Page 23 |

primal: minimize

() + ( )

dual: maximize −

∗

() −

∗

(− ) • follows from Lagrange duality applied to reformulated primal

minimize

() + ()

subject to

=

dual function for the formulated problem is:

inf

, ( () +

+ () − ) = −

∗

(− ) −

∗

() • Slater��s condition (for convex , ): strong duality holds if there exists an ˆ with ˆ �� int dom , ˆ �� int dom

this also guarantees that the dual optimum is attained if optimal value is finite

Conjugate functions 5.21

Page 24 |

minimize

()

subject to

− ��

() + ( − )

dual: maximize − −

∗

() −

∗

(− )

()

equality

= {0} 0

norm inequality

− �� 1 unit ·-ball

∗

conic inequality

≼ −

∗ ()

Conjugate functions 5.22

Page 25 |

minimize

()+ − • take () = −

in general problem minimize

() + ( ) • conjugate of · is indicator of unit ball for dual norm

∗

() = + () where = { | ∗ �� 1} • hence, dual problem can be written as

maximize − −

∗

(− )

subject to

∗ �� 1

Conjugate functions 5.23

Page 26 |

minimize

() + ()

subject to

=

assume , are convex and Slater��s condition holds

�� dom

2. and = are minimizers of the Lagrangian () +

+ () −

:

− �� (), �� ( )

if is closed, this can be written symmetrically as

− �� (), ��

∗

()

Conjugate functions 5.24

Page 27 |

• J.-B. Hiriart-Urruty, C. Lemar��chal,

(1993), chapter X.

• D.P. Bertsekas, A. Nedić, A.E. Ozdaglar,

(2003), chapter 7.

• R. T. Rockafellar,

Conjugate functions 5.25

- Full text of "Talbot genealogy [the genealogical history of Peter Talbot, the emigrant, and some of his descendants]"
- METHOD
- Language and Style in Writing
- Meta-Analysis in Human Neuroimaging: Computational Modeling of Large-Scale Databases
- Runway to Realway: Visual Analysis of Fashion
- Securing an Oracle Database
- A Culture of Excess
- COLLEGE OF COMPUTER TRAINING (CCT)
- TOP OFFICIALS (TOPOFF)
- Member List 2014
- Instruments and guidelines for qualitative fieldwork
- gar003, Chapter 3 Systems Design: Job-Order Costing
- A Comparison of Software and Hardware Techniques for x86 Virtualization
- Curriculum Vitae Robert G. Smith Contact Information Address: George Mason University, College of Education and Human Developmen
- The Anatomy of a Search Engine
- Rose-Hulman Institute of Technology Dr. Rostamkolai ECE 370 MACHINES & POWER
- WORKBOOK- ANSWER KEY
- OPM-V-A-1 What Should an Agency Do To Prepare For a Pandemic Health Crisis? Has the agency...
- Mega-Fun Fractions.final
- In situ X-ray Radiography for Compression Test of Foam Aluminum
- CONVERGENCE OF BACKTRACKING LINE SEARCH David F. Gleich Tis is a summary of Teorem 11.7 from Griva, Nash, and Sofer. f : R n

- ERF
- GCC box
- tobacco
- rice
- disease resistance
- salt tolerance
- copyright
- the fair use
- permission use in the legal scenes
- authorized important patterns
- collective management
- online service provider safe harbors
- Tall fescue
- Zn
- Cd
- Combined pollution
- Seed germination
- Seedling growth
- Physiological processes
- Wireless Mesh Networks
- Topology control

All Rights Reserved Powered by Free Document Search and Download

Copyright © 2011This site does not host pdf,doc,ppt,xls,rtf,txt files all document are the property of their respective owners. complaint#nuokui.com