EDP Sciences 1775437703 20260406 fre

laboutique.edpsciences.fr-R002493 03 01 01 R002493 00 ED E107 00 01 01 Modern Optimization Methods 01 eng 08 168 03 22 6612243 17 01 P15 EDP Sciences & Science Press 01 01 P15 EDP Sciences & Science Press 11 00 20231113 02 01 002613 03 9782759831753 15 9782759831753 03 01 D1 EDP Sciences 45 03 laboutique.edpsciences.fr-002613 03 01 01 002613 03 9782759831753 15 9782759831753 10 EA E107 10 00 01 R002493 ED E107 1 10 01 02 Current Natural Sciences 01 01 Modern Optimization Methods 1 A01 01 A2211 Qingna LI LI, Qingna Qingna LI Qingna LI is a full professor in School of Mathematics and Statistics, Beijing Institute of Technology. Her research interests include continuous optimization, specifically in matrix optimization, sparse optimization and the applications in various areas including wireless communication, statistics, artificial intelligence and so on. 01 eng 08 157 03 02 24 Izibook:Subject Mathématiques 24 Izibook:Subject Ingénierie 24 Izibook:SubjectAndCategoryAndTags |Mathématiques| 24 Izibook:SubjectAndCategoryAndTags |Ingénierie| 20 optimisation;conditions d'optimalité;algorithmes;méthode de Newton non lisse;optimisation numérique;recherche opérationnelle;hypergraph matching;support vector machine;hyperparameter selection;machine learning 10 2011 MAT029030 01 510 06 05 06 03 00 <blockquote>With the fast development of big data and artificial intelligence, a natural question is how do we analyze data more efficiently? One of the efficient ways is to use optimization. What is optimization? Optimization exists everywhere. People optimize. As long as you have choices, you do optimization. Optimization is the key of operations research. This book introduces the basic definitions and theory about numerical optimization, including optimality conditions for unconstrained and constrained optimization, as well as algorithms for unconstrained and constrained problems. Moreover, it also includes the nonsmooth Newton’s method, which plays an important role in large-scale numerical optimization. Finally, based on the author’s research experiences, several latest applications about optimization are introduced, including optimization algorithms for hypergraph matching, support vector machine and bilevel optimization approach for hyperparameter selection in machine learning. With these optimization tools, one can deal with data more efficiently.</blockquote> 02 00 With the fast development of big data and artificial intelligence, a natural question is how do we analyze data more efficiently? One of the efficient ways is to use optimization. This book presents several methods of optimization. 04 00 Preface..................................................... III<o:p></o:p>CHAPTER 1<o:p></o:p>Introduction................................................. 1<o:p></o:p>1.1 About Optimization ...................................... 1<o:p></o:p>1.2 Classification of Optimization ............................... 4<o:p></o:p>1.3 Preliminaries in Convex Analysis ............................ 10<o:p></o:p>1.4 Exercises............................................... 15<o:p></o:p>CHAPTER 2<o:p></o:p>Fundamentals of Optimization ................................... 17<o:p></o:p>2.1 Unconstrained Optimization Problem ......................... 17<o:p></o:p>2.2 What isa Solution? ...................................... 18<o:p></o:p>2.2.1 Definitions of Different Solutions ....................... 18<o:p></o:p>2.2.2 Recognizing a Local Minimum ......................... 20<o:p></o:p>2.2.3 Nonsmooth Problems ................................ 23<o:p></o:p>2.3 Overview of Algorithms ................................... 25<o:p></o:p>2.3.1 Line Search Strategy ................................ 26<o:p></o:p>2.3.2 Trust Region Strategy ............................... 30<o:p></o:p>2.4 Convergence ............................................ 31<o:p></o:p>2.5 Scaling................................................ 32<o:p></o:p>2.6 Exercises............................................... 33<o:p></o:p>CHAPTER 3<o:p></o:p>Line Search Methods .......................................... 35<o:p></o:p>3.1 Step Length ............................................ 35<o:p></o:p>3.1.1 The Wolfe Conditions ............................... 37<o:p></o:p>3.1.2 The Goldstein Conditions ............................ 40<o:p></o:p>3.1.3 Sufficient Decrease and Backtracking .................... 41<o:p></o:p>3.2 Convergence of Line Search Methods ......................... 42<o:p></o:p>3.3 Rate of Convergence ...................................... 44<o:p></o:p>3.3.1 Steepest Descent Method ............................. 44<o:p></o:p>3.3.2 Newton’s Method................................... 46<o:p></o:p>3.3.3 Quasi-Newton Methods .............................. 48<o:p></o:p>3.4 Exercises............................................... 50<o:p></o:p>CHAPTER 4<o:p></o:p>Trust Region Methods ......................................... 51<o:p></o:p>4.1 Outline of the Trust Region Approach ........................ 52<o:p></o:p>4.2 Algorithms Based on the Cauchy Point ........................ 54<o:p></o:p>4.2.1 The Cauchy Point .................................. 54<o:p></o:p>4.2.2 The Dogleg Method ................................. 56<o:p></o:p>4.2.3 Two-Dimensional Subspace Minimization ................. 58<o:p></o:p>4.3 Global Convergence ...................................... 59<o:p></o:p>4.3.1 Reduction Obtained by the Cauchy Point ................ 59<o:p></o:p>4.3.2 Convergence to Stationary Points....................... 61<o:p></o:p>4.4 Local Convergence ....................................... 65<o:p></o:p>4.5 Other Enhancements......................................65<o:p></o:p>4.6 Exercises...............................................68<o:p></o:p>CHAPTER 5<o:p></o:p>Conjugate Gradient Methods.................................... 69<o:p></o:p>5.1 LinearConjugate Gradient Method........................... 69<o:p></o:p>5.1.1Conjugate Direction Method .......................... 69<o:p></o:p>5.1.2 Conjugate Gradient Method........................... 72<o:p></o:p>5.1.3 A Practical Form of the Conjugate Gradient Method ........ 75<o:p></o:p>5.1.4 Rate of Convergence ................................ 76<o:p></o:p>5.1.5 Preconditioning .................................... 77<o:p></o:p>5.2 Nonlinear Conjugate Gradient Methods ....................... 78<o:p></o:p>5.2.1 The Polak-Ribiere Method and Variants.................. 80<o:p></o:p>5.2.2Global Convergence ................................. 81<o:p></o:p>5.3 Exercises............................................... 83<o:p></o:p>CHAPTER 6<o:p></o:p>Semismooth Newton’s Method ................................... 85<o:p></o:p>6.1 Semismoothness ......................................... 85<o:p></o:p>6.2 Nonsmooth Version of Newton’s Method....................... 87<o:p></o:p>6.3 Support Vector Machine ................................... 89<o:p></o:p>6.4 Semismooth Newton’s Method for SVM ....................... 91<o:p></o:p>6.5 Exercises............................................... 96<o:p></o:p>CHAPTER 7<o:p></o:p>Theory of Constrained Optimization ............................... 97<o:p></o:p>7.1 Local and Global Solutions ................................. 97<o:p></o:p>7.1.1 Smoothness ....................................... 98<o:p></o:p>7.2 Examples .............................................. 997.3 Tangent Cone and Constraint Qualifications .................... 103<o:p></o:p>7.4 First-Order Optimality Conditions ........................... 105<o:p></o:p>7.5 Second-Order Conditions .................................. 106<o:p></o:p>7.6 Duality................................................ 109<o:p></o:p>7.7 KKT Condition ......................................... 112<o:p></o:p>7.8 Dual Problem ........................................... 114<o:p></o:p>7.9 Exercises............................................... 118<o:p></o:p>CHAPTER 8<o:p></o:p>Penalty and Augmented Lagrangian Methods ........................ 119<o:p></o:p>8.1 The Quadratic Penalty Method.............................. 119<o:p></o:p>8.2 Exact Penalty Method .................................... 122<o:p></o:p>8.3 Augmented Lagrangian Method ............................. 123<o:p></o:p>8.4 Quadratic Penalty Method for Hypergraph Matching ............. 125<o:p></o:p>8.4.1 Hypergraph Matching ............................... 126<o:p></o:p>8.4.2 Mathematical Formulation ............................ 126<o:p></o:p>8.4.3 Relaxation Problem ................................. 128<o:p></o:p>8.4.4 Quadratic Penalty Method for (8.21) .................... 129<o:p></o:p>8.4.5 Numerical Results .................................. 130<o:p></o:p>8.5 Augmented Lagrangian Method for SVM ...................... 132<o:p></o:p>8.5.1 Support Vecotr Machine ............................. 132<o:p></o:p>8.5.2 Mathematical Formulation ............................ 133<o:p></o:p>8.5.3 Augmented Lagrangian Method (ALM) .................. 133<o:p></o:p>8.5.4 Semismooth Newton’s Method for the Subproblem.......... 136<o:p></o:p>8.5.5 Reducing the Computational Cost ...................... 137<o:p></o:p>8.5.6 Convergence Result of ALM ........................... 138<o:p></o:p>8.5.7 Numerical Results on LIBLINEAR...................... 139<o:p></o:p>8.6 Exercises............................................... 141<o:p></o:p>CHAPTER 9<o:p></o:p>Bilevel Optimization and Its Applications ........................... 143<o:p></o:p>9.1 Introduction ............................................ 143<o:p></o:p>9.2 Bilevel Model for a Case of Hyperparameter Selection in SVC ....... 145<o:p></o:p>9.2.1 An MPEC Formulation .............................. 147<o:p></o:p>9.3 The Global Relaxation Method (GRM)........................ 148<o:p></o:p>9.4 MPEC-MFCQ: A Hidden Property ........................... 149<o:p></o:p>9.5 Numerical Results........................................ 150<o:p></o:p>Bibliography................................................. 153<o:p></o:p> Preface..................................................... III<o:p></o:p>CHAPTER 1<o:p></o:p>Introduction................................................. 1<o:p></o:p>1.1 About Optimization ...................................... 1<o:p></o:p>1.2Classification of Optimization ............................... 4<o:p></o:p>1.3Preliminaries in Convex Analysis ............................ 10<o:p></o:p>1.4Exercises............................................... 15<o:p></o:p>CHAPTER 2<o:p></o:p>Fundamentals of Optimization ................................... 17<o:p></o:p>2.1Unconstrained Optimization Problem ......................... 17<o:p></o:p>2.2 What isa Solution? ...................................... 18<o:p></o:p>2.2.1Definitions of Different Solutions ....................... 18<o:p></o:p>2.2.2Recognizing a Local Minimum ......................... 20<o:p></o:p>2.2.3Non smooth Problems ................................ 23<o:p></o:p>2.3Overview of Algorithms ................................... 25<o:p></o:p>2.3.1 Line Search Strategy ................................ 26<o:p></o:p>2.3.2 Trust Region Strategy ............................... 30<o:p></o:p>2.4Convergence ............................................ 31<o:p></o:p>2.5 Scaling................................................ 32<o:p></o:p>2.6Exercises............................................... 33<o:p></o:p>CHAPTER 3<o:p></o:p>Line Search Methods .......................................... 35<o:p></o:p>3.1 StepLength ............................................ 35<o:p></o:p>3.1.1 The Wolfe Conditions ............................... 37<o:p></o:p>3.1.2 The Goldstein Conditions ............................ 40<o:p></o:p>3.1.3Sufficient Decrease and Backtracking .................... 41<o:p></o:p>3.2Convergence of Line Search Methods ......................... 42<o:p></o:p>3.3 Rate of Convergence ...................................... 44<o:p></o:p>3.3.1Steepest Descent Method ............................. 44<o:p></o:p>3.3.2Newton’s Method................................... 46<o:p></o:p>3.3.3Quasi-Newton Methods .............................. 48<o:p></o:p>3.4Exercises............................................... 50<o:p></o:p>CHAPTER 4<o:p></o:p>Trust Region Methods ......................................... 51<o:p></o:p>4.1 Outlineof the Trust Region Approach ........................ 52<o:p></o:p>4.2Algorithms Based on the Cauchy Point ........................ 54<o:p></o:p>4.2.1 The Cauchy Point .................................. 54<o:p></o:p>4.2.2 The Dogleg Method ................................. 56<o:p></o:p>4.2.3Two-Dimensional Subspace Minimization ................. 58<o:p></o:p>4.3 Global Convergence ...................................... 59<o:p></o:p>4.3.1Reduction Obtained by the Cauchy Point ................ 59<o:p></o:p>4.3.2Convergence to Stationary Points....................... 61<o:p></o:p>4.4 Local Convergence ....................................... 65<o:p></o:p>4.5 Other Enhancements......................................65<o:p></o:p>4.6 Exercises...............................................68<o:p></o:p>CHAPTER 5<o:p></o:p>Conjugate Gradient Methods.................................... 69<o:p></o:p>5.1 Linear Conjugate Gradient Method........................... 69<o:p></o:p>5.1.1Conjugate Direction Method .......................... 69<o:p></o:p>5.1.2Conjugate Gradient Method........................... 72<o:p></o:p>5.1.3 A Practical Form of the Conjugate Gradient Method ........ 75<o:p></o:p>5.1.4 Rate of Convergence ................................ 76<o:p></o:p>5.1.5Preconditioning .................................... 77<o:p></o:p>5.2Nonlinear Conjugate Gradient Methods ....................... 78<o:p></o:p>5.2.1 ThePolak-Ribiere Method and Variants.................. 80<o:p></o:p>5.2.2Global Convergence ................................. 81<o:p></o:p>5.3Exercises............................................... 83<o:p></o:p>CHAPTER 6<o:p></o:p>Semi smooth Newton’s Method ................................... 85<o:p></o:p>6.1Semi smoothness ......................................... 85<o:p></o:p>6.2Non smooth Version of Newton’s Method....................... 87<o:p></o:p>6.3 Support Vector Machine ................................... 89<o:p></o:p>6.4Semi smooth Newton’s Method for SVM ....................... 91<o:p></o:p>6.5Exercises............................................... 96<o:p></o:p>CHAPTER 7<o:p></o:p>Theory of Constrained Optimization ............................... 97<o:p></o:p>7.1 Local and Global Solutions ................................. 97<o:p></o:p>7.1.1Smoothness ....................................... 98<o:p></o:p>7.2Examples .............................................. 99<o:p></o:p>VI Contents<o:p></o:p>7.3 Tangent Cone and Constraint Qualifications .................... 103<o:p></o:p>7.4First-Order Optimality Conditions ........................... 105<o:p></o:p>7.5Second-Order Conditions .................................. 106<o:p></o:p>7.6 Duality................................................ 109<o:p></o:p>7.7 KKTCondition ......................................... 112<o:p></o:p>7.8 DualProblem ........................................... 114<o:p></o:p>7.9Exercises............................................... 118<o:p></o:p>CHAPTER 8<o:p></o:p>Penalty and Augmented Lagrangian Methods ........................ 119<o:p></o:p>8.1 The Quadratic Penalty Method.............................. 119<o:p></o:p>8.2 ExactPenalty Method .................................... 122<o:p></o:p>8.3Augmented Lagrangian Method ............................. 123<o:p></o:p>8.4Quadratic Penalty Method for Hypergraph Matching ............. 125<o:p></o:p>8.4.1Hypergraph Matching ............................... 126<o:p></o:p>8.4.2Mathematical Formulation ............................ 126<o:p></o:p>8.4.3Relaxation Problem ................................. 128<o:p></o:p>8.4.4Quadratic Penalty Method for (8.21) .................... 129<o:p></o:p>8.4.5Numerical Results .................................. 130<o:p></o:p>8.5Augmented Lagrangian Method for SVM ...................... 132<o:p></o:p>8.5.1Support Vector Machine ............................. 132<o:p></o:p>8.5.2Mathematical Formulation ............................ 133<o:p></o:p>8.5.3Augmented Lagrangian Method (ALM) .................. 133<o:p></o:p>8.5.4Semismooth Newton’s Method for the Subproblem.......... 136<o:p></o:p>8.5.5Reducing the Computational Cost ...................... 137<o:p></o:p>8.5.6Convergence Result of ALM ........................... 138<o:p></o:p>8.5.7Numerical Results on LIBLINEAR...................... 139<o:p></o:p>8.6Exercises............................................... 141<o:p></o:p>CHAPTER 9<o:p></o:p>Bilevel Optimization and Its Applications ........................... 143<o:p></o:p>9.1Introduction ............................................ 143<o:p></o:p>9.2 Bilevel Model for a Case of Hyperparameter Selection in SVC ....... 145<o:p></o:p>9.2.1 AnMPEC Formulation .............................. 147<o:p></o:p>9.3 The Global Relaxation Method (GRM)........................ 148<o:p></o:p>9.4MPEC-MFCQ: A Hidden Property ........................... 149<o:p></o:p>9.5 Numerical Results........................................ 150<o:p></o:p>Bibliography................................................. 153<o:p></o:p> 21 00 06 01 https://laboutique.edpsciences.fr/produit/1376/9782759831753/modern-optimization-methods 01 00 03 02 https://laboutique.edpsciences.fr/system/product_pictures/data/009/983/047/original/9782759831746-Modern_Optimization_Methods_couv-sofedis.jpg 17 14 20240913T175127+0200 01 P15 EDP Sciences & Science Press 01 01 P15 EDP Sciences & Science Press 04 11 00 20231113 01 00 20231113 19 00 20231113 2026 06 3052868830012 01 WORLD 13 03 9782759831746 15 9782759831746 06 03 9782759831760 15 9782759831760 WORLD 08 EDP Sciences & Science Press 04 01 00 20231113 03 01 D1 EDP Sciences 20 04 05 01 02 1 00 65.99 01 5.50 3.44 EUR 04 06 01 02 1 00 231.99 01 5.50 3.44 EUR 01 04 06 01 02 1 00 255.99 01 5.50 13.35 USD 01 laboutique.edpsciences.fr-002614 03 01 01 002614 03 9782759831760 15 9782759831760 10 EA 10 10 01 02 Current Natural Sciences 01 01 Modern Optimization Methods 1 A01 01 A2211 Qingna LI LI, Qingna Qingna LI Qingna LI is a full professor in School of Mathematics and Statistics, Beijing Institute of Technology. Her research interests include continuous optimization, specifically in matrix optimization, sparse optimization and the applications in various areas including wireless communication, statistics, artificial intelligence and so on. 01 eng 08 157 03 02 24 Izibook:Subject Mathématiques 24 Izibook:Subject Ingénierie 24 Izibook:SubjectAndCategoryAndTags |Mathématiques| 24 Izibook:SubjectAndCategoryAndTags |Ingénierie| 20 optimisation;conditions d'optimalité;algorithmes;méthode de Newton non lisse;optimisation numérique;recherche opérationnelle;hypergraph matching;support vector machine;hyperparameter selection;machine learning 10 2011 MAT029030 01 510 06 05 06 03 00 <blockquote>With the fast development of big data and artificial intelligence, a natural question is how do we analyze data more efficiently? One of the efficient ways is to use optimization. What is optimization? Optimization exists everywhere. People optimize. As long as you have choices, you do optimization. Optimization is the key of operations research. This book introduces the basic definitions and theory about numerical optimization, including optimality conditions for unconstrained and constrained optimization, as well as algorithms for unconstrained and constrained problems. Moreover, it also includes the nonsmooth Newton’s method, which plays an important role in large-scale numerical optimization. Finally, based on the author’s research experiences, several latest applications about optimization are introduced, including optimization algorithms for hypergraph matching, support vector machine and bilevel optimization approach for hyperparameter selection in machine learning. With these optimization tools, one can deal with data more efficiently.</blockquote> 02 00 With the fast development of big data and artificial intelligence, a natural question is how do we analyze data more efficiently? One of the efficient ways is to use optimization. This book presents several methods of optimization. 04 00 Preface..................................................... III<o:p></o:p>CHAPTER 1<o:p></o:p>Introduction................................................. 1<o:p></o:p>1.1 About Optimization ...................................... 1<o:p></o:p>1.2 Classification of Optimization ............................... 4<o:p></o:p>1.3 Preliminaries in Convex Analysis ............................ 10<o:p></o:p>1.4 Exercises............................................... 15<o:p></o:p>CHAPTER 2<o:p></o:p>Fundamentals of Optimization ................................... 17<o:p></o:p>2.1 Unconstrained Optimization Problem ......................... 17<o:p></o:p>2.2 What isa Solution? ...................................... 18<o:p></o:p>2.2.1 Definitions of Different Solutions ....................... 18<o:p></o:p>2.2.2 Recognizing a Local Minimum ......................... 20<o:p></o:p>2.2.3 Nonsmooth Problems ................................ 23<o:p></o:p>2.3 Overview of Algorithms ................................... 25<o:p></o:p>2.3.1 Line Search Strategy ................................ 26<o:p></o:p>2.3.2 Trust Region Strategy ............................... 30<o:p></o:p>2.4 Convergence ............................................ 31<o:p></o:p>2.5 Scaling................................................ 32<o:p></o:p>2.6 Exercises............................................... 33<o:p></o:p>CHAPTER 3<o:p></o:p>Line Search Methods .......................................... 35<o:p></o:p>3.1 Step Length ............................................ 35<o:p></o:p>3.1.1 The Wolfe Conditions ............................... 37<o:p></o:p>3.1.2 The Goldstein Conditions ............................ 40<o:p></o:p>3.1.3 Sufficient Decrease and Backtracking .................... 41<o:p></o:p>3.2 Convergence of Line Search Methods ......................... 42<o:p></o:p>3.3 Rate of Convergence ...................................... 44<o:p></o:p>3.3.1 Steepest Descent Method ............................. 44<o:p></o:p>3.3.2 Newton’s Method................................... 46<o:p></o:p>3.3.3 Quasi-Newton Methods .............................. 48<o:p></o:p>3.4 Exercises............................................... 50<o:p></o:p>CHAPTER 4<o:p></o:p>Trust Region Methods ......................................... 51<o:p></o:p>4.1 Outline of the Trust Region Approach ........................ 52<o:p></o:p>4.2 Algorithms Based on the Cauchy Point ........................ 54<o:p></o:p>4.2.1 The Cauchy Point .................................. 54<o:p></o:p>4.2.2 The Dogleg Method ................................. 56<o:p></o:p>4.2.3 Two-Dimensional Subspace Minimization ................. 58<o:p></o:p>4.3 Global Convergence ...................................... 59<o:p></o:p>4.3.1 Reduction Obtained by the Cauchy Point ................ 59<o:p></o:p>4.3.2 Convergence to Stationary Points....................... 61<o:p></o:p>4.4 Local Convergence ....................................... 65<o:p></o:p>4.5 Other Enhancements......................................65<o:p></o:p>4.6 Exercises...............................................68<o:p></o:p>CHAPTER 5<o:p></o:p>Conjugate Gradient Methods.................................... 69<o:p></o:p>5.1 LinearConjugate Gradient Method........................... 69<o:p></o:p>5.1.1Conjugate Direction Method .......................... 69<o:p></o:p>5.1.2 Conjugate Gradient Method........................... 72<o:p></o:p>5.1.3 A Practical Form of the Conjugate Gradient Method ........ 75<o:p></o:p>5.1.4 Rate of Convergence ................................ 76<o:p></o:p>5.1.5 Preconditioning .................................... 77<o:p></o:p>5.2 Nonlinear Conjugate Gradient Methods ....................... 78<o:p></o:p>5.2.1 The Polak-Ribiere Method and Variants.................. 80<o:p></o:p>5.2.2Global Convergence ................................. 81<o:p></o:p>5.3 Exercises............................................... 83<o:p></o:p>CHAPTER 6<o:p></o:p>Semismooth Newton’s Method ................................... 85<o:p></o:p>6.1 Semismoothness ......................................... 85<o:p></o:p>6.2 Nonsmooth Version of Newton’s Method....................... 87<o:p></o:p>6.3 Support Vector Machine ................................... 89<o:p></o:p>6.4 Semismooth Newton’s Method for SVM ....................... 91<o:p></o:p>6.5 Exercises............................................... 96<o:p></o:p>CHAPTER 7<o:p></o:p>Theory of Constrained Optimization ............................... 97<o:p></o:p>7.1 Local and Global Solutions ................................. 97<o:p></o:p>7.1.1 Smoothness ....................................... 98<o:p></o:p>7.2 Examples .............................................. 997.3 Tangent Cone and Constraint Qualifications .................... 103<o:p></o:p>7.4 First-Order Optimality Conditions ........................... 105<o:p></o:p>7.5 Second-Order Conditions .................................. 106<o:p></o:p>7.6 Duality................................................ 109<o:p></o:p>7.7 KKT Condition ......................................... 112<o:p></o:p>7.8 Dual Problem ........................................... 114<o:p></o:p>7.9 Exercises............................................... 118<o:p></o:p>CHAPTER 8<o:p></o:p>Penalty and Augmented Lagrangian Methods ........................ 119<o:p></o:p>8.1 The Quadratic Penalty Method.............................. 119<o:p></o:p>8.2 Exact Penalty Method .................................... 122<o:p></o:p>8.3 Augmented Lagrangian Method ............................. 123<o:p></o:p>8.4 Quadratic Penalty Method for Hypergraph Matching ............. 125<o:p></o:p>8.4.1 Hypergraph Matching ............................... 126<o:p></o:p>8.4.2 Mathematical Formulation ............................ 126<o:p></o:p>8.4.3 Relaxation Problem ................................. 128<o:p></o:p>8.4.4 Quadratic Penalty Method for (8.21) .................... 129<o:p></o:p>8.4.5 Numerical Results .................................. 130<o:p></o:p>8.5 Augmented Lagrangian Method for SVM ...................... 132<o:p></o:p>8.5.1 Support Vecotr Machine ............................. 132<o:p></o:p>8.5.2 Mathematical Formulation ............................ 133<o:p></o:p>8.5.3 Augmented Lagrangian Method (ALM) .................. 133<o:p></o:p>8.5.4 Semismooth Newton’s Method for the Subproblem.......... 136<o:p></o:p>8.5.5 Reducing the Computational Cost ...................... 137<o:p></o:p>8.5.6 Convergence Result of ALM ........................... 138<o:p></o:p>8.5.7 Numerical Results on LIBLINEAR...................... 139<o:p></o:p>8.6 Exercises............................................... 141<o:p></o:p>CHAPTER 9<o:p></o:p>Bilevel Optimization and Its Applications ........................... 143<o:p></o:p>9.1 Introduction ............................................ 143<o:p></o:p>9.2 Bilevel Model for a Case of Hyperparameter Selection in SVC ....... 145<o:p></o:p>9.2.1 An MPEC Formulation .............................. 147<o:p></o:p>9.3 The Global Relaxation Method (GRM)........................ 148<o:p></o:p>9.4 MPEC-MFCQ: A Hidden Property ........................... 149<o:p></o:p>9.5 Numerical Results........................................ 150<o:p></o:p>Bibliography................................................. 153<o:p></o:p> Preface..................................................... III<o:p></o:p>CHAPTER 1<o:p></o:p>Introduction................................................. 1<o:p></o:p>1.1 About Optimization ...................................... 1<o:p></o:p>1.2Classification of Optimization ............................... 4<o:p></o:p>1.3Preliminaries in Convex Analysis ............................ 10<o:p></o:p>1.4Exercises............................................... 15<o:p></o:p>CHAPTER 2<o:p></o:p>Fundamentals of Optimization ................................... 17<o:p></o:p>2.1Unconstrained Optimization Problem ......................... 17<o:p></o:p>2.2 What isa Solution? ...................................... 18<o:p></o:p>2.2.1Definitions of Different Solutions ....................... 18<o:p></o:p>2.2.2Recognizing a Local Minimum ......................... 20<o:p></o:p>2.2.3Non smooth Problems ................................ 23<o:p></o:p>2.3Overview of Algorithms ................................... 25<o:p></o:p>2.3.1 Line Search Strategy ................................ 26<o:p></o:p>2.3.2 Trust Region Strategy ............................... 30<o:p></o:p>2.4Convergence ............................................ 31<o:p></o:p>2.5 Scaling................................................ 32<o:p></o:p>2.6Exercises............................................... 33<o:p></o:p>CHAPTER 3<o:p></o:p>Line Search Methods .......................................... 35<o:p></o:p>3.1 StepLength ............................................ 35<o:p></o:p>3.1.1 The Wolfe Conditions ............................... 37<o:p></o:p>3.1.2 The Goldstein Conditions ............................ 40<o:p></o:p>3.1.3Sufficient Decrease and Backtracking .................... 41<o:p></o:p>3.2Convergence of Line Search Methods ......................... 42<o:p></o:p>3.3 Rate of Convergence ...................................... 44<o:p></o:p>3.3.1Steepest Descent Method ............................. 44<o:p></o:p>3.3.2Newton’s Method................................... 46<o:p></o:p>3.3.3Quasi-Newton Methods .............................. 48<o:p></o:p>3.4Exercises............................................... 50<o:p></o:p>CHAPTER 4<o:p></o:p>Trust Region Methods ......................................... 51<o:p></o:p>4.1 Outlineof the Trust Region Approach ........................ 52<o:p></o:p>4.2Algorithms Based on the Cauchy Point ........................ 54<o:p></o:p>4.2.1 The Cauchy Point .................................. 54<o:p></o:p>4.2.2 The Dogleg Method ................................. 56<o:p></o:p>4.2.3Two-Dimensional Subspace Minimization ................. 58<o:p></o:p>4.3 Global Convergence ...................................... 59<o:p></o:p>4.3.1Reduction Obtained by the Cauchy Point ................ 59<o:p></o:p>4.3.2Convergence to Stationary Points....................... 61<o:p></o:p>4.4 Local Convergence ....................................... 65<o:p></o:p>4.5 Other Enhancements......................................65<o:p></o:p>4.6 Exercises...............................................68<o:p></o:p>CHAPTER 5<o:p></o:p>Conjugate Gradient Methods.................................... 69<o:p></o:p>5.1 Linear Conjugate Gradient Method........................... 69<o:p></o:p>5.1.1Conjugate Direction Method .......................... 69<o:p></o:p>5.1.2Conjugate Gradient Method........................... 72<o:p></o:p>5.1.3 A Practical Form of the Conjugate Gradient Method ........ 75<o:p></o:p>5.1.4 Rate of Convergence ................................ 76<o:p></o:p>5.1.5Preconditioning .................................... 77<o:p></o:p>5.2Nonlinear Conjugate Gradient Methods ....................... 78<o:p></o:p>5.2.1 ThePolak-Ribiere Method and Variants.................. 80<o:p></o:p>5.2.2Global Convergence ................................. 81<o:p></o:p>5.3Exercises............................................... 83<o:p></o:p>CHAPTER 6<o:p></o:p>Semi smooth Newton’s Method ................................... 85<o:p></o:p>6.1Semi smoothness ......................................... 85<o:p></o:p>6.2Non smooth Version of Newton’s Method....................... 87<o:p></o:p>6.3 Support Vector Machine ................................... 89<o:p></o:p>6.4Semi smooth Newton’s Method for SVM ....................... 91<o:p></o:p>6.5Exercises............................................... 96<o:p></o:p>CHAPTER 7<o:p></o:p>Theory of Constrained Optimization ............................... 97<o:p></o:p>7.1 Local and Global Solutions ................................. 97<o:p></o:p>7.1.1Smoothness ....................................... 98<o:p></o:p>7.2Examples .............................................. 99<o:p></o:p>VI Contents<o:p></o:p>7.3 Tangent Cone and Constraint Qualifications .................... 103<o:p></o:p>7.4First-Order Optimality Conditions ........................... 105<o:p></o:p>7.5Second-Order Conditions .................................. 106<o:p></o:p>7.6 Duality................................................ 109<o:p></o:p>7.7 KKTCondition ......................................... 112<o:p></o:p>7.8 DualProblem ........................................... 114<o:p></o:p>7.9Exercises............................................... 118<o:p></o:p>CHAPTER 8<o:p></o:p>Penalty and Augmented Lagrangian Methods ........................ 119<o:p></o:p>8.1 The Quadratic Penalty Method.............................. 119<o:p></o:p>8.2 ExactPenalty Method .................................... 122<o:p></o:p>8.3Augmented Lagrangian Method ............................. 123<o:p></o:p>8.4Quadratic Penalty Method for Hypergraph Matching ............. 125<o:p></o:p>8.4.1Hypergraph Matching ............................... 126<o:p></o:p>8.4.2Mathematical Formulation ............................ 126<o:p></o:p>8.4.3Relaxation Problem ................................. 128<o:p></o:p>8.4.4Quadratic Penalty Method for (8.21) .................... 129<o:p></o:p>8.4.5Numerical Results .................................. 130<o:p></o:p>8.5Augmented Lagrangian Method for SVM ...................... 132<o:p></o:p>8.5.1Support Vector Machine ............................. 132<o:p></o:p>8.5.2Mathematical Formulation ............................ 133<o:p></o:p>8.5.3Augmented Lagrangian Method (ALM) .................. 133<o:p></o:p>8.5.4Semismooth Newton’s Method for the Subproblem.......... 136<o:p></o:p>8.5.5Reducing the Computational Cost ...................... 137<o:p></o:p>8.5.6Convergence Result of ALM ........................... 138<o:p></o:p>8.5.7Numerical Results on LIBLINEAR...................... 139<o:p></o:p>8.6Exercises............................................... 141<o:p></o:p>CHAPTER 9<o:p></o:p>Bilevel Optimization and Its Applications ........................... 143<o:p></o:p>9.1Introduction ............................................ 143<o:p></o:p>9.2 Bilevel Model for a Case of Hyperparameter Selection in SVC ....... 145<o:p></o:p>9.2.1 AnMPEC Formulation .............................. 147<o:p></o:p>9.3 The Global Relaxation Method (GRM)........................ 148<o:p></o:p>9.4MPEC-MFCQ: A Hidden Property ........................... 149<o:p></o:p>9.5 Numerical Results........................................ 150<o:p></o:p>Bibliography................................................. 153<o:p></o:p> 21 00 06 01 https://laboutique.edpsciences.fr/produit/1376/9782759831753/modern-optimization-methods 01 00 03 02 https://laboutique.edpsciences.fr/system/product_pictures/data/009/983/047/original/9782759831746-Modern_Optimization_Methods_couv-sofedis.jpg 17 14 20240913T175127+0200 01 P15 EDP Sciences & Science Press 01 01 P15 EDP Sciences & Science Press 04 11 00 20231113 01 00 20231113 19 00 20231113 2026 06 3052868830012 01 WORLD 13 03 9782759831746 15 9782759831746 06 03 9782759831753 15 9782759831753 WORLD 08 EDP Sciences & Science Press 04 01 00 20231113 03 01 D1 EDP Sciences 31 04 05 01 02 1 00 65.99 01 5.50 3.44 EUR 04 06 01 02 1 00 231.99 01 5.50 3.44 EUR 01 04 06 01 02 1 00 255.99 01 5.50 13.35 USD 01