Abstract
We consider the second-order cone function (SOCF) defined by , with parameters , , , and . Every SOCF is concave. We give necessary and sufficient conditions for strict concavity of . The parameters and are not uniquely determined. We show that every SOCF can be written in the form . We give necessary and sufficient conditions for the parameters , , , , and to be uniquely determined. We also give necessary and sufficient conditions for to be bounded above.
1. Introduction
Second-order cone programming is an important convex optimization problem [1–4]. A second-order cone constraint has the following form: , where is the Euclidean norm. This second-order cone constraint is equivalent to the inequality , where is what we call a second-order cone function. The solution set of the constraint is convex, and the function is concave [1, 5].
In the following definition, we use to denote the set of real numbers and to denote the set of matrices with real entries. Of course, and are positive integers.
Definition 1. A second-order cone function (SOCF) is a function that can be written aswith parameters , , , and .
In second-order cone programming, a linear function of is minimized subject to one or more second-order cone constraints, along with the constraint , where and . The solution set of is an affine subspace, and we will show that the restriction of a SOCF to an affine subspace is another SOCF. Thus, from a mathematical point of view, the constraint is not necessary, although in applications it can be convenient. In this paper, we do not consider the constraint but instead focus on understanding the family of SOCFs.
There are interior-point methods for solving second-order cone programming problems. These methods usually use SOCFs to impose the second-order cone constraints [2, 5–7]. Solvers for second-order cone programming problems include CVXOPT and MATLAB [8, 9]. The study of second-order cone programming and its applications has continued to generate interest for over three decades [3, 10–15].
The current research was started to get a deeper understanding of SOCFs to improve interior-point algorithms for finding the weighted analytic center of a system of second-order cone constraints [7, 16, 17]. The current work can lead to improved algorithms.
In this paper, we give a thorough description of the family of SOCFs. In equation (1), the parameters and are not uniquely determined, since for any orthogonal matrix . We show that every SOCF can be written in the following form:with the parameters and , and the positive semidefinite replace the parameters and . We show that these new parameters are unique if and only if is positive definite.
It is known that every SOCF is concave [1, 5]. We show that is strictly concave if and only if and , where denotes the column space of . In terms of the new parameters, the SOCF is strictly concave if and only if is positive definite and .
In the case where is positive definite, we show that is bounded above if and only if . We show that the convex set is bounded if and only if is positive definite and .
Our results have computational implications for convex optimization problems involving second-order constraints such as the problem of minimizing weighted barrier functions presented in [16, 17]. This is related to the problem of finding a weighted analytic center for second-order cone constraints given in [7]. There are also computational implications for the problem of computing the region of weighted analytic centers of a system of several second-order cone constraints. This is under investigation as part of our current research is an extension of the work given in [7].
In the problems presented in [7, 16, 17], the boundedness of the feasible region guarantees the existence of a minimizer, and the strict convexity of the barrier function guarantees the uniqueness of the minimizer. Also, the strict convexity of the barrier function affects how quickly we can find the minimizer using these algorithms. The determination of the strict concavity of is related to the strict convexity of the barrier function. The boundedness of the feasible region of the SOC constraints system is also related to the boundedness of . If a single is bounded, then the feasible region of the SOC constraints system is also bounded.
Convex optimization algorithms perform well and more efficiently when the problem is known to be bounded and the objective function is strictly convex. If a second-order cone function is strictly concave, its gradient and Hessian matrix is defined, and the Hessian is invertible. The corresponding barrier function is similarly well-behaved, and Newton’s method and Newton-based methods work well for the problem. However, many optimization problems are not bounded or have objective functions that are not strictly convex. Our results would allow one to recognize convex optimization problems involving second-order cone constraints (as in [7, 16, 17]) that can be solved efficiently, or to assist in reformulating those that are hard to solve.
2. Properties of Second-Order Cone Functions
The SOCFs of (that is, ) are the simplest to understand, and give insight into the general case.
Example 1. We consider defined by equation (1) with and . Thus, , and , so . Figure 1 shows several graphs with various values of the real parameters , , , and . If , then is smooth and strictly concave, as shown by the solid graphs. If , then is piecewise linear with a corner at , as shown by the dashed graphs. Note that for any value of , so the solid graphs in Figure 1 (with ) pass a distance of 0.2 below the corner of the dashed graphs (with ), as indicated by the double arrows.
One important property of SOCFs is that their restriction to an affine subspace is another SOCF. We will frequently restrict to a 1-dimensional affine subspace.
Remark 2. Let be written in the form of equation (1). The restriction of to the affine subspace , for some and iswhich is a SOCF on with the variable .
We recall that a function is concave provided that for all , and all . The function is strictly concave if the inequality is strict. A twice differentiable function is concave if for all , and strictly concave if the inequality is strict.
Lemma 3. Let be the general SOCF of one variable, defined by with parameters , and . The function is concave for all parameters, and is strictly concave if and only if and .
Proof. If , then is linear, and hence concave but not strictly concave.
Assume . Then, , where is the point in closest to . Let be the distance from to . Thus, by the Pythagorean theorem, and . The constant is a positive real number. The geometry is shown in Figure 2. Note that if and only if . If , then is piecewise linear with a downward bend at , and hence concave but not strictly concave.
So far, we have proved that is concave but not strictly concave if or .
Assume and . Then, , and is strictly concave, sinceis defined and negative for all .
Theorem 4. Every second-order cone function is concave. Furthermore, is strictly concave if and only if and , by using the parameters in Definition 1.
Proof. Let , and we define that . Let be defined by . It follows directly from the definition that is (strictly) concave if and only if is (strictly) concave for all . Note that . If , then is linear. If , then, we havewhere , , , and are all real numbers. Thus, is a second-order cone function of one variable. By Lemma 3, is concave for all choices of and , and hence is concave.
Since , it follows that . If , then is singular, and there exists such that and hence is linear. If , then there exists such that . Thus, and , and is piecewise linear with a downward corner. Thus, if or (or both), we can find such that is concave but not strictly concave, and hence is not strictly concave.
Now, assume and . It follows that and for all . Lemma 3 implies that is strictly concave for all , and it follows that is strictly concave.
Note that cannot satisfy and . Therefore, any SOCF with is concave but not strictly concave.
Example 2. We give four examples of SOCFs on , with different truth values of or . These SOCFs have and , so (Figure 3).(a) and yields .(b) and yields .(c) and yields .(d) and yields .
(a)
(b)
(c)
(d)
Notice that, we have frequently rewritten in terms of a square root, as shown in Examples 1 and 2. We have also noted that for any orthogonal matrix , so many different choices of and define the same SOCF. The next theorem describes a useful way to write a SOCF.
This theorem uses the Moore–Penrose Inverse of a matrix, also called the pseudoinverse, which has many interesting properties found in [18]. For example, is the least squares solution to , where is the pseudoinverse of .
The next theorem mentions the well-known fact that is a positive semidefinite matrix, which means that it is symmetric with non-negative eigenvalues. A positive definite matrix is a symmetric matrix with all positive eigenvalues. If , then is positive definite if and only if the rank of is .
Theorem 5. Every SOCF of the form is identically equal towhere is positive semidefinite, , and .
Proof. It is well-known that the least squares solution to is , and that is the orthogonal projection of onto . That is, is the point in that is closest to . Thus, the distance squared from to is the distance squared from to plus the distance squared from to . That is,The last equality uses the definitions of and . The results are as follows.
Remark 6. For , note that if and only if is positive definite. The definition of in Theorem 5 makes it clear that if and only if . Therefore, Theorem 4 implies that a SOCF written in the form of equation (6) is strictly concave if and only if is positive definite and .
Example 3. The left half of Figure 4 shows the critical point and one contour of the SOCF , withThe right part of the same figure shows the geometry behind Theorem 5, which describes how to write the function in the form . The calculations show thatThe image of the square in under is the light blue parallelogram in , shown on the right side of Figure 4. The vectors in are the first (blue) and second (red) columns of . These span the column space of in . The dot in is , and the dot in the column space is , which is the orthogonal projection of onto . The other dot in is . The distance from to is , so . The ellipse on the left is the contour of with height . The image of the ellipse under is the circle on the right, which is the set of points in the column space that are at a distance of 2 from .
The proof Theorem 5, to follow, is subtle. While it is obvious that changing one parameter will change the function , it is difficult to eliminate the possibility that more than one parameter can be changed while leaving the function unchanged. For example, with the form of equation (1), the function is unchanged when and for an orthogonal matrix . The strategy in the proof is to uniquely determine one parameter at a time in a specific order.
Theorem 7. We assume a SOCF is written in the form of equation (6), and that the same SOCF is written with possibly different parameters satisfying the same requirements, sofor all .
(i)If (the zero matrix), then , , , and is arbitrary.(ii)If , then , , , , and .As a consequence, the parameterization of a SOCF in the form of equation (6) is unique if and only if is positive definite.
Proof. We recall that are positive semidefinite. It follows that if and only if . Also, we recall that are non-negative real numbers.
For nonzero and , we consider the function and its asymptotic behavior as . If , then . If , then, we have
The third equation uses the fact that , and the fourth equation uses the Taylor series as . The fourth equation describes the slant asymptote of the graph of , and is crucial for the remainder of the proof.
For all , equation (11) implies that
Which is a similar expression where is replaced by holds. If , then . If , then the slope of the slant asymptote is the same for both sets of parameters, so again . This holds for all , so .
For all , equation (11) implies thatalong with a similar expression where is replaced by , etc. If , then there is some vector such that . This leads to a contradiction since the slope of the slant asymptote in equation (13) would be different. Thus, .
Assume . Then, , since , and . Thus, .
Assume . Then, there exists that satisfies . Using equation (13) with , we find that . At this point, we conclude that from the equality of the two expressions for , that for all . By expanding the quadratic term and canceling like terms, we find that for all . Thus, and .
Now, we show that the parameterization of is unique if and only if is positive definite. If is not positive definite, there exists such that . If is positive definite, then and is invertible, so and all of the parameters are unique.
Example 4. Let be the SOCF on defined by , , , , and . Note that is not positive definite. The null space of is . The parameterization is not unique since any yields the same SOCF.
While many choices of and in the form of equation (1) yield the same function, there is a canonical choice for and starting with the function in the form of equation (6). We recall that a positive semidefinite matrix has a unique positive semidefinite square root, denoted by .
Theorem 8. Let be positive semidefinite, , and . Then, for
The last row of is all 0s, and the last component of is .
Proof. Note that is symmetric, andThus, .
Remark 9. It follows from this theorem that any SOCF can be defined in the form of equation (1) with . While is an matrix with any , using is never needed.
We recall that any nonconstant SOCF is not bounded below, since it is concave. We give necessary and sufficient conditions for a SOCF to be bounded above with the two theorems. The next theorem assumes that is positive definite, and the case where is positive semidefinite is handled in Theorem 13.
Theorem 10. The SOCF can be written in the form of equation (6).with positive definite is bounded above if and only if .(1)If and , then is the unique critical point of , and is the global maximum value of .(2)If and , then every point in the ray is a critical point of , on which attains its maximum value of .(3)If and , then is the unique critical point of , but is not bounded above.(4)If and , then is the unique critical point of , and is the global maximum value of .(5)If and , then has no critical points and does not have a global maximum value, but is bounded above by .(6)If and , then has no critical points and is not bounded above.
Proof. To simplify the proof, we will analyze the SOCF . Note that , and , so we can easily relate the critical points, and the upper bounds, of and .
Case 1. . In this case, . Let be any nonzero vector in . Since , the function is not differentiable at . Thus, is a critical point of at which is not differentiable, and has a critical point at . To determine if has a global maximum at 0, we define by . Note that is a linear function giving the value of along a ray starting at with the direction vector . The function is bounded above if and only if the slope of is nonpositive for all directions .
Let . Note that is an ellipsoid centered at 0, since is positive definite. Furthermore, , so is bounded above if and only if the maximum value of , restricted to , is nonpositive. We compute this maximum value by using the method of Lagrange multipliers. The extreme values of restricted to occur at places where . This is equivalent to or since on . Thus, the extrema of are at , where is determined by . Thus, , so . There are two antipodal points on , , with extreme values of restricted to . We see that . The maximum value of restricted to is , which occurs at . Thus, the maximum slope of occurs when is a positive scalar multiple of , and that the maximum slope has the same sign as . Thus, is bounded above if and only if .
If , then 0 is the unique critical point of , and is the global maximum value of . Thus, is the unique critical point of , and is the global maximum value of . This proves part 1 in the theorem. If , then the linear function has a slope 0 when , and achieves its maximum value of 0 at each point on the ray from 0 through . Every point in this ray, , is a critical point. Translating this result to the original proves part 2. If , then the slope of is positive for some . Thus, has an isolated critical point at 0, but is not bounded above. This proves part 3 of the theorem.
Case 2. . The gradient of at isIn this case, is smooth, and the critical points of are solutions to . Since is positive definite, is strictly concave by Theorem 4, and has at most one critical point. If has a critical point then it must be a global maximum and hence is bounded above. We denote the critical point of as , if it exists, which satisfies . It follows that the critical point is a scalar multiple of . Let . The scalar satisfies . If , then the unique solution is , and if , then there are no solutions for . Thus, if , the function has the critical point , and the critical point of is , and a calculation of completes the proof of part 4.
We have already seen that , and therefore , has no critical points when . The results about boundedness and upper bounds need the following asymptotic analysis. When is large, then is large of order because is positive definite, and . The Taylor expansion shows that . Thus, a SOCF with is always less than the corresponding SOCF with , and the difference approaches 0 as . We now present parts 5 and 6 of the theorem which are given as follows.
Example 5. Figure 5 shows the contour diagrams of 6 SOCFs of the form , with . The eigenvalues of are , so is positive definite and Theorem 10 applies with the parameters and . A calculation shows that . The other parameters are in the top row, in the bottom row, in the left column, in the middle column, and in the right column. These values of give , and , respectively. In the top row, is always the critical point and . In the top middle figure, the contour with height 0 is the ray in the direction . In the bottom left figure, we find that and . In the bottom middle figure, is bounded above by 0.
Theorem 11. The SOCF written in the form of equation (6)with positive semidefinite is bounded above if and only if and .
Proof. Since is symmetric, the fundamental theorem of linear algebra states that the null space of is the orthogonal complement of the column space of . We can write any as for unique and . Similarly, we split . For a fixed , we write as . Thus, , and the general second-order cone function isAssume . Then, and is not bounded since is an unbounded linear function.
Assume , so . We define byNote that , restricted to is a nonsingular map, so we can apply Theorem 10 to as follows. The pseudoinverse of satisfies the following equation:Thus, the restriction of to is the inverse of the restriction of to . Theorem 10 says that is bounded above if and only if . Note that for any .
We have shown that is not bounded above if . We have also shown that if , then is bounded above if and only if . These two statements can be combined into one: is bounded above if and only if and .
Remark 12. If is posivite definite, then and . Thus, Theorem 13, in the case where is posivite definite, implies that is bounded above if and only if , which is the first part of Theorem 10.
Example 6. We consider the SOCF on with , , and asNote that is not bounded above if . If , then and as . Thus, is bounded above if and only if and .
This observation is predicted by Theorem 11. The column space of is , so is equivalent to . The pseudoinverse of is , so , and is equivalent to .
One of the main uses of SOCFs is to define convex sets for optimization problems. Optimization over a bounded set is very different from optimization over an unbounded set, so we finish this paper with a simple characterization.
Theorem 13. Let be defined by , where is positive semidefinite. Then the set is closed and convex. Assuming is not the empty set, is bounded if and only if is positive definite and .
Proof. The set is convex since is concave, and it is closed since is continuous. If is not positive definite, then Theorem 11 implies that is not bounded, since is unbounded if , and satisfies for all if . If is a positive definite, then Theorem 10 implies that is bounded if and only if .
Remark 14. In the case where is positive definite and , the compact might be trivial. Let . The set is the empty set if , is the singleton set if , and has a nonempty interior if .
3. Conclusion
The second-order cone function has important applications in optimization problems. Our work gives necessary and sufficient conditions for strict concavity of a second-order cone function. We show that every SOCF can be written in the following form: , which has unique parameters in many cases. This alternative parameterization gives a deep understanding of the family of SOCFs. This alternative description leads to new results on SOCFs. We characterize the critical points and global maxima of , depending on the parameters. We give necessary and sufficient conditions for to be bounded above, and for the set to be bounded. Our results can lead to improved algorithms for optimization problems involving second-order cone constraints.
Data Availability
The data used to support the findings of this study are included within the article.
Disclosure
The Cornell University arXiv posted a preprint of this article [19]. This research was performed as part of the employment of the authors at Northern Arizona University.
Conflicts of Interest
The authors declare that they have no conflicts of interest.