7
Jul 18

Euclidean space geometry: Cauchy-Schwarz inequality

category: Matrix algebra, MT3042 Optimisation theory, Optimisation theory No Comments

Euclidean space geometry: Cauchy-Schwarz inequality

At first glance, the scalar product and the distance notion are largely independent entities. As a matter of fact, they are intimately related, and the Cauchy-Schwarz inequality provides one of the links.

Cauchy-Schwarz inequality

Statement (Cauchy-Schwarz inequality) (I) For any vectors x,y one has |{x\cdot y}|\leq \Vert x\Vert\Vert y\Vert .

(II) If the inequality sign turns into equality, |{x\cdot y}|=\Vert x\Vert\Vert y\Vert , then y is proportional to x: y=ax.

Proof. (I) If at least one of the vectors is zero, both sides of the inequality are 0 and there is nothing to prove. To exclude the trivial case, suppose that none of x,y is zero and, therefore, \Vert x\Vert,\Vert y\Vert are positive. Consider a real-valued function of a real number t defined by f(t)=\Vert tx+y\Vert^2. Here we have a norm of a linear combination f(t)=t^2\Vert x\Vert ^2+2tx\cdot y+\Vert y\Vert^2.

We see that f(t) is a parabola with branches looking upward (because the senior coefficient \Vert x\Vert^2 is positive). By nonnegativity of the squared norm, f(t)\geq 0 and the parabola lies above the horizontal axis in the (f,t) plane. Hence, the quadratic equation f(t)=0 may have at most one real root. This means that the discriminant of the equation is non-positive: D=[x\cdot y]^2-\Vert x\Vert^2\Vert y\Vert^2\leq 0. Applying square roots to both sides of [x\cdot y]^2\leq\Vert x\Vert^2\Vert y\Vert^2 we finish the proof of the first part.

(II) In case of the equality sign the discriminant is 0. Therefore the parabola touches the horizontal axis where f(t)=\Vert tx+y\Vert^2=0. But we know that this implies tx+y=0 which is just another way of writing y=ax.

Do you think this proof is tricky? During the long history of development of mathematics, mathematicians have invented many tricks, small and large. No matter how smart you are, you cannot reinvent all of them. Studying the existing tricks is a necessity. By the way, the definition of what is tricky and what is not is strictly personal and time-dependent.

Consequences of the Cauchy-Schwarz inequality

Exercise 1. Prove the triangle inequality.

Proof. From the expression for the norm of a linear combination

\Vert x+y\Vert^2=\Vert x\Vert^2+2x\cdot y+\Vert y\Vert^2           (using Cauchy-Schwarz)

\leq\Vert x\Vert^2+2\Vert x\Vert\Vert y\Vert+\Vert y\Vert^2=(\Vert x\Vert+\Vert y\Vert )^2

which gives \Vert x+y\Vert\leq\Vert x\Vert+\Vert y\Vert.

Definition. For any two nonzero vectors from the Cauchy-Schwarz inequality we have \left|\frac{x\cdot y}{\Vert x\Vert\Vert y\Vert}\right|\leq 1. Therefore we can define the cosine of the angle \widehat{x,y} between x,y by

\cos (\widehat{x,y})=\frac{x\cdot y}{\Vert x\Vert\Vert y\Vert}.

This definition agrees with the definition of orthogonality: {x\cdot y=0} means that the angle between x,y is \pi/2 and \cos (\widehat{x,y}) should be zero.

Remark. Every bit of information about scalar products, norms and the Cauchy-Schwarz is true in the infinite-dimensional case. For applications in Optimization see this post and that post.

Playing with balls

Once we have a norm, we can define the distance between x,y by \text{dist}(x,y)=\Vert x-y\Vert . For any c\in R^n and r>0 let us put B(c,r)=\{ x\in R^n:\Vert x-c\Vert <r\}. This is a set of points whose distance from c is less than r. Therefore it is a ball centered at c and with radius r.

Exercise 2. (Application of the triangle inequality) Consider two balls B(C,R) and B(c,r), where R>r, so the first ball is larger. Under what condition the smaller ball is contained in the larger ball:

(1) B(c,r)\subset B(C,R)?

Solution. There are two possible ways to solve this exercise. One is to look at the geometry first and then try to translate it to math. The other is just try to see the solution from calculations. We follow the second way.

The inclusion relationship in terms of sets (1) is equivalent to a point-wise statement: any element x of the smaller ball belongs to the larger ball. This means that we have to start with the assumption \Vert x-c\Vert<r and arrive to \Vert x-C\Vert<R. Using the triangle inequality we have

\Vert x-C\Vert=\Vert x-c+c-C\Vert\leq \Vert x-c\Vert +\Vert c-C\Vert <r+\Vert c-C\Vert.

We want \Vert x-C\Vert to be smaller than R. This is achieved if we require r+\Vert c-C\Vert\leq R.

Exercise 3. A set A\subset R^n is called open if any its element a\in A belongs to A together with some ball B(a,\varepsilon ). Prove that B(c,r) is open.

Proof. Take any a\in B(c,r). We have to produce \varepsilon such B(a,\varepsilon )\subset B(c,r). From the previous exercise we have this inclusion if \varepsilon =r-\Vert a-c\Vert.

Post Views: 104

Leave a Reply

You must be logged in to post a comment.