第 2 次作业¶

理论部分(范数与二次型)¶

习题 1¶

假设 $M , P \in \mathbb { R } ^ { n \times n }$ 为对称阵， $P$ 为正交阵，

\[ A = \left( \begin{array}{c c} M & P M \\ \hline M P & P M P \end{array} \right) \in \mathbb {R} ^ {2 n \times 2 n}. \]

(i) 证明 $A ^ { \mathrm { T } } = A$ .

(ii) 假设 $U \in \mathbb { R } ^ { m \times m } , V \in \mathbb { R } ^ { n \times n }$ 是正交矩阵, $D \in \mathbb { R } ^ { m \times n }$ , 证明 $\| U D V \| _ { 2 } = \| D \| _ { 2 } , \| U D V \| _ { F } =$ $\| D \| _ { F }$

(iii) 证明 $\| A \| _ { F } = 2 \| M \| _ { F }$ . $\| A \| _ { 2 } \leq 2 \| M \| _ { 2 }$ . ( 提示：将 $A$ 分解，并利用 (ii) 结论)

(iv) 假设 $n = 4 , M = { \mathrm { d i a g } } _ { 4 \times 4 } ( - 2 , 1 , 0 , 0 ) , P = ( e _ { 4 } | e _ { 3 } | e _ { 2 } | e _ { 1 } )$ . 证明 $\| A \| _ { p } = 2 \forall p \in [ 1 , \infty )$

解. (i) 解法 $I$ ：分解 $A$

\[ A = \left( \begin{array}{c} I \\ P \end{array} \right) M \left( \begin{array}{c c} I & P \end{array} \right) = \left( \begin{array}{c c} I & \\ & P \end{array} \right) \left( \begin{array}{c c} M & M \\ M & M \end{array} \right) \left( \begin{array}{c c} I & \\ & P \end{array} \right) = A ^ {\mathrm {T}} \]

解法 2：不分解 A

\[ A ^ {\mathrm {T}} = \left( \begin{array}{c c} M ^ {\mathrm {T}} & (M P) ^ {\mathrm {T}} \\ \hline (P M) ^ {\mathrm {T}} & (P M P) ^ {\mathrm {T}} \end{array} \right) = \left( \begin{array}{c c} M ^ {\mathrm {T}} & P ^ {\mathrm {T}} M ^ {\mathrm {T}} \\ \hline M ^ {\mathrm {T}} P ^ {\mathrm {T}} & P ^ {\mathrm {T}} M ^ {\mathrm {T}} P ^ {\mathrm {T}} \end{array} \right) = \left( \begin{array}{c c} M & P M \\ \hline M P & P M P \end{array} \right) = A \]

(ii)即证明 2 范数与 $F$ 范数满足正交不变性

对 2 范数，即证 $\| U D \| _ { 2 } = \| D \| _ { 2 } , \| D V \| _ { 2 } = \| D \| _ { 2 }$

\[ \left\| U D \right\| _ {2} = \sqrt {\lambda_ {\max } \left(D ^ {\mathrm {T}} U ^ {\mathrm {T}} U D\right)} = \sqrt {\lambda_ {\max } \left(D ^ {\mathrm {T}} D\right)} = \left\| D \right\| _ {2} \]

又因为 $\left\| D V \right\| _ { 2 } = x ^ { \mathrm { { T } } } V ^ { \mathrm { { T } } } V x = x ^ { \mathrm { { T } } } x = \| x \| _ { 2 }$

\[ \| D V \| _ {2} = \sup _ {\| x \| _ {2} = 1} \| D V x \| _ {2} = \sup _ {\| V x \| _ {2} = 1} \| D V x \| _ {2} = \| D \| _ {2} \]

对 $F$ 范数，

\[ \left\| U D V \right\| _ {F} = \sqrt {\operatorname {t r} \left(V ^ {\mathrm {T}} D ^ {\mathrm {T}} U ^ {\mathrm {T}} U D V\right)} = \sqrt {\operatorname {t r} \left(V V ^ {\mathrm {T}} D ^ {\mathrm {T}} D\right)} = \sqrt {\operatorname {t r} \left(V ^ {\mathrm {T}} V D ^ {\mathrm {T}} D\right)} = \sqrt {\operatorname {t r} \left(D ^ {\mathrm {T}} D\right)} = \| D \| _ {F} \]

(iii) 解法 1：分解 $A$ ，由于 $\left( \begin{array} { c c } { { I } } & { { } } \\ { { } } & { { P } } \end{array} \right)$ 为正交阵，由 (ii) 得

\[ \| A \| _ {F} = \left\| \left( \begin{array}{c c} M & M \\ M & M \end{array} \right) \right\| _ {F} \qquad \| A \| _ {2} = \left\| \left( \begin{array}{c c} M & M \\ M & M \end{array} \right) \right\| _ {2} \]

\[ \| A \| _ {F} ^ {2} = \| M \| _ {F} ^ {2} + \| M \| _ {F} ^ {2} + \| M \| _ {F} ^ {2} + \| M \| _ {F} ^ {2} = 4 \| M \| _ {F} ^ {2} \]

或

\[ \begin{array}{l} \| A \| _ {F} = \left\| \left( \begin{array}{c c} M & M \\ M & M \end{array} \right) \right\| _ {F} = \left\| \left( \begin{array}{c} I \\ I \end{array} \right) M \left( \begin{array}{c c} I & I \end{array} \right) \right\| _ {F} \\ = \sqrt {\operatorname {t r} \left(\left( \begin{array}{c} I \\ I \end{array} \right) M \left( \begin{array}{c c} I & I \end{array} \right) \left( \begin{array}{c} I \\ I \end{array} \right) M \left( \begin{array}{c c} I & I \end{array} \right)\right)} \\ = \sqrt {2 \operatorname {t r} \left(M \left( \begin{array}{c c} I & I \end{array} \right) \left( \begin{array}{c} I \\ I \end{array} \right) M\right)} \\ = 2 \sqrt {\operatorname {t r} (M ^ {2})} = 2 \| M \| _ {F} \\ \end{array} \]

记 $B=\left( \begin{array}{c c} M & M \\ M & M \end{array} \right),w=\left( \begin{array}{c} x \\ y \end{array} \right)$

\[\|A\|_2 = \|B\|_2 = \sup_{\|w\|_{2} \neq 0} \frac{\|Bw\|_2}{\|w\|2} = \sup_{\|w\|_2 = 1} \|Bw\|_2\]

\[ \begin{array}{l} \left\| B \left( \begin{array}{c} x \\ y \end{array} \right) \right\| _ {2} ^ {2} = \left\| \left( \begin{array}{c} M x + M y \\ M x + M y \end{array} \right) \right\| _ {2} ^ {2} \\ = 2 \| M x + M y \| _ {2} ^ {2} \\ \leqslant 2 \left(\| M x \| _ {2} + \| M y \| _ {2}\right) ^ {2} \\ \leq 4 \| M x \| _ {2} ^ {2} + 4 \| M y \| _ {2} ^ {2} \\ \leq 4 \| M \| _ {2} ^ {2} \| x \| _ {2} ^ {2} + 4 \| M \| _ {2} ^ {2} \| y \| _ {2} ^ {2} (\text {相容性}) \\ = 4 \| M \| _ {2} ^ {2} \| w \| _ {2} ^ {2} = 4 \| M \| _ {2} ^ {2} \\ \end{array} \]

因此 $\begin{array} { r } { \| A \| _ { 2 } = \operatorname* { s u p } _ { \| w \| _ { 2 } = 1 } \| B w \| _ { 2 } = 2 \| M \| _ { 2 } } \end{array}$

解法 2：不分解 $A$

\[ \begin{array}{l} \| A \| _ {F} ^ {2} = \| M \| _ {F} ^ {2} + \| M P \| _ {F} ^ {2} + \| P M \| _ {F} ^ {2} + \| P M P \| _ {F} ^ {2} = 4 \| M \| _ {F} ^ {2} \\ \| A w \| _ {2} ^ {2} = \left\| \left( \begin{array}{c} M x + P M y \\ M P x + P M P y \end{array} \right) \right\| _ {2} ^ {2} \\ = \| M x + P M y \| _ {2} ^ {2} + \| M P x + P M P y \| _ {2} ^ {2} \\ \leq \left(\| M x \| _ {2} + \| P M y \| _ {2}\right) ^ {2} + \left(\| M P x \| _ {2} + \| P M P y \| _ {2}\right) ^ {2} \\ \leq 2 \| M x \| _ {2} ^ {2} + 2 \| P M y \| _ {2} ^ {2} + 2 \| M P x \| _ {2} ^ {2} + 2 \| P M P y \| _ {2} ^ {2} \\ \leq 2 \| M \| _ {2} ^ {2} \| x \| _ {2} ^ {2} + 2 \| P M \| _ {2} ^ {2} \| y \| _ {2} ^ {2} + 2 \| M P \| _ {2} ^ {2} \| x \| _ {2} ^ {2} + 2 \| P M P \| _ {2} ^ {2} \| y \| _ {2} ^ {2} \\ = 2 \| M \| _ {2} ^ {2} \| x \| _ {2} ^ {2} + 2 \| M \| _ {2} ^ {2} \| y \| _ {2} ^ {2} + 2 \| M \| _ {2} ^ {2} \| x \| _ {2} ^ {2} + 2 \| M \| _ {2} ^ {2} \| y \| _ {2} ^ {2} \\ = 4 \| M \| _ {2} ^ {2} \left(\| x \| _ {2} ^ {2} + \| y \| _ {2} ^ {2}\right) \\ = 4 \| M \| _ {2} ^ {2} \| w \| _ {2} ^ {2} \\ \end{array} \]

(iv)

\[ M = \left( \begin{array}{c c c c} - 2 & & & \\ & 1 & & \\ & & 0 & \\ & & & 0 \end{array} \right) P = \left( \begin{array}{c c c c} & & & 1 \\ & & 1 & \\ & 1 & & \\ 1 & & & \end{array} \right) A = \left( \begin{array}{c c c c c c c c} - 2 & & & & & & & 0 \\ & 1 & & & & & 0& \\ & & 0 & & & 1 & & \\ & & & 0 & - 2 & & & \\ \hline & & & - 2 & 0 & & & \\ & & 1 & & & 0 & & \\ & 0 & & & & & 1 & \\ 0 & & & & & & & -2 \end{array} \right) \]

\[ \begin{array}{l} \| A x \| _ {p} ^ {p} = \left\| (- 2 x _ {1}, x _ {2}, x _ {6}, - 2 x _ {5}, - 2 x _ {4}, x _ {3}, x _ {7}, - 2 x _ {8}) ^ {\mathrm {T}} \right\| _ {p} ^ {p} \\ = \left| - 2 x _ {1} \right| ^ {p} + \left| x _ {2} \right| ^ {p} + \left| x _ {6} \right| ^ {p} + \left| - 2 x _ {5} \right| ^ {p} + \left| - 2 x _ {4} \right| ^ {p} + \left| x _ {3} \right| ^ {p} + \left| x _ {7} \right| ^ {p} + \left| - 2 x _ {8} \right| ^ {p} \\ = 2 ^ {p} \left| x _ {1} \right| ^ {p} + \left| x _ {2} \right| ^ {p} + \left| x _ {3} \right| ^ {p} + 2 ^ {p} \left| x _ {4} \right| ^ {p} + 2 ^ {p} \left| x _ {5} \right| ^ {p} + \left| x _ {6} \right| ^ {p} + \left| x _ {7} \right| ^ {p} + 2 ^ {p} \left| x _ {8} \right| ^ {p} \\ \leq 2 ^ {p} \left(\left| x _ {1} \right| ^ {p} + \left| x _ {2} \right| ^ {p} + \left| x _ {3} \right| ^ {p} + \left| x _ {4} \right| ^ {p} + \left| x _ {5} \right| ^ {p} + \left| x _ {6} \right| ^ {p} + \left| x _ {7} \right| ^ {p} + \left| x _ {8} \right| ^ {p}\right) \\ = 2 ^ {p} \| x \| _ {p} ^ {p}. \\ \end{array} \]

因此 $\| A \| _ { p } = \operatorname* { s u p } _ { \| x \| _ { p } = 1 } \| A x \| _ { p } = 2$

(即如果一个变换只将某些维度倍乘并交换顺序，不作维度间相加的操作，那矩阵范数即最大拉伸倍数)

习题 2¶

假设 $P \in \mathbb { R } ^ { n \times n } \backslash \{ 0 \}$ 是一个投影矩阵.

(i) 证明 $P y = y ,\forall y \in \mathcal { R } ( P )$ . $P x - x \in { \mathcal { N } } ( P ) \forall x \in \mathbb { R } ^ { n } .$

( 即证明投影 $P$ 沿着零空间 $\mathcal { N } ( P )$ 投影到列空间 $\mathcal { R } ( P )$ )

(ii) 证明 $P$ 的特征值 $\lambda \in \Lambda ( P ) \subseteq \{ 0 , 1 \}$ . 假设 $\mathcal { R } ( P ) = \operatorname { s p a n } \left( u _ { 1 } , \ldots , u _ { r } \right) , \mathcal { N } ( P ) = \operatorname { s p a n } \left( v _ { r + 1 } , \ldots , v _ { n } \right)$ ，试找到 $P$ 的特征分解 $P = X D X ^ { - 1 }$ 并证明 $\operatorname { t r } ( P ) = \operatorname { r a n k } ( P )$ . ( 提示：利用 (i) 结论. )

(iii) 证明当 $P \neq I _ { n }$ ， $\operatorname* { d e t } ( P ) = 0$ .

(iv) 证明当 $P$ 是正交投影矩阵（ $P ^ { 2 } = P = P ^ { \mathrm { T } } $ ）时， $I _ { n } - 2 P$ 是正交矩阵.

(v) 假设 $A \ \in \ \mathbb { R } ^ { n \times m }$ ， $m \leq n$ ， $\operatorname { r a n k } ( A ) = m$ . $P = A \left( A ^ { \mathrm { T } } A \right) ^ { - 1 } A ^ { \mathrm { T } }$ 证明 $P$ 是正交投影矩阵，$\operatorname { r a n k } ( P ) = m$ . ( 提示：利用 (ii) 结论. )

解. $( i ) \forall y \in { \mathcal { R } } ( P )$ 即对 $x \in \mathbb { R } ^ { n }$ , $y = P x$ , $P y = P ^ { 2 } x = P x = y .$

\[ \forall x \in \mathbb {R} ^ {n}, P (P x - x) = P ^ {2} x - P x = P x - P x = 0. \]

(ii) 对 $\lambda \in \Lambda ( P ) , x \in \mathbb { R } ^ { n } \backslash \{ 0 \} ,$ , 有 $P x = \lambda x$ . 由于 $P = P ^ { 2 }$ , $\lambda x = P x = P ( P x ) = P ( \lambda x ) = \lambda P x =$ $\lambda ^ { 2 } x$ . 因为 $x \neq 0 \in \mathbb { R } ^ { n }$ , 故 $\lambda = \lambda ^ { 2 }$ , $\lambda \in \{ 0 , 1 \}$ . 因此 $\Lambda ( P ) \subseteq \{ 0 , 1 \}$ .

由 $( i )$ 可知，$\forall i =1,...r,u_i \in { \mathcal { R } } ( P ),Pu_i=u_i. \forall j =r+1,...n,v_j \in { \mathcal { N } } ( P ),Pv_j=0.$

故令 $X : = ( u _ { 1 } | \cdots | u _ { r } | v _ { r + 1 } | \cdots | v _ { n } ) \in \mathbb { R } ^ { n \times n }$ $, D \ : = \mathrm { d i a g } _ { n \times n } ( 1 , \ldots , 1 , 0 \ldots , 0 ) \ \in \ \mathbb { R } ^ { n \times n }$ ，此时

\[ P = X D X ^ {- 1} \]

（注：也可理解为 SVD(后续课程会讲)，即 $P = U D V ^ { \mathrm { T } } , U \in \mathcal { R } ( P ) , V \in \mathcal { N } ( P ) )$ ）

\[ \operatorname {t r} (P) = \operatorname {t r} \left(X D X ^ {- 1}\right) = \operatorname {t r} (D) = r. \]

(iii) 反证 $\operatorname* { d e t } ( P ) \neq 0 \Longrightarrow P = I _ { n }$ . 由于 $\operatorname* { d e t } ( P ) \neq 0 ,$ $P$ 可逆. 故由 $P ^ { 2 } = P$ , 得 $P ^ { - 1 } P ^ { 2 } = P ^ { - 1 } P$ ,$P = I _ { n }$ .

(iv) 由于 $P$ 是正交投影矩阵, $P ^ { 2 } = P = P ^ { \mathrm { T } }$ . 令 $Q : = I _ { n } - 2 P , Q ^ { \mathrm { T } } = I _ { n } - 2 P ^ { \mathrm { T } } = Q \ , Q ^ { 2 } =$ $I _ { n } - 4 P + 4 P ^ { 2 } = I _ { n }$ . 因此, $Q ^ { \mathrm { T } } Q = Q Q ^ { \mathrm { T } } = I _ { n }$ .

$( \nu ) P ^ { 2 } = A \left( A ^ { \mathrm { T } } A \right) ^ { - 1 } A ^ { \mathrm { T } } A \left( A ^ { \mathrm { T } } A \right) ^ { - 1 } A ^ { \mathrm { T } } = A \left( A ^ { \mathrm { T } } A \right) ^ { - 1 } A ^ { \mathrm { T } } = P$

\[ P ^ {\mathrm {T}} = A \left(\left(A ^ {\mathrm {T}} A\right) ^ {- 1}\right) ^ {\mathrm {T}} A ^ {\mathrm {T}} = A \left(\left(A ^ {\mathrm {T}} A\right) ^ {\mathrm {T}}\right) ^ {- 1} A ^ {\mathrm {T}} = A \left(A ^ {\mathrm {T}} A\right) ^ {- 1} A ^ {\mathrm {T}} = P. \]

由 (ii) $, \operatorname { r a n k } ( P ) = \operatorname { t r } ( P ) = \operatorname { t r } ( A \left( A ^ { T } A \right) ^ { - 1 } A ^ { T } ) = \operatorname { t r } ( \left( A ^ { T } A \right) ^ { - 1 } A ^ { T } A ) = \operatorname { t r } ( I _ { m } ) = m .$

习题 3¶

求向量 $( 1 , 1 , 1 ) ^ { T }$ 投影到一维子空间 $s p a n \{ ( 1 , - 1 , 1 ) ^ { T } \}$ 的正交投影。

解. 首先求得投影矩阵

\[ P _ {\pi} = \frac {1}{3} \left( \begin{array}{c c c} 1 & - 1 & 1 \\ - 1 & 1 & - 1 \\ 1 & - 1 & 1 \end{array} \right) \]

向量 $( 1 , 1 , 1 ) ^ { T }$ 投影到一维子空间 $s p a n \{ ( 1 , - 1 , 1 ) ^ { T } \}$ 的正交投影为

\[ P _ {\pi} \left( \begin{array}{c} 1 \\ 1 \\ 1 \end{array} \right) = \frac {1}{3} \left( \begin{array}{c c c} 1 & - 1 & 1 \\ - 1 & 1 & - 1 \\ 1 & - 1 & 1 \end{array} \right) \left( \begin{array}{c} 1 \\ 1 \\ 1 \end{array} \right) = \frac {1}{3} \left( \begin{array}{c} 1 \\ - 1 \\ 1 \end{array} \right) \]

习题 4¶

证明：正交矩阵和范数有关的性质：如果矩阵${U}\in \mathbb{R}^{m\times m}, {V}\in \mathbb{R}^{n\times n}$是正交矩阵，${M}\in\mathbb{R}^{m\times n}$，${x}\in\mathbb{R}^m$

$\left\lVert{U}\right\rVert_2 = 1$，$\left\lVert{U}\right\rVert_F = \sqrt{m}$
$\left\lVert{U}{x}\right\rVert_2=\left\lVert{x}\right\rVert_2$
$\left\lVert{U}{M}{V}\right\rVert_2=\left\lVert{M}\right\rVert_2$，$\left\lVert{U}{M}{V}\right\rVert_F=\left\lVert{M}\right\rVert_F$

解

因为 $U^T U = I$，故结论显然成立；
$\|Ux\|_2 = \sqrt{x^T U^T U x} = \sqrt{x^T I x} =\|X\|_2$；
$\|UMV\|_2 = \sqrt{V^T M^T U^T U M V} = \sqrt{V^T M^T M V} = \|V^T M^T M V\|_2$，由于 $V$ 是正交矩阵，故 $V^T M^T M V$ 与 $M^T M$ 具有相同的特征值，因此 $\|V^T M^T M V\|_2 = \| M^T M \|_2$；同理可得 $\left\lVert{U}{M}{V}\right\rVert_F=\left\lVert{M}\right\rVert_F$。