Our goal is to prove that d f d x ⃗ ( x ⃗ ∗ ) = 0 \frac{df}{d\vec{x}}(\vec{x}^*) = 0 d x df ( x ∗ ) = 0 x ⃗ ∗ = min f ( x ⃗ ) ∈ S ⊆ R n \vec{x}^* = \min f(\vec{x}) \in S \sube \mathbb{R}^n x ∗ = min f ( x ) ∈ S ⊆ R n
Since S S S
0 < ∣ ∣ Δ x ⃗ ∣ ∣ < ϵ 0 < ||\Delta \vec{x}|| < \epsilon 0 < ∣∣Δ x ∣∣ < ϵ such that x ⃗ ∗ ± Δ x ⃗ ∈ S \vec{x}^* \pm \Delta \vec{x} \in S x ∗ ± Δ x ∈ S
Then using Taylor’s theorem
f ( x ⃗ ∗ + Δ x ⃗ ) = f ( x ⃗ ∗ ) + d f d x ⃗ ( x ⃗ ∗ ) Δ x + O ( Δ x ⃗ ) f(\vec{x}^* + \Delta\vec{x})=f(\vec{x}^*)+\frac{df}{d\vec{x}}(\vec{x}^*) \Delta x+O(\Delta \vec{x}) f ( x ∗ + Δ x ) = f ( x ∗ ) + d x df ( x ∗ ) Δ x + O ( Δ x ) We know that f ( x ⃗ ∗ + Δ x ⃗ ) ≥ f ( x ⃗ ∗ ) f(\vec{x}^* + \Delta \vec{x}) \ge f(\vec{x}^*) f ( x ∗ + Δ x ) ≥ f ( x ∗ )
f ( x ⃗ ∗ ) ≤ f ( x ⃗ ∗ ) + d f d x ⃗ ( x ⃗ ∗ ) Δ x + O ( Δ x ⃗ ) 0 ≤ d f d x ⃗ ( x ⃗ ∗ ) Δ x ⃗ + O ( Δ x ⃗ ) 0 ≤ d f d x ⃗ ( x ⃗ ∗ ) Δ x ⃗ + 1 2 d 2 f d x ⃗ 2 ( x ⃗ ∗ ) Δ x ⃗ 2 + 1 6 d 3 f d x ⃗ 3 ( x ⃗ ∗ ) Δ x ⃗ 3 + ⋯ divide each side by Δ x ⃗ 0 ≤ d f d x ⃗ ( x ⃗ ∗ ) + 1 2 d 2 f d x ⃗ 2 ( x ⃗ ∗ ) Δ x ⃗ + 1 6 d 3 f d x ⃗ 3 ( x ⃗ ∗ ) Δ x ⃗ 2 + ⋯ as Δ x ⃗ → 0 , 0 ≤ d f d x ⃗ ( x ⃗ ∗ ) \begin{split}
f(\vec{x}^*) &\le f(\vec{x}^*)+\frac{df}{d\vec{x}}(\vec{x}^*) \Delta x+O(\Delta \vec{x}) \\
0 &\le \frac{df}{d\vec{x}}(\vec{x}^*) \Delta \vec{x} + O(\Delta \vec{x}) \\
0 &\le \frac{df}{d\vec{x}}(\vec{x}^*) \Delta \vec{x} + \frac{1}{2}\frac{d^2f}{d\vec{x}^2}(\vec{x}^*) \Delta \vec{x}^2+\frac{1}{6}\frac{d^3f}{d\vec{x}^3}(\vec{x}^*) \Delta \vec{x}^3+ \cdots \\
&\text{divide each side by $\Delta \vec{x}$} \\
0 &\le \frac{df}{d\vec{x}}(\vec{x}^*) + \frac{1}{2}\frac{d^2f}{d\vec{x}^2}(\vec{x}^*) \Delta \vec{x}+\frac{1}{6}\frac{d^3f}{d\vec{x}^3}(\vec{x}^*) \Delta \vec{x}^2+ \cdots \\
&\text{as } \Delta \vec{x} \rightarrow 0, \\
0 &\le \frac{df}{d\vec{x}}(\vec{x}^*)
\end{split} f ( x ∗ ) 0 0 0 0 ≤ f ( x ∗ ) + d x df ( x ∗ ) Δ x + O ( Δ x ) ≤ d x df ( x ∗ ) Δ x + O ( Δ x ) ≤ d x df ( x ∗ ) Δ x + 2 1 d x 2 d 2 f ( x ∗ ) Δ x 2 + 6 1 d x 3 d 3 f ( x ∗ ) Δ x 3 + ⋯ divide each side by Δ x ≤ d x df ( x ∗ ) + 2 1 d x 2 d 2 f ( x ∗ ) Δ x + 6 1 d x 3 d 3 f ( x ∗ ) Δ x 2 + ⋯ as Δ x → 0 , ≤ d x df ( x ∗ ) If we use the same proof for f ( x ⃗ ∗ − Δ x ⃗ ) f(\vec{x}^*-\Delta \vec{x}) f ( x ∗ − Δ x )
0 ≤ − d f d x ⃗ ( x ⃗ ∗ ) \begin{split}
0 &\le -\frac{df}{d\vec{x}}(\vec{x}^*)
\end{split} 0 ≤ − d x df ( x ∗ )