# CG -- The Method of Steepest Descent

### Definitions

![Pasted image 20221225215518.png](/img/user/attachment/Pasted image 20221225215518.png)

**Whenever you read "residual", think "direction of steepest descent"**

### Procedure

- Starting from an initial point
- compute
, which is the direction - Select next point

#### How to decide

Choose the vector which has the minimized increase of

#### Final procedure

![Pasted image 20221225220257.png](/img/user/attachment/Pasted image 20221225220257.png)

### Convergency

CG -- Eigenvalues (Eigenvectors) and convergency

#### Several Special cases

We are using the Final procedure.

##### Special Case 1

If

![Pasted image 20221228163248.png](/img/user/attachment/Pasted image 20221228163248.png)

**It takes only one step to converge to the exact solution**

##### General Formula

If

Then

##### Special case 2

If

##### Special case 3

All the eigenvectors have a common eigenvalue

#### General Convergency

We have the formula General Formula

We define energy norm to help

##### Transfer minimizing to a problem related to the energy norm

Recall for an arbitrary point

(From Conjugate Gradient >

{ #6f2cce}

)

That is

Thus, minimizing

![500](/img/user/attachment/Pasted image 20221228165621.png)

Recall

##### Express as condition number slop (we defined)

Here we only consider

We define

- the spectral condition number as
, - The slop of
as

![Pasted image 20221228170942.png](/img/user/attachment/Pasted image 20221228170942.png)

##### Plotting w.r.t. and

![Pasted image 20221228171025.png](/img/user/attachment/Pasted image 20221228171025.png)

The worst case is when

That is, the larger its condition number