# Hand Computation, Conceptual Debugging, and Coding Projects

*The 3 types of problems that I would have students work out back when I was teaching ML.*

*Want to get notified about new posts? Join the mailing list.*

Back when I was teaching ML, there were 3 main types of problems that I would have students work out. Only one of those problem types involved coding.

**1) HAND COMPUTATION**

For instance: Find the new function after 1 iteration of gradient descent when fitting `f(x)=1/(1 + e^(ax+b))`

to the data `[(0,0.1), (1,0.2), (2,0.5)]`

using

- initial guess
`f(x)=1/(1 + e^(-x))`

- learning rate
`alpha=0.1`

- MSE as the loss function

**2) CONCEPTUAL DEBUGGING**

Questions in the format “suppose you make a particular mistake when coding up your algorithm. What will happen?”

For instance: What will happen if you add the gradient instead of subtracting it, i.e., you code up your gradient descent as

```
param --> param + alpha * dMSE/d(param)
```

instead of

```
param --> param - alpha * dMSE/d(param)?
```

(Answer: The loss will go up instead of down)

You can also flip these questions the other way: given the symptom, diagnose the cause.

For instance: Suppose you have a loss curve that’s going up instead of down. Which of the following could be the root cause?

- I. The gradient is being added instead of subtracted
- II. The learning rate is too small
- III. The learning rate is too big

(Answer: I and III.)

**3) CODING PROJECTS**

I’d start out having students write the code to carry out a procedure they previously worked out (an iteration of) by hand.

Afterwards, we’d tackle more complicated cases where it would be infeasible to work out even a single iteration by hand.

*Want to get notified about new posts? Join the mailing list.*