## Product Rule

Let $h(x) = f(x)g(x)$. Using first principles $$\begin{eqnarray} h\,'(x) & = & \lim_{h \rightarrow 0} \frac{h(x + h) - h(x)}{h} \\ & = & \lim_{h \rightarrow 0} \frac{f(x + h)g(x + h) - f(x)g(x)}{h} \\ & = & \lim_{h \rightarrow 0} \frac{f(x + h)g(x + h) \color{Red}{ + f(x + h)g(x) - f(x + h)g(x)} - f(x)g(x)}{h} \\ & = & \lim_{h \rightarrow 0} \frac{f(x + h) \left [ g(x + h) - g(x) \right ] + \left [ f(x + h) - f(x) \right ] g(x)}{h} \\ & = & \lim_{h \rightarrow 0} \frac{f(x + h) \left [ g(x + h) - g(x) \right ]}{h} + \lim_{h \rightarrow 0} \frac{\left [ f(x + h) - f(x) \right ] g(x)}{h} \\ & = & \lim_{h \rightarrow 0} f(x + h) \frac{g(x + h) - g(x)}{h} + g(x) \lim_{h \rightarrow 0} \frac{f(x + h) - f(x)}{h} \\ & = & f(x) \lim_{h \rightarrow 0} \frac{g(x + h) - g(x)}{h} + g(x) \lim_{h \rightarrow 0} \frac{f(x + h) - f(x)}{h} \\ & = & f(x) g\,'(x) + g(x) f\,'(x) \\ \end{eqnarray}$$

Note that there's absolutely no reason to memorize the quotient rule for $h(x) = f(x) / g(x)$ since $h(x)$ is equivalent to $f(x) g(x)^{-1}$ to which the product rule applies.

## Chain Rule

Let $h(x) = f(g(x))$. The slope at $x_0$ is $$\begin{eqnarray} \label{chain_rule_setup} h\,'(x) & = & \lim_{x \rightarrow x_0} \frac{f(g(x)) - f(g(x_0))}{x - x_0} \\ & = & \lim_{x \rightarrow x_0} \frac{f(g(x)) - f(g(x_0))}{\color{Red}{g(x) - g(x_0)}} \frac{\color{Red}{g(x) - g(x_0)}}{x - x_0}\\ & = & \lim_{x \rightarrow x_0} \frac{f(g(x)) - f(g(x_0))}{g(x) - g(x_0)} \lim_{x \rightarrow x_0} \frac{g(x) - g(x_0)}{x - x_0}\\ & = & \lim_{x \rightarrow x_0} \frac{f(g(x)) - f(g(x_0))}{g(x) - g(x_0)} g\,'(x_0)\\ \end{eqnarray}$$

Let $y = g(x)$ so $y_0 = g(x_0)$. $\lim_{x \rightarrow x_0} g(x) = g(x_0)$ or, equivalently, $\lim_{x \rightarrow x_0} y = y_0$ $$$$\lim_{x \rightarrow x_0} \frac{f(g(x)) - f(g(x_0))}{g(x) - g(x_0)} = \lim_{y \rightarrow y_0} \frac{f(y) - f(y_0)}{y - y_0} = f\,'(y_0)$$$$

which can be substituted into $\ref{chain_rule_setup}$ to give the chain rule: $$\frac{\partial}{\partial x} f(g(x)) = f\,'(y)g\,'(x)$$ where $y = g(x)$.

## Newton's Method

Use the tangent of the function at a guess of the root to determine the next guess at the root where $x_A$ is the initial guess and $x_B$ is the subsequent guess

$$$$\label{y_A} y_A = mx_A + b$$$$ $$$$\label{y_B} y_B = mx_B + b$$$$

The slope and intercept of the tangent is (obviously) equal at A and B, therefore subtracting $\ref{y_B}$ from $\ref{y_A}$ gives

$$$$y_A - y_B = m( x_A - x_B ).$$$$

$y_B = 0$, $m = f^\prime$ and $y_A = f(x_A)$, therefore

$$$$f(x_A) = f'(x_A)( x_A - x_B ).$$$$

Solving for $x_B$ (the next guess):

$$$$x_B = x_A - \frac{f(x_A)}{f'(x_A)}.$$$$

Generalized:

$$$$x_{i + 1} = x_i - \frac{f(x_i)}{f'(x_i)}.$$$$

## Separation Of Variables

Based on the premise that the solution is a product of functions that are each a function of just a single independent variable. eg, the heat equation with T dependent sink:

$$\begin{eqnarray} \frac{\partial u}{\partial t} - \frac{\partial^2 u}{\partial x^2} = -u, & u = u(x, t) \\ \end{eqnarray}$$

Let $u(x, t)$ be $f(x)g(t)$, so

$$$$f(x)g'(t) - f''(x)g(t) = -f(x)g(t)$$$$

Separate the variables

$$\begin{eqnarray} \frac{g \,'}{g} + 1 & = & \frac{f \,'\,'}{f} \end{eqnarray}$$

The LHS equals the RHS iff they equal a constant ratio $\lambda$

$$\begin{eqnarray} \frac{g \,'}{g} + 1 & = & \lambda & \Longrightarrow & g\,' + (1 - \lambda)g = 0 \\ \frac{f \,'\,'}{f} & = & \lambda & \Longrightarrow & f\,'\,' - \lambda f = 0 \\ \end{eqnarray}$$