Product Rule

Let $ h(x) = f(x)g(x) $. Using first principles $$ \begin{eqnarray} h\,'(x) & = & \lim_{h \rightarrow 0} \frac{h(x + h) - h(x)}{h} \\ & = & \lim_{h \rightarrow 0} \frac{f(x + h)g(x + h) - f(x)g(x)}{h} \\ & = & \lim_{h \rightarrow 0} \frac{f(x + h)g(x + h) \color{Red}{ + f(x + h)g(x) - f(x + h)g(x)} - f(x)g(x)}{h} \\ & = & \lim_{h \rightarrow 0} \frac{f(x + h) \left [ g(x + h) - g(x) \right ] + \left [ f(x + h) - f(x) \right ] g(x)}{h} \\ & = & \lim_{h \rightarrow 0} \frac{f(x + h) \left [ g(x + h) - g(x) \right ]}{h} + \lim_{h \rightarrow 0} \frac{\left [ f(x + h) - f(x) \right ] g(x)}{h} \\ & = & \lim_{h \rightarrow 0} f(x + h) \frac{g(x + h) - g(x)}{h} + g(x) \lim_{h \rightarrow 0} \frac{f(x + h) - f(x)}{h} \\ & = & f(x) \lim_{h \rightarrow 0} \frac{g(x + h) - g(x)}{h} + g(x) \lim_{h \rightarrow 0} \frac{f(x + h) - f(x)}{h} \\ & = & f(x) g\,'(x) + g(x) f\,'(x) \\ \end{eqnarray} $$

Note that there's absolutely no reason to memorize the quotient rule for $ h(x) = f(x) / g(x) $ since $ h(x) $ is equivalent to $ f(x) g(x)^{-1} $ to which the product rule applies.

Chain Rule

Let $ h(x) = f(g(x)) $. The slope at $ x_0 $ is $$ \begin{eqnarray} \label{chain_rule_setup} h\,'(x) & = & \lim_{x \rightarrow x_0} \frac{f(g(x)) - f(g(x_0))}{x - x_0} \\ & = & \lim_{x \rightarrow x_0} \frac{f(g(x)) - f(g(x_0))}{\color{Red}{g(x) - g(x_0)}} \frac{\color{Red}{g(x) - g(x_0)}}{x - x_0}\\ & = & \lim_{x \rightarrow x_0} \frac{f(g(x)) - f(g(x_0))}{g(x) - g(x_0)} \lim_{x \rightarrow x_0} \frac{g(x) - g(x_0)}{x - x_0}\\ & = & \lim_{x \rightarrow x_0} \frac{f(g(x)) - f(g(x_0))}{g(x) - g(x_0)} g\,'(x_0)\\ \end{eqnarray} $$

Let $ y = g(x) $ so $ y_0 = g(x_0) $. $ \lim_{x \rightarrow x_0} g(x) = g(x_0) $ or, equivalently, $ \lim_{x \rightarrow x_0} y = y_0 $ $$ \begin{equation} \lim_{x \rightarrow x_0} \frac{f(g(x)) - f(g(x_0))}{g(x) - g(x_0)} = \lim_{y \rightarrow y_0} \frac{f(y) - f(y_0)}{y - y_0} = f\,'(y_0) \end{equation} $$

which can be substituted into $ \ref{chain_rule_setup} $ to give the chain rule: $$ \frac{\partial}{\partial x} f(g(x)) = f\,'(y)g\,'(x) $$ where $ y = g(x) $.

Newton's Method

Use the tangent of the function at a guess of the root to determine the next guess at the root where $ x_A $ is the initial guess and $ x_B $ is the subsequent guess

$$ \begin{equation} \label{y_A} y_A = mx_A + b \end{equation} $$ $$ \begin{equation} \label{y_B} y_B = mx_B + b \end{equation} $$

The slope and intercept of the tangent is (obviously) equal at A and B, therefore subtracting $ \ref{y_B} $ from $ \ref{y_A} $ gives

$$ \begin{equation} y_A - y_B = m( x_A - x_B ). \end{equation} $$

$ y_B = 0 $, $ m = f^\prime $ and $ y_A = f(x_A) $, therefore

$$ \begin{equation} f(x_A) = f'(x_A)( x_A - x_B ). \end{equation} $$

Solving for $ x_B $ (the next guess):

$$ \begin{equation} x_B = x_A - \frac{f(x_A)}{f'(x_A)}. \end{equation} $$

Generalized:

$$ \begin{equation} x_{i + 1} = x_i - \frac{f(x_i)}{f'(x_i)}. \end{equation} $$

Separation Of Variables

Based on the premise that the solution is a product of functions that are each a function of just a single independent variable. eg, the heat equation with T dependent sink:

$$ \begin{eqnarray} \frac{\partial u}{\partial t} - \frac{\partial^2 u}{\partial x^2} = -u, & u = u(x, t) \\ \end{eqnarray} $$

Let $ u(x, t) $ be $ f(x)g(t) $, so

$$ \begin{equation} f(x)g'(t) - f''(x)g(t) = -f(x)g(t) \end{equation} $$

Separate the variables

$$ \begin{eqnarray} \frac{g \,'}{g} + 1 & = & \frac{f \,'\,'}{f} \end{eqnarray} $$

The LHS equals the RHS iff they equal a constant ratio $ \lambda $

$$ \begin{eqnarray} \frac{g \,'}{g} + 1 & = & \lambda & \Longrightarrow & g\,' + (1 - \lambda)g = 0 \\ \frac{f \,'\,'}{f} & = & \lambda & \Longrightarrow & f\,'\,' - \lambda f = 0 \\ \end{eqnarray} $$