Hands-On Mathematics for Deep Learning
上QQ阅读APP看书,第一时间看更新

First and second derivatives

Now that we know how to find the derivative of a function, it is important to know that we can take the derivative more than once. 

The first derivative, as we know, gives us the gradient (slope of the tangent line) of a function at any given point (x) on the curve—in other words, whether the curve's altitude (that is, y or f(x)) is increasing or decreasing. A positive slope tells us f(x) is increasing as x increases and a negative slope tells us f(x) is decreasing as x increases, and a slope of 0 tells us nothing about the curve's direction, other than that it is likely at a turning point (local minimum or local maximum). This can be written as follows: 

  • If , then f(x) is increasing at x = t.
  • If , then f(x) is decreasing at x = t.
  • If , then x=t is a critical point of f(x).

For example, let . The derivative of this function is shown here:

At x = 0, the derivative is 9, which tells us the function is increasing at this point. But at x = 1 the derivative is -3 telling us that the function is decreasing at this point. 

The second derivative is the derivative of the derivative of the function. We write this as  or . As before, where the first derivative told us whether the function was increasing or decreasing, the second derivative gives us the same information about the first derivative—whether it is increasing or decreasing. 

If the second derivative is positive, then as x increases, the first derivative is increasing; and if the second derivative is negative, then as x increases, the first derivative is decreasing. 

To help us visualize this, when the second derivative is positive, the curve is concave up (parabola open upward) at a point, whereas when it is negative, the curve is concave down (parabola open downward). And as before, when the second derivative is equal to zero, we learn nothing new. This point could be a local maximum, a local minimum, or an inflection point. This is written as follows: 

  • If , then f(x) is concave up at x=t.
  • If , then f(x) is concave down at x=t.
  • If , then at x=t we obtain no new information about f(x).

For example, let's take the second derivative of the same function we used, as follows:

At x = 0, the second derivative is -24, which tells us the function is concave down here. But at x = 2, it is equal to 24, telling us the function is concave up.

Earlier, we learned that when x is a critical point of a function we learn nothing new about the function at that point, but we can use it to find out whether it is a local maximum or a local minimum. These rules can be written as follows:

  • If  and , then f(x) has a local minimum at x=t.
  • If  and , then f(x) has a local maximum at x=t.
  • If  and , then at x=t we learn nothing new about f(x).