Convexity Intuition

A function is convex if a straight line between any two points on its curve never dips below the curve itself — visually, it "curves upward" everywhere, like x². Convexity is the property that makes gradient descent trustworthy: on a convex function, ANY local minimum you find via gradient descent is guaranteed to be THE global minimum — no risk of getting stuck in a worse local dip.

def is_convex_numerically(f, a, b, c):
    line_at_b = f(a) + (f(c) - f(a)) * (b - a) / (c - a)
    return f(b) <= line_at_b

This checks the definition directly for one triple of points: if f(b) sits below (or on) the straight line from (a, f(a)) to (c, f(c)), that's consistent with convexity at that point.

For a single-variable function, there's a cleaner test: convex exactly where the second derivative is non-negative (curving upward, not downward). The second derivative is just the derivative of the derivative — numerically, extend the central-difference idea one step further:

def second_derivative(f, x, h=1e-4):
    return (f(x + h) - 2*f(x) + f(x - h)) / h**2

For f(x) = x², the second derivative is the constant 2 everywhere — that's why x² is convex EVERYWHERE, not just at some points. Functions can also be convex only in certain regions (like x³ for x > 0) — that's exactly why gradient descent's global-minimum guarantee only holds when the WHOLE function is convex, not just near where you started.