The simplest method is to use finite difference approximations. A simple two-point estimation is to compute the slope of a nearby secant line through the points and. Choosing a small number h, h represents a small change in x, and it can be either positive or negative. The slope of this line is This expression is Newton's difference quotient. The slope of this secant line differs from the slope of the tangent line by an amount that is approximately proportional to h. As h approaches zero, the slope of the secant line approaches the slope of the tangent line. Therefore, the true derivative offatx is the limit of the value of the difference quotient as the secant lines get closer and closer to being a tangent line: Since immediately substituting 0 for h results in indeterminate form, calculating the derivative directly can be unintuitive. Equivalently, the slope could be estimated by employing positions and x. Another two-point formula is to compute the slope of a nearby secant line through the points and. The slope of this line is This formula is known as the symmetric difference quotient. In this case the first-order errors cancel, so the slope of these secant lines differ from the slope of the tangent line by an amount that is approximately proportional to. Hence for small values of h this is a more accurate approximation to the tangent line than the one-sided estimation. However, although the slope is being computed at x, the value of the function at x is not involved. The estimation error is given by where is some point between and. This error does not include the rounding error due to numbers being represented and calculations being performed in limited precision. The symmetric difference quotient is employed as the method of approximating the derivative in a number of calculators, including TI-82, TI-83, TI-84, TI-85, all of which use this method with h = 0.001.
Step size
An important consideration in practice when the function is calculated using floating-point arithmetic is the choice of step size, h. If chosen too small, the subtraction will yield a large rounding error. In fact, all the finite-difference formulae are ill-conditioned and due to cancellation will produce a value of zero ifh is small enough. If too large, the calculation of the slope of the secant line will be more accurately calculated, but the estimate of the slope of the tangent by using the secant could be worse. For the numerical derivative formula evaluated at x and x + h, a choice for h that is small without producing a large rounding error is , where the machine epsilonε is typically of the order of 2.2. A formula for h that balances the rounding error against the secant error for optimum accuracy is , and to employ it will require knowledge of the function. For single precision the problems are exacerbated because, although x may be a representable floating-point number, x + halmost certainly will not be. This means that x + h will be changed to a nearby machine-representable number, with the consequence that − x will not equal h; the two function evaluations will not be exactly h apart. In this regard, since most decimal fractions are recurring sequences in binary a seemingly round step such as h = 0.1 will not be a round number in binary; it is 0.000110011001100...2 A possible approach is as follows: h := sqrt * x; xph := x + h; dx := xph - x; slope := - F) / dx; However, with computers, compiler optimization facilities may fail to attend to the details of actual computer arithmetic and instead apply the axioms of mathematics to deduce that dx and h are the same. With C and similar languages, a directive that xph is a volatile variable will prevent this.
Other methods
Higher-order methods
Higher-order methods for approximating the derivative, as well as methods for higher derivatives, exist. Given below is the five-point method for the first derivative : where. For other stencil configurations and derivative orders, the is a tool that can be used to generate derivative approximation methods for any stencil with any derivative order.
Higher derivatives
Using Newton's difference quotient, the following can be shown :
Complex-variable methods
The classical finite-difference approximations for numerical differentiation are ill-conditioned. However, if is a holomorphic function, real-valued on the real line, which can be evaluated at points in the complex plane near, then there are stable methods. For example, the first derivative can be calculated by the complex-step derivative formula: This formula can be obtained by Taylor series expansion: The complex-step derivative formula is only valid for calculating first-order derivatives. A generalization of the above for calculating derivatives of any order employ multicomplex numbers, resulting in multicomplex derivatives. In general, derivatives of any order can be calculated using Cauchy's integral formula: where the integration is done numerically. Using complex variables for numerical differentiation was started by Lyness and Moler in 1967. A method based on numerical inversion of a complex Laplace transform was developed by Abate and Dubner. An algorithm that can be used without requiring knowledge about the method or the character of the function was developed by Fornberg.
Differential quadrature
Differential quadrature is the approximation of derivatives by using weighted sums of function values. The name is in analogy with quadrature, meaning numerical integration, where weighted sums are used in methods such as Simpson's method or the Trapezoidal rule. There are various methods for determining the weight coefficients. Differential quadrature is used to solve partial differential equations.
