using Symbolics

, LinearAlgebra

, PlutoUI

, ForwardDiff

, Zygote

1.9 s

TableOfContents(title="Differentiating the Determinant and the Inverse", indent=true, depth=4, aside=true)

19.1 μs

Math: The gradient of the determinant

md"""

# Math: The gradient of the determinant

"""

152 μs

Theorem: $\nabla (det A) = cofactor (A) = (det A) A^{- T} = adj (A^{T})$
$d (det A) = tr (det (A) A^{- 1} d A) = tr(adj (A) d A) = tr(cofactor (A)^{T} d A)$
Note:the adjugate is defined as $(det A) A^{- 1}$ ; it is the transpose of the cofactor.

md"""

Theorem:

``\nabla(\det A) = \text{cofactor}(A) = (\det A)A^{-T} = \text{adj}(A^T)`` $(br)

``d(\det A) = \text{tr}( \det(A)A^{-1}dA) = \text{tr(adj}(A) dA) = \text{tr(cofactor}(A)^T dA)`` $(br)

Note:the adjugate is defined as ``(\det A)A^{-1}``; it is the transpose of the cofactor.

"""

228 μs

adj (generic function with 1 method)

adj(A) = det(A) * inv(A) # adjugate function

246 μs

cofactor (generic function with 1 method)

cofactor(A) = adj(A)'

231 μs

Formulas at a glance:
$A^{- 1}$ = adj( $A$ )/det( $A$ ) = cofactor( $A$ ) $^{T}$ /det( $A$ )
adj( $A$ ) = det( $A$ ) $A^{- 1}$ = cofactor( $A$ ) $^{T}$
cofactor( $A$ ) = $A^{- T}$ /det( $A$ ) = adj( $A$ ) $^{T}$

md"""

**Formulas at a glance:** $(br)

``A^{-1}`` = adj(``A``)/det(``A``) = cofactor(``A``)``^T``/det(``A``) $(br)

adj(``A``) = det(``A``)``A^{-1}`` = cofactor(``A``)``^T`` $(br)

cofactor(``A``) = ``A^{-T}``/det(``A``) = adj(``A``)``^{T}``

"""

352 μs

The adjugate is the transpose of the cofactor matrix. You may remember that the cofactor matrix has as (i,j) entry $(- 1)^{i + j}$ times the determinant obtained by deleting row i and column j.

md"""

The adjugate is the transpose of the cofactor matrix.

You may remember that the cofactor matrix has as (i,j) entry ``(-1)^{i+j}`` times the determinant obtained by deleting row i and column j.

"""

164 μs

Symbolics.Num

$a$

$b$

$c$

$d$

@variables a b c d

868 ns

$[\begin{array}{cc} a & c \\ b & d \end{array}]$

M = [ a c;b d]

1.1 μs

$[\begin{array}{cc} d & - b \\ - c & a \end{array}]$

simplify.(cofactor(M))

8.5 ms

$[\begin{array}{cc} d & - c \\ - b & a \end{array}]$

simplify.(adj( M))

7.8 ms

$[\begin{array}{cc} \frac{d}{a d - b c} & \frac{- c}{a d - b c} \\ \frac{- b}{a d - b c} & \frac{a}{a d - b c} \end{array}]$

simplify.(inv(M))

5.2 ms

Numerical Demonstration

md"""

## Numerical Demonstration

"""

5.3 ms

5×5 Matrix{Float64}:
 6.23738e-6  9.80317e-6  9.49748e-6  2.16775e-6  7.13224e-6
 4.29949e-7  1.04401e-6  9.43453e-6  3.49108e-6  3.30058e-6
 2.43275e-6  6.93099e-6  9.77647e-6  4.73139e-6  1.52485e-6
 4.08123e-6  5.79363e-6  5.63512e-6  9.05209e-6  5.3378e-6
 3.20542e-7  9.13855e-6  1.20064e-6  2.504e-6    3.50599e-6

A = rand(5,5); dA = rand(5,5) * .00001

1.3 μs

-1.88723e-6

-1.88722e-6

det(A+dA) - det(A), tr( adj(A) * dA )

13.6 μs

Forward Diff Autodiff

md"""

## Forward Diff Autodiff

"""

137 μs

5×5 Matrix{Float64}:
  0.227566    -0.231878    0.00127616   0.0537115   -0.113963
 -0.441303     0.453903    0.109316    -0.188827     0.132947
  0.0485198   -0.0217404  -0.0335765    0.068198    -0.0835962
 -0.00220219  -0.0833639   0.0166832    0.0266121   -0.0155698
  0.124369    -0.124389   -0.105723     0.00347312   0.0503901

ForwardDiff.gradient(A->det(A), A)

11.4 μs

5×5 Matrix{Float64}:
  0.227566    -0.231878    0.00127616   0.0537115   -0.113963
 -0.441303     0.453903    0.109316    -0.188827     0.132947
  0.0485198   -0.0217404  -0.0335765    0.068198    -0.0835962
 -0.00220219  -0.0833639   0.0166832    0.0266121   -0.0155698
  0.124369    -0.124389   -0.105723     0.00347312   0.0503901

adj(A')

6.9 μs

Reverse Mode Autodiff

md"""

## Reverse Mode Autodiff

"""

135 μs

Zygote does reverse ad

md"""

Zygote does reverse ad

"""

163 μs