check numerical stability properties when dG/d(u,p) is highly stiff

[gszep]

The implicit layers approach to calculating gradients requires solving a linear system with A::Matrix \ x::Vector where A is the Jacobian dG/d(u,p) where G = [ F, det(J) ]. Let's make a plot of the percentage error between finite difference and autodiff as the condition number for A is increased. @rveltz I use the same linear solvers as you, do you know what happens when your matrix J = dF/du is ill-conditioned?

[gszep]

problems with adjoints https://arxiv.org/abs/2103.15341

[rveltz]

The continuation steps through it most likely. This solver is well described here:
Keller, Herbert B. “The Bordering Algorithm and Path Following Near Singular Points of Higher Nullity.” SIAM Journal on Scientific and Statistical Computing 4, no. 4 (December 1983): 573–82. https://doi.org/10.1137/0904039.