University of Hertfordshire

From the same journal

By the same authors

Differentiating through Conjugate Gradient

Research output: Contribution to journalArticlepeer-review


View graph of relations
Original languageEnglish
Pages (from-to)988-994
Number of pages7
JournalOptimization Methods and Software
Early online date26 Jan 2018
Publication statusPublished - 2 Nov 2018


We show that, although the Conjugate Gradient (CG) Algorithm has a singularity at the solution, it is possible to differentiate forward through the algorithm automatically by re-declaring all the variables as truncated Taylor series, the type of active variable widely used in Automatic Differentiation (AD) tools such as ADOL-C. If exact arithmetic is used, this approach gives a complete sequence of correct directional derivatives of the solution, to arbitrary order, in a single cycle of at most n iterations, where n is the number of dimensions. In the inexact case the approach emphasizes the need for a means by which the programmer can communicate certain conditions involving derivative values directly to an AD tool.


This is the pre-print version of an article published by Taylor & Francis in Optimization Methods and Software on 6 January 2018, available online at:

ID: 12798171