Having an input-output-gradient dataset, I’d like to build an input-output model, e.g. PCE or GP. Is it possible to do this with the current version of OpenTURNS, directly or indirectly (for exemple by fitting the OT PCE outside OT from the function computing its outputs, the function computing its derivatives, the database and elasticnet regression)? If not, is it in the roadmap to implement gradient-enhanced surrogate models?
1 Like
Hi!
No, it is not available in OT up to my knowledge. As far as I remember, you are the first one to require this feature. This is because we do not get the gradient if the code very often.
The easy part to implement that based on the PCE is to define the least squares problem (assuming we want to use LS) and to solve it. The difficult part is to express the gradient of the PCE depending on its parameters. It might not be as difficult as it seems since the lib knows how to evaluate the gradient of polynomials. Then the LS problem has 2n equations : the first n for the function values and the last ones for the derivatives.
Do you know of bibliographic references on this topic?
Regards,
Michaël
Hello Michael,
Sorry, I completely forgot to follow up this thread!
Some colleagues have told me about the idea you’re talking about (include the derivatives information in the linear system of the LS problem), but I can’t think of any articles.
However, I remember an article I read a few months ago about gradient-enhanced sparse Poincaré chaos expansion (authors: Nora Lüthen, Olivier Roustant, Fabrice Gamboa, Bertrand Iooss, Stefano Marelli, Bruno Sudret). In the introduction, you will find a paragraph with many references which could be a good starting point. I’ll try to read them in September.
In some practical situations, partial derivatives of the model output with respect to each input are easily accessible, for example by algorithmic differentiation of the numerical model in the reverse (adjoint) mode (Griewank and Walther, 2008). This technique allows for computing all partial derivatives of the model output at a cost independent of the number of input variables. Since PCEs are such a well-established metamodelling tool, there have been many efforts to leverage the additional information contained in model derivatives to improve the performance of PCE. The idea of including derivative information into sparse regression problems, often called gradient-enhanced ℓ1-minimization, is tested by Jakeman et al. Jakeman et al. (2015) for one numerical example with uniform inputs, and analyzed theoretically and numerically by Peng et al. Peng et al. (2016) for Hermite PCE. Both report favorable results. Roderick et al. (Roderick et al., 2010) and Li et al. (Li et al., 2011) apply polynomial regression (PCE) in the context of nuclear engineering. They include derivative information into the least-squares regression formulation and observe that most polynomial families are not orthogonal with respect to the H1 inner product. This may deteriorate certain properties of the regression matrix. To alleviate this issue, Guo et al. Guo et al. (2018) develop a preconditioning procedure for gradient-enhanced sparse regression with certain polynomial families, with the goal of improving the orthogonality properties of the regression matrix. In all these approaches, the utilization of derivative information is not straightforward, but requires specific polynomial families and/or specialized sampling and preconditioning, because the partial derivatives of a PCE basis do in general not form an orthogonal system. Gejadze et al. (Gejadze et al., 2019) have derived derivative-enhanced projection methods to compute the PCE coefficients but their method is restricted to Hermite polynomials and low polynomial degree.