Introducing an AI-Driven UQ and Sensitivity Analysis App Built with OpenTURNS

Announcements
,
1

Dear OpenTURNS Community,

I wanted to share with you a new web application that I’ve developed during my Engineering PhD at the University of Warwick. This app aims to simplify uncertainty quantification (UQ) and Sensitivity Analysis (SA) for a wide range of models, and I’m excited to share it with anyone who may find it useful.

Try it here: (https://uncertaintycat.streamlit.app) → Best to copy-paste the URL into a new tab as the redirection here is working not as expected.

What Does the App Do?

  • Input Your Own Models: The app allows you to input your own mathematical models directly into an integrated code editor. You can define your models in Python code and there’s no need to write additional code for UQ and sensitivity analyses.

  • Analyses Powered by OpenTURNS: Once your model is entered, the app performs a suite of UQ and sensitivity analyses using OpenTURNS, including:

    • Monte Carlo Simulations
    • Sobol Indices
    • Morris Method
    • Taylor Expansion
    • Expectation Convergence Analysis
    • Correlation Analysis
    • HSIC Analysis

  • AI-Driven Interpretation: A standout feature is the AI-driven interpretation of results. The app integrates advanced language models (defaulting to Google’s Gemma 2) to provide insightful explanations and recommendations based on your analysis results.

Benchmark Examples Included

To help you get started, I’ve included several well-known benchmark cases within the app, many of which are familiar to the OpenTURNS community:

  • Borehole Function
  • Morris Function
  • Damped Oscillator
  • Truss Model
  • Undamped Oscillator

Why Did I Create This App?

During my PhD research, I frequently needed to perform UQ on various models without the overhead of constantly creating new code and re-interpreting the results for my sponsor company. I developed this app to streamline that process, making it easier for anyone to run UQ and sensitivity analyses on their models quickly and efficiently.

Key Features

  • User-Friendly Interface: Paste or write your model code directly into the app’s editor or modify one of the existing examples. The app handles the rest, running analyses and generating results without additional coding. Please check the readme on how to define custom models, but hopefully, it is somewhat intuitive. The included examples demonstrate how to set up models using OpenTURNS.
  • Powered by OpenTURNS: The backend utilizes OpenTURNS extensively.
  • AI Integration for Insights: The app doesn’t just crunch numbers—it interprets them. By integrating AI for results analysis and interpretation, it provides insights and recommendations (with the usual caveats of AI-generated content), improving your understanding of the model’s behavior under uncertainty.
  • Free and Accessible: The app is completely free to use. I’m hosting it on Streamlit and maintaining it, and I’m considering making it open source if there’s enough interest from the community? Also, I am using a free-tier access to the LLMs so there can be usage limits - let me know if that is the case?

Get Involved

The idea here is to make UQ more accessible and insightful for wide audiences, especially those who are new to OpenTURNS or UQ in general. Whether you’re a researcher, practitioner, or just curious, I hope this tool proves useful in your work.

Feel free to reach out with any questions, suggestions, or if you’re interested in contributing.

Thank you for your time, and I hope this can help!

1 Like
2

Nice work, curious to see the api being used, could you share the sources ?

3

Hi and welcome on this forum!
I could make some tests: the results are impressive.
I noticed some bugs: is there any place where I can submit them?
Regards,
Michaël

4

hi @schueller @MichaelBaudin

Thanks for your messages!

Here is the repository: [GitHub - MLegkovskis/UncertaintyCat: UncertaintyCat is an interactive application designed to make Uncertainty Quantification (UQ) accessible to everyone! Whether you're an engineer, data scientist, or researcher, UncertaintyCat simplifies complex UQ techniques, providing AI-driven insights and a user-friendly interface.]

I would really welcome your contributions (just open PR into the main branch) - happy to add you as maintainers on the project - lmk. It is getting a tiny bit of traction with daily users. And I would really appreciate your help on the actual “UQ-side-of-things”, as my experience is on the application side and I have definitely made some mistakes in the actual use of OpenTURNS for some analyses etc so @MichaelBaudin if you could raise a PR for that bug you observed, would be greatly appreciated.

In terms of the API calls - it is very straightforward:

For example, when performing Sobol Sensitivity Analysis, the app generates a prompt based on the analysis results and sends it to the Groq API to receive an interpretation. Here’s a simplified example of how the API call is made:

from modules.api_utils import call_groq_api

def sobol_sensitivity_analysis(...):
    # ... perform Sobol analysis ...

    # Prepare the prompt with analysis results
    prompt = f"""
    Please interpret the following Sobol indices:
    
    First-order indices:
    {first_order_md_table}
    
    Total-order indices:
    {total_order_md_table}
    
    Second-order indices:
    {second_order_md_table}
    
    Provide insights based on these results.
    """
    
    # Call the Groq API to get AI-driven interpretation
    interpretation = call_groq_api(prompt, model_name='gemma2-9b-it')
    
    # Display the interpretation in the app
    st.markdown(interpretation)

In this snippet:

  • call_groq_api is a utility function that sends the prepared prompt to the Groq API and retrieves the AI-generated markdown interpretation.
  • The model_name parameter specifies which language model to use (defaulting to Google’s Gemma 2).
  • The returned interpretation is then displayed within the Streamlit app using st.markdown().

PS I have also added support for creation of PCE surrogates (based on Chaospy) + Morris dimensionality reduction (based on Salib) → please check it out too.

Thank you!

1 Like
5

Hi!
Thank you very much for giving us access to your code: this is very stimulating.

I was wondering why the tool uses Chaospy to create a polynomial chaos expansion (Chaospy is a very solid option indeed, but that is worth questioning). As far as I can read from PCE_surrogate_model.py, the code computes the expansion using quadrature using either a full quadrature or Smolyak quadrature. This can be done using OpenTURNS as shown for full quadrature here at plot_chaos_cantilever_beam_integration.

Using Smolyak quadrature can be done using a SmolyakExperiment. An example of creating such an experiment is shown here in plot_smolyak_quadrature.html. One of the features of OpenTURNS’s Smolyak quadrature is that we can use any marginal quadrature we want: we do not necessarily have to use the same marginal quadrature for all marginals. Furthermore, we can “Smolyak ise” a list of multidimensionnal quadratures. Is there any specific Chaospy feature that is missing on OpenTURNS’s PCE side (I can think of several ones, but I am interested by your opinion on this topic)?

As far as Morris’s method is concerned, I read the source code at morris_sensitivity_analysis.py. Did you know of otmorris? This would be a straightforward option. Is there any specific Salib feature that is not implemented here?

I was surprised by the fact that the code uses Salib to estimate Sobol’ indices in sobol_sensitivity_analysis.py. An example of estimating Sobol’ indices is presented in plot_sensitivity_sobol. Furthermore, we can use the SobolSimulationAlgorithm which enables one to set a convergence criteria which stops the simulation based on a pre-defined accuracy criteria. An example of this is presented in plot_sensitivity_sobol_simulationalgorithm.html. This might be convenient within a graphical interface, because the user may want to set a pre-defined time limit (using the setMaximumTimeDuration() method). We used this for Persalys and it reveals to be quite convenient in practice so that we often do not have to set the sample size beforehand. Is there any specific Salib feature here that is not available in OpenTURNS?

Best regards,
Michaël

6

Hi Michael,

Thank you so much for your thoughtful and detailed feedback—I really appreciate it!

On Polynomial Chaos Expansion (PCE):

Yes, Chaospy is a solid option, and I initially used it before I became familiar with OpenTURNS. While I’ve since tried to shift as much of the app’s functionality as possible into OpenTURNS, I’ve kept Chaospy for PCE due to a key usability feature: Chaospy generates human-readable polynomial surrogates, which are directly formatted for easy copy-pasting into the main application. So, this is what I am able to do:

  terms_list = []
  for exponent_tuple, coeff in coeff_dict.items():
      # Build the term
      term_str = ''
      # Handle the coefficient
      if coeff >= 0 and terms_list:
          term_str += '    + '
      else:
          term_str += '    '
      term_str += f'{coeff}'
      # For each variable, if exponent is non-zero, include it
      for var_name, exp in zip(var_names, exponent_tuple):
          if exp != 0:
              term_str += f' * ({var_name} ** {int(exp)})'
      terms_list.append(term_str)

  # Combine all terms into a single expression
  polynomial_expression = '\n'.join(terms_list)

surrogate_model_code = f'''
def function_of_interest(X):
    {', '.join(var_names)} = X
    Y = (
{polynomial_expression}
    )
    return [Y]

And it gives me this (which I can then pump into my app):

def function_of_interest(X):
    E, F, L, I = X
    Y = (
    -0.164649718956575
    + 0.0026897830842560894 * (I ** 1)
    ...
    + 8.623324814809858e-28 * (E ** 1) * (F ** 2)
    + 1.3336918598569558e-14 * (E ** 2)
    -4.458498341340005e-17 * (E ** 2) * (I ** 1)
    + 1.9355468865117568e-16 * (E ** 2) * (L ** 1)
    + 3.6571574269008106e-19 * (E ** 2) * (F ** 1)
    -4.3944481768581105e-22 * (E ** 3)
    )
    return [Y]

# Problem definition

problem = {
    'num_vars': 4,
    'names': ['E', 'F', 'L', 'I'],
    'distributions': [
        {'type': 'Beta', 'params': [0.9, 3.1, 28000000.0, 48000000.0]},
        {'type': 'LogNormalMuSigma', 'params': [30000.0, 9000.0, 15000.0]},
        {'type': 'Uniform', 'params': [250.0, 260.0]},
        {'type': 'Beta', 'params': [2.5, 4, 310.0, 450.0]}
    ]
}

In the current app design, users need to manually transfer the surrogate model back into the main application for the actual analysis, and Chaospy’s clear output greatly facilitates this process. I did successfully implement OpenTURNS for PCE generation, but I struggled to extract the surrogate model definition in a similarly user-friendly format. Since the app relies on users verifying and reusing this output, Chaospy proved to be a good choice.

That said, I’d be full open to exploring ways to achieve this level of usability with OpenTURNS, especially if there’s a straightforward way to extract human-readable polynomial expressions.

On Morris Method:

I wasn’t aware of otmorris—thank you for pointing that out! My use of Salib here was due to familiarity and ease of access. When I initially explored the OpenTURNS library, I couldn’t locate an implementation of Morris and assumed it wasn’t included. I now realize I may have overlooked otmorris or misunderstood how to use it. I’ll definitely investigate integrating this feature into the app to align with OpenTURNS as much as possible. Do I need to pip install otmorris?

On Sobol Indices:

You’re right again—this was the very first module I created, and my choice of Salib came down to simply it being up there in the google search results. Salib is heavily featured in UQ resources online, and I used it as a starting point. However, I completely agree with your suggestion to migrate this functionality to OpenTURNS, particularly given the advantages of SobolSimulationAlgorithm for dynamically setting convergence criteria and managing computational efficiency. I think this will greatly enhance the user experience, especially for those working with time constraints/complex models.

Moving Forward:

Overall, I do agree with standardizing the app around OpenTURNS as much as possible. If you’re open to it, I’d be keen to collaborate more directly, somehow.

What is your view on the app though overall - who do you think may be the target audience? What would you suggest doing with the project overall? Can it become some sort of a non-official frontend/UI/entry-point for OpenTurns (once we standardize) perhaps?

Thank you again for taking the time!

7

Hi!
Thank you very much for your detailed reply: this brings many unique informations.
On the PCE topic, the PCE metamodel is already available as a string is available in the library. I consider the example available at plot_chaos_ishigami. Then:

print(chaosResult.getMetaModel())

This prints:

(3.51725 + 1.62286 * (1.73205 * x0) + 0.000962831 * (1.73205 * x2) - 0.589896 * (-1.11803 + 3.3541 * x1^2) - 1.28859 * (-3.96863 * x0 + 6.61438 * x0^3) + 1.36247 * (1.73205 * x0 * (-1.11803 + 3.3541 * x2^2)) - 1.93978 * (1.125 - 11.25 * x1^2 + 13.125 * x1^4) + 0.000695357 * ((-3.96863 * x1 + 6.61438 * x1^3) * 1.73205 * x2) + 0.18832 * (6.21867 * x0 - 29.0205 * x0^3 + 26.1184 * x0^5) - 1.08118 * ((-3.96863 * x0 + 6.61438 * x0^3) * (-1.11803 + 3.3541 * x2^2)) + 0.410692 * (1.73205 * x0 * (1.125 - 11.25 * x2^2 + 13.125 * x2^4)) + 1.35858 * (-1.12673 + 23.6614 * x1^2 - 70.9843 * x1^4 + 52.0551 * x1^6) - 0.00237001 * (-1.12673 + 23.6614 * x2^2 - 70.9843 * x2^4 + 52.0551 * x2^6) + 0.160285 * ((6.21867 * x0 - 29.0205 * x0^3 + 26.1184 * x0^5) * (-1.11803 + 3.3541 * x2^2)) - 0.319932 * ((-3.96863 * x0 + 6.61438 * x0^3) * (1.125 - 11.25 * x2^2 + 13.125 * x2^4)) - 0.0241258 * (1.73205 * x0 * (1.125 - 11.25 * x1^2 + 13.125 * x1^4) * (-1.11803 + 3.3541 * x2^2)) - 0.000126094 * (1.73205 * x1 * (-1.12673 + 23.6614 * x2^2 - 70.9843 * x2^4 + 52.0551 * x2^6)) + 0.0295804 * ((-3.96863 * x0 + 6.61438 * x0^3) * (6.21867 * x2 - 29.0205 * x2^3 + 26.1184 * x2^5)) - 0.0184612 * (1.73205 * x0 * (-8.47215 * x2 + 76.2494 * x2^3 - 167.749 * x2^5 + 103.844 * x2^7)) - 0.355647 * (1.12741 - 40.5868 * x1^2 + 223.228 * x1^4 - 386.928 * x1^6 + 207.283 * x1^8) - 0.019878 * ((-3.96863 * x1 + 6.61438 * x1^3) * (6.21867 * x2 - 29.0205 * x2^3 + 26.1184 * x2^5)))o(| y0 = [x0]->[0.31830988618379069122*x0]
| y1 = [x1]->[0.31830988618379069122*x1]
| y2 = [x2]->[0.31830988618379069122*x2]
)

Notice that the second part of the string describes the iso-probabilistic transformation which is one of the ingredients that makes the PCE so stable (but this is not sufficient, see below). That is also available from chaosResult.getMetaModel().__str__(). I think, however, that using this string as an implementation of the metamodel is not a good idea (to say the least), because it can lead to an unstable evaluation. I wrote more details on this topic in the following document:

Do not evaluate the printed PCE.pdf (212.8 KB)

Exporting the metamodel into a file is easy with OpenTURNS. For example, we can create a Study, add the metamodel and save it on the disk. Then the Study can be read, and the metamodel can be created (we can use pickle if needed). This is for example what approximately happens in Persalys (see exportStandalonePythonScript) with the difference that a Persalys study file is used instead. But the Streamlit app requires to have a completely standalone Python script, which prevent us from using a file containing the metamodel. What we can still do is to create a Python script using OpenTURNS, recreating the PCE when it needs to be evaluated. It is easy to create a script which contains any dataset. The data required to create a PCE are: 1) the input distribution which defines the iso-probabilistic transformation, 2) the set of indices corresponding to the model selection (if LARS is involved), 3) the set of estimated coefficients. I created the following script for you:

This creates the “pce_metamodel.py” python script. Its content defines a standalone PCE of the Ishigami function:

from openturns import *
def pceModel(X):
    # Define the distribution
    marginals = []
    marginals.append(Uniform(-3.141592653589793, 3.141592653589793))
    marginals.append(Uniform(-3.141592653589793, 3.141592653589793))
    marginals.append(Uniform(-3.141592653589793, 3.141592653589793))
    distribution = JointDistribution(marginals)
    # Set the indices
    indices = [0,1,6,7,9,10,15,30,32,34,35,40,49,51,65,77,83,89,98,109,119,121,125,128,156,163]#26
    # Define the coefficients
    coefficients = Sample([[3.5080413504746915], [1.6143454720819665], [-0.011977451617001271], [-0.5853961996906742], [-0.016533164114624046], [-1.2849614959577127], [1.3687745585574793], [-1.9578673552520318], [0.007818143465962605], [-0.013988939678967135], [0.18670006158189084], [-1.0708624964041402], [0.39138426252265074], [-0.01365068249265241], [0.0037816035001760153], [1.3343816320512953], [-0.017770803638288542], [0.16145682554646218], [-0.33151563222883595], [-0.022312021201981218], [0.0010618753230878404], [-0.026372655726279715], [-0.01805834859738003], [0.008337136150391757], [-0.3629216338749513], [-0.025780331326596802]])

    # Set the basis
    inputDimension = distribution.getDimension()
    polynomials = PolynomialFamilyCollection(inputDimension)
    for i in range(inputDimension):
        marginalPolynomial = StandardDistributionPolynomialFactory(marginals[i])
        polynomials[i] = marginalPolynomial
    enumerate = LinearEnumerateFunction(inputDimension)
    basis = OrthogonalProductPolynomialFactory(polynomials, enumerate)

    # Set the function collection
    function_collection = []
    numberOfIndices = len(indices)
    for i in range(numberOfIndices):
        function_collection.append(basis.build(indices[i]))

    # Set the composed metamodel, set the transformation, create the metamodel
    composedMetaModel = DualLinearCombinationFunction(function_collection, coefficients)
    measure = basis.getMeasure()
    transformation = DistributionTransformation(distribution, measure)
    metaModel = ComposedFunction(composedMetaModel, transformation)

    # Evaluate the metamodel
    Y = metaModel(X)
    return Y

The next script uses it:

# %%
from openturns.usecases import ishigami_function
import openturns as ot

# %%
# We load the Ishigami model :
im = ishigami_function.IshigamiModel()

# %%
with open("pce_metamodel.py","r") as f:
    pythonCode = f.read()

print("Code :")
print(pythonCode)
exec(pythonCode)

# %%
x = im.inputDistribution.getRealization()
y = pceModel(x)
print("y = ")
print(y)

# %%
# Define the PCE metamodel as a function
pceFunction = ot.PythonFunction(3, 1, pceModel)


# %%
sampleSize = 1000
inputSample = im.inputDistribution.getSample(sampleSize)
outputSample = pceFunction(inputSample)
print(outputSample)
print("Exact mean = ", im.expectation)
print("PCE Mean =", outputSample.computeMean())

This prints:

Exact mean =  3.5
PCE Mean = [3.63089]

This is slightly slower than necessary, because the PCE is fully re-created each time it is evaluated.
Regards,
Michaël
PS
@jschueller: Don’t you think that this might be useful for other cases where we want to export a surrogate model?

8

On the Morris method, you can you pip to install it, or Conda:

https://anaconda.org/conda-forge/otmorris

Overall, the app you created is extremely useful from my point of view. This is because it makes it possible to investigate small model in very short time. Many of the A, C and C’ steps of the UQ methodology can be tested very smoothly. This is an intermediate step between OpenTURNS and Persalys. One of the bonuses is that the interpretation is made easier from the IA engine the app uses: this might be very useful for engineers on the hurry or for beginners. From a user point of view, I feel however that making a fully fledged GUI must be limited for such an app (think for example of taking into account the dependency in the probabilistic model, calibrating a parametric model, propagating the uncertainties through a model having a time-depending output, exploring a large data set, evaluating a CPU-consuming model on a cluster, etc.). Moreover, making it usable for professionals having confidentiality concerns might be almost impossible (some of users at CEA, including for example B.Kerleguer made this very clear, see its analysis here at last year’s Users Day). If, however, there is a simple method to analyze a model, then the app adds some value to the user. This may be helpful for example for teaching purposes where time is very short (say <1h for example).

One of the steps that is missing from my point of view is the B step, i.e. from a dataset, fit a distribution. Then this distribution can be propagated though the model. OpenTURNS makes this straightforward. See e.g. here for the results available in Persalys. Making a very simple app for this task using OpenTURNS should be simple and efficient.

Regards,
Michaël