Exceptions in a PythonFunction

Python usage
1

Hi!
I want to see what happens when we build a surrogate model while ignoring some of the observations which lead to failures in the model evaluation. This can be emulated by considering a toy model and raising an exception when appropriate. Unfortunately, I am not able to catch the exception produced by the PythonFunction.
To see this, please consider the code:

import openturns as ot

def rungeWithFailurePy(x):
    x0 = x[0]
    y = 1.0 / (1.0 + 25.0 * x0 ** 2)
    yThresholdRunge = 0.8
    if y > yThresholdRunge:
        raise ValueError(f"Cannot evaluate y at input {x0:.4f}")
    return [y]

rungeModelWithFailure = ot.PythonFunction(1, 1, rungeWithFailurePy)

sampleSize = 50
inputDistribution = ot.Uniform(-1.0, 1.0)
inputSample = inputDistribution.getSample(sampleSize)
outputSample = ot.Sample(sampleSize, 1)
outputSampleLabel = [True] * sampleSize
for i in range(sampleSize):
    try:
        # outputSample[i] = rungeWithFailurePy(inputSample[i])  # OK
        outputSample[i] = rungeModelWithFailure(inputSample[i])  # Fails
    except ValueError:
        outputSampleLabel[i] = False
        print(f"Cannot evaluate y at input {inputSample[i, 0]:.4f}")

When I use rungeModelWithFailure (i.e. the ot.PythonFunction), this prints:

[...]
ValueError: Cannot evaluate y at input 0.0484
RuntimeError: InternalException : Python exception: ValueError: Cannot evaluate y at input 0.0484

When I use rungeWithFailurePy (i.e. the Python function), this prints:

Cannot evaluate y at input -0.0133
Cannot evaluate y at input -0.0905
Cannot evaluate y at input 0.0900
Cannot evaluate y at input -0.0374
Cannot evaluate y at input -0.0302

So I can catch the exception raised by the basic Python function, but I cannot catch the exception raised by the OpenTURNS PythonFunction. Why is that?
Regards,
Michaël

2

HI Michael,

OpenTURNS raises a “RuntimeError”, you can catch this type of error but I do not why it does not raise the initial error. But still the output is strange.

Antoine

3

Hi Antoine,
Thank you for the help!
I tried this:

sampleSize = 50
inputDistribution = ot.Uniform(-1.0, 1.0)
inputSample = inputDistribution.getSample(sampleSize)
outputSample = ot.Sample(sampleSize, 1)
outputSampleLabel = [True] * sampleSize
for i in range(sampleSize):
    try:
        # outputSample[i] = rungeWithFailurePy(inputSample[i])  # OK
        outputSample[i] = rungeModelWithFailure(inputSample[i])  # Fails
    except RuntimeError:
        outputSampleLabel[i] = False
        print(f"Cannot evaluate y at input {inputSample[i, 0]:.4f}")
    except ValueError:
        outputSampleLabel[i] = False
        print(f"Cannot evaluate y at input {inputSample[i, 0]:.4f}")

But then the ValueError exception is raised:

ValueError: Cannot evaluate y at input 0.0491
ValueError: Cannot evaluate y at input 0.0606
etc.

It seems to me that it should not and should be catched by the except ValueError. Should I organize my “try/catch” statements in some other way?
Regards,
Michaël

4

Hi Michael,

I noticed that i haven’t answered you.

The second except (ValueError) is not catched, only the RuntimeError, but it is OpenTURNS that prints the original error : openturns/python/src/openturns/PythonWrappingFunctions.hxx at 3fa82375be08ef98a90b5528aa1138bfe1ccf3b9 · openturns/openturns · GitHub

Maybe to check if this is really the ValueError you want to catch, you can test the text in the error. But the original print is still here because it comes frome OT, you can redirect the stdout. It is not really elegant but it works:

import sys
import os
f_origin = sys.stdout
print_null = open(os.devnull, 'w')

sampleSize = 50
inputDistribution = ot.Uniform(-1.0, 1.0)
inputSample = inputDistribution.getSample(sampleSize)
outputSample = ot.Sample(sampleSize, 1)
outputSampleLabel = [True] * sampleSize
for i in range(sampleSize):
    try:
        # outputSample[i] = rungeWithFailurePy(inputSample[i])  # OK
        sys.stdout = print_null
        outputSample[i] = rungeModelWithFailure(inputSample[i])  # Fails
        sys.stdout = f_origin
    except RuntimeError as err:
        sys.stdout = f_origin
        outputSampleLabel[i] = False
        if 'ValueError' in err.args[0]:
            outputSampleLabel[i] = False
            print(f"Cannot evaluate y at input {inputSample[i, 0]:.4f}")

Regards,
Antoine