Quantitative sensitivity analysis

econsa provides several algorithms for quantitative sensitivity analysis.

Sobol indices

We implement the methods outlined in [8].

Constributor: Tim Mensinger (timmens)

Shapley values

We implement the methods outlined in [12].

Constributor: Linda Maokomatanda (lindamaok899)

Capabilities for computation of Shapley effects.

This module contains functions to estimate shapley effects for models with dependent inputs.

econsa.shapley.get_shapley(method, model, x_all, x_cond, n_perms, n_inputs, n_output, n_outer, n_inner)[source]

Shapley value function.

This function calculates Shapley effects and their standard errors for models with both dependent and independent inputs. We allow for two ways to calculate Shapley effects: by examining all permutations of the given inputs or alternatively, by randomly sampling permutations of inputs.

The function is a translation of the exact and random permutation funtions in the sensitivity package in R, and takes the method (exact or random) as an argument and therefore estimates shapley effects in both ways.

The functions where obtained from R’s sensitiity package for the shapleyPermEx and shapleyPermRand functions.

Contributor: Linda Maokomatanda

Parameters

  • method (string) – Specifies which method you want to use to estimate shapley effects, the exact or random permutations method. When the number of inputs is small, it is better to use the exact method, and random otherwise.

  • model (string) – The model/function you will calculate the shapley effects on.

  • x_all (string (n)) – A function that takes n as an argument and generates a n-sample of a d-dimensional input vector.

  • x_cond (string (n, Sj, Sjc, xjc)) – A function that takes n, Sj, Sjc, xjc as arguments and generates a n- sample an input vector corresponding to the indices in Sj conditional on the input values xjc with the index set Sjc.

  • n_perms (scalar) – This is an input for the number of permutations you want the model to make. For the exact method, this argument is none as the number of permutations is determined by how many inputs you have, and for the random method, this is determined exogeniously.

  • n_inputs (scalar) – The number of input vectors for which shapley estimates are being estimated.

  • n_output (scalar) – Monte Carlo (MC) sample size to estimate the output variance of the model output Y.

  • n_outer (scalar) – The outer Monte Carlo sample size to estimate the cost function for c(J) = E[Var[Y|X]].

  • n_inner (scalar) – The inner Monte Carlo sample size to estimate the cost function for c(J) = Var[Y|X].

Returns

effects – n dimensional DataFrame with the estimated shapley effects, the standard errors and the confidence intervals for the input vectors.

Return type

DataFrame

Quantile based sensitivity measures

We implement the methods outlined in [7].

Constributor: Yulei Li (Yuleii)

Capabilities for quantile-based sensitivity analysis.

This module contains functions to calculate the global sensitivity measures based on quantiles of the output introduced by Kucherenko et al.(2019). Both the brute force and double loop reordering MC estimators are included.

econsa.quantile_measures.mc_quantile_measures(estimator, func, n_params, loc, scale, dist_type, n_draws, sampling_scheme='sobol', seed=0, skip=0)[source]

Compute the MC/QMC estimators of quantile-based global sensitivity measures.

The algorithm is described in Section 4 of Kucherenko et al.(2019).

Parameters

  • estimator (string) – Specify the Monte Carlo estimator. One of [“brute force”, “DLR”], where “DLR” denotes to the double loop reordering approach.

  • func (callable) – Objective function to estimate the quantile-based measures. Must be broadcastable.

  • n_params (int) – Number of parameters of objective function.

  • loc (float or np.ndarray) –

    The location(loc) keyword passed to scipy.stats.norm function to shift the location of “standardized” distribution. Specifically, for normal distribution it specifies the mean array with the length of n_params.

  • scale (float or np.ndarray) – The scale keyword passed to scipy.stats.norm function to adjust the scale of “standardized” distribution. Specifically, for normal distribution it specifies the covariance matrix of shape (n_params, n_params).

  • dist_type (str) – The distribution type of inputs. Options are “Normal”, “Exponential” and “Uniform”.

  • n_draws (int) – Number of Monte Carlo draws. For double loop reordering estimator, S. Kucherenko and S. Song(2017). suggests that n_draws should always be equal to \(2^p\) to preserve the uniformity properties , where \(p\) is an integer. In this function \(p\) should be integers between 6 and 15 if estimator is “DLR”.

  • sampling_scheme (str, optional) – One of [“random”, “sobol”], default “sobol”.

  • seed (int, optional) – Random number generator seed.

  • skip (int, optional) – Number of values to skip of Sobol sequence. Default is 0.

Returns

df_measures – DataFrame containing quantile-based sensitivity measures.

Return type

pd.DataFrame