improver.ensemble_copula_coupling.ensemble_copula_coupling module

This module defines the plugins required for Ensemble Copula Coupling.

class improver.ensemble_copula_coupling.ensemble_copula_coupling.EnsembleReordering

Bases: object

Plugin for applying the reordering step of Ensemble Copula Coupling, in order to generate ensemble realizations with multivariate structure from percentiles. The percentiles are assumed to be in ascending order.

Reference: Schefzik, R., Thorarinsdottir, T.L. & Gneiting, T., 2013. Uncertainty Quantification in Complex Simulation Models Using Ensemble Copula Coupling. Statistical Science, 28(4), pp.616-640.

__init__()

Initialise the class

static _recycle_raw_ensemble_realizations(post_processed_forecast_percentiles, raw_forecast_realizations, percentile_coord)

Function to determine whether there is a mismatch between the number of percentiles and the number of raw forecast realizations. If more percentiles are requested than ensemble realizations, then the ensemble realizations are recycled. This assumes that the identity of the ensemble realizations within the raw ensemble forecast is random, such that the raw ensemble realizations are exchangeable. If fewer percentiles are requested than ensemble realizations, then only the first n ensemble realizations are used.

Parameters:

• post_processed_forecast_percentiles (iris.cube.Cube) – Cube for post-processed percentiles. The percentiles are assumed to be in ascending order.
• raw_forecast_realizations (iris.cube.Cube) – Cube containing the raw (not post-processed) forecasts.
• percentile_coord (String) – Name of required percentile coordinate.

Returns:

Cube for the raw ensemble forecast, where the raw ensemble realizations have either been recycled or constrained, depending upon the number of percentiles present in the post-processed forecast cube.

Return type:

raw_forecast_realizations (iris cube.Cube)

process(post_processed_forecast, raw_forecast, random_ordering=False, random_seed=None)

Reorder post-processed forecast using the ordering of the raw ensemble.

Parameters:

• post_processed_forecast (Iris Cube or CubeList) – The cube or cubelist containing the post-processed forecast realizations.
• raw_forecast (Iris Cube or CubeList) – The cube or cubelist containing the raw (not post-processed) forecast.
• random_ordering (Logical) – If random_ordering is True, the post-processed forecasts are reordered randomly, rather than using the ordering of the raw ensemble.
• random_seed (Integer or None) – If random_seed is an integer, the integer value is used for the random seed. If random_seed is None, no random seed is set, so the random values generated are not reproducible.

Returns:

Cube containing the new ensemble realizations where all points within the dataset have been reordered in comparison to the input percentiles.

Return type:

post-processed_forecast_realizations (cube)

static rank_ecc(post_processed_forecast_percentiles, raw_forecast_realizations, random_ordering=False, random_seed=None)

Function to apply Ensemble Copula Coupling. This ranks the post-processed forecast realizations based on a ranking determined from the raw forecast realizations.

Parameters:

• post_processed_forecast_percentiles (cube) – Cube for post-processed percentiles. The percentiles are assumed to be in ascending order.
• raw_forecast_realizations (cube) – Cube containing the raw (not post-processed) forecasts. The probabilistic dimension is assumed to be the zeroth dimension.
• random_ordering (Logical) – If random_ordering is True, the post-processed forecasts are reordered randomly, rather than using the ordering of the raw ensemble.
• random_seed (Integer or None) – If random_seed is an integer, the integer value is used for the random seed. If random_seed is None, no random seed is set, so the random values generated are not reproducible.

Returns:

Cube for post-processed realizations where at a particular grid point, the ranking of the values within the ensemble matches the ranking from the raw ensemble.

Return type:

iris.cube.Cube

class improver.ensemble_copula_coupling.ensemble_copula_coupling.GeneratePercentilesFromMeanAndVariance

Bases: object

Plugin focussing on generating percentiles from mean and variance. In combination with the EnsembleReordering plugin, this is Ensemble Copula Coupling.

__init__()

Initialise the class.

static _mean_and_variance_to_percentiles(calibrated_forecast_predictor, calibrated_forecast_variance, percentiles)

Function returning percentiles based on the supplied mean and variance. The percentiles are created by assuming a Gaussian distribution and calculating the value of the phenomenon at specific points within the distribution.

Parameters:

• calibrated_forecast_predictor (cube) – Predictor for the calibrated forecast i.e. the mean.
• calibrated_forecast_variance (cube) – Variance for the calibrated forecast.
• percentiles (List) – Percentiles at which to calculate the value of the phenomenon at.

Returns:

Cube containing the values for the phenomenon at each of the percentiles requested.

Return type:

percentile_cube (Iris cube)

process(calibrated_forecast_predictor_and_variance, no_of_percentiles)

Generate ensemble percentiles from the mean and variance.

Parameters:

• calibrated_forecast_predictor_and_variance (Iris CubeList) – CubeList containing the calibrated forecast predictor and calibrated forecast variance.
• raw_forecast (Iris Cube or CubeList) – Cube or CubeList that is expected to be the raw (uncalibrated) forecast.

Returns:

Cube for calibrated percentiles. The percentile coordinate is always the zeroth dimension.

Return type:

calibrated_forecast_percentiles (iris.cube.Cube)

class improver.ensemble_copula_coupling.ensemble_copula_coupling.GeneratePercentilesFromProbabilities(ecc_bounds_warning=False)

Bases: object

Class for generating percentiles from probabilities. In combination with the Ensemble Reordering plugin, this is a variant Ensemble Copula Coupling.

This class includes the ability to interpolate between probabilities specified using multiple thresholds in order to generate the percentiles, see Figure 1 from Flowerdew, 2014.

Scientific Reference: Flowerdew, J., 2014. Calibrated ensemble reliability whilst preserving spatial structure. Tellus Series A, Dynamic Meteorology and Oceanography, 66, 22662.

__init__(ecc_bounds_warning=False)

Initialise the class.

Keyword Arguments:  ecc_bounds_warning (bool) – If true and ECC bounds are exceeded by the percentile values, a warning will be generated rather than an exception. Default value is FALSE.

_add_bounds_to_thresholds_and_probabilities(threshold_points, probabilities_for_cdf, bounds_pairing)

Padding of the lower and upper bounds of the distribution for a given phenomenon for the threshold_points, and padding of probabilities of 0 and 1 to the forecast probabilities.

Parameters:

• threshold_points (Numpy array) – Array of threshold values used to calculate the probabilities.
• probabilities_for_cdf (Numpy array) – Array containing the probabilities used for constructing an cumulative distribution function i.e. probabilities below threshold.
• bounds_pairing (Tuple) – Lower and upper bound to be used as the ends of the cumulative distribution function.

Returns:

tuple containing:

threshold_points (Numpy array):

Array of threshold values padded with the lower and upper bound of the distribution.

probabilities_for_cdf (Numpy array):

Array containing the probabilities padded with 0 and 1 at each end.

Return type:

(tuple)

_probabilities_to_percentiles(forecast_probabilities, percentiles, bounds_pairing)

Conversion of probabilities to percentiles through the construction of an cumulative distribution function. This is effectively constructed by linear interpolation from the probabilities associated with each threshold to a set of percentiles.

Parameters:

• forecast_probabilities (Iris cube) – Cube with a threshold coordinate.
• percentiles (Numpy array) – Array of percentiles, at which the corresponding values will be calculated.
• bounds_pairing (Tuple) – Lower and upper bound to be used as the ends of the cumulative distribution function.

Returns:

Cube containing values for the required diagnostic e.g. air_temperature at the required percentiles.

Return type:

percentile_cube (Iris cube)

process(forecast_probabilities, no_of_percentiles=None, percentiles=None, sampling='quantile')

1. Concatenates cubes with a threshold coordinate.
2. Creates a list of percentiles.
3. Accesses the lower and upper bound pair to find the ends of the cumulative distribution function.
4. Convert the threshold coordinate into values at a set of percentiles using linear interpolation, see Figure 1 from Flowerdew, 2014.

Parameters:

• forecast_probabilities (Iris CubeList or Iris Cube) – Cube or CubeList expected to contain a threshold coordinate.
• no_of_percentiles (Integer or None) – Number of percentiles. If None and percentiles is not set, the number of thresholds within the input forecast_probabilities cube is used as the number of percentiles. This argument is mutually exclusive with percentiles.
• percentiles (list of floats) – The desired percentile values in the interval [0, 100]. This argument is mutually exclusive with no_of_percentiles.
• sampling (String) –

  Type of sampling of the distribution to produce a set of percentiles e.g. quantile or random.

  Accepted options for sampling are:

  • Quantile: A regular set of equally-spaced percentiles aimed

    at dividing a Cumulative Distribution Function into blocks of equal probability.

  • Random: A random set of ordered percentiles.

Returns:

Cube with forecast values at the desired set of percentiles. The threshold coordinate is always the zeroth dimension.

Return type:

forecast_at_percentiles (Iris cube)

class improver.ensemble_copula_coupling.ensemble_copula_coupling.GenerateProbabilitiesFromMeanAndVariance

Bases: object

Plugin to generate probabilities relative to given thresholds from the mean and variance of a distribution.

__init__()

Initialise the class.

static _check_template_cube(cube)

The template cube is expected to contain a leading threshold dimension followed by spatial (y/x) dimensions. This check raises an error if this is not the case. If the cube contains the expected dimensions, a threshold leading order is enforced.

Parameters:cube (iris.cube.Cube) – A cube whose dimensions are checked to ensure they match what is expected. Raises:ValueError – If cube is not of the expected dimensions.

static _check_unit_compatibility(mean_values, variance_values, probability_cube_template)

The mean, variance, and threshold values come from three different cubes. They should all be in the same units, but this is a sanity check to ensure this is the case, converting units of the means and variances if possible. This has been written specifically for this plugin as we are comparing squared units in the case of the variance.

Parameters:

• mean_values (iris.cube.Cube) – Cube of mean values.
• variance_values (iris.cube.Cube) – Cube of variance values.
• probability_cube_template (iris.cube.Cube) – Cube containing threshold values.

Raises:

ValueError – If units of input cubes are not compatible.

static _mean_and_variance_to_probabilities(mean_values, variance_values, probability_cube_template)

Function returning probabilities relative to provided thresholds based on the supplied mean and variance. A Gaussian distribution is assumed.

Parameters:

• mean_values (iris.cube.Cube) – Predictor for the calibrated forecast i.e. the mean.
• variance_values (iris.cube.Cube) – Variance for the calibrated forecast.
• probability_cube_template (iris.cube.Cube) – A probability cube that has the threshold coordinate, and attribute relative_to_threshold, that match the desired output cube format.

Returns:

Cube containing the data expressed as probabilities relative to the provided thresholds in the way described by relative_to_threshold.

Return type:

probability_cube (iris.cube.Cube)

process(mean_values, variance_values, probability_cube_template)

Generate probabilties from the mean and variance of distribution.

Parameters:

• mean_values (iris.cube.Cube) – Cube containing the distribution mean values of a diagnostic, e.g. the mean over realizations.
• variance_values (iris.cube.Cube) – Cube containing the distribution variance values of a diagnostic, e.g. the variance across realizations.
• probability_cube_template (iris.cube.Cube) – A probability cube that has the threshold coordinate, and attribute relative_to_threshold, that match the desired output cube format.

Returns:

A cube of diagnostic data expressed as probabilities relative to the thresholds found in the probability_cube_template.

Return type:

probability_cube (iris.cube.Cube)

class improver.ensemble_copula_coupling.ensemble_copula_coupling.RebadgePercentilesAsRealizations

Bases: object

Class to rebadge percentiles as ensemble realizations. This will allow the quantisation to percentiles to be completed, without a subsequent EnsembleReordering step to restore spatial correlations, if required.

__init__()

Initialise the class.

static process(cube, ensemble_realization_numbers=None)

Rebadge percentiles as ensemble realizations. The ensemble realization numbering will depend upon the number of percentiles in the input cube i.e. 0, 1, 2, 3, …, n-1, if there are n percentiles.

Parameters:

• cube (Iris.cube.Cube) –
• containing a percentile coordinate, which will be rebadged as (Cube) –
• realization. (ensemble) –

Raises:

InvalidCubeError – If the realization coordinate already exists on the cube.

class improver.ensemble_copula_coupling.ensemble_copula_coupling.ResamplePercentiles(ecc_bounds_warning=False)

Bases: object

Class for resampling percentiles from an existing set of percentiles. In combination with the Ensemble Reordering plugin, this is a variant of Ensemble Copula Coupling.

This class includes the ability to linearly interpolate from an input set of percentiles to a different output set of percentiles.

__init__(ecc_bounds_warning=False)

Initialise the class.

Keyword Arguments:  ecc_bounds_warning (bool) – If true and ECC bounds are exceeded by the percentile values, a warning will be generated rather than an exception. Default value is FALSE.

_add_bounds_to_percentiles_and_forecast_at_percentiles(percentiles, forecast_at_percentiles, bounds_pairing)

Padding of the lower and upper bounds of the percentiles for a given phenomenon, and padding of forecast values using the constant lower and upper bounds.

Parameters:

• percentiles (Numpy array) – Array of percentiles from a Cumulative Distribution Function.
• forecast_at_percentiles (Numpy array) – Array containing the underlying forecast values at each percentile.
• bounds_pairing (Tuple) – Lower and upper bound to be used as the ends of the cumulative distribution function.

_interpolate_percentiles(forecast_at_percentiles, desired_percentiles, bounds_pairing, percentile_coord)

Interpolation of forecast for a set of percentiles from an initial set of percentiles to a new set of percentiles. This is constructed by linearly interpolating between the original set of percentiles to a new set of percentiles.

Parameters:

• forecast_at_percentiles (Iris CubeList or Iris Cube) – Cube or CubeList expected to contain a percentile coordinate.
• desired_percentiles (Numpy array) – Array of the desired percentiles.
• bounds_pairing (Tuple) – Lower and upper bound to be used as the ends of the cumulative distribution function.
• percentile_coord (String) – Name of required percentile coordinate.

Returns:

Cube containing values for the required diagnostic e.g. air_temperature at the required percentiles.

Return type:

percentile_cube (iris cube.Cube)

process(forecast_at_percentiles, no_of_percentiles=None, sampling='quantile')

1. Creates a list of percentiles.
2. Accesses the lower and upper bound pair of the forecast values, in order to specify lower and upper bounds for the percentiles.
3. Interpolate the percentile coordinate into an alternative set of percentiles using linear interpolation.

Parameters:

• forecast_at_percentiles (Iris Cube) – Cube expected to contain a percentile coordinate.
• no_of_percentiles (Integer or None) – Number of percentiles If None, the number of percentiles within the input forecast_at_percentiles cube is used as the number of percentiles.
• sampling (String) –

  Type of sampling of the distribution to produce a set of percentiles e.g. quantile or random.

  Accepted options for sampling are:

  • Quantile: A regular set of equally-spaced percentiles aimed

    at dividing a Cumulative Distribution Function into blocks of equal probability.

  • Random: A random set of ordered percentiles.

Returns:

Cube with forecast values at the desired set of percentiles. The percentile coordinate is always the zeroth dimension.

Return type:

forecast_at_percentiles (iris.cube.Cube)