improver.blending.weighted_blend module

Module containing classes for doing weighted blending by collapsing a whole dimension.

class improver.blending.weighted_blend.MaxProbabilityAggregator[source]

Bases: object

Class for the Aggregator used to calculate the maximum weighted probability.

  1. Find the weighted probabilities for each point in the dimension of interest by multiplying each probability by the corresponding weight.
  2. Find the maximum weighted probability and return the array with one less dimension than the input array.
__init__()[source]

Initialise class.

static aggregate(data, axis, arr_weights)[source]
Max probability aggregator method. Used to find the maximum
weighted probability along a given axis.
Parameters:
  • data (np.array) – Array containing the data to blend
  • axis (int) – The index of the coordinate dimension in the cube. This dimension will be aggregated over.
  • arr_weights (np.array) – Array of weights, same size as the axis dimension of data.
Returns:

The data collapsed along the axis dimension, containing the maximum weighted probability.
Return type:

result (np.array)
class improver.blending.weighted_blend.PercentileBlendingAggregator[source]

Bases: object

Class for the percentile blending aggregator

This class implements the method described by Combining Probabilities by Caroline Jones, 2017. This method implements blending in probability space.

The steps are:
  1. At each geographic point in the cube we take the percentile threshold’s values across the percentile dimensional coordinate. We recalculate, using linear interpolation, their probabilities in the pdf of the other points across the coordinate we are blending over. Thus at each point we have a set of thresholds and their corresponding probability values in each of the probability spaces across the blending coordinate.
  2. We do a weighted blend across all the probability spaces, combining all the thresholds in all the points in the coordinate we are blending over. This gives us an array of thresholds and an array of blended probailities for each of the grid points.
  3. We convert back to the original percentile values, again using linear interpolation, resulting in blended values at each of the original percentiles.

References

Combining Probabilities by Caroline Jones, 2017: https://github.com/metoppv/improver/files/1128018/ Combining_Probabilities.pdf

__init__()[source]

Initialise class.

static aggregate(data, axis, arr_percent, arr_weights, perc_dim)[source]
Blend percentile aggregate function to blend percentile data
along a given axis of a cube.
Parameters:
  • data (np.array) – Array containing the data to blend
  • axis (int) – The index of the coordinate dimension in the cube. This dimension will be aggregated over.
  • arr_percent (np.array) – Array of percentile values e.g [0, 20.0, 50.0, 70.0, 100.0], same size as the percentile dimension of data.
  • arr_weights (np.array) – Array of weights, same size as the axis dimension of data.
  • perc_dim (int) – The index of the percentile coordinate
  • percent and weights have special meaning in Aggregator ((Note) – hence the rename.)
Returns:

containing the weighted percentile blend data across the chosen coord. The dimension associated with axis has been collapsed, and the rest of the dimensions remain.
Return type:

result (np.array)
static blend_percentiles(perc_values, percentiles, weights)[source]
Blend percentiles function, to calculate the weighted blend across
a given axis of percentile data for a single grid point.
Parameters:
  • perc_values (np.array) – Array containing the percentile values to blend, with shape: (length of coord to blend, num of percentiles)
  • percentiles (np.array) – Array of percentile values e.g [0, 20.0, 50.0, 70.0, 100.0], same size as the percentile dimension of data.
  • weights (np.array) – Array of weights, same size as the axis dimension of data, that we will blend over.
Returns:

Array containing the weighted percentile blend data across the chosen coord
Return type:

new_combined_perc (np.array)
class improver.blending.weighted_blend.WeightedBlendAcrossWholeDimension(coord, weighting_mode, cycletime=None, timeblending=False)[source]

Bases: object

Apply a Weighted blend to a cube, collapsing across the whole dimension. Uses one of two methods, either weighted average, or the maximum of the weighted probabilities.

__init__(coord, weighting_mode, cycletime=None, timeblending=False)[source]

Set up for a Weighted Blending plugin

Parameters:
  • coord (string) – The name of the coordinate dimension over which the cube will be blended.
  • weighting_mode (string) –
    One of ‘weighted_maximum’ or ‘weighted_mean’:
    • Weighted mean: a normal weighted average over the coordinate of interest.
    • Weighted_maximum: the points in the coordinate of interest are multiplied by the weights and then the maximum is taken.
Keyword Arguments:
 
  • cycletime (str) – The cycletime in a YYYYMMDDTHHMMZ format e.g. 20171122T0100Z.
  • timeblending (bool) – With the default of False the cube being blended will be checked to ensure that slices across the blending coordinate all have the same validity time. Setting this to True will bypass this test, as is necessary for triangular time blending.
Raises:

ValueError – If an invalid weighting_mode is given.
check_compatible_time_points(cube)[source]

Check that the time coordinate only contains a single time. Data varying over the blending coordinate should all be for the same validity time unless we are triangular time blending. In this case the timeblending flag should be true and this function will not raise an exception.

Parameters:cube (iris.cube.Cube) – The cube upon which the compatibility of the time coords is being checked.
Raises:ValueError – If blending over forecast reference time on a cube with multiple times.
check_percentile_coord(cube)[source]

Determines if the cube to be blended has a percentile dimension coordinate.

Parameters:

cube (iris.cube.Cube) – The cube to be checked for a percentile coordinate.
Returns:

None if no percentile dimension coordinate is found. If such a coordinate is found it is returned.
Return type:

None or perc_coord (iris.coords.DimCoord)
Raises:
  • ValueError – If there is a percentile coord and it is not a dimension coord in the cube.
  • ValueError – If there is a percentile dimension with only one point, we need at least two points in order to do the blending.
  • ValueError – If there is a percentile dimension on the cube and the mode for blending is ‘weighted_maximum’
static check_weights(weights, blend_dim)[source]

Checks that weights across the blending dimension sum up to 1.

Parameters:
  • weights (np.array) – Array of weights shaped to match the data cube.
  • blend_dim (integer) – The dimension in the weights array that is being collapsed.
Raises:

ValueError – Raised if the weights do not sum to 1 over the blending dimension.
non_percentile_weights(cube, weights, custom_aggregator=False)[source]

Given a 1 or multidimensional cube of weights, reshape and broadcast these in such a way as to make them applicable to the data cube. If no weights are provided, an array of weights is returned that equally weights all slices across the blending coordinate of the cube.

The output of this function is different depending upon the method being used to blend the data.

weighted_mean:
reshape and broadcast to match data shape.
weighted_maximum:
reshape and broadcast to match data shape, before reordering in anticipation of the blending coord being shifted to the -1 position by the custom aggregator.
Parameters:
  • cube (iris.cube.Cube) – The data cube on which a coordinate is being blended.
  • weights (iris.cube.Cube or None) – Cube of blending weights or None.
Returns:

An array of weights that matches the cube data shape.
Return type:

weights_array (np.array)
percentile_weighted_mean(cube, weights, perc_coord)[source]

Blend percentile data using the weights provided.

Parameters:
  • cube (iris.cube.Cube) – The cube which is being blended over self.coord.
  • weights (iris.cube.Cube) – Cube of blending weights.
  • perc_coord (iris.coords.DimCoord) – The percentile coordinate for this cube.
Returns:

The cube with percentile values blended over self.coord, with suitable weightings applied.
Return type:

cube_new (iris.cube.Cube)
percentile_weights(cube, weights, perc_coord)[source]

Given a 1, or multidimensional cube of weights, reshape and broadcast these in such a way as to make them applicable to the data cube. If no weights are provided, an array of weights is returned that equally weights all slices across the blending coordinate of the cube.

For percentiles the dimensionality of the weights cube is checked against the cube without including the percentile coordinate for which no weights are likely to ever be provided (e.g. we don’t want to weight different percentiles differently across the blending coordinate). Reshape and broadcast to match the data shape excluding the percentile dimension before finally broadcasting to match at the end.

Parameters:
  • cube (iris.cube.Cube) – The data cube on which a coordinate is being blended.
  • weights (iris.cube.Cube or None) – Cube of blending weights or None.
Returns:

An array of weights that matches the cube data shape.
Return type:

weights_array (np.array)
process(cube, weights=None)[source]
Calculate weighted blend across the chosen coord, for either
probabilistic or percentile data. If there is a percentile coordinate on the cube, it will blend using the PercentileBlendingAggregator but the percentile coordinate must have at least two points.
Parameters:

cube (iris.cube.Cube) – Cube to blend across the coord.
Keyword Arguments:
 

weights (iris.cube.Cube) – Cube of blending weights. If None, the diagnostic cube is blended with equal weights across the blending dimension.
Returns:

containing the weighted blend across the chosen coord.
Return type:

result (iris.cube.Cube)
Raises:
  • TypeError – If the first argument not a cube.
  • CoordinateNotFoundError – If coordinate to be collapsed not found in cube.
  • CoordinateNotFoundError – If coordinate to be collapsed not found in provided weights cube.
  • ValueError – If coordinate to be collapsed is not a dimension.
shape_weights(cube, weights)[source]

The function shapes weights to match the diagnostic cube. A 1D cube of weights that vary across the blending coordinate will be broadcast to match the complete multidimensional cube shape. A multidimensional cube of weights will be checked to ensure that the coordinate names match between the two cubes. If they match the order will be enforced and then the shape will be checked. If the shapes match the weights will be returned as an array.

Parameters:
  • cube (iris.cube.Cube) – The data cube on which a coordinate is being blended.
  • weights (iris.cube.Cube) – Cube of blending weights.
Returns:

An array of weights that matches the cube data shape.
Return type:

weights_array (np.array)
Raises:
  • ValueError – If weights cube coordinates do not match the diagnostic cube in the case of a multidimensional weights cube.
  • ValueError – If weights cube shape is not broadcastable to the data cube shape.
weighted_maximum(cube, weights)[source]

Blend data using a weighted maximum using the weights provided. This entails scaling the data by the weights before then taking a maximum across the blending coordinate self.coord.

Parameters:
  • cube (iris.cube.Cube) – The cube which is being blended over self.coord.
  • weights (iris.cube.Cube) – Cube of blending weights.
Returns:

The cube with values blended over self.coord, with suitable weightings applied.
Return type:

cube_new (iris.cube.Cube)
weighted_mean(cube, weights)[source]

Blend data using a weighted mean using the weights provided.

Parameters:
  • cube (iris.cube.Cube) – The cube which is being blended over self.coord.
  • weights (iris.cube.Cube or None) – Cube of blending weights or None.
Returns:

The cube with values blended over self.coord, with suitable weightings applied.
Return type:

cube_new (iris.cube.Cube)
improver.blending.weighted_blend.conform_metadata(cube, cube_orig, coord, cycletime=None)[source]

Ensure that the metadata conforms after blending together across the chosen coordinate.

The metadata adjustments are:
  • Forecast reference time: If a cycletime is not present, the most recent available forecast_reference_time is used. If a cycletime is present, the cycletime is used as the forecast_reference_time instead.
  • Forecast period: If a forecast_period coordinate is present, and cycletime is not present, the lowest forecast_period is used. If a forecast_period coordinate is present, and the cycletime is present, forecast_periods are forceably calculated from the time and forecast_reference_time coordinate. This is because, if the cycletime is present, then the forecast_reference_time will also have been just re-calculated, so the forecast_period coordinate needs to be reset to match the newly calculated forecast_reference_time.
  • Forecast reference time and time: If forecast_reference_time and time coordinates are present, then a forecast_period coordinate is calculated and added to the cube.
  • Model_id, model_realization and realization coordinates are removed.
  • A title attribute is added to the cube if none is found. Otherwise the attributes are unchanged.
Parameters:
  • cube (iris.cube.Cube) – Cube containing the metadata to be adjusted.
  • cube_orig (iris.cube.Cube) – Cube containing metadata that may be useful for adjusting metadata on the cube variable.
  • coord (str) – Coordinate that has been blended. This allows specific metadata changes to be limited to whichever coordinate is being blended.
Keyword Arguments:
 

cycletime (str) – The cycletime in a YYYYMMDDTHHMMZ format e.g. 20171122T0100Z.
Returns:

Cube containing the adjusted metadata.
Return type:

cube (iris.cube.Cube)
improver.blending.weighted_blend.rationalise_blend_time_coords(cubelist, blend_coord, cycletime=None, weighting_coord=None)[source]

Updates time coordinates on unmerged input cubes before blending depending on the coordinate over which the blend will be performed. Modifies cubes in place.

If blend_coord is forecast_reference_time, ensures the cube does not have a forecast_period dimension. If weighting_coord is forecast_period, equalises forecast_reference_time on each cube before blending.

Parameters:
  • cubelist (iris.cube.CubeList) – List of cubes containing data to be blended
  • blend_coord (str) – Name of coordinate over which the blend will be performed
Kwargs:
cycletime (str or None):
The cycletime in a YYYYMMDDTHHMMZ format e.g. 20171122T0100Z
weighting_coord (str or None):
The coordinate across which weights will be scaled in a multi-model blend.
Raises:ValueError – if forecast_reference_time (to be unified) is a dimension coordinate