Skip to content

Issue #915: Produce BCVs #918

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Ed (mo-gill) wants to merge 28 commits into
base: main
Choose a base branch
from
Open

Issue #915: Produce BCVs #918

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
34913c0
#915: Add skeleton of coord replacement func.
mo-gill May 11, 2026
0e9bfc9
#915: Iterate function and rename func in mappings cfg.
mo-gill May 11, 2026
a2acaef
#915: Correction to argument name, add filenames to ukcm2 json.
mo-gill May 11, 2026
76ae7f3
#915: Revert and refine processor function
mo-gill May 12, 2026
c3acd7d
#915: Add guard when loading the reference cube.
mo-gill May 12, 2026
fc1d578
#915: Minor adjustment to func.
mo-gill May 13, 2026
ea5832e
#915: Rename func in mapping.
mo-gill May 13, 2026
e3a017f
#915: Add guard for fixed time in load cubes.
mo-gill May 13, 2026
24f56e6
#915: Introduce fx stream to model config
mo-gill May 14, 2026
f7df065
#915: Skip time constraint enforcement for seaice ancils.
mo-gill May 14, 2026
33ed78f
#915: Add replacement coord refs to ancil variables in model config.
mo-gill May 14, 2026
748a4dd
#915: Add guard to prevent err with PP ancils.
mo-gill May 14, 2026
aab1b11
#915: Add mapping for leaf area index.
mo-gill May 15, 2026
6890f9b
Revert "#915: Introduce fx stream to model config"
mo-gill May 18, 2026
41e1818
#915: Change fx streams in streams_config to afx.
mo-gill May 18, 2026
72744f5
#915: Add ancils required by variables.
mo-gill May 18, 2026
108fc6c
#902: move sim_review from deprecated (#914)
mo-laurenboon May 11, 2026
b70e7c8
Potential fix for concatenate error.
mo-gill May 14, 2026
ff974e0
Make comment less verbose.
mo-gill May 15, 2026
c51d452
Merge branch 'main' into 915_produce_BCVs
mo-gill Jun 18, 2026
481cb61
#915: Update filenames for ukcm2 ancils introduced.
mo-gill Jun 30, 2026
c976f19
#915: Add back ancil file name i removed by mistake.
mo-gill Jun 30, 2026
77009e7
#915: Update ancil file used in func test.
mo-gill Jul 1, 2026
e92ee2c
#915: Change back name after renaming ancils.
mo-gill Jul 1, 2026
ba763de
#915: Revert ancil names in config after decision to change names.
mo-gill Jul 1, 2026
46774c0
#915: Update name of an ancil.
mo-gill Jul 1, 2026
30e0136
#915: Minor reversion of guard logic.
mo-gill Jul 2, 2026
a3ede49
#915: Simplify ancil grid check exclusion.
mo-gill Jul 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions cdds/cdds/common/constants.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@
'so_0', # ocean_zostoga
'rho_0_mean', # ocean_zostoga
'deptho_0_mean', # ocean_zostoga
'u_coord_reference', # nemo_eORCA1_grid-U.nc
'v_coord_reference', # nemo_eORCA1_grid-V.nc
]
APPROVED_VARS_PREFIX = 'approved_variables'
APPROVED_VARS_FILENAME_TEMPLATE = APPROVED_VARS_PREFIX + '_{dt}.txt'
Expand Down
9 changes: 8 additions & 1 deletion cdds/cdds/common/plugins/cmip7/data/model/UKCM2-0-LL.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,8 @@
"ancil_filenames": [
"qrparm.landfrac.pp",
"qrparm.soil.pp",
"qrparm.orog.pp"
"qrparm.orog.pp",
"slthick_rootd_sftgif.pp"
],
"hybrid_heights_files": [
"atmosphere_theta_levels_85.txt",
Expand All @@ -216,12 +217,18 @@
"levels": 75,
"replacement_coordinates_file": "",
"ancil_filenames": [
"nemo_eORCA1_grid-V.nc",
"nemo_eORCA1_grid-U.nc",
"ocean_constants.nc",
"ocean_byte_masks.nc",
"ocean_basin.nc",
"diaptr_basin_masks.nc",
"ocean_zostoga.nc"
],
"ancil_variables": [
"u_coord_reference",
"v_coord_reference"
Comment on lines +228 to +230

@matthew-mizielinski Matthew Mizielinski (matthew-mizielinski) Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to add this to UKCM2a-0-HH as well

@matthew-mizielinski Matthew Mizielinski (matthew-mizielinski) Jul 1, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

separate issue

],

@mo-gill Ed (mo-gill) May 14, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needed otherwise moo filter threw an error trying to extract these from MASS.

"bounds_coordinates": {
"onm-grid-T": [
"bounds_nav_lon",
Expand Down
12 changes: 6 additions & 6 deletions cdds/cdds/common/plugins/cmip7/data/streams/streams_config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -585,19 +585,19 @@
"mrsll_tavg-sl-hxy-lnd@mon": "ap5",
"mrso_tavg-u-hxy-lnd@day": "ap6",
"mrso_tavg-u-hxy-lnd@mon": "ap5",
"mrsofc_ti-u-hxy-lnd@fx": "fx",
"mrsofc_ti-u-hxy-lnd@fx": "afx",
"mrsol_tavg-d100cm-hxy-lnd@3hr": "ap8",
"mrsol_tavg-d10cm-hxy-lnd@day": "ap6",
"mrsol_tavg-d10cm-hxy-lnd@mon": "ap5",
"mrsol_tavg-sl-hxy-lnd@day": "ap6",
"mrsol_tavg-sl-hxy-lnd@mon": "ap5",
"mrsol_tpt-d10cm-hxy-lnd@3hr": "ap8",
"orog_ti-u-hxy-u@fx": "fx",
"rootd_ti-u-hxy-lnd@fx": "fx",
"orog_ti-u-hxy-u@fx": "afx",
"rootd_ti-u-hxy-lnd@fx": "afx",
"sftgif_tavg-u-hxy-u@mon": "ap5",
"sftgif_ti-u-hxy-u@fx": "fx",
"sftlaf_ti-u-hxy-u@fx": "fx",
"slthick_ti-sl-hxy-lnd@fx": "fx",
"sftgif_ti-u-hxy-u@fx": "afx",
"sftlaf_ti-u-hxy-u@fx": "afx",
"slthick_ti-sl-hxy-lnd@fx": "afx",
"srfrad_tavg-u-hxy-u@3hr": "UNKNOWN",
"sweLut_tavg-u-hxy-multi@mon": "ap5",
"tran_tavg-u-hxy-lnd@mon": "ap5",
Expand Down
4 changes: 3 additions & 1 deletion mip_convert/mip_convert/load/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,13 @@ def load(filenames, variable_metadata):
input_variables.update({loadable.constraint: cube})

# Ensure all 'input variables' are on the same grid.
# (Excludes ancillary variables from the grid check.)
for axis in ['Y', 'X']:
if cube.coords(axis=axis):
try:
var_names = [cube.coord(axis=axis).var_name
for cube in list(input_variables.values())]
for cube in input_variables.values()
if cube.var_name not in variable_metadata.ancil_variables]
except iris.exceptions.CoordinateNotFoundError:
# crude hack to account for ancils that don't have all coordinates
var_names = []
Expand Down
18 changes: 12 additions & 6 deletions mip_convert/mip_convert/load/iris_load_util.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -198,10 +198,13 @@ def load_cubes(all_input_data, run_bounds, loadable, ancil_variables):
logger.debug('Loading cube using Iris constraints')
load_constraints = constraint_constructor.load_constraints(loadable)

# Static coord-reference ancils have a fixed timestamp that won't match
# later cycle run_bounds, so skip time filtering for them.
effective_run_bounds = None if loadable.name.endswith('_coord_reference') else run_bounds
if loadable.is_pp():
merged_cubes = load_cubes_from_pp(all_input_data, load_constraints, run_bounds, ancil_variables)
else:
merged_cubes = load_cubes_from_nc(all_input_data, load_constraints, run_bounds)
merged_cubes = load_cubes_from_nc(all_input_data, load_constraints, effective_run_bounds)

@mo-gill Ed (mo-gill) May 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went with the name change from run_bounds to effective_run_boundsto prevent join(run_bounds) (Ln 211) throwing an error due to being used with the None which gets assigned on line 203. Hoping the rename isn't confusing.


if not merged_cubes:
error_msg = 'No cubes found using constraints "{}" within "{}"'
Expand Down Expand Up @@ -391,18 +394,21 @@ def load_cubes_from_nc(all_input_data, load_constraints, run_bounds):

cubes = iris.cube.CubeList()
if merged_cubes:
# Apply the time constraint.
time_constraint = setup_time_constraint(run_bounds)
# Apply the time constraint (skipped if run_bounds is None, e.g. for static ancils).
time_constraint = setup_time_constraint(run_bounds) if run_bounds is not None else None
for merged_cube in merged_cubes:
promote_aux_time_coord_to_dim(merged_cube)
# Add the fill_value as an attribute on the cube to workaround the
# fact that iris.util.new_axis resets the fill_value.
if hasattr(merged_cube.lazy_data(), 'fill_value'):
merged_cube.attributes['fill_value'] = merged_cube.lazy_data().fill_value

cube = apply_time_constraint(merged_cube, time_constraint)
if cube is not None:
cubes.append(cube)
if time_constraint is None:
cubes.append(merged_cube)
else:
cube = apply_time_constraint(merged_cube, time_constraint)
if cube is not None:
cubes.append(cube)

@mo-gill Ed (mo-gill) May 14, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting time_constraint to None was necessary for the nemo ancil that lacked run bounds. However, passing None to setup_time_constraint lead to TypeError: 'NoneType' object is not subscriptable when iris evaluated it. The guard should now only be constructing the time constraint when run_bounds is not None, and appending the cube directly otherwise.

return cubes


Expand Down
6 changes: 6 additions & 0 deletions mip_convert/mip_convert/plugins/base/data/processors.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2497,3 +2497,9 @@ def calc_rootd(soil_cube, frac_cube, ice_class=None):
)

return rootd_cube


def apply_ocean_coordinates(cube, coordinate_cube):

@matthew-mizielinski Matthew Mizielinski (matthew-mizielinski) Jul 1, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docstring

cube.coord(axis='Y').points = coordinate_cube.coord(axis='Y').points.copy()
cube.coord(axis='X').points = coordinate_cube.coord(axis='X').points.copy()
return cube
11 changes: 11 additions & 0 deletions mip_convert/mip_convert/plugins/ukcm2/data/UKCM2_land_mappings.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -344,3 +344,14 @@ positive =
reviewer = none
status = embargoed
units = m

[lai_tavg-u-hxy-lnd]

@matthew-mizielinski Matthew Mizielinski (matthew-mizielinski) Jul 1, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks wrong (section 19 not in UKCM2)

comment =
component = land
dimension = longitude latitude time
expression = land_class_mean(m01s19i014[lbtim_ia=240,lbproc=128], m01s19i013[lbtim_ia=240,lbproc=128], land_class='veg')
mip_table_id = land
positive =
reviewer = none
status = embargoed
units = 1
4 changes: 2 additions & 2 deletions mip_convert/mip_convert/plugins/ukcm2/data/UKCM2_seaIce_mappings.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -767,7 +767,7 @@ units = 1
comment =
component = seaIce
dimension = longitude latitude time
expression = apply_SI3_U_coordinates(siu)
expression = apply_ocean_coordinates(siu, u_coord_reference)
mip_table_id = seaIce
positive =
reviewer = none
Expand All @@ -778,7 +778,7 @@ units = m s-1
comment =
component = seaIce
dimension = longitude latitude time
expression = apply_SI3_V_coordinates(siv)
expression = apply_ocean_coordinates(siv, v_coord_reference)
mip_table_id = seaIce
positive =
reviewer = none
Expand Down