netcdf_utils module for NetCDF input

See also

• netcdf_input

• topotools module for working with topography data

• Meteorological Forcing Python API

The netcdf_utils module provides the Python layer for reading NetCDF topography and meteorological forcing files in GeoClaw. It handles CF attribute parsing, coordinate convention detection, unit resolution (recognized non-contract units are converted via a scale factor; missing or unrecognized units raise), and descriptor writing so that Fortran only needs to open the file and read pre-validated indices.

The main classes are:

• NetCDFInspector — base class; coordinate discovery, fill-value resolution, crop-bound validation.

• TopoInspector — topography subclass; fill-value checking within the crop region, unit enforcement.

• MetInspector — meteorological subclass; wind/pressure variable discovery, CF datetime decoding, unit resolution (recognized non-contract units converted, with a storm-format fallback for missing units; unrecognized units raise), and a magnitude sanity check.

• CFNormalizer — adds or repairs CF attributes in place without modifying data values.

• DescriptorWriter — writes the key=value topo descriptor lines or the &file_info / &variable_info Fortran namelist blocks for met forcing.

Documentation auto-generated from the module docstrings

NetCDF utilities for GeoClaw NetCDF input.

Provides inspector classes that inspect NetCDF files for coordinate metadata, unit consistency, and fill values without loading data arrays. The inspectors produce metadata dataclasses consumed by DescriptorWriter (not yet implemented — pending confirmation of Fortran topo.data parser).

Classes

NetCDFInspector

Base class: opens a file, discovers coordinate variables, detects lon/lat conventions, resolves fill value, validates crop bounds.

TopoInspector(NetCDFInspector)

Adds bathymetry-specific checks: fill values within crop region (fatal), unit verification against GEOCLAW_NETCDF_UNITS[‘topo’].

MetInspector(NetCDFInspector)

Adds met-forcing checks: multi-variable grid/time consistency, unit verification and conversion to contract units, CF time -> seconds offset, ensemble dimension detection.

CFNormalizer

Standalone utility: renames coordinates to CF standard names, adds/fixes standard_name / axis / units / _FillValue attributes, resolves _FillValue vs missing_value conflicts. Does not resample or reproject.

Metadata dataclasses

FileMetadata, TopoMetadata, MetVariableInfo, MetMetadata

class clawpack.geoclaw.netcdf_utils.CFNormalizer(ds: Dataset)

Bases: object

Normalize CF metadata in an xarray Dataset.

Performs in-memory attribute fixups: * Renames coordinate variables to CF standard names where unambiguous. * Adds standard_name, axis, and units attributes to coordinate

System Message: ERROR/3 (/Users/rjl/clawpack_src/clawpack_master/geoclaw/src/python/geoclaw/netcdf_utils.py:docstring of clawpack.geoclaw.netcdf_utils.CFNormalizer, line 6)

Unexpected indentation.

variables if missing.

System Message: WARNING/2 (/Users/rjl/clawpack_src/clawpack_master/geoclaw/src/python/geoclaw/netcdf_utils.py:docstring of clawpack.geoclaw.netcdf_utils.CFNormalizer, line 7)

Block quote ends without a blank line; unexpected unindent.

• Resolves _FillValue vs missing_value conflicts (CF precedence: _FillValue wins; missing_value is promoted if _FillValue absent; conflict triggers a warning).

Does NOT resample, reproject, or modify data values.

Parameters

dsxr.Dataset

Dataset to normalise. A copy is made; the original is not modified.

Examples

>>> ds_norm = CFNormalizer(ds).normalize()

normalize() → Dataset

Apply all normalizations and return the modified dataset.

class clawpack.geoclaw.netcdf_utils.DTopoInspector(path: str | Path, var_name: str | None = None, time_reference: str | None = None, crop_bounds: tuple[float, float, float, float] | None = None, assume_units: str | None = None)

Bases: NetCDFInspector

Interrogate a NetCDF moving-topography (dtopo) file.

Required roles: longitude, latitude, time, and the deformation (dZ) variable. Beyond the base class this:

• Discovers the deformation variable (by common names, else the unique 3-D data variable).

• Requires time to be the variable’s slowest (first) dimension.

• Validates that the time axis is uniformly spaced — the Fortran dtopo machinery reconstructs times as t0 + k*dt.

• Converts the time axis to simulation seconds: numeric time values pass through unchanged; CF datetime values require an explicit time_reference and become seconds since that reference.

Parameters

pathstr or Path

Path to the NetCDF file.

var_namestr, optional

Name of the deformation variable; auto-discovered if omitted.

time_referencestr or datetime-like, optional

Reference datetime when the file’s time axis decodes to datetimes.

crop_boundstuple of four floats, optional

(lon_min, lon_max, lat_min, lat_max); validated against file extent.

inspect_dtopo() → DTopoMetadata

Fully inspect the dtopo file and return a DTopoMetadata.

Verifies deformation units (meters), records the source unit on self.source_units, and stores a scale_factor in the metadata. A recognised non-meter unit yields a scale_factor (applied in memory by DTopography.read or by Fortran via the descriptor); a missing or unrecognised unit still raises (units are never assumed).

class clawpack.geoclaw.netcdf_utils.DTopoMetadata(source_file: Path, x_name: str, y_name: str, time_name: str | None, lon_wrap: int | None, y_increasing: bool, dim_order: list[str], fill_value: float | None, crop_bounds: tuple[float, float, float, float] | None, var_name: str, t0: float, dt: float, mt: int, lon_wrap_offset: float = 0.0, scale_factor: float = 1.0)

Bases: FileMetadata

FileMetadata plus dtopo-specific fields.

The time axis is collapsed to (t0, dt) in simulation seconds: the Fortran dtopo machinery requires uniform time spacing and reconstructs times as t0 + k*dt, so Fortran never parses the file’s time variable.

dt: float

lon_wrap_offset: float = 0.0

mt: int

scale_factor: float = 1.0

t0: float

var_name: str

class clawpack.geoclaw.netcdf_utils.DescriptorWriter

Bases: object

Write NetCDF descriptor metadata for GeoClaw input files.

For topo (type 4) entries the descriptor is a block of key = value lines written immediately after the topo_type line in topo.data. Fortran’s read_netcdf_descriptor parses these lines until it encounters a blank line.

Usage — topo:

meta = TopoInspector(path, var_name='z').inspect_topo()
with open('topo.data', 'a') as f:
    DescriptorWriter.write_topo_descriptor(f, meta)

Usage — met (storm file):

meta = MetInspector(path, var_map).inspect_met()
with open('storm.nc', 'w') as f:
    f.write('netcdf\n')   # format header written by caller
    DescriptorWriter.write_met_descriptor(f, meta)

static write_dtopo_descriptor(f, meta: DTopoMetadata) → None

Write the key=value descriptor block for one dtopo file.

Same format and parser contract as the topo descriptor (a blank line terminates the block), written immediately after the 9-line per-file block in dtopo.data. The time axis is carried as (t0, dt) in simulation seconds so Fortran never parses CF time.

Parameters

ffile-like object

Open for writing (text mode).

metaDTopoMetadata

Output of DTopoInspector.inspect_dtopo().

static write_met_descriptor(f, meta: MetMetadata) → None

Write the &file_info / &variable_info namelist-style body of a GeoClaw NetCDF met/storm descriptor file.

The caller is responsible for writing the netcdf format header line before calling this method.

Parameters

ffile-like object

Open for writing (text mode).

metaMetMetadata

Output of MetInspector.inspect_met().

static write_topo_descriptor(f, meta: TopoMetadata) → None

Write the key=value descriptor block for one topo file.

f should be positioned immediately after the topo_type line has been written. A trailing blank line is written to terminate the block for the Fortran parser.

Parameters

ffile-like object

Open for writing (text mode).

metaTopoMetadata

Output of TopoInspector.inspect_topo().

class clawpack.geoclaw.netcdf_utils.FileMetadata(source_file: Path, x_name: str, y_name: str, time_name: str | None, lon_wrap: int | None, y_increasing: bool, dim_order: list[str], fill_value: float | None, crop_bounds: tuple[float, float, float, float] | None)

Bases: object

Coordinate and convention metadata for a single NetCDF file.

crop_bounds: tuple[float, float, float, float] | None

dim_order: list[str]

fill_value: float | None

lon_wrap: int | None

source_file: Path

time_name: str | None

x_name: str

y_increasing: bool

y_name: str

class clawpack.geoclaw.netcdf_utils.MetInspector(path: str | Path, variable_map: dict[str, str] | None = None, crop_bounds: tuple[float, float, float, float] | None = None, time_reference: str | None = None, fill_action: str = 'warn', assume_units: bool = False, format_units: dict[str, str] | None = None, skip_sanity_check: bool = False)

Bases: NetCDFInspector

Interrogate a NetCDF meteorological forcing file.

Additional checks beyond the base class:

• Verifies that all requested variables share the same spatial grid and time axis.

• Validates and (if needed) converts units to contract units via units.py.

• Converts CF time to seconds from a user-provided reference offset.

• Detects ensemble/member dimensions and raises if any are non-singleton.

Parameters

pathstr or Path

Path to the NetCDF file.

variable_mapdict, optional

Maps GeoClaw role strings to variable names in the file, e.g.:

{'wind_u': 'u10', 'wind_v': 'v10', 'pressure': 'msl'}

Roles not given (or the whole map, when omitted) are auto-discovered by CF standard_name first, then by common variable names. An explicit entry overrides discovery for that role.

crop_boundstuple of four floats, optional

(lon_min, lon_max, lat_min, lat_max).

time_referencestr or datetime-like, optional

Reference datetime for the time_offset calculation. The time_offset written to the descriptor will be the number of seconds between time_reference and the first time in the file. Defaults to the Unix epoch (1970-01-01T00:00:00).

fill_actionstr, optional

‘abort’ or ‘warn’. Met files may legitimately have fill values at the domain edges (Fortran handles edge fill). Default is ‘warn’.

assume_unitsbool, optional

Explicit escape hatch for a file whose forcing variables have no units attribute. When True, each variable is assumed to already be in its contract unit instead of raising. This must be set deliberately; missing units are never silently assumed.

format_unitsdict, optional

{geoclaw_role: unit_string} giving the units documented by the storm format (e.g. NWS13/OWI pressure is mbar). Used only for a variable that has no units attribute: the format’s documented unit is assumed and converted (e.g. mbar -> Pa). Takes precedence over assume_units for the roles it covers.

skip_sanity_checkbool, optional

Skip the post-unit-resolution magnitude sanity check on pressure/wind (see _check_magnitude). Escape hatch for exotic-but-valid files; defaults to False.

inspect_met() → MetMetadata

Fully inspect the met file and return a MetMetadata instance.

Roles missing from variable_map are auto-discovered (CF standard_name first, then common variable names).

class clawpack.geoclaw.netcdf_utils.MetMetadata(source_file: Path, x_name: str, y_name: str, time_name: str | None, lon_wrap: int | None, y_increasing: bool, dim_order: list[str], fill_value: float | None, crop_bounds: tuple[float, float, float, float] | None, variables: list[MetVariableInfo], time_offset: float, fill_action: str, time_scale: float = 1.0)

Bases: FileMetadata

FileMetadata plus met-forcing-specific fields.

fill_action: str

time_offset: float

time_scale: float = 1.0

variables: list[MetVariableInfo]

class clawpack.geoclaw.netcdf_utils.MetVariableInfo(var_name: str, geoclaw_role: str, source_units: str, scale_factor: float = 1.0)

Bases: object

Maps one NetCDF variable to its GeoClaw role.

geoclaw_role: str

scale_factor: float = 1.0

source_units: str

var_name: str

class clawpack.geoclaw.netcdf_utils.NetCDFInspector(path: str | Path, crop_bounds: tuple[float, float, float, float] | None = None)

Bases: object

Open a NetCDF file and inspect its coordinate metadata.

No data arrays are loaded; only coordinate values and variable attributes are accessed. Dask-lazy chunking is enabled so that any accidental downstream .compute() calls are bounded.

Parameters

pathstr or Path

Path to the NetCDF file.

crop_boundstuple of four floats, optional

(lon_min, lon_max, lat_min, lat_max) in the same convention as the file. When provided, bounds are validated against file extent.

close() → None

ds: Dataset

inspect(var_name: str, time_name_override: str | None = None) → FileMetadata

Interrogate var_name and return a FileMetadata instance.

Parameters

var_namestr

Name of the primary data variable (used to determine dim_order).

time_name_overridestr, optional

Force a specific time coordinate name instead of auto-detecting.

class clawpack.geoclaw.netcdf_utils.TopoInspector(path: str | Path, var_name: str | None = None, crop_bounds: tuple[float, float, float, float] | None = None, assume_units: str | None = None, skip_sanity_check: bool = False)

Bases: NetCDFInspector

Interrogate a NetCDF bathymetry/topography file.

Additional checks beyond the base class:

• Verifies the data variable’s units attribute matches the contract unit (meters). If units are convertible via units.py, records the source units in the metadata; Fortran will need a conversion factor. If units are unrecognised, raises ValueError.

• Checks for fill values (NaN) within the crop region and raises ValueError — silent NaN in bathymetry is numerically fatal.

Parameters

pathstr or Path

Path to the NetCDF topography file.

var_namestr, optional

Name of the elevation/bathymetry variable (e.g. 'z', 'elevation'). If omitted, the variable is auto-detected by searching for known CF standard_name values (surface_altitude, height_above_mean_sea_level, …) and then falling back to common names (z, elevation, topo, …). A ValueError is raised if no match is found.

crop_boundstuple of four floats, optional

(lon_min, lon_max, lat_min, lat_max).

assume_unitsstr, optional

Explicit escape hatch for a file whose elevation variable has no units attribute. When given (e.g. 'm'), that unit is assumed instead of raising. This must be set deliberately by the caller; missing units are never silently assumed.

skip_sanity_checkbool, optional

Skip the post-unit-resolution magnitude sanity check (see _check_magnitude). Escape hatch for exotic-but-valid files; defaults to False.

inspect_topo() → TopoMetadata

Fully inspect the topo file and return a TopoMetadata instance.

topo_entries() → list[list]

Return a list of ready-to-use topo entries for topofiles.

Each entry is [4, filepath, TopoMetadata]. When no wrapping is needed this returns a list of one entry. When crop_bounds straddle the file’s lon cut point, returns two entries pointing to the same file with different lon_wrap_offset and crop_bounds values.

crop_bounds on the inspector are in domain coordinates; this method converts them to file coordinates before storing in the returned metadata. Fortran can then use crop_bounds directly against file coordinate arrays before applying lon_wrap_offset.

class clawpack.geoclaw.netcdf_utils.TopoMetadata(source_file: Path, x_name: str, y_name: str, time_name: str | None, lon_wrap: int | None, y_increasing: bool, dim_order: list[str], fill_value: float | None, crop_bounds: tuple[float, float, float, float] | None, var_name: str, source_units: str, fill_action: str, lon_wrap_offset: float = 0.0, scale_factor: float = 1.0)

Bases: FileMetadata

FileMetadata plus topo-specific fields.

fill_action: str

lon_wrap_offset: float = 0.0

scale_factor: float = 1.0

source_units: str

var_name: str

clawpack.geoclaw.netcdf_utils.compression_encoding(compression, chunksizes=None) → dict

Translate a compression argument into an xarray encoding fragment.

GeoClaw’s NetCDF writers pass a user-facing compression option through here to build the per-variable encoding dict for to_netcdf. netCDF zlib compression is transparent to readers (the netCDF-C library, and hence the Fortran reader, decompresses on read) and, being chunked, keeps the file randomly accessible – unlike an externally gzipped .nc.

Parameters

compressionNone | bool | int | dict

• None / False -> no compression (empty fragment; files stay bit-identical to the uncompressed default).

• True -> the recommended default, zlib level 1 with the byte shuffle filter (level 1 captures nearly all the size gain; higher levels cost CPU for little more).

• int -> zlib at that complevel (1–9) with shuffle.

• dict -> used verbatim (full control, e.g. {'zlib': True, 'complevel': 4, 'shuffle': False}).

chunksizestuple, optional

netCDF chunk shape, applied only when compression is enabled and the caller supplies it (zlib requires chunking; a per-time-slice chunk also matches how Fortran reads dtopo/met time slices). A dict compression that already sets chunksizes is left untouched.

Returns

dict

Encoding keys to merge into the variable’s encoding entry.

clawpack.geoclaw.netcdf_utils.suppress_netcdf4_shape_warning()

Silence netCDF4-python’s spurious NumPy 2.5 shape-assignment warning.

netCDF4.Variable.__setitem__ reshapes the incoming array in place via data.shape = ... (see netCDF4/_netCDF4.pyx), which NumPy >= 2.5 now deprecates in favor of np.reshape. This is entirely internal to netCDF4-python (as of 1.7.4, the latest release) and not something callers can avoid, so wrap var[:] = ... assignments in this context manager rather than letting the warning leak into test output. Safe to remove once netCDF4-python ships a fix upstream.