API reference#

Network#

Implement plugin model class

class hybridurb.networksutils.network.NetworkModel(root=None, mode='w', config_fn=None, data_libs=None, logger=<Logger hybridurb.networksutils.network (WARNING)>, deltares_data=False, crs: int | str = 4326)#

General and basic API for models in HydroMT

property crs#: Returns coordinate reference system embedded in region.

read()#: Method to read the complete model schematization and configuration from file.

read_forcing()#: Read forcing at <root/?/> and parse to dict of xr.DataArray

read_graphmodel()#: Read graphmodel at <root/?/> and parse to networkx DiGraph

read_results()#: Read results at <root/?/> and parse to dict of xr.DataArray

read_states()#: Read states at <root/?/> and parse to dict of xr.DataArray

read_staticgeoms()#: Read staticgeoms at <root/?/> and parse to dict of geopandas

read_staticmaps()#: Read staticmaps at <root/?/> and parse to xarray Dataset

setup_basemaps(region, report: str = None, **kwargs)#

Define the model region. and setup the base graph - with geometry and id

Adds model layer: * region geom: model region

Parameters#

region: dict: Dictionary describing region of interest, e.g. {‘bbox’: [xmin, ymin, xmax, ymax]}. See parse_region() for all options.
**kwargs: Keyword arguments passed to _setup_graph(**kwargs)

setup_dag(G: Graph = None, targets=None, target_query: str = None, weight: str = None, loads: list = [], report: str = None, algorithm: str = 'simple', **kwargs)#

This component prepares subgraph as Directed Acyclic Graphs (dag) using shortest path step 1: add a supernode to subgraph (representing graph - subgraph) step 2: use shortest path to prepare dag edges (source: node in subgraph; target: super node)

in progress

Parameters#

G : nx.Graph targets : None or String or list, optional (default = None)

DAG super targets. If None, a target node will be any node with out_degree == 0. If a string, use this to query part of graph as a super target node. If a list, use targets as a list of target nodes.

weightNone or string, optional (default = None): Weight used for shortest path. If None, every edge has weight/distance/cost 1. If a string, use this edge attribute as the edge weight. Any edge attribute not present defaults to 1.
loadsNone or list of strings, optional (default - None): Load used from edges/nodes attributes. If None, every node/edge has a load equal to the total number of nodes upstream (nnodes), and number of edges upstream (nedges). If a list, use the attributes in the list as load. The attribute can be either from edges or from nodes. Any attributes that are not present defaults to 0.
algorithmstring, optional (default = ‘simple’): The algorithm to use to compute the dag. Supported options: ‘simple’, ‘flowpath’. Other inputs produce a ValueError. if ‘simple’, the input graph is treated as a undirected graph, the resulting graph might alter the original edges direction If ‘flowpath’, the input graph is treated as a directed graph, the resulting graph do not alter the original edges direction. Both option will have less edges. the missing edges indicates their insignificance in direction, i.e. both way are possible. # FIXME

Arguments#

use_super_targetbool: whether to add a super target at the ends of all targets. True if the weight of DAG exist for all edges. False if the weight of DAG also need to consider attribute specified for targets.

Arguments#

G: nx.DiGraph: Directional Graph
subgraph_fn:: Directional Graph node that are used as target to find predecessors
target_query: bool: Whether to include the input node in the results
**kwargs:: weight = None, for setup_dag # FIXME report = None, for plotting

This component update edges attributes to the graph model

Parameters#

graph_class: str: Class in networkx for creating the graph.
edges_fn: str: Name of the edges shp file specified in data yml.
edges_id_col: str: Columns name used in the shp file as the id of edges.
nodes_fn: str Not Implemented: Name of the nodes shp file specified in data yml.
nodes_id_col: str Not Implemented: Columns name used in the shp file as the id of nodes.

Notes#

This function can only be called once

setup_node_attributes(nodes_fn: None | str | GeoDataFrame, id_col: str | None = None, attribute_cols: list | None = None, snap_offset: float | None = None, **kwargs)#: This component update nodes attributes to the graph model

See Also#

_setup_nodes

setup_nodes(nodes_fn: None | str | GeoDataFrame, id_col: str, attribute_cols: list = None, snap_offset: float = 1e-05, **kwargs)#: This component setup nodes locations and attributes to the graph model

See Also#

_setup_nodes

setup_partition(G: Graph = None, subgraph_fn: str = None, algorithm: str = 'simple', report: str = None, contracted: bool = False, **kwargs)#

This component prepares the partition based on the connectivity of graph

Parameters#

subgraph_fnstr: Name of the subgraph instance. If None, self._graphmodel will be used for partition; the function will update the self._graphmodel if String and new, self._graphmodel will be used for partition; the function will create the instance in self._subgraphmodels if String and old, self._subgraphmodels[subgraph_fn] will be used for partition; the function will update the instance in self._subgraphmodels[subgraph_fn]
algorithmstr: Algorithm to derive partitions from subgraph. Options: [‘simple’, ‘louvain’ ] testing methods: “simple” : based on connected components, every connected components will be considered as a partition (undirected) “flowpath” : based on direction of the edges, the graph is devided into a few partitions, each of which represents a target node with all of its sources. “louvain”: based on louvain algorithm (work in progress) (undirected). “It measures the relative density of edges inside communities with respect to edges outside communities. Optimizing this value theoretically results in the best possible grouping of the nodes of a given network.”(from wikipedia)
contractedbool: Specify whether to build contracted graph from the parititons. So a new contracted graph is created, with a node representing a part of the graph; edges represernting the connectivity between the parts If True, each partition will be contracted to a node If False, no contraction is performed

setup_pruning(G: Graph = None, subgraph_fn: str = None, algorithm: str = 'simple', edge_prune_query: str = None, node_prune_query: str = None, weight: str = None, loads: list = [], max_loads: float = [], report: str = None, **kwargs)#

This component prepares the pruning the 1D flow network based on aggregation

Parameters#

G: nx.Graph, optional (default = None): Networkx graph to allow the function being called by another function. If None, the network graph from self._graphmodel or self._subgraphmodels[subgraph_fn] will be used
subgraph_fnstr, optional (default - None): Name of the subgraph instance. If None, self._graphmodel will be used for partition; the function will update the self._graphmodel if String and new, self._graphmodel will be used for partition; the function will create the instance in self._subgraphmodels if String and old, self._subgraphmodels[subgraph_fn] will be used for partition; the function will update the instance in self._subgraphmodels[subgraph_fn]
algorithmstr, optional (default - simple): # FIXME: after creating dag, some edges are missing, also the edge attributes. Therefore, the loads are imcomplete. will not have influence if the loads are applied on nodes. Algorithm to derive partitions from subgraph. Options: [‘simple’, ‘auto’] testing methods: “simple” : based on user specifie. The algorithm will prune the edges/nodes based on edge_prune_query and node_prune_query. “flowpath”: based on direction defined for edges. “auto” : based on analysis of network. The algrithm will prune the arborescence of the tree. A threshold can be set to determine whether an arborescence is small enough to be pruned.
weightNone or string, optional (default = None): Weight used for shortest path. Used in setup_dag. If None, every edge has weight/distance/cost 1. If a string, use this edge attribute as the edge weight. Any edge attribute not present defaults to 1.
loadsNone or list of strings, optional (default - None): Load used from edges/nodes attributes. Used in setup_dag. If None, every node/edge has a load equal to the total number of nodes upstream (nnodes), and number of edges upstream (nedges). If a list, use the attributes in the list as load. The attribute can be either from edges or from nodes. Any attributes that are not present defaults to 0.

setup_subgraph(G: Graph = None, subgraph_fn: str = None, edge_query: str = None, edge_target: str = None, node_query: str = None, node_target: str = None, algorithm: str = None, target_query: str = None, weight: str = None, report: str = None, **kwargs)#

This component prepares the subgraph from a graph

In progress

Parameters#

subgraph_fnstr: Name of the subgraph instance. If None, the function will update the self._graphmodel if String and new, the function will create the instance in self._subgraphmodels if String and old, the function will update the instance in self._subgraphmodels

Arguments#

edge_querystr: Conditions to query the subgraph from graph. Applies on existing attributes of edges, so make sure to add the attributes first.
edge_targetstr: Conditions to request from subgraph the connected components that has the edge_target id Applies on existing id of edges.
node_querystr: Conditions to query the subgraph from graph. Applies on existing attributes of nodes, so make sure to add the attributes first.
node_targetstr: Conditions to request from subgraph the connected components that has the node_target id Applies on existing id of nodes.
algorithmstr = None: Algorithms to apply addtional processing on subgraph. Supported options: ‘patch’, ‘mst’, ‘dag’. If None, do not apply any method - disconnected subgraph If “patch”, patch paths from queried nodes in SG to an signed outlet if “mst”, steiner tree approximation for nodes in SG if “dag” dijkstra shortest length resulted Directed Acyclic Graphs (DAG) # FIXME: add weight as argument; combine setup_dag function

weight : str = None

write()#: Method to write the complete model schematization and configuration to file.

write_forcing()#: write forcing at <root/?/> in model ready format

write_graphmodel()#: write graphmodel at <root/?/> in model ready format

write_results()#: write results at <root/?/> in model ready format

write_states()#: write states at <root/?/> in model ready format

write_staticgeoms()#: Write staticmaps at <root/?/> in model ready format

write_staticmaps()#: Write staticmaps at <root/?/> in model ready format

class hybridurb.networksutils.network_from_delft3dfm.Delft3dfmNetworkWrapper(config_fn: Path)#

This is a wrapper class for the Delft3D flexible mesh (FM) type Network.

Methods: setup_delft3dfm setup_network

get_geoms_from_model()#

Retrieves geometries from the model.

Returns:: Geometries from the model. If the function is unable to retrieve the geometries, it returns None.

get_network_from_geoms()#: setup network. Note will always overwrite.

get_network_properties()#: calculate physical properties that can only be computed in the network

hybridurb.networksutils.network_from_delft3dfm.mystring = '\n\n'#

get_geoms_from_model:: input: region output: # geoms/*.geojson
setup_network:: _config: yaml secifying which files are read as nodes and which as edges input: geoms/*.geojson output: model/graph.gpickle
simplify_network: get flow path: _config: yaml secifying which files are read as nodes and which as edges input: geoms/*.geojson output: model/graph.gpickle

Runner#

class hybridurb.runners.runner.NowcastRunner(root_dir: Path, t0: str, name: str = '', crs='EPSG:4326')#

Initializes the NowcastRunner class.

Parameters:

root_dir – Path object. Root directory containing model, input, and output subdirectories.
t0 – String. The start time for simulation, in the format ‘yyyymmddHHMM’, e.g. ‘201605300900’.
name – String. A name for the runner instance, defaults to an empty string.
crs – String. Coordinate reference system for the geographic data, defaults to ‘EPSG:4326’.

This method initializes various directory paths, the start time, and the name for the runner instance. It also calls the _initialise_model method to initialize the graph, lambda and statistical models.

run()#

Runs the HybridUrb model using a bottom-up approach that includes detailed physical processes.

This method runs the simulation by reading rainfall data, running the graph model, preparing predictors, performing logistic regression, and writing the output to a CSV file in the output directory. It also prints the time taken for the simulation.

Equations used for bottom up appraoch:

‘n_F ~ C(Cat)+NS+PS+OT+d’: n_F: probability of flooding C(Cat): category of the noes NS: saturation of the nodes PS: saturation of the pipes OT: return periods of the outfall location d: depth of the nodes

Reads rainfall input and model files:

Rainfall inputs:: [T0]_ens[ENSEMBLE_ID].csv and grid_to_nodes_mapping.csv. The former contains rainfall intensity (unit mm/h) per rainfall catchment for each ensemble member. The latter contains the mapping of rainfall catchments.
Model files:: GraphModel.gpickle: nx.Graph. graph model of the sewer network. LambdaModel.pickle lambda model for boxcox transformation. StatsModel.pickle statistical model for the logistic regression.

Writes output:

Probabilistic output:: [T0]_ypred_summary.csv A summary table containing the probability of flooding of each node for each rainfall ensemble. Note that the output is not a timeseries.

Returns:: None

run_fews()#

Runs the entire FEWS process, including the preadaptor, model run, and postadaptor.

This method calls the run_preadaptor, run, and run_postadaptor methods in sequence to perform the entire FEWS process. It prints messages before and after each of these steps.

Returns:: None

run_postadaptor()#

Runs a post-adaptor for FEWS. Converts outputs generated by HybridUrb to standard netcdf files supported by Delft-FEWS.

This method prepares the output data, writes it to a netCDF file in the output directory, and also writes a grid file containing the node locations to a netCDF file in the output directory.

Probabilistic nputs generated by HybridUrb:: [T0]_ypred_summary.csv A summary table containing the probability of flooding of each node for each rainfall ensemble. Note that the output is not a timeseries.
Outputs supported by Delft-FEWS:: [T0]_predictions_at_nodes.nc. It contains the probabilistic predictions per model nodes. It has the dimension of realization which contains ENSEMBLE_ID.

T0 (str) is the start time of the input, supported format: yyyymmddHHMM, e.g. 201605300900; ENSEMBLE_ID (int) is the identifier of each ensemble member, known as dimension realization in [T0]_precipitation_on_catchments.nc

Returns:: None

run_preadaptor()#

Runs a pre-adaptor for FEWS. Converts rainfall inputs prepared by Delft-FEWS into rainfall inputs supported by HybridUrb.

This method reads the rainfall data file prepared by Delft-FEWS, converts the unit from mm to mm/hour, and writes the converted data to a CSV file in the input directory.

Rainfall inputs prepared by Delft-FEWS:: [T0]_precipitation_on_catchments.nc. It contains the rainfall quantity (unit mm) per rainfall catchments.
Rainfall inputs supported by HybridUrb:: [T0]_ens[ENSEMBLE_ID].csv and grid_to_nodes_mapping.csv. The former contains rainfall intensity (unit mm/h) per rainfall catchment for each ensemble member. The latter contains the mapping of rainfall catchments.

Returns:: None

hybridurb.runners.runner.predict(rgRR, rgID, X, lams, model, equ, event_id=None)#

Predict the probability of flooding for a given rainfall event rgRR.

Parameters:

rgRR – pd.DataFrame timeseries of the rainfall event. must have time as index and meteo catchments as columns
rgID – pd.DataFrame mapping table to map rainfall at gauges to rainfall at nodes. must contain rg_id. index is node_id
X – nx.Graph graph model (calibrated)
lams – lambda parameter (calibrated)
model – statistical model (calirbated)
equ – equation used for the model
event_id – str used as the identifier for the results

Returns:

y_pred

Utils#

hybridurb.networksutils.delft3dfm_utils.convert_geometries(geometries) → dict#

Converts geometries and calculates additional attributes.

Returns: dict: A dictionary containing the converted geometries.

hybridurb.networksutils.delft3dfm_utils.extract_geometries(model: DFlowFMModel, clip_region: GeoDataFrame | None = None)#

Extracts geometries from the DFlowFM model. If a region is provided, the geometries will be clipped to this region.

Parameters: model (DFlowFMModel): A DFlowFMModel instance containing the model whoes geometries will be extracted. clip_region (gpd.GeoDataFrame, optional): A GeoPandas DataFrame containing the region to which the geometries will be clipped.

If no region is provided, all geometries from the model will be returned.

Returns: dict: A dictionary where the keys are the names of the geometries and the values are GeoPandas DataFrames containing the geometries.

Each DataFrame has a ‘geometry’ column containing the geometry objects, and the rest of the columns contain attributes of the geometries.

hybridurb.networksutils.delft3dfm_utils.get_geometries_from_model(model_root: Path, mdu_fn: str, crs: CRS, clip_region: GeoDataFrame | None = None, geoms_dir: str | None = 'geoms_dir')#

Retrieves geometries from a given model directory.

Parameters:

model_root – The model root directory.
mdu_fn – The mdu file as relative path to model_root.
crs – The coordinate reference system.
clip_region – The region to clip geoms. Defaults to None.
geoms_dir – the dir to save the geoms.

Returns:

The geometries from the model directory.

hybridurb.networksutils.delft3dfm_utils.save_geometries(geometries, out_dir: Path)#: save geometries to dir

API reference#

Network#

Parameters#

Parameters#

Arguments#

See Also#

Arguments#

See Also#

See Also#

Parameters#

Notes#

See Also#

See Also#

Parameters#

Parameters#

Parameters#

Arguments#

Runner#

Utils#