Gauge (updating) Unit

The Gauge (Updating) Unit is a method of specifying a time series of observed water level or discharge at a given node (or nodes) so that the model can self-adjust to meet these conditions; it can also be used to project error in model forecasting.

Data

Theory and Guidance

In real-time forecasting, there is a need to improve existing models in real time by an updating process; this enables improvements on the model failing to meet observed data for whatever reason, e.g. geomorphology, inability for one model to accurately represent every situation. It also provides an updated state from which to begin a forecast.

The Gauge Unit is a method of specifying a time series of observed water level or discharge at a given node (or nodes) so that the model can self-adjust to meet these conditions; it can also be used to project the error to use for the model forecast.

This is not intended to be a substitute for model calibration; updating methods are reliant on a well-calibrated model to produce more accurate results.

It is proposed that the Flood Modeller inputs used in updating will be made generic in order that in the future different updating methods can be introduced and feed the correct to the Flood Modeller simulation at run-time. Currently, one updating method has been introduced, which serves as the Flood Modeller default.

The general response will be simple, independent of the eventual update method used - for RIVER units, a discharge-time series will be applied at the location of the gauge unit, acting as a "correction flow" to be added or removed from the model. Where the updating is based on stage, rather than level, rating information must be supplied within the gauge unit to associate stage with discharge. For reservoir units, extra volume is added or removed from the reservoir to adjust it to the observed level.

Connectivity

The gauge unit node is superimposed on top of an existing node in much the same way as the QH-Rating unit does currently, i.e. the node in question would still be attached to exactly two other units. Flow is added into the system by means of a lateral inflow to a river unit, or by an external volume source in the case of a reservoir. The current constraint is that the Gauge Unit must be attached to a downstream node of a river unit, or to a reservoir unit.

Methodology

General terminology

In general, a simulation will be undertaken from a start time T_S to an end time T_E. The duration of the entire simulation will be termed the simulation period .

The time at which gauged data is available up to, T₀, is termed the time of forecast and denotes the time where updating must cease and forecasting may begin.

The time beyond T₀ will be known as the forecast period, and the time prior to this will be termed the pre-forecast period.

Updating and Forecasting

There are two processes to be modelled - updating and forecasting. Updating can occur up to the time of forecasting and involves the process of altering model parameters to bring the simulated results into line with observed results - in the proposed Flood Modeller method this will be by way of generating an additional flow into the system. Forecasting will use the error estimated during the update period and use this to project into the future. The user may select whether he wants to update only or to forecast (which involves updating during the pre-forecast period) as well.

For updating on RIVER units, in either case, a simulation will involve one simulation up to the time of forecast with no effect from the Gauge Unit, from which the model will derive the relevant parameters for updating; a further simulation will then be performed using the additional flow derived from the updating parameters during the pre-forecast period, projected into the forecast period if appropriate. Thus the updating/forecasting run will consist of the original model simulation augmented by extra inflows at the gauged site(s).

For updating on RESERVOIR units, there is no concept of a forecasting period and therefore no forecasting is performed. The model interactively updates itself based on observed readings at specified times, which may be sparse.

Multiple Gauge sites

If there are multiple gauged RIVER sites in one model, then it is possible that these will not be independent, and thus updating each site simultaneously would lead to incorrect updating due to double-counting. The method therefore involves running a number of iterations, updating each site simultaneously. The number of extra iterations should be at least equal to the maximum number of non-independent sites. A simple illustration of this is provided in Appendix A. The user is able to specify the number of updating iterations (defaulting to the number of RIVER updating sites + 1) in the gauge control file.

In the case of RESERVOIR units, the required volume deficit or excess correction is made instantaneously at the time of reading and therefore no extra iterations are required.

Missing data

The updating scheme is able to cope with the occasional absence of data at a gauged site. The options are to interpolate across missing bands, for the gauge site to be ignored or that the updating is temporarily switched off until data is specified again. If updating is switched off, this will be phased out gradually to reduce the risk of instabilities. The modeller will enter which strategy to use for a missed reading, and the maximum acceptable time interval beyond which a reading is treated as 'missing', within the data file.

For updating based on RESERVOIR levels, it is recommended that the limit strategy is set to "PARTIAL" and the missing data strategy is set to "DISABLE". This enables sparse data to be entered, therefore invalid data will be temporarily ignored, and missing data will temporarily disable updating. When a valid reading is detected, updating will be switched back on.

Inaccurate readings

The gauge unit may be able to set an upper and lower bound, outside of which, any reading will be ignored as unfeasible and the scheme will either treat this as missing data or discard the site entirely. Consideration should also be given to whether the gradient of the gauged readings (with respect to time) exceeds a preset maximum, to prevent sudden 'spiking', which could occur after as period of missing data. Such a site should be disregarded for the entire simulation, since there is no obvious correction to apply. The modeller will enter which strategy to use within the data file.

Updating based on water level (RIVER units)

In order to update on water level for RIVER units by adding an extra flow to the model, it is necessary to provide some rating information to equate flows to water levels. The updating technique will still determine the difference in level, and will use the rating information to derive the additional flow. Thus if the simulated water level is h S and the updated level is h U, then the additional flow DQ is given by

Thus if the provided rating curve is "accurate", then this assumption is perfectly valid. Practically however, all that is required is a best approximation. As long as the function is reasonably well-behaved, one would expect convergence to the correct level, although this may take more than one iteration. Further problems may be encountered if there exist multiple values of Q for the same h , for example if the flow is reversing. Such a multi-valued function will be disallowed by the simulation engine/interface.

State saving

The current states used at each gauging site are able to be saved in an 'updating initial conditions' file (default on, with file name resultsfilename.uic, although both can be overridden) so that an updating simulation may be stopped and 'hot-started' from the same place, whilst giving the same results had it run all the way through. This file is produced in much the same way as the standard results files, i.e. the parameters are saved at each save interval, and may be extracted 'on the fly' during a subsequent simulation.

Datafile Format

Line 1: Keywords 'GAUGE', '#revision#', revno

Line 2: label1

Line 3: method

Line 4: gauge control file identifier

Line 5: keyword 'STAGE' or 'FLOW', limit_strategy, limit_q, lower, upper, missing_strategy

Line 6: n₁, [t_lag], [h_offset], [t_m], [smooth], [qmult], [qfmax], [t_miss], [qmin]

Lines 7 to 6+n₁: f_i, t_i

Line 7+n₁ : n₂ , [smooth], [extend], [hr_offset], [qrmult]

Lines 8+n1 to
7+n₁+n₂: q_i, h_i

Where

revno = revision number of Gauge unit in flow engine

label1 = node label of the gauge boundary unit

method = update/forecast method to be used. 'uRIVER' - update on readings associated with a RIVER unit; 'uRESERVOIR' - update on water levels associated with a RESERVOIR unit; Blank=default. 'OFF' - no updating is performed. update/forecast method to be used. Blank=default. 'OFF' - no updating is performed.

limit_strategy = strategy to apply on gauge data exceeding user-specified limits: 'NONE'=limits do not apply; 'STRICT' [default]=updating is disabled for entire simulation if one or more non-conformity is encountered; 'PARTIAL'=non-conformant data is ignored.

limit_q = Quantity to which limits apply: 'GRADIENT'=rate of change of gauged data w.r.t. time (df/dt; t in hours); 'VALUE' [default]=gauged data

lower = lower limit of acceptable gauge data (if lower≥0 and 'GRADIENT' is selected, lower will be set to -upper ). Units are in quantity/hour if 'GRADIENT' selected.

upper = upper limit of acceptable gauge data. Units are in quantity/hour if 'GRADIENT' selected.

missing_strategy = strategy to apply on finding missing data - 'INTERP' [default] - interpolate across missing data; 'DISCARD' - discard the gauge unit entirely; 'DISABLE' - disable updating until the next reading is found.

n₁ = number of ensuing data pairs

t_lag = time to reduce values in table by

h_offset = datum offset from which measurement was taken

t_m = units of time (chosen from the standard Flood Modeller list)

smooth = keyword 'LINEAR' or 'SPLINE' to denote interpolation method of supplied table

qmult = multiplier to apply to supplied flow values

qfmax = factor applied to updated flow to limit the adjusted flow to qfmax*(existing flow).

t_miss = maximum time interval between gauge data (in seconds) - an interval greater than this will lead to data between these two values being treated as missing.

qmin = minimum flow allowed at updating node - any adjusted flow calculated as falling below this level will be capped at this level.

f_ii^th = measured flow or stage value

t_i = corresponding i^th time value (or time/date if t_m ='DATES')

n₂ = number of ensuing rating data pairs (can be zero for flow measurement)
extend = keyword 'EXTEND', 'NOEXTEND' to apply to rating continuation methodology

hr_offset = datum offset to which rating applies

qrmult = multiplier to apply to rating flow values

q_ii^th = flow value in rating table

h_i = corresponding i^th stage value in rating table

The method file identifier contains any driver/parameter information specific to the chosen method. The format will be dependent on the method selected.

Event File flags

The following flags are available in the Event File, and may be set from the run forms interface:

UpdatingDiagnostics =<binary>. If set to 1, a table of the values used in updating will be written to a new updating diagnostics file (*.iud).

UpdatingZZN =<string> . [Optional] file name of the results files (zzn/zzl) written after the first updating iteration (for use in generating hydraulic initial conditions). The path may be specified as relative or absolute. The zzn [or any 3-digit] extension is optional and will be appended or overwritten by zzn (& zzl) as appropriate.
UpdatingInitialConditions =<string>. [Optional] file name of the updating initial conditions file to be read. The path may be specified as relative or absolute.

WriteUpdatingInitialConditions =<binary>. If set to 1, a uic file is written, otherwise not. The default is 1.

OutputUICFilename =<string>. Filename/path of the uic file. If missing or blank, and WriteUpdatingConditions is 1, the default results file name + '.uic' is used.

DestroyInvalidUICFile =<binary>. If set to 1, the output uic file will not be retained should the simulation fail, or updating is not performed. The default is 1.

UpdatingColdStart =<binary>. If set to 1, updating is performed on the initial timestep, t₀. Otherwise, the first updating occurs after one save interval. The default is 1.

UnupdatedSnapshotTime =<time>,

UnupdatedSnapshotFile =<string>. Simulation time and output filename respectively at which to write 'unupdated' snapshots. Similar to snapshot file/times, but only written on first updating iteration. If no updating is present in the model, then there is effectively only one "updating iteration", and so this will be written anyway.

Datafile example

Gauge control file format

The gauge control file is a simple ASCII file consisting of a list of keyword=value lines. Valid keywords are as follows:

AnalysisPeriod (real) - time of the Analysis period (hours).

UpdateIterations (integer) - number of updating iterations to perform. If not entered, this will default to the number of gauge sites + 1.

PhaseDecay (real) - decay factor for the phase error.

AmplitudeDecay (real) - decay factor for the amplitude error.

Alpha (real) - phase error weighting factor.

The list must contain the keyword AnalysisPeriod ; all other keywords are optional. The ordering of keywords within the list is insignificant.

Gauge control file example

Flood Modeller Default method

The default method that is currently used for Flood Modeller updating is based on the approach by Rungo et al. The updating procedure involves estimation of a phase and amplitude error of the measured quantity during the pre-forecast period. The estimation of both these components of the error helps distinguish between those errors, e.g. if one was correcting the error purely on amplitude, then a phase error could be interpreted as a large amplitude error.

For a phase error to be detected, it is necessary to compare the results from a number of previous timesteps, thus the procedure is to minimise an error function over a period of time, termed the Analysis Period.

We can estimate the correction to the simulated solution S(t) to achieve the measured solution M(t) by the addition of the errors due to amplitude and phase E^A(t) and E^P(t) respectively, thus

The actual phase error, i.e. in units of time, E^q(t), is given by

The goal is therefore to minimise the following function over a period of time

which is achieved by partially differentiating with respect to E^A and Eq in turn and setting each derivative to zero. The i subscript denotes the value at the relevant point in time, and the summation is performed over the appropriate analysis period, which is to be determined by the modeller during the calibration phase of the modelling. The analysis period should be large enough to capture any phase error.

A time series of amplitude and phase errors is therefore produced which can then be used to derive a time series of the correction term. It also gives a phase and amplitude error at the time of forecast, which can be used to project into the forecast period. The most simple way of doing this is by applying decay terms λ_A and λ_P , which can be zero or positive, and are applied to the amplitude error and phase error respectively at the time of forecast, i.e.

Appendix A - Multiple Gauge site illustration

The following is a simple illustration of how updating multiple gauge sites may be performed by simultaneous updating and iterating towards a solution. Consider four gauge sites, named Site A-D in upstream to downstream order, all with measured values of 100m³/s and simulated values of 110m³/s from the raw simulation (i.e. prior to updating). Assuming that (i) adjusting the discharge at each site has an instantaneous effect on any gauged downstream site, (ii) backwater effects can be discounted and (iii) the updating procedure has no way of determining the independency or otherwise of the sites, then the first iteration would be to remove 10m³/s at each of the sites. The correct adjustment however would be to remove 10m³/s from Site A only. After the first iteration, this would give the correct discharge at Site A, but would overpredict the adjustment at each of the remaining sites. On the second iteration, the adjustment can be made to Site B, which will mean that Sites A and B are correct, but over-correction has again occurred at Sites C and D, and so on. After the fourth iteration, the correct adjustment should have been made to all four of the sites. This is illustrated in the table below. "Sim 0" refers to the raw simulation without any updating; "Sim 1-4" refer to successive simulations with iterative updating; "Point adjustment" indicates the extra discharge that will be added at this node based on the current iteration; "Cumulative point adjustment" is the extra discharge that will be added, including all previous iterations; Cumulative u/s influence is the extra discharge that has been added from all the upstream nodes.