Constraints in the AEMO tables

Two kinds of table

Standing data describes the equations: their sides, their versions, their sets. It changes when AEMO edits a constraint, not every five minutes. Solution data records what NEMDE did with them: one row per constraint per interval per process. Every table name below links to its full column list in the catalog.

Standing data: the GENERIC_CONSTRAINT package

  GENCONDATA  (one row per GENCONID + EFFECTIVEDATE + VERSIONNO)
    |  operator (<=, >=, =), description, violation weight, which processes use it
    |
    +-- SPDCONNECTIONPOINTCONSTRAINT   LHS unit terms: connection point, bid type, FACTOR
    +-- SPDINTERCONNECTORCONSTRAINT    LHS interconnector terms: FACTOR
    +-- SPDREGIONCONSTRAINT            LHS regional FCAS terms: region, bid type, FACTOR
    +-- GENERICCONSTRAINTRHS           RHS term list, per SCOPE (DS / PD / ST)
    |     +-- GENERICEQUATIONRHS       reusable equations called by X-type terms
    |     +-- EMSMASTER                what each SPD_ID (SCADA point) is
    +-- GENCONSET                      set membership: GENCONSETID contains GENCONID
          +-- GENCONSETINVOKE          when each set is switched on and off

GENCONDATA

The constraint's header: operator in CONSTRAINTTYPE, the plain-English DESCRIPTION, the violation weight, flags for dispatch, pre-dispatch and PASA use. Keyed by GENCONID, EFFECTIVEDATE, VERSIONNO.

SPDCONNECTIONPOINTCONSTRAINT, SPDINTERCONNECTORCONSTRAINT, SPDREGIONCONSTRAINT

The three LHS tables, one per kind of term. Units are keyed by connection point, not DUID; the bridge is below.

GENERICCONSTRAINTRHS

The RHS program of Chapter 9, one row per term, per scope.

GENCONSET and GENCONSETINVOKE

Membership and switching, Chapter 7. AEMO's own description calls the invocation table "the key table for determining what constraints are active in dispatch".

Solution data: what NEMDE published

DISPATCHCONSTRAINT

The five-minute ground truth: LHS, RHS, MARGINALVALUE, VIOLATIONDEGREE, and the exact GENCONID_EFFECTIVEDATE and GENCONID_VERSIONNO that applied. Keyed by constraint, interval, intervention flag and run number.

DISPATCHPRICE, DISPATCHREGIONSUM, DISPATCHINTERCONNECTORRES

Regional prices for energy and the ten FCAS services; regional demand and supply; interconnector flows, losses, and the export and import limits with the constraint ids that set them.

DISPATCH_LOCAL_PRICE

Per DUID: the local price adjustment (sum of MV times factor) and a locally-constrained flag.

DISPATCHCASESOLUTION, DISPATCHBLOCKEDCONSTRAINT, CONSTRAINTRELAXATION_OCD

Run-level totals of violation; constraints blocked from a run; and the relaxed RHS values from an over-constrained dispatch re-run.

P5MIN_CONSTRAINTSOLUTION, PREDISPATCHCONSTRAINT, STPASA_CONSTRAINTSOLUTION

The same columns from the forecast processes: five-minute pre-dispatch (an hour ahead), 30-minute pre-dispatch (to the end of the next market day) and short-term PASA (seven days). Chapter 11.

The price-setter file

Not an MMS table but an XML feed: which offer bands set each price, with their increase weights. Chapter 5.

From a unit to its constraints

The LHS tables name connection points, not units. To ask "which constraints touch this DUID" you walk DUDETAILSUMMARY (DUID to CONNECTIONPOINTID, region, station, participant, schedule type) into the SPD tables on connection point, restricted to the constraint version in force. The unit registry in the catalog is that join, precomputed.

Conventions that bite

Market time

All timestamps are Australian Eastern Standard Time with no daylight saving. The market day starts at 04:00; DISPATCHINTERVAL is the date plus a period number 1 to 288 counted from the interval ending 04:05. A stamp names the interval by its end: start-of-interval readings (DISPATCH_UNIT_SCADA.SCADAVALUE, DISPATCHLOAD.INITIALMW) describe five minutes before the stamp; targets, prices and marginal values describe the stamp itself.

Metering days are different

NEM12 interval metering data is organised by calendar day from midnight: interval 1 is the period ending 00:05 (5-minute meters) or 00:30 (30-minute meters), and each value is energy accumulated over the period. Join metering to dispatch by trading interval, not by day number.

Intervention

Solution tables carry an INTERVENTION flag. When AEMO directs a unit or intervenes for security, NEMDE publishes a second, "what would have happened" run for pricing; the intervention run is flagged 1. Filter to 0 for the physical dispatch.

Absence means not binding

DISPATCHCONSTRAINT holds every interconnector constraint but only the binding intra-regional ones. A constraint missing from an interval was slack, not inactive. Reindex to the full grid before computing bind rates.

Binding and violating

MARGINALVALUE != 0 and VIOLATIONDEGREE != 0. Never recompute.

Exclusions

Swamped rows (abs(RHS) >= 9,500), ramping ids beginning #, and rows flagged inconsistent or informational, all out before any statistic.

Versions

The version applied in an interval is explicit in the solution row. When joining to standing data that came from monthly archives, fall back to the latest version you hold, because archives are change-sets and may not carry every version.

Change-sets, not snapshots

Monthly archives of the GENERIC_CONSTRAINT package contain what changed that month. Standing coverage builds up over months, must be ingested in chronological order, and a re-ingest of an old month after a new one lets stale drafts overwrite amendments under the same key. Registration tables (DUDETAIL, PARTICIPANT) are the exception: full snapshots each month.

Restrict before you join

The SPD tables and DUDETAILSUMMARY are versioned change-sets spanning years. Pick the version you want, deduplicate, and only then join; the unrestricted join is a memory bomb.

PERIODID in pre-dispatch

Inconsistent across files. Join pre-dispatch constraint rows on DATETIME.