Technology

Technical perspective

Contextual intelligence for the energy system.

Connecting physical infrastructure, changing conditions and economic choices in one modeling foundation.

The architecture, research foundations and evaluation principles shaping our approach to contextual intelligence.

Research & further reading

01 System context

The relationships are part of the data.

A meter reading, an equipment record and a market schedule can describe different aspects of the same asset. Their meaning depends on where that asset sits, what it connects to and when each piece of information applies.

ArayNex’s starting point is a unified representation of the physical and economic system. By contextual intelligence, we mean interpreting observations and potential actions in the context of the system as a whole. A weather change can affect generation, loading and available flexibility; an existing market commitment can determine how much of that flexibility remains usable.

One model means a consistent system context. Specialized models can work within it while referring to the same assets, time horizons and assumptions.

Structure
Assets, electrical connections, locations, equipment parameters and operating limits.
State
Measurements and estimates of voltage, flows, asset condition and available energy, tied to a point in time.
Conditions
Weather, generation and demand trajectories, planned outages and the uncertainty around them.
Commitments
Procurement positions, operating schedules, service obligations and the market rules that apply to an action.

Graph representations offer a way to encode assets as entities and connections as relationships. Learning over that structure can preserve dependencies that would be lost by treating every measurement as an independent input. A useful representation also distinguishes an electrical connection from a contractual relationship: both matter, but they impose different constraints. [1]

This context is intended to carry across networks, renewable generation, storage, computing, industry and mobility. The asset models and obligations differ by setting; the need to connect physical capability with economic choices remains.

02 Observations and state

Establish the system state from incomplete evidence.

Network records establish an initial structure. Telemetry describes behavior over time. Field imagery can provide evidence about asset identity, location or condition. Bringing these sources together requires matching them to the same equipment and distinguishing what was measured, inferred or confirmed in the field.

Keep the evidence attached.

A useful observation carries its source, timestamp, units, location and uncertainty. The time when an event occurred may differ from the time its record arrived. An equipment inspection can change an assumption about the asset without replacing a current sensor reading. Conflicting records need reconciliation, rather than silent overwriting.

Account for what is not observed.

State estimation relates available measurements to the network that produced them. Weighted least squares is an established engineering baseline: measurements contribute according to their uncertainty, and residuals help identify inconsistencies. It provides a meaningful comparison for learned approaches. [2]

The measurement relationship
z=h(x;G,θ)+ε

Measurements z relate to the electrical state x through topology G and equipment parameters θ. The term ε represents measurement error.

Incomplete measurements can leave uncertainty about how the system is actually operating. Prior estimates and physical models can help reduce this uncertainty, but cannot replace direct observations. The representation should retain that uncertainty and identify when another observation or field check is needed.

03 Physics and learning

Learn within the physical principles that govern the system.

Connectivity, power balance and equipment dynamics provide information that a time series alone cannot supply. Our research direction brings those relationships into learning and evaluates learned estimates against engineering models.

Physics-informed neural networks (PINNs) incorporate governing equations into training alongside observed data. Depending on the formulation, a training objective can account for measurement error, equation residuals, and initial or boundary conditions. The original PINN work establishes this approach for differential equations. [3]

Learning from data and physical laws
ℒ=ℒdata+λℒphysics

ℒdata measures disagreement with observations and prescribed initial or boundary values. ℒphysics penalizes violations of the governing equations, while λ controls its relative influence during training. Appropriate scaling balances these contributions, and the resulting model is validated against the system’s physical behavior.

Match the model to the question.

Network state
Power-flow equations relate power injections, voltages and network flows at a given operating point. For unbalanced distribution networks, these relationships are modeled explicitly for each phase.
Asset dynamics
State changes depend on the asset: stored energy, temperature or electromechanical dynamics evolve across different timescales.
Learned approximation
A surrogate can estimate a state or response within a defined domain. Its inputs, training conditions and physical residuals remain part of the evaluation.

PINNs have been used to model rotor angle and frequency dynamics and estimate uncertain parameters such as inertia and damping. Each formulation is evaluated within a defined system and range of operating conditions. [4]

Models are compared with numerical time-domain solvers on accuracy, computational cost and flexibility across operating conditions. A model’s practical value depends on its training and retraining costs, inference speed and ability to generalize across operating conditions. [5]

Physical loss terms introduce their own optimization challenges. [6] Model outputs are evaluated against numerical baselines, checked against the governing equations and tested beyond the conditions represented in training. These checks assess accuracy and physical feasibility independently of training loss.

04 Scenarios and uncertainty

Evaluate futures that can occur together.

Weather influences both supply and demand. Renewable output, temperature-sensitive load and equipment availability can change together. Independent forecasts can miss the combinations that put the greatest pressure on the system.

Probabilistic forecasting represents a range of outcomes. For an operating plan, that range needs to extend across relevant locations and time periods. Wind forecasts describe possible generation trajectories across locations and time, while probabilistic demand forecasts capture uncertainty in electricity consumption. [7] [8]

Preserve sequence and dependence.

Storage availability depends on what happened in earlier intervals. A demand peak coinciding with low renewable output creates a different requirement from the same two events occurring apart. Network conditions, asset states and existing commitments provide a common context for evaluating these trajectories and the dependencies that shape the system’s response.

How stored energy carries forward
Et+1=Et+ηcCt−Dtηd

Et is stored energy at the start of interval t. Ct and Dt are energy charged from and discharged to the network during that interval; ηc and ηd are the respective conversion efficiencies. Stored energy links dispatch decisions across time: what happens in one interval affects what remains available in the next. The model assumes no energy loss through self-discharge. [12]

A forecast distribution and a stress test answer different questions. The first describes estimated probabilities; the second examines a specified condition, such as an outage. Keeping that distinction explicit prevents an assumed scenario from being mistaken for a calibrated likelihood.

Forecast evaluation should consider calibration and how informative the predicted ranges are. Proper scoring rules, including the continuous ranked probability score, provide tools for this assessment. Results also need examination by horizon, season and operating regime, so average scores do not conceal important failures. [9] For joint forecasts, evaluation also checks whether relationships across locations and time are preserved. [10]

05 Operations and markets

A useful choice must be physically possible.

Buying energy does not increase a connection’s transfer limit. Discharging storage changes what is available later. Reserving capacity for one commitment can make it unavailable for another.

Constrained optimization makes these relationships explicit. An objective expresses the cost or service outcome being considered; constraints define which responses are allowed. AC optimal power flow is one established formulation, combining power balance with network, voltage and generator limits. [11]

Choosing within the feasible set
minu∈ℱ(c)J(u;c)

An operating plan u is evaluated within system context c: the network, asset states, forecasts and commitments. J expresses the cost or other objective being minimized. The feasible set ℱ(c) contains only plans that satisfy the specified physical, operational and market constraints, including those linking decisions across time.

Physical feasibility
Balance, network loading, voltage limits, equipment availability and site connection limits.
Across time
State of charge, conversion losses, ramping, maintenance windows and future service obligations.
Economic exposure
Purchases and sales, deviation costs, tariffs, contractual positions and the applicable market rules.

Single-period feasibility is only part of an operating plan. Multi-period formulations connect generation and storage choices across time; open frameworks such as PyPSA combine these linked decisions with power-flow equations and asset constraints in a single optimization model. The appropriate physical approximation depends on the question, and any simplification needs to remain explicit.

Separate speed from feasibility.

A learned approximation may accelerate evaluation or provide a starting point for an optimizer. Its output still needs checking against the applicable equations and limits. In neural AC optimal power flow, physical constraints shape model training, while worst-case violation analysis provides a separate check on the feasibility of the resulting solutions. [13]

Judge forecasts by the choices they change.

An error near a binding constraint can change an operating decision more than a larger error under unconstrained conditions. Decision-focused learning studies this relationship between prediction and downstream optimization. It motivates assessing forecast accuracy and decision quality separately. [14]

ArayNex’s intended output is a response that can be examined: feasible alternatives, the constraints that determine them, the assumptions behind their cost and the commitments they create. Recommendations should retain operator review and decision authority, with their validity tied to the system conditions used to evaluate them.

06 Learning from outcomes

Update the understanding behind the next response.

Measurements, field findings and operating outcomes provide new context. Comparing them with expectations can reveal a forecast error, an incorrect equipment parameter, a topology change or an action that was never carried out.

A changed system
A replaced asset or confirmed switch change updates the physical representation and the assumptions of subsequent studies. Residual-based state estimation can help identify errors in the assumed network topology. [15]
A persistent error
A forecast bias or changing error distribution prompts investigation of data, features and calibration before deciding what to retrain. [16]
A different action
A recommendation that was modified or not executed must be distinguished from a model that predicted the executed action incorrectly.

Our design principle is to retain the observation, context version, recommendation and recorded action together. A prediction about one network configuration should not be assessed against another without accounting for the change.

Comparing an outcome with its prediction
rt=yt−y^t

The residual rt is the difference between an observed outcome yt and its prediction ŷt. The comparison must refer to the same quantity, time and operating conditions. A residual can reflect measurement noise, model error or a changed system; its cause needs investigation before updating the model.

An update then becomes something that can be tested. Replay it against historical periods using the information available at each point, compare it with the previous version, and examine whether any improvement survives changes in season, topology or operating conditions. This keeps contextual learning connected to evidence. [17]

07 Evaluation

Measure the foundation and the decisions it informs.

The foundation needs assessment both as individual models and as a connected system. A faster estimate is useful only if its error is understood. A lower-cost schedule is meaningful only under the constraints and implementation costs that would apply in practice.

Representation
Asset identity, connectivity and time alignment against trusted records; performance with missing, stale and conflicting observations.
State estimation
Voltage and flow errors against held-out measurements or an engineering baseline, reported by condition and network configuration.
Uncertainty
Calibration, interval coverage and forecast scores across horizons, with chronological separation between training and test periods.
Feasibility
Equation residuals and constraint violations, with units, tolerances, unsuccessful solves and any correction of model outputs reported.
Decision quality
Realized cost and service outcomes against a baseline with the same information, accounting for losses, deviations and implementation costs.
Computation
Data preparation, inference, optimization and verification time together; training and recalibration costs reported separately.

Compare under the same conditions.

A fair comparison fixes the information available at the time, the feasible actions and the operating constraints. It also asks which part of the foundation adds value: the network representation, physical information, uncertainty treatment or decision objective. Testing components separately helps identify the source of a gain or failure.

PGLib-OPF supplies standardized AC optimal-power-flow cases and formulations for reproducible algorithm comparisons. RTS-GMLC provides a complementary reference system for studies involving operating chronology and reliability. Their different scopes matter; neither substitutes for validation in the intended operating environment. [18] [19]

This perspective describes ArayNex’s technical direction and evaluation principles.

08 Research foundations

Research & further reading.

Selected papers, engineering references and open systems, grouped by their role in this perspective.

Representation and estimation

  1. Relational inductive biases, deep learning, and graph networks

    Battaglia et al. · 2018 · arXiv

    A foundation for learning about entities and their relationships. Explains why the structure connecting assets can matter as much as their individual attributes.

    Back to text
  2. State Estimation

    pandapower · Official documentation

    An engineering reference for estimating network states from measurements with different uncertainties, including weighted least squares and bad-data checks.

    Back to text

Physical models and learning

  1. Physics Informed Deep Learning, Part I

    Raissi, Perdikaris & Karniadakis · 2017 · arXiv

    Introduces neural learning informed by governing differential equations. Establishes the underlying PINN approach, rather than a power-system-specific implementation.

    Back to text
  2. Physics-Informed Neural Networks for Power Systems

    Misyris, Venzke & Chatzivasileiadis · 2019, revised 2020 · arXiv

    Applies PINNs to power-system dynamics and parameter estimation. Its single-machine study provides a concrete domain example with a defined experimental scope.

    Back to text
  3. Physics-Informed Neural Networks for Time-Domain Simulations: Accuracy, Computational Cost, and Flexibility

    Stiasny & Chatzivasileiadis · 2023 · arXiv

    Compares learned time-domain models with conventional solvers. Useful for separating inference speed, accuracy and the conditions under which a model remains useful.

    Back to text
  4. Characterizing possible failure modes in physics-informed neural networks

    Krishnapriyan et al. · 2021 · NeurIPS

    Examines difficult optimization landscapes in PINNs. Shows why incorporating physical equations in training does not, by itself, ensure a reliable solution.

    Back to text

Forecasting and uncertainty

  1. Wind Energy: Forecasting Challenges for Its Operational Management

    Pinson · 2013 · Statistical Science

    Connects probabilistic wind forecasts with operating decisions, including the importance of trajectories across locations and time.

    Back to text
  2. Probabilistic electric load forecasting: A tutorial review

    Hong & Fan · 2016 · International Journal of Forecasting

    Reviews demand-forecasting methods, evaluation and common misunderstandings when moving beyond a single predicted value.

    Back to text
  3. Strictly Proper Scoring Rules, Prediction, and Estimation

    Gneiting & Raftery · 2007 · JASA

    The statistical basis for assessing probability forecasts, including the continuous ranked probability score.

    Back to text
  4. Variogram-Based Proper Scoring Rules for Probabilistic Forecasts of Multivariate Quantities

    Scheuerer & Hamill · 2015 · Monthly Weather Review

    Examines scores that assess dependence in joint forecasts, including their sensitivity to errors in correlations across locations. Complements evaluation of individual forecast distributions.

    Back to text

Optimization and decisions

  1. MATPOWER User’s Manual — Optimal Power Flow

    MATPOWER · Official manual

    Formulates AC optimal power flow as cost minimization subject to power balance and limits on voltages, generation and network flows.

    Back to text
  2. PyPSA: Python for Power System Analysis

    Brown, Hörsch & Schlachtberger · 2018 · Journal of Open Research Software

    An open-source framework for modeling power networks and optimizing generation, storage and transmission across time, from operational dispatch to long-term investment planning.

    Back to text
  3. Physics-Informed Neural Networks for AC Optimal Power Flow

    Nellikkath & Chatzivasileiadis · 2021 · arXiv

    Studies physical equations in neural OPF training and methods for assessing worst-case constraint violations. Connects learned approximations with explicit verification.

    Back to text
  4. Smart “Predict, then Optimize”

    Elmachtoub & Grigas · 2017, revised 2020 · arXiv

    Studies prediction through the downstream decision error it creates. Relevant to judging economic consequences alongside conventional forecasting scores.

    Back to text

Learning from outcomes

  1. Topology Error Detection and Robust State Estimation Using Nonlinear Least Absolute Value

    Park, Mohammadi-Ghazi & Lavaei · 2019 · American Control Conference

    Studies how measurement residuals in AC state estimation can localize topology errors. Connects model discrepancies with investigation of the assumed network.

    Back to text
  2. A Survey on Concept Drift Adaptation

    Gama et al. · 2014 · ACM Computing Surveys

    Reviews how changing relationships in data affect predictive models, covering drift detection, adaptation and evaluation. Provides background for investigating persistent changes in forecast error.

    Back to text
  3. On evaluating stream learning algorithms

    Gama, Sebastião & Rodrigues · 2013 · Machine Learning

    Develops sequential methods for evaluating and comparing models as data arrives, including tracking prediction error under changing conditions. Supports testing predictions before using their outcomes to update a model.

    Back to text

Evaluation and reference systems

  1. The Power Grid Library for Benchmarking AC Optimal Power Flow Algorithms

    Babaeinejadsarookolaee et al. · 2019, revised 2021 · arXiv

    Standardized AC optimal-power-flow formulations and network cases for reproducible algorithm comparisons.

    Back to text
  2. Reliability Test System — Grid Modernization Lab Consortium

    GridMod / RTS-GMLC · Open reference dataset

    A documented system for production-cost and reliability studies. Adds time-series study data to the network and resource representation.

    Back to text