Purpose: Answer the three questions a distribution control room needs within a minute of a recloser locking out — who is out, what isolates the fault, and who can be restored by backfeeding from an adjacent feeder.
Domain: This is the core of an OMS (outage management system), the flagship GIS application in electric utilities. The vocabulary is the real vocabulary: feeder, lateral, recloser, sectionalizer, tie switch, normally-open point, backfeed, SAIDI.
Dataset: A deterministic synthetic circuit over Redlands — one substation, two radial feeders (267 elements, 2,495 customers), 22 fused laterals, three reclosers, seven sectionalizing switches, and one normally-open tie between the feeder tails.
Why not the road network: A distribution feeder is directed (power flows radially outward from the substation), its edges carry device state (open/closed), and it needs per-element identity to attribute customers and devices. The Redlands road graph has none of those — every edge there is bidirectional, anonymous, and attribute-free — so this gets its own dataset and its own kernel.
Statelessness: No database and no persistence. The circuit is generated once from a fixed-seed LCG, exactly like the OpenStreetMap road network powering the Route Planner, so every run is identical.
The tie is the point: Without a normally-open point between the two feeders there is nothing to restore from and the restoration search would have no candidate to evaluate. It is what makes the network a mesh on paper and a tree in operation.
Algorithm
Downstream trace: A breadth-first sweep from the faulted element's downstream node, forbidden from crossing back through the fault itself and never traversing an open device. On a radial circuit that is exactly the de-energized subtree. The faulted element is included; customers affected is the summed customer count over the set.
Upstream trace: Rather than searching from the fault, the kernel sweeps outward from the source and tags every reached node with the element that energizes it. Walking those tags back from the fault yields the ordered path to the substation breaker in one pass.
Isolation: Two halves. Upstream, the first protective device (fuse, then recloser, then breaker) encountered walking toward the source — that is the device that actually clears the fault. Downstream, the frontier of closed switches: traversal stops at each one, so only switches that genuinely bound the de-energized section are returned. Returning every downstream switch would fragment the feeder and leave nothing for a tie to pick up.
Energization sweep: A connectivity sweep from the source with a set of device-state overrides applied. An element is energized when either endpoint is reachable through closed devices, which correctly counts a closed tie sitting between two live sections.
Directed storage, undirected energization: The single most important modelling idea here. fromNodeId/toNodeId record the nominal flow direction so upstream and downstream have meaning, but energization is pure connectivity — a closed tie backfeeds against the nominal direction. Tracing connectivity directionally would silently report a successful backfeed as reaching nobody.
Restoration search: Open the isolation devices, measure the post-isolation baseline, then evaluate every normally-open tie one at a time and keep the one that serves the most customers. A candidate is rejected outright if it re-energizes the faulted element — restoring customers by backfeeding into a fault is the one outcome worse than leaving them out.
SAIDI estimate: SAIDI is customer-minutes lost divided by total customers, so minutes avoided is customersRestored × assumedRepairMinutes / totalCustomers. The repair duration is an operator assumption, labelled as such in the UI, not a measurement.
API Layer
Endpoints:GET /api/v1/outage/network, POST /api/v1/outage/trace, POST /api/v1/outage/restore.
Versioning:Asp.Versioning with the version in the route template, matching every other API in the portfolio.
Payload guards:[RequestSizeLimit(4_000_000)] on both POSTs and [EnableRateLimiting("expensive")] on the controller, since the restoration search runs one full connectivity sweep per candidate tie.
Caching: The circuit is a compile-time constant, so GET /network carries [ResponseCache(Duration = 3600)].
Error contract: Only ArgumentException is caught, returning 400 with { "error": "..." }. The service's validation messages are therefore the public API contract and are written as user-facing text — "The faulted element was not found in the network.", "Device type must be between 0 and 6."
Device types cross the wire as integers to match the native struct exactly, with a DeviceTypes constant class in Portfolio.Common and a mirrored table in the frontend legend, so the magic numbers live in exactly two named places.
Services and Native Boundary
Kernel:network_trace_kernel, a C++20 shared library exposing Trace_Downstream, Trace_Upstream, Trace_FindIsolationDevices, and Trace_ComputeEnergizedSet through a stable C ABI with #pragma pack(push, 8) structs and status-code returns (plus -5 for "faulted element not found").
Where the boundary sits: The restoration search loop is managed and readable; each evaluation is a native connectivity sweep. That is the right split — the search is a handful of candidates, the sweep is the hot inner work.
Graph layout: The kernel densifies arbitrary node ids into 0..n-1 once and stores incidence as CSR (offsets plus a flat element-index array), caching both endpoint indices per element so the traversals run on integer array indexing with no hashing in the inner loop. Node ids are looked up with .find(), never operator[] — a default-constructed entry would be a phantom node in the traversal.
Marshalling: Blittable [StructLayout(Sequential, Pack = 8)] structs with preallocated managed output arrays and explicit [In]/[Out] direction attributes. Native code never allocates across the boundary.
Managed fallback:NetworkTraceNativeBridge.IsAvailable gates the native path, and the managed implementation mirrors the kernel's traversal order element for element, so the two produce byte-identical id lists. The fallback is production behaviour, not a test convenience: the deployed container does not build the native libraries, so what runs in production today is the managed path.
Measured speedup — and the honest answer: Native is faster on the shipped circuit and slower on a large one. Same machine, warm, medians of three runs, with byte-identical result checksums on every run:
Native versus managed benchmark results for the network trace kernel, by workload.
Every downstream, upstream, isolation and energized id list matches the managed path exactly — the checksums are equal, not merely close, because this is integer graph traversal with no floating-point arithmetic anywhere.
Why the crossover happens: A trace is three separate P/Invoke calls, and each one re-marshals the whole element array and rebuilds the graph. At 267 elements that overhead is negligible and the kernel's cache-friendly CSR traversal wins by ~1.3×. At 4,603 elements the marshalling and rebuild dominate and the native path loses. The single-call sweep, which pays that cost once, is a wash. The fix is an ABI change — one entry point that returns all three traces from a single marshalled array, or a caller-owned opaque graph handle built once and reused — not a faster inner loop.
What this says about native code generally: The first version of this kernel used std::unordered_map<int, std::vector<int>> for adjacency and measured 2.5× slower than the C# it was meant to accelerate, because it heap-allocated once per node and hashed on every edge visit. .NET's allocator and Dictionary are genuinely fast. Writing C++ buys nothing on its own; it buys the ability to control layout, and only if you actually do.
Known wart:IsNativeInvocationException includes InvalidOperationException, which is exactly what the negative-status check throws. A genuine native error therefore logs one warning and silently degrades to managed rather than surfacing. This is consistent across all kernels and is a deliberate availability-over-visibility trade, but it means native failures are invisible in production metrics.
Engineering Decisions and Tradeoffs
A separate dataset, not a reuse of the road graph: Bending the OSM road network into a feeder would have been faster and wrong. Radial topology, device state, and per-element customer attribution are the entire problem; a graph without them cannot express it.
All three traces come from one engine: The service requires the downstream, upstream and isolation calls to all succeed natively before reporting NativeAccelerated. Mixing a native downstream set with a managed isolation frontier would make the flag a half-truth.
The isolation frontier stops at the first switch: Returning every switch downstream of the fault would technically isolate it and would also chop the feeder into pieces no tie can reach. Stopping at the frontier is what a real switching order does.
Restoration rejects any plan that re-energizes the fault: The search maximises customers served, and without that guard the optimum on a fault near the tie is to backfeed straight into the fault. The constraint is cheap to state and non-negotiable in the domain.
Conservation is defined against the post-isolation state:CustomersRestored + CustomersStillOut equals the customers out after isolation, not the customers the fault trace reported. Opening an upstream recloser de-energizes a superset of the faulted section, and the switching plan has to be measured against the state it actually starts from.
SAIDI is labelled an estimate: The repair duration is an operator assumption. Presenting an assumption-driven number as a regulated metric would be the wrong signal to anyone who reports SAIDI for a living.
Self-contained SVG: The single-line diagram is hand-built SVG with CSS custom properties for colour, so it follows the light/dark theme and the page carries no map-tile dependency. Hovering an element shows its label, device type, state and customer count — utility software lives on that interaction.
Interview Discussion Points
The network is stored with directed from/to but energization is computed undirected. Why, and what would break if you traced connectivity directionally?
A distribution feeder is radial in operation but meshed on paper. How does that change the data model versus a road network?
What is the difference between isolating a fault and restoring customers, and why do they need separate algorithms?
Closing a tie switch backfeeds from an adjacent feeder. What real-world constraints would stop you — conductor ampacity, voltage drop, protection coordination — and how would you model them?
SAIDI is customer-minutes ÷ total customers. Why do utilities optimize for it, and what behavior does that incentivize that might not serve customers well?
The restoration search evaluates every tie one at a time. When does that stop working, and what would you do for a network with 200 ties?
How would you extend this to handle a partial trace when the SCADA state of a device is unknown?
This kernel is 1.3× faster than managed C# at 267 elements and 0.87× at 4,603. What does that tell you about where the cost actually is, and what would you change first?
The first version of the kernel was 2.5× slower than the C# it replaced. How would you have caught that before shipping it?