DolphinDB Streaming SQL: Real-Time at the Speed of Thought

DolphinDB
2026-05-28

In financial trading, IoT monitoring, and real-time risk control, modern systems must deliver personalized real-time views to hundreds or thousands of concurrent clients—from live market dashboards and dynamic leaderboards to complex multi-dimensional monitoring metrics. Traditional approaches fall into two camps: either rerunning complete queries for every client or constructing intricate streaming pipelines with manual state management. Both lead to high development costs, excessive resource consumption, and unpredictable latency.

Beginning with version 3.00.4, DolphinDB introduces Streaming SQL —a transformative solution to these challenges. Register your business SQL once (with full support for joins, filters, sorting, and aggregations), and the engine automatically tracks changes in underlying data, pushing the latest results in real time to all subscribed clients. Simply put: write SQL once, receive continuous results.

Why Streaming SQL?

Traditional SQL operates as a bounded query model: execute once, receive a static result set. With continuously arriving data, this model forces repetitive query execution.

Streaming SQL extends familiar SQL semantics with real-time and incremental capabilities:

  • Persistent Execution Model : Rather than a one-off query that "finishes and exits," Streaming SQL runs as a long-lived job. The engine continuously monitors data changes, feeds incremental updates through the processing pipeline, and delivers fresh results to clients—until you explicitly terminate the query.
  • Delta Computation : For real-time analytics, Streaming SQL avoids rescanning historical data. Instead, it employs delta computation: only changed portions are recalculated, with deltas pushed to subscribers. This design dramatically reduces computational and network overhead, enabling true millisecond-level latency.
  • Native Real-Time Features : Built-in support for late-arriving and out-of-order events, multiple time windows, and Top-K/sorting operations—the essential patterns of real-time analysis.

No new framework to master, no business logic rewrites required. If you can write SQL, you can build real-time data applications immediately.

Core Advantages of DolphinDB Streaming SQL

State-Driven Incremental Processing

DolphinDB Streaming SQL embeds continuous result updates directly into SQL syntax. Once registered, a query becomes a long-running job. As underlying tables change, the engine maintains necessary intermediate state and indexes, recomputing only affected keys and time windows before pushing incremental results downstream. This state-driven incremental model eliminates repeated full scans and redundant data transfers, sustaining millisecond-level latency at stable throughput.

Rich Analytical Expressiveness

Streaming SQL preserves the relational and analytical semantics you already know. Write multi-table JOINs, filters, group aggregations, and sorting using standard syntax. Window and cumulative functions—such as rolling/sliding windows, cumavg, and cumsum—integrate naturally into real-time statistics. Top-K ranking requirements are expressed directly in SQL without additional operators or external state management. For time-sensitive workloads, queries organize by event time with explicit policies for late-arriving and out-of-order data, ensuring metrics remain both low-latency and explainable.

Flexibility and Maintainability

The engine supports hot query updates and parallel views, allowing you to modify SQL logic without downtime while projecting a single data source into multiple concurrent business views.

Typical Use Cases

  • Real-Time Market Dashboards: Deliver personalized views for individual traders—watchlists, leaderboards, and technical indicators. As market ticks arrive, the system computes and pushes deltas instantly, eliminating the need for client-side polling.
  • Operational Monitoring & Alerting: Continuously track device status and business KPIs through multi-dimensional windowed aggregations. When thresholds are breached, alerts are computed and delivered within milliseconds.
  • Low-Latency Metric Maintenance: In high-frequency trading and quantitative strategies, maintain complex technical indicators—such as moving averages and cumulative returns—in real time to support split-second decision-making.
  • Dynamic Leaderboards & Top-K Rankings: Maintain and stream rankings like Top 10 by trading volume or Top 20 by percentage gain. When rankings shift, updates are pushed incrementally—no full recomputation required.

Quick Start

The example below shows how to use Streaming SQL to compute a running average in real time and push updates to subscribers.

// step 1: create a shared table as the data source
t = table(1..10 as id, rand(100,10) as val)
share t as st

// step 2: declare the shared table as a Streaming SQL input
declareStreamingSQLTable(st)

// step 3: register a Streaming SQL query
// compute a cumulative average over column `val`
qId = registerStreamingSQL("select cumavg(val) as avg_val from st", "sql_avg")

// step 4: subscribe to the Streaming SQL results
// the engine tracks changes on `st` and updates results in real time
subscribeStreamingSQL(queryId=qId)

// (upstream data ingestion into `st` is omitted here)

// step 5: read the live result table
select * from objByName(qId + "_result")

// ===== environment cleanup =====
// stop receiving updates for this query
unsubscribeStreamingSQL(queryId=qId)
// revoke the registered Streaming SQL query
revokeStreamingSQL(queryId=qId)
// revoke the previously declared Streaming SQL table
revokeStreamingSQLTable("st")

Try It Now

DolphinDB Streaming SQL is available in the latest release. Upgrade to DolphinDB V3.00.04 to get started.

  • 📖 Docs: https://docs.dolphindb.cn/en/Streaming/Streaming_SQL.html
  • 💬 Support: support@dolphindb.com
  • 🚀 Download: https://dolphindb.com/product#downloads-top
  • 🌐 Learn more: https://dolphindb.com/