In this post, we explore DolphinDB’s streaming engine architecture and walk through practical examples.

This article shows how DolphinDB's AI DataLoader eliminates the data pipeline bottleneck. We'll build a production-ready stock price prediction model, demonstrating how to seamlessly connect database-scale factor data with PyTorch—without the usual memory management headaches.

This article walks through exactly that transformation, using DolphinDB as the computing layer. We’ll cover both the batch path (historical data, processed in parallel) and the streaming path (real-time data, processed with sub-millisecond latency), with the same core logic running in both.

Using simulated tick-by-tick trade data from the Shanghai Stock Exchange on a single trading day in 2020, we’ll calculate minute-level capital flow — splitting buy and sell orders into “large” and “small” buckets based on a 50,000-share threshold — and stream the results live into a dashboard.

To understand where the breaking points actually are, we benchmarked three of the most widely used TSDBs — DolphinDB, InfluxDB, and TimescaleDB — using the TSBS framework, simulating a trucking fleet across five device-scale scenarios ranging from 100 to 10 million devices.

In this article, we’ll walk through how to build a real-time IV calculation and volatility smile construction pipeline for commodity options, using DolphinDB’s streaming framework. By the end, you’ll have a working architecture that goes from raw CTP tick data to a smoothed volatility curve — refreshed every minute, live.

In quantitative trading, factor discovery is the foundation of alpha generation. Whether for high-frequency crypto strategies or medium-term systematic portfolios, the ability to efficiently compute, iterate, and deploy factors directly determines research velocity and production readiness.

Factor replication from research reports has traditionally been a core method for analysts seeking to expand their analytical frameworks and identify new alpha-generating opportunities.

High-frequency market data contains a level of market microstructure detail that daily OHLC data simply can't match — order book dynamics, trade impact, informed order flow, intraday liquidity patterns. The problem is that building strategies directly on tick data is expensive: you need serious infrastructure, the signal-to-noise ratio is brutal, turnover costs eat returns, and scaling up is genuinely hard.