Stop Backtesting in Theory, Simulate the Real Crypto Market with DolphinDB

In the fast-paced world of cryptocurrency trading, the difference between profit and loss often comes down to milliseconds. Quantitative traders need robust infrastructure to test strategies before risking real capital—but traditional backtesting frameworks struggle with the unique challenges of crypto markets: massive tick data volumes, complex order matching logic, and the need for real-time simulation.

This is where having the right technical foundation matters. At DolphinDB, we've built an integrated backtesting and simulation solution specifically designed for mid- to high-frequency cryptocurrency trading. By combining our high-performance time-series database with a sophisticated order matching engine, we're helping traders move from strategy conception to live deployment faster and more confidently.

In this guide, we'll walk you through the complete workflow—from setting up your first backtest to running real-time simulations with full code versioning and team collaboration features.

The Challenge: Why Crypto Backtesting Is Different

Before deploying any quantitative strategy to live trading, extensive validation is essential. But cryptocurrency backtesting presents two critical challenges that traditional equity backtesting frameworks weren't built to handle:

Massive Data Storage and Processing

High-frequency snapshot data in crypto markets generates enormous data volumes. A single trading pair can produce millions of records per day. This demands exceptional framework-level performance for both data processing and computation.

Complex Matching Logic

Order execution isn't as simple as using the latest price. Realistic backtesting must account for order price, traded volume, market volatility, and liquidity conditions—requiring a sophisticated order matching simulator.

The DolphinDB Solution: End-to-End Backtesting and Simulation

Leveraging our distributed architecture and low-latency streaming engine, DolphinDB integrates a backtesting engine with an order matching simulator to deliver a complete solution. Here's what it includes:

Market data replay and processing
Strategy development and code management
Result visualization and analytics
Strategy permission management for teams

The platform supports both snapshot and minute-level cryptocurrency data, enabling individuals and quantitative teams to develop strategies efficiently with minimal code and a gentle learning curve. Beyond native DLang, strategies can be written in Python and C++, offering flexibility without sacrificing performance.

1. Understanding the Cryptocurrency Backtesting Engine

The cryptocurrency backtesting engine is delivered as a plugin with a straightforward logical flow:

The engine replays market data streams in chronological order and dispatches them to the order matching simulator and market data callback functions.
Market data callback functions process strategy logic and submit orders.
The engine performs risk control management.
Orders that pass risk control are sent to the order matching simulator for execution.
The engine tracks positions and funds in real time, and returns strategy returns and trade details after backtesting completes.

The engine supports both snapshot and minute-level data for spot and futures across multiple exchanges. It can manage multiple accounts and contract types within a single engine instance.

One of the biggest pain points in backtesting is data preparation. Market data must conform to strict schemas, often requiring substantial cleansing and transformation. Since strategy researchers should focus on designing and optimizing strategy logic—not wrestling with data pipelines—we've streamlined this entirely.

Our solution provides not just raw exchange data, but fully processed and replay-ready snapshot and minute-level market data. This means you can focus exclusively on your custom strategy functions and configuration to quickly complete backtesting.

2. Strategy Development and Code Management

All market data access, backtest execution, trade simulation, and code management functions are encapsulated in the CryptocurrencySolution module. Based on the cryptocurrency data environment built in Engineering Always-on Market Data Infrastructure for Crypto Trading, you can directly use this module to perform strategy backtesting.

The sub-modules are described below:

Sub-module	Catagory	Description
`CryptocurrencySolution::utils`	Common functions	Includes common functions such as engine deletion and management, funding rate retrieval, base information tables, OHLC data downsampling, backtest and simulation strategy retrieval, strategy log retrieval, and database/stream table status checks.
`CryptocurrencySolution::setting`	Unified settings	Includes unified strategy configuration, GitLab address configuration, module lists, and database/table metadata.
`CryptocurrencySolution::runBacktest`	Backtesting	Includes market data retrieval and processing functions, and backtest execution functions.
`CryptocurrencySolution::simulatedTrading`	Simulation	Includes market data retrieval and processing functions, and simulation start/stop function.
`CryptocurrencySolution::manageScripts`	Code storage	Includes strategy upload, retrieval, and deletion functions, backtest result storage and retrieval functions, and backtest/simulation code submission and storage function.

First, obtain the home directory using getHomeDir(), place the attached module folder in the corresponding path, and load it using use:

use CryptocurrencySolution::manageScripts
go

To meet strategy storage and management requirements, this solution integrates with GitLab. When submitting a strategy for backtesting or simulation, the system automatically uploads the strategy code to a specified Git repository, enabling unified management, version tracking, and rapid retrieval. For details, see Section 4.

If strategy code storage is not required, you can use the following modules for backtesting or simulation only:

// Backtesting
use CryptocurrencySolution::runBacktest
go
// Simulation
use CryptocurrencySolution::simulatedTrading
go

2.1 Writing Cryptocurrency Strategies

The backtesting engine adopts an event-driven mechanism and provides the following event functions. You can customize callbacks for strategy initialization, daily pre-market processing, snapshot and OHLC market data, and order and trade reports.

Event Function	Description
`initialize(mutable context)`	Strategy initialization function: triggered once. The parameter context represents the logical context. In this function, you can use context to initialize global variables or subscribe to metric computation.
`beforeTrading(mutable context)`	Callback executed during the daily session rollover; triggered once when the trading day switches.
`onSnapshot(mutable context, msg)`	Snapshot market data callback.
`onBar(mutable context, msg)`	Low- to mid-frequency market data callback.
`onOrder(mutable context, orders)`	Order report callback, triggered whenever an order’s status changes.
`onTrade(mutable context, trades)`	Trade execution callback, triggered when a trade occurs.
`finalize(mutable context)`	Triggered once before strategy termination.

Note:

If GitLab-based strategy code management is required, the strategy must be declared using module, and the module name must match strategyName.
Required modules used in callbacks must be listed in getUsedModuleName() in CryptocurrencySolution::setting. Otherwise, strategy code will fail to be uploaded.

module testMinStrategy   //Declare the strategy module when enabling code management
def initialize(mutable context){
}
def beforeTrading(mutable context){
}
def onBar(mutable context,msg, indicator){
}
def onSnapshot( mutable context,msg, indicator){
}
def onOrder( mutable context,orders){
}
def onTrade(mutable context,trades){
}
def finalize (mutable context){
}
strategyName = "testMinStrategy"

2.2 Cryptocurrency Strategy Configuration

The userConfig strategy configuration must include the strategy type, backtest start and end dates, market data type, initial capital, and the cryptocurrency asset to be backtested. The CryptocurrencySolution::setting module provides default strategy configuration, which can be obtained using the following function:

startDate = 2025.08.24
endDate = 2025.08.25
dataType = 3 // Minute-level market data; set to 1 for snapshot data
userConfig = CryptocurrencySolution::setting::getUnifiedConfig(startDate, endDate, dataType)

An example of the userConfig is shown below:

sym = ["BTCUSDT"]
userConfig = dict(STRING,ANY)
userConfig["startDate"] = 2025.08.24
userConfig["endDate"] = 2025.08.25
userConfig["strategyGroup"] = "cryptocurrency"
cash = dict(STRING,DOUBLE)
cash["spot"] = 1000000.
cash["futures"] = 1000000.
cash["option"] = 1000000.
userConfig["cash"] = cash
userConfig["dataType"] = 3    // 1: snapshot; 3: minute-level
userConfig["Universe"] = sym  // Symbol for backtesting (required)
p = dict(STRING,ANY)
p["Universe"] = userConfig["Universe"] + "_futures"
p["log"] = table(10000:0,[`tradeDate,`time,`info],[DATE,TIMESTAMP,STRING]) // log table for debugging
userConfig["context"] = p    // Pass the asset universe

To backtest historical market data, you can customize the start and end dates in userConfig. This module automatically loads the default strategy configuration and allows user-provided userConfig values to override the defaults. By default, Binance data is used, and the backtest period is the most recent 10 days.

userConfig = dict(STRING,ANY)
userConfig["startDate"] = 2025.08.24
userConfig["endDate"] = 2025.08.25
// Example: backtest an asset (used to define the asset universe)
p = dict(STRING,ANY)
p["Universe"] = ["XRPUSDT_futures"]
userConfig["context"] = p
strategyType = 0
engine, straname_ =
    CryptocurrencySolution::manageScripts::runCryptoAndUploadToGit(
        strategyName, eventCallbacks, strategyType, userConfig
    )

To downsample OHLC data, you can use the built-in downsampling function and specify the barMinutes parameter:

strategyType = 0
engine, straname_ =
    CryptocurrencySolution::manageScripts::runCryptoAndUploadToGit(
        strategyName, eventCallbacks, strategyType, , barMinutes = 1
    )

To use market data from the OKX exchange, specify exchange = "OKX". Otherwise, it defaults to "Binance":

engine, straname_ =
    CryptocurrencySolution::manageScripts::runCryptoAndUploadToGit(
        strategyName, eventCallbacks, strategyType, exchange = "OKX"
    )

2.3 Strategy Backtest Execution and Result Retrieval

After setting the strategyName , eventCallbacks , and strategyType parameters, call runCryptoAndUploadToGit to execute the backtest.

strategyName = "testMinStrategy"
eventCallbacks = {
    "initialize": initialize,
    "beforeTrading": beforeTrading,
    "onBar": onBar,
    "onOrder": onOrder,
    "onTrade": onTrade,
    "finalize": finalize
}
// Execute backtest and upload strategy files to GitLab
strategyType = 0
engine, straname_ =
    CryptocurrencySolution::manageScripts::runCryptoAndUploadToGit(
        strategyName, eventCallbacks, strategyType
    )

During backtest execution, the strategy files are automatically uploaded to the specified GitLab repository, and the corresponding commit ID is stored in the gitstrategy table of the database dfs://CryptocurrencyStrategy.
If strategy code management is not required, you can use runCryptoBacktest from the CryptocurrencySolution::runBacktest module. This function automatically loads default strategy configuration and allows you to override it with a custom userConfig. By default, it performs backtesting on Binance data for the most recent 10 days.

// userConfig = dict(STRING,ANY)
// userConfig["startDate"] = 2025.11.10
// userConfig["endDate"] = 2025.11.20
engine, straname_ =
    CryptocurrencySolution::runBacktest::runCryptoBacktest(
        strategyName, eventCallbacks, userConfig
    )

Retrieve backtest results

After the backtest completes, you can retrieve daily positions, daily equity, return summary, trade details, and strategy configurations using the following methods.

// Trade details
tradeDetails = Backtest::getTradeDetails(engine, "futures")
// Unfilled orders
tradeDetails[tradeDetails.orderStatus == -3]
// Current open (unfilled) orders
Backtest::getOpenOrders(engine, "futures")
// Daily positions
Backtest::getDailyPosition(engine, "futures")
// Overall backtest summary
Backtest::getReturnSummary(engine)

After the backtest completes, the strategy return summary and configuration are stored in dfs://CryptocurrencyStrategy/backtestStrategy for comparative analysis.
Visualization of backtest results is described in Section 3.

2.4 Submitting Simulated Trading

After real-time market data is ingested into stream tables, set strategyType = 1 to enable simulated trading. The strategy name will automatically be suffixed with SimulatedTrading, and the strategy code will be managed and stored.

You can use the overwrite parameter to control whether strategies with the same name are automatically deleted (defaults to false). By default, Binance data is used and simulation starts from the current day.

strategyType = 1
engine, straname_ =
    CryptocurrencySolution::manageScripts::runCryptoAndUploadToGit(
        strategyName, eventCallbacks, strategyType,
        userConfig = NULL, overwrite = false
    )

After simulation starts, trade details, real-time positions, and equity are written in real time to the corresponding simulation tables, for example:

strategyName + "_tradeDetails"
strategyName + "_position"

If strategy code management is not required, you can run simulated trading using the submitCryptoSimulatedTrading function in the CryptocurrencySolution::simulatedTrading module. This function automatically loads the default strategy configuration and allows you to override it with a custom userConfig. To run a simulation based on an existing backtest strategy, use the submitCryptoSimulatedTradingByStrategyName function, which submits the backtest strategy (strategyName_) in simulation mode.

// Directly submit simulated trading
engine, straname_ = submitCryptoSimulatedTrading(strategyName, eventCallbacks)
// Submit simulated trading based on an existing backtest strategy
engine, straname_ = submitCryptoSimulatedTradingByStrategyName(strategyName_, overwrite = false)

Simulation result dashboards are described in Section 3.
If you need to run simulated trading on downsampled OHLC data, use the time-series engine to process the original stream tables. For details, see klineMergeScript.dos in the Appendix.

2.5 Retrieving Strategy Logs

To simplify debugging, you can write logs inside strategy callback functions:

def beforeTrading(mutable context){
    // Write logs to the engine-maintained log table
    context.log.tableInsert(context["tradeDate"], now(), "beforeTrading")
}

Use getStrategyLog to retrieve strategy logs. If you define a custom userConfig , you must manually add log settings.

CryptocurrencySolution::utils::getStrategyLog(straname_)

Sample strategy logs:

2.6 Deleting Strategies

To avoid excessive engine creation, each user is limited to a maximum of 5 strategy engines. Unneeded engines can be removed using dropMyBacktestEngine. For simulated strategies, the corresponding result tables are also deleted. You can only delete strategies created by yourself.

use CryptocurrencySolution::utils
straname_ = 'admin_testMinStrategy'
dropMyBacktestEngine(straname_)

2.7 Strategy Code Management

To meet requirements for strategy code storage and versioning, we integrate backend modules with a GitLab repository. When a backtest or simulation is submitted, the backend automatically uploads the strategy event callback functions to a user-defined GitLab repository and records version information in the corresponding database tables. All GitLab-related strategy management functions are encapsulated in the CryptocurrencySolution::manageScripts module.

First, you must create a GitLab account and obtain a GitLab Personal Access Token for authentication. See Personal access tokens | GitLab Docs for details. When using strategy code management, you can either pass gitInfo or configure it globally in the CryptocurrencySolution::setting module. Example:

gitInfo = dict(STRING,ANY)
gitInfo[`gitLabSite] = "https://dolphindb.net"   // GitLab URL
gitInfo[`repoId] = 620                          // Project ID
gitInfo[`repoBranch] = "main"                   // Branch name
gitInfo[`privateToken] = "zQx-BzyJrHxd_sbnEFNR"  // Access token

Below are the main functions provided by this module, including uploading, retrieving, and deleting strategies.

Upload strategy files

def uploadOrupdateToGit(
    strategyName, strategyType, eventCallbacks,
    gitInfo = NULL, commitMsg = NULL, timeout = 10000
)

Parameter description

Parameter	Type/Form	Description	Required
strategyName	SYMBOL	Strategy name	Yes
strategyType	INT	Strategy type (0: backtest, 1: simulation)	Yes
eventCallbacks	DICT	Event callback functions	Yes
gitInfo	DICT	GitLab configuration; an example is displayed at the start of the section.	No
commitMsg	STRING	Commit mesaage	No
timeout	INT	Request timeout	No

As described in Section 2, you can run backtests or simulations via runCryptoAndUploadToGit, which automatically uploads strategy code. Strategy versions (both new and updated) are recorded in dfs://CryptocurrencyStrategy/gitstrategy. You can retrieve specific versions using the stored commit IDs.

Retrieve strategy files

def getStrategyFromGit(
    strategyName, commitId = NULL, gitInfo = NULL, timeout = 10000
)

Parameter description

Parameter	Type/Form	Description	Required
strategyName	SYMBOL	Strategy name	Yes
commitId	STRING	ID of the commit that uploads the strategy	No
gitInfo	DICT	GitLab configuration; an example is displayed at the start of the section.	No
timeout	INT	Request timeout	No

By default, this function retrieves the latest version of the strategy. To fetch a specific version, specify commitId based on the records in dfs://CryptocurrencyStrategy/gitstrategy.

Delete strategy files

def deleteStrategyFromGit(
    strategyName, gitInfo = NULL, timeout = 10000
)

Parameter description

Parameter	Type/Form	Description	Required
strategyName	SYMBOL	Strategy name	Yes
gitInfo	DICT	GitLab configuration	No
timeout	INT	Request timeout	No

This function deletes the corresponding strategy file from GitLab. Use with caution. You can only delete strategies uploaded by yourself.

Submit backtests or simulations with code upload

def runCryptoAndUploadToGit(
    strategyName, eventCallbacks, strategyType, userConfig = NULL,
    barMinutes = 1, overwrite = false, gitInfo = NULL,
    commitMsg = NULL, timeout = 10000
)

Parameter description

Parameter	Type/Form	Description	Required
strategyName	SYMBOL	Strategy name	Yes
eventCallbacks	DICT	Event callback functions	Yes
strategyType	INT	Strategy type (0: backtest, 1: simulation)	Yes
userConfig	DICT	Strategy configuration (Only start/end time can be modified; defaults are defined in setting)	No
exchange	STRING	Exchange providing the source data (default: "Binance").	No
barMinutes	INT	Frequency in minutes, effective for OHLC data only.	No
overwrite	BOOL	Whether to delete existing simulation strategies with the same name (default: false).	No
gitInfo	DICT	GitLab configuration	No
commitMsg	STRING	Commit mesaage	No
timeout	INT	Request timeout	No

This function wraps backtest/simulation submission and strategy code upload, using default strategy configuration. You only need to define strategyName , eventCallbacks , and strategyType to quickly deploy a cryptocurrency strategy. For details, see Section 2.3.

3. Strategy Result Visualization

To help you analyze backtesting and simulated trading results more intuitively, we provide separate dashboards for backtesting and simulation. These dashboards enable visual strategy management. You only need to import the dashboard files included in the Appendix—no additional configuration is required.

3.1 Backtesting Strategy Visualization

The backtesting dashboard displays all backtesting strategies and their results. It includes the following content:

Strategy list

Shows the strategy names, engine status, error messages, and related information.

Backtesting results

Select a strategy name and click Query to retrieve the backtesting results.

Submit strategy for simulation

You can submit strategies that have been backtested for simulated trading. The simulation strategies will appear in the simulation dashboard.

Delete backtesting strategies

You can delete unneeded backtesting strategies.

3.2 Simulated Trading Strategy Visualization

The simulation dashboard displays all simulated trading strategies and their results. Its structure is the same as the backtesting dashboard. The page refreshes once per second to display real-time strategy results.

Simulation strategy list
Real-time trades and positions
Delete simulation strategies

You can delete unneeded simulation strategies.

Stop simulated trading

Simulation strategies that are no longer needed can be stopped. After stopping, the strategy will appear in the backtesting dashboard.

4. Strategy Permission Management

To support strategy isolation within quantitative teams, we implement user-level permission isolation for both backtesting and simulation strategies. Each user can only view the backtesting and simulation results they created, and strategy code is stored in user-defined GitLab repositories.

To use the cryptocurrency data environment for backtesting or simulation, contact the admin to create a user account with the required read permissions on the relevant databases and tables. Then, you can log in as the created account and import the dashboard files to view backtesting and simulation strategies. A permission granting script is shown below:

// Executed by admin
login("admin","DolphinDB123456")
// Create a new user
createUser("user1","123456",,false)
// Grant database and table permissions
grant(`user1, DB_READ, "dfs://CryptocurrencyDay")
grant(`user1, DB_INSERT,"dfs://CryptocurrencyStrategy")
grant(`user1, TABLE_READ, "P1-node1:Cryptocurrency_depthTradeMerge")
// Function view permissions
grant("user1", VIEW_EXEC, "CryptocurrencySolution::utils::getSimulatedTradingEngine")
grant("user1", VIEW_EXEC, "CryptocurrencySolution::utils::getAllBacktestEngine")
// Dashboard permissions
dashboard_grant_functionviews(`user1, NULL, false)
// Grant permissions to group member bk
grant(`bk, VIEW_EXEC, objs="CryptocurrencySolution::utils::getSimulatedTradingEngine")
grant(`bk, DB_INSERT, "dfs://CryptocurrencyStrategy")   // Permission on database storing backtesting results
grant(`bk, VIEW_OWNER)   // Strategy storage requires function view permission

Note: If you want to share strategy backtesting results, use the dashboard sharing feature. Shared dashboards are view-only.

For centralized management, the following functions are restricted to the admin user:

Unsubscribe all minute-level or snapshot stream tables.

CryptocurrencySolution::utils::unsubscribeTableByAdmin(tbName)

Retrieve return comparisons for all simulated strategies.

CryptocurrencySolution::utils::getReturnComparision(startDate, endDate)

Rebuild the database for strategy IDs and backtesting results. (Write permissions must be re-granted to the specific user after rebuilding.

CryptocurrencySolution::utils::createdatabase()

Conclusion: From Concept to Live Trading, Faster

Building a successful quantitative crypto trading operation requires more than just good ideas—it demands robust infrastructure that can handle massive data volumes, complex execution logic, and seamless collaboration across teams.

With DolphinDB's integrated backtesting and simulation solution, you get:

Performance at scale : Handle millions of ticks per day without breaking a sweat
Realistic execution : Sophisticated order matching that mirrors real market conditions
Rapid iteration : From strategy idea to validated backtest in minutes, not days
Team collaboration : Built-in version control and permission management
Seamless deployment : One-parameter switch from backtest to live simulation

Whether you're an individual trader testing your first momentum strategy or a quantitative fund managing dozens of algorithms across multiple exchanges, this platform provides the foundation you need to move confidently from concept to live trading.

Ready to start building? Check out our documentation at http://www.dolphindb.com/ or reach out to our team at info@dolphindb.com to get started.