Table Import and Export

Saving Data

We can save a table object to disk in binary mode with the function saveTable, or save to a text file with the function saveText . Compared to saveTable, saveText requires much more space and is more time consuming. However, it is convenient if users need to take a quick look at the data. It is recommended for saving small data sets.

To save a table to a DFS database, please first use function createPartitionedTable to create a DFS table, and then use function append! or tableInsert.

The example below shows how efficiently save saves a table with 20 million records to disk, compared with saveText.

n=20000000
syms=`IBM`C`MS`MSFT`JPM`ORCL`BIDU`SOHU`GE`EBAY`GOOG`FORD
timestamp=09:30:00+rand(18000,n)
sym=rand(syms,n)
qty=100*(1+rand(100,n))
price=5.0+rand(100.0,n)
t1=table(timestamp,sym,qty,price);
$ timer saveTable("C:/DolphinDB/Data", t1, `trades);
Time elapsed: 208.963 ms

$ saveText(t1, "C:/DolphinDB/Data/trades.csv");
Time elapsed: 3231.914 ms

Loading Data

We can use the function loadTable to load previously saved data, or use the function loadText to import a text file. loadText is more time consuming than loadTable (binary mode).

The example below shows how efficiently loadTable loads a table with 20 million records, compared with loadText.

$ n=20000000
$ syms=`IBM`C`MS`MSFT`JPM`ORCL`BIDU`SOHU`GE`EBAY`GOOG`FORD
$ timestamp=09:30:00+rand(18000,n)
$ sym=rand(syms,n)
$ qty=100*(1+rand(100,n))
$ price=5.0+rand(100.0,n)
$ t1=table(timestamp,sym,qty,price)
$ saveTable("C:/DolphinDB/Data", t1, `trades);

$ timer tt1 = loadTable("C:/DolphinDB/Data",`trades,,true);
Time elapsed: 179.423 ms

$ saveText(t1, "C:/DolphinDB/Data/trades.txt");
$ timer tt2=loadText("C:/DolphinDB/Data/trades.txt");
Time elapsed: 7609.7 ms