Example 9. Batch job dependency



A batch job may depend on the results of other batch jobs. We can ensure this batch job is run only after all the dependencies are successfully completed by setting the parameter blocking to true in function getJobReturn for all batch jobs. DolphinDB infers job dependencies based on batch job script, so we don't need to explicitly specify job dependencies with job IDs.


In this example we have 3 batch jobs: job1, job2, and job3. Job1 takes 5 seconds to finish. Job2 takes 4 seconds to finish. Job1 and job2 calculate the sum of their respective input variables. Job3 sums up the results of job1 and job2. Job3 should not run until both job1 and job2 are completed successfully.



def job1(x){

       sleep(5000)

       return x.sum()

}

def job2(y){

       sleep(4000)

       return y.sum()

}

def job3(x, y){

       job1Id = submitJob("job1","",job1{x})

       job2Id = submitJob("job2","",job2{y})

       job1result=getJobReturn(job1Id,true)

       job2result=getJobReturn(job2Id,true)

       return job1result + job2result

}


x=[1,2,3]

y=[1,2,3,4]

job3Id = submitJob("job3", "", job3{x,y})

getJobReturn(job3Id,true);



The result is 16, and it should take a bit more than 5 seconds to finish.

Download source code here.