Addons/math/mt/Benchmarking

Benchmarking process uses matrices of size(s) and generator given, and produces scalar result for each size. Benchmarking result is a sum of estimated execution durations (in seconds) for some verbs set. Verbs set is defined by self-test verb test_mt_ . An execution duration for each verb is estimated as proposed in : "the minimum run-time of 3-5 executions of the program when the machine is lightly loaded.".

Benchmarking process outputs some useful information while progress. This output is directed into:

• console
• log file: ~temp/mt.benchmark.*.log
• raw measurements data file: ~temp/mt.benchmark.*.raw
• formatted result file: ~temp/mt.benchmark.*.result

Typical workflow to compare different hardware environments performance for the same J interpreter version

  # on one host:
  user@i5-3230M:~/j64-805> ./jconsole.sh
     load 'math/mt'
     mkmat=. _1 1 0 3 _6 4&gemat_mt_
     bmkrun=. mkmat benchmark_mt_
     sizes=. 100 150 200 300 400 500
     ] out1=. 'i5-3230M_j64-805' bmkrun sizes
     NB. a lot of output, very long execution...
  75.305 283.243 1040.378 9666.966 26840.712 66151.747
     exit ''

  # on another host:
  user@i7-4790:~/j64-805> ./jconsole.sh
     load 'math/mt'
     mkmat=. _1 1 0 3 _6 4&gemat_mt_
     bmkrun=. mkmat benchmark_mt_
     sizes=. 100 150 200 300 400 500
     ] out2=. 'i7-4790_j64-805' bmkrun sizes
     NB. a lot of output, very long execution...
  42.323 145.205 395.336 1782.877 8592.176 21163.120
     NB. now we can compare a speed-up of 2nd host against 1st one:
     sizes ,. 75.305 283.243 1040.378 9666.966 26840.712 66151.747 % out2
  100 1.77929
  150 1.95064
  200 2.63163
  300 5.42212
  400 3.12386
  500  3.1258
     exit ''

Typical workflow to compare different J interpreter versions performance for the same hardware environment

  # in one interpreter:
  user@i7-4790:~/j64-805> ./jconsole.sh
     load 'math/mt'
     mkmat=. _1 1 0 3 _6 4&gemat_mt_
     bmkrun=. mkmat benchmark_mt_
     sizes=. 100 150 200 300 400 500
     ] out1=. 'i7-4790_j64-805' bmkrun sizes
     NB. a lot of output, very long execution...
  42.323 145.205 395.336 1782.877 8592.176 21163.120
     exit ''

  # in another interpreter:
  user@i7-4790:~/j64-806> ./jconsole.sh
     load 'math/mt'
     mkmat=. _1 1 0 3 _6 4&gemat_mt_
     bmkrun=. mkmat benchmark_mt_
     sizes=. 100 150 200 300 400 500
     ] out2=. 'i7-4790_j64-806' bmkrun sizes
     NB. a lot of output, very long execution...
  43.179 150.100 400.637 1814.688 8729.023 20626.823
     NB. now we can compare a speed-up of 2nd interpreter against 1st one:
     sizes ,. 42.323 145.205 395.336 1782.877 8592.176 21163.120 % out2
  100 0.980176
  150 0.967388
  200 0.986769
  300  0.98247
  400 0.984323
  500    1.026
     exit ''