Guides/Batch Test

Batch test is a code-centric comparative performance tester of code cases over data series. It operates at a number of cases (expressions of verbs that return the same result, but may differ in performance) and series (sets of input data for the cases). The result is tabulation, plot and error log of each case against each series.

Fibonacci

require 'user/batchtest/batchtest.ijs'
(coclass [ coerase@<) 'pfib1'
coextend 'pbatchtest'

create=: 3 : 0
  CASPR=: 'fib'
  create_pbatchtest_ f. ''
  TITLE=: 'Fibonacci numbers'
  SHOWWHAT=: ;:'tableout errorsout plot'
  XPOSE=: 1
)

x=: +/ .*

fib0=: +/@((] ! -) i.@>:@>.@-:)

fib1=: 3 : '{.{: x/ x~^:(I.|.#:>:y.) 2 2$0 1 1 1x'

fib1a=: 3 : '{.{.(+/ . *)^:(>:y.)~2 2$0 1 1 1x'

fib2=: 3 : '{. +/\@|.^:(>:y.) 0 1x'

fib2a=: 3 : '{.(+/ , {.)^:y. 1 0x'

series045x=: 45x
series450x=: 450x

btest >coname''   NB. run with Ctrl+W

+--+------------------+------------------+
|  |045x              |450x              |
+--+------------------+------------------+
|0 |0.000771886 202368|   0.036872 148608|
|1 |0.000184381  19968|0.000296686  32896|
|1a|0.000566832   8128| 0.00569458  12352|
|2 |0.000341943   5440| 0.00327025   6400|
|2a| 0.00015393   4544| 0.00144041   5504|
+--+------------------+------------------+

Locales as Hashtable

require 'user/batchtest/batchtest.ijs'
(coclass [ coerase@<) 'ptest1'
coextend 'pbatchtest'

create=: 3 : 0
  create_pbatchtest_ f. ''
  TITLE=: 'Locale as Hashtable'
  SHOWWHAT=: ;:'tableout errorsout plot'
  setup''
)

case1Set=: testset
case2Get=: testget

series00128=: 128
series00256=: 256
series00512=: 512
series01024=: 1024
series02048=: 2048
series04096=: 4096
series08192=: 8192
NB. series16384=: 16384
NB. series32768=: 32768
NB. series65536=: 65536

setup=: 3 : 0
  9!:33]0
  servals=. (<SERPR) ".@,&> SERIES
  r=. ''
  for_i. i.#SERIES do.
    loc=. 'L',>i{SERIES
    coerase <loc
    for_j. i.i{servals do.
      r=. r,'v',(":j),'_',loc,'_=:',(":j),'['
    end.
  end.
  ".}:r
)

at=: , '_'&([,~,)

testset=: 3 : 0
  loc=. 'L',_5{.!.'0'":y.
  for_i. i.1000 do.
    (('w',":i)at loc) =: i
  end.
)

testget=: 3 : 0
  loc=. 'L',_5{.!.'0'":y.
  for_i. i.1000 do.
    assert. i -: (('w',":i)at loc)~
  end.
)

btest >coname''   NB. run with Ctrl+W

+-----+----------------+--------------+
|     |1Set            |2Get          |
+-----+----------------+--------------+
|00128|0.0132919 200704|0.0124541 8576|
|00256|0.0127991 200704|0.0128463 8576|
|00512|0.0130433 200704|0.0124672 8576|
|01024|0.0128223 200704|0.0149256 8576|
|02048|0.0132651 200704|0.0125723 8576|
|04096|0.0133852 200704|0.0131075 8576|
|08192|0.0135649 200704|0.0126402 8576|
+-----+----------------+--------------+

Array Loop

require 'user/batchtest/batchtest.ijs'
(coclass [ coerase@<) 'ptest1'
coextend 'pbatchtest'

create=: 3 : 0
  create_pbatchtest_ f. ''
  TITLE=: 'Array Loop'
  SHOWWHAT=: ;:'tableout errorsout plot'
)

('series',"1 0'ABCDEF')=: 1000*1+i.6

case1=: 3 : 0
  i=. _1 [ r=. y.$0
  while. y.>i=.i+1 do.
    r=. 1 i} r
  end.
  +/r
)

case2=: 3 : 0
  r=. y.$0
  for_i. i.y. do.
    r=. 1 i} r
  end.
  +/r
)

case3=: 3 : 0
  r=. i.y.
  for_i. r do.
    r=. 1 i} r
  end.
  +/r
)

case4=: 3 : 0
  r=: i.y.
  aux4^:y.0
  +/r
)

aux4=: 3 : 0
  r=: 1 y.} r
  y.+1
)

NB. uncomment below for Ctrl+W
btest >coname''

+-+----------------+---------------+----------------+---------------+
| |1               |2              |3               |4              |
+-+----------------+---------------+----------------+---------------+
|A|0.00823149  4992|0.0062589  9856|0.00638712 11968|0.0114934  8448|
|B| 0.0168521  4992|0.0126086 13952| 0.0136556 20160|0.0223621 10304|
|C| 0.0263352  7040|0.0186228 24192| 0.0196114 36544|0.0343245 18496|
|D|  0.033245  7040|0.0250593 24192|  0.026507 36544|0.0475977 18496|
|E| 0.0413385 11136|0.0315959 44672| 0.0325404 69312|0.0572617 34880|
|F| 0.0509235 11136|0.0378191 44672| 0.0397827 69312|0.0687132 34880|
+-+----------------+---------------+----------------+---------------+

Dyadic X Argument

For dyadic cases a fixed right argment is given in create with string representation.

XARG=: '2000'

Bar Chart

Another output is barsout which has the same look as tableout but magnitudes are shown as bars.

  SHOWWHAT=: ;:'... barsout ...'

+--+--------+---------------------+
|  |A       |B                    |
+--+--------+---------------------+
|0 |**  *** |******     ********* |
|1 |*   *** |**         ******    |
|2 |*   *** |***        ********* |
|2a|**  ****|*****      **********|
|3 |*** ****|********** ********* |
|3a|**  *** |*****      ********* |
|4 |*   *   |**         ***       |
+--+--------+---------------------+

Equivalence

eqtest allows to quickly verify that all cases return the same result. It takes the class name and matches each by each result for the first series.

   eqtest'pcopy1'
1 1 1 1
1 1 1 1
1 1 1 1
1 1 1 1

See Also

• Addons/general/unittest, Unit Test Framework

Contributed by Oleg Kobchenko