### Author Topic: Integer/looping benchmark  (Read 5364 times)

#### Ed Davis

• Guest
##### Integer/looping benchmark
« on: April 20, 2016, 12:21:39 AM »
Here is the latest update of the integer intensive interpreter benchmark. Interestingly, the fastest interpreter I tested is over 7 times slower than optimized C.

What is the purpose of this?

I enjoy fiddling with compilers/interpreters, especially those that are simple enough that I can understand. I also enjoy writing my own.  I wanted to find out what makes some interpreters so much faster than others, and why are others so slow. Examining the source to some of these interpreters has helped me learn some of the reasons, and has helped to improve the speed of those I'm working on.

Here is the test code in fairly standard Basic:

Code: [Select]
`accum = 0count = 0while count < 1545    leftedge   = -420    rightedge  =  300    topedge    =  300    bottomedge = -300    xstep      =  7    ystep      =  15    maxiter    =  200    y0 = topedge    while y0 > bottomedge        x0 = leftedge        while x0 < rightedge            y = 0            x = 0            thechar = 32            xx = 0            yy = 0            i = 0            while i < maxiter and xx + yy <= 800                xx = int((x * x) / 200)                yy = int((y * y) / 200)                if xx + yy > 800 then                    thechar = 48 + i                    if i > 9 then                        thechar = 64                    end if                else                    temp = xx - yy + x0                    if (x < 0 and y > 0) or (x > 0 and y < 0) then                        y = int(-1 * (-1 * x * y) / 100) + y0                    else                        y = int(x * y / 100) + y0                    end if                    x = temp                end if                i = i + 1            wend            x0 = x0 + xstep            accum = accum + thechar        wend        y0 = y0 - ystep    wend    if count mod 300 = 0 then        print accum,    end if    count = count + 1wendprint accum`
This is the output:

200574 60372774 120544974 180717174 240889374 301061574 309886830

This is a pretty intense integer benchmark!

Note this funky code:
Code: [Select]
`    if (x < 0 and y > 0) or (x > 0 and y < 0) then        y = int(-1 * (-1 * x * y) / 100) + y0    else        y = int(x * y / 100) + y0    end if`
It turns out that there is no consensus as to whether integer division, when one of the operands is negative, should round towards zero, negative infinity, or positive infinity.

Python and Ruby both round towards negative infinity.  C rounds towards zero.

In order to get the same output from each language (to verify that each language was essentially computing the same thing and doing similar work), I had to figure out how to preclude one of the operands from being negative.

The code checks, and if either x or y is < 0 (but not both), then it multiplies by minus one to force positive division, and then by minus one again at the end to restore the sign.

Below are the tests that I have run.  I would welcome your additions and/or suggestions.

Test machine:
Windows 7, Service Pack 1, 64-bit
Intel Core i7-3720QM CPU @2.60GHz
16.0 GB (15.9 usable)

In the tables below
• Native means that the code is compiled to machine code, one way or another, and an .exe is created.
• JIT means that the code is only compiled to machine code on a Just In Time basis.
• VM means that the code is compiled into virtual machine code, and that VM code is executed by a VM interpreter.
• Interp means that the code is not compiled but interpreted - it may be tokenized though.
• ? means I don't know how this processor works.  Updates are appreciated!
Basic compiler/interpreters:

 FreeBasic 1.09 seconds Native Basic Compiler BCX 1.12 seconds Native Basic to C translator VisualBasic.Net 1.52 seconds JIT Oxygen Basic 3.77 seconds Native PowerBasic 4.03 seconds Native QB64 32.43 seconds Native Basic to C++ translator JWillia basic 109    seconds VM Basic interpreter ChipMunkBasic 216    seconds Interp Yabasic 278    seconds VM SdlBasic 310    seconds ? SmallBASIC 354    seconds ? SpecBAS 358    seconds VM Basic interpreter BBCBasic 531    seconds ? ThinBasic 543    seconds ? FBSL 551    seconds ? Basic interpreter Scriba 618    seconds ? Basic interpreter RCBasic 778    seconds ? LBB 1502    seconds ? Liberty Basic Booster DDS5 3033    seconds Interp Extended Tiny Basic my-basic 3302    seconds ? MiniBASIC 5471    seconds Interp nuBASIC 8132    seconds ? Basic256 18852    seconds ? bscript 21558    seconds ? Basic interpreter

Various Toy implementations - mainly to experiment with speeds of various implementations:

 Toy7.c 7.80 seconds VM gcc goto's, TOS, superinstructions Toy6.c 19.76 seconds VM gcc goto's Toy.bas 33.35 seconds VM compiled with FreeBASIC -lang qb -O 3 Toy5.c 53.48 seconds VM Simple stack-based VM, standard C switch Toy.bas Oxygen ver. 82.34 seconds VM console integer version - float version fails Toy - tokenized, no VM 137    seconds Interp Tokenizing interpreter - no VM Toy - pure interpreter 1154    seconds Interp pure-interpreter - re-lexes each token - no VM

All the rest:

 gcc C -O3 or -O2 1.00 seconds Native FreeBasic 1.09 seconds Native Basic Compiler BCX 1.12 seconds Native Basic to C translator Nim 1.12 seconds Native gcc C -O  or -O1 1.13 seconds Native Java 1.23 seconds JIT C# 1.34 seconds JIT VisualBasic.Net 1.52 seconds JIT FreePascal 1.92 seconds Native C, no options 1.96 seconds Native C -Os 2.38 seconds Native Borland C 2.80 seconds Native TinyC 3.23 seconds Native euc -gcc -con 3.58 seconds Native Euphoria to C translator Oxygen Basic 3.77 seconds Native PowerBasic 4.03 seconds Native Fast Toy.c 7.80 seconds VM gcc goto's, TOS, superinstructions Javascript via node.js 10.23 seconds JIT pe 64-bit 14.26 seconds VM pe 32-bit 17.96 seconds VM Toy6.c 19.76 seconds VM gcc goto's Java -Xint 21.24 seconds VM Java without JIT QB64 32.43 seconds Native Basic to C++ translator Toy.bas 33.35 seconds VM compiled with FreeBASIC -lang qb -O 3 Euphoria v4.1 beta 2 35.66 seconds VM Pike 44.39 seconds VM C like interpreter Toy5.c 53.48 seconds VM TinyPas.c 58.97 seconds VM Pascal-S converted to C. Ruby 60.25 seconds VM SAL 67.22 seconds VM vspl 68.76 seconds VM Lua 80.14 seconds VM php 81.23 seconds VM Toy.bas Oxygen ver. 82.34 seconds VM console integer version - float version fails Wren 84.57 seconds VM PL0.c 97.33 seconds VM Wirth's 1976 Tiny Pascal converted to C C4 97.80 seconds VM C subset interpreter UnderC 103    seconds VM C++ interpreter JWillia basic 109    seconds VM Basic interpreter Toy - tokenized, no VM 137    seconds Interp Tokenizing interpreter - no VM hoc 140    seconds VM Higher Order Calculator Lily 162    seconds VM NaaLaa 168    seconds VM CInt 201    seconds VM C++ interpreter ChipMunkBasic 216    seconds Interp Python 274    seconds VM Yabasic 278    seconds VM SdlBasic 310    seconds ? Hanson-calc 321    seconds VM Hanson's version of hoc SmallBASIC 354    seconds ? SpecBAS 358    seconds VM Basic interpreter BBCBasic 531    seconds ? ThinBasic 543    seconds ? FBSL 551    seconds ? Basic interpreter Ch 582    seconds ? C interpreter Scriba 618    seconds ? Basic interpreter RCBasic 778    seconds ? SI 1020    seconds Interp C subset interpreter Toy - pure interpreter 1154    seconds Interp pure-interpreter - re-lexes each token - no VM LBB 1502    seconds ? Liberty Basic Booster LittleC 2160    seconds Interp C subset interpreter PicoC 2352    seconds ? C interpreter DDS5 3033    seconds Interp Extended Tiny Basic my-basic 3302    seconds ? MiniBASIC 5471    seconds Interp nuBASIC 8132    seconds ? Basic256 18852    seconds ? bscript 21558    seconds ? Basic interpreter

#### jcfuller

• Guest
##### Re: Integer/looping benchmark
« Reply #1 on: April 20, 2016, 09:53:49 AM »
Ed,
Nice work!!
Are  the BCX numbers relative to the c/c++ compiler used.
If so please list the compiler for the BCX test.
James

#### Ed Davis

• Guest
##### Re: Integer/looping benchmark
« Reply #2 on: April 20, 2016, 11:06:22 AM »
Are  the BCX numbers relative to the c/c++ compiler used.

Most assuredly.

Quote
If so please list the compiler for the BCX test.

3 tries for each one, taking the lowest number.

gcc (tdm64-1) 5.1.0

-s -Ofast -m64  1.17
-s -Ofast -m32  1.43

gcc (tdm64-2) 4.8.1

-s -Ofast -m64  1.12
-s -Ofast -m32  1.43

I've noticed that 5.1 is slightly slower that 4.81 in some things, however 5.1 beats 4.81 in a few of my tests too.

#### wang renxin

• Guest
##### Re: Integer/looping benchmark
« Reply #3 on: April 20, 2016, 01:49:43 PM »
Great job Ed. FYI. MY-BASIC works as an interpreter.

#### ZXDunny

• Guest
##### Re: Integer/looping benchmark
« Reply #4 on: April 20, 2016, 01:55:34 PM »
FWIW, I've made quite a lot of changes to code flow in specbas just recently - cached GOTO/GOSUB and incomplete boolean evaluation so will likely shave a few seconds off that time. I'll release it soon.

D.

#### Ed Davis

• Guest
##### Re: Integer/looping benchmark
« Reply #5 on: April 20, 2016, 03:20:59 PM »
Great job Ed. FYI. MY-BASIC works as an interpreter.

Thank you, and thanks for the clarification.

I've played with MY-BASIC somewhat, and you've done a good job.  I especially like some of the newer features you have added lately.

#### Ed Davis

• Guest
##### Re: Integer/looping benchmark
« Reply #6 on: April 20, 2016, 03:22:30 PM »
FWIW, I've made quite a lot of changes to code flow in specbas just recently - cached GOTO/GOSUB and incomplete boolean evaluation so will likely shave a few seconds off that time. I'll release it soon.

Looking forward to it!  I keep it around, because I always enjoy running your graphic programs!

#### wang renxin

• Guest
##### Re: Integer/looping benchmark
« Reply #7 on: April 21, 2016, 01:22:22 AM »
I especially like some of the newer features you have added lately.

Thanks, MB will keep evolving.