Here is the latest update of the integer intensive interpreter benchmark. Interestingly, the fastest interpreter I tested is over 7 times slower than optimized C.
What is the purpose of this?
I enjoy fiddling with compilers/interpreters, especially those that are simple enough that I can understand. I also enjoy writing my own. I wanted to find out what makes some interpreters so much faster than others, and why are others so slow. Examining the source to some of these interpreters has helped me learn some of the reasons, and has helped to improve the speed of those I'm working on.
Here is the test code in fairly standard Basic:
accum = 0
count = 0
while count < 1545
leftedge = -420
rightedge = 300
topedge = 300
bottomedge = -300
xstep = 7
ystep = 15
maxiter = 200
y0 = topedge
while y0 > bottomedge
x0 = leftedge
while x0 < rightedge
y = 0
x = 0
thechar = 32
xx = 0
yy = 0
i = 0
while i < maxiter and xx + yy <= 800
xx = int((x * x) / 200)
yy = int((y * y) / 200)
if xx + yy > 800 then
thechar = 48 + i
if i > 9 then
thechar = 64
end if
else
temp = xx - yy + x0
if (x < 0 and y > 0) or (x > 0 and y < 0) then
y = int(-1 * (-1 * x * y) / 100) + y0
else
y = int(x * y / 100) + y0
end if
x = temp
end if
i = i + 1
wend
x0 = x0 + xstep
accum = accum + thechar
wend
y0 = y0 - ystep
wend
if count mod 300 = 0 then
print accum,
end if
count = count + 1
wend
print accum
This is the output:
200574 60372774 120544974 180717174 240889374 301061574 309886830
This is a pretty intense integer benchmark!
Note this funky code:
if (x < 0 and y > 0) or (x > 0 and y < 0) then
y = int(-1 * (-1 * x * y) / 100) + y0
else
y = int(x * y / 100) + y0
end if
It turns out that there is no consensus as to whether integer division, when one of the operands is negative, should round towards zero, negative infinity, or positive infinity.
Python and Ruby both round towards negative infinity. C rounds towards zero.
In order to get the same output from each language (to verify that each language was essentially computing the same thing and doing similar work), I had to figure out how to preclude one of the operands from being negative.
The code checks, and if either x or y is < 0 (but not both), then it multiplies by minus one to force positive division, and then by minus one again at the end to restore the sign.
Below are the tests that I have run. I would welcome your additions and/or suggestions.
Test machine:
Windows 7, Service Pack 1, 64-bit
Intel Core i7-3720QM CPU @2.60GHz
16.0 GB (15.9 usable)
In the tables below- Native means that the code is compiled to machine code, one way or another, and an .exe is created.
- JIT means that the code is only compiled to machine code on a Just In Time basis.
- VM means that the code is compiled into virtual machine code, and that VM code is executed by a VM interpreter.
- Interp means that the code is not compiled but interpreted - it may be tokenized though.
- ? means I don't know how this processor works. Updates are appreciated!
Basic compiler/interpreters:| FreeBasic | 1.09 seconds | Native | Basic Compiler |
| BCX | 1.12 seconds | Native | Basic to C translator |
| VisualBasic.Net | 1.52 seconds | JIT | |
| Oxygen Basic | 3.77 seconds | Native | |
| PowerBasic | 4.03 seconds | Native | |
| QB64 | 32.43 seconds | Native | Basic to C++ translator |
| JWillia basic | 109 seconds | VM | Basic interpreter |
| ChipMunkBasic | 216 seconds | Interp | |
| Yabasic | 278 seconds | VM | |
| SdlBasic | 310 seconds | ? | |
| SmallBASIC | 354 seconds | ? | |
| SpecBAS | 358 seconds | VM | Basic interpreter |
| BBCBasic | 531 seconds | ? | |
| ThinBasic | 543 seconds | ? | |
| FBSL | 551 seconds | ? | Basic interpreter |
| Scriba | 618 seconds | ? | Basic interpreter |
| RCBasic | 778 seconds | ? | |
| LBB | 1502 seconds | ? | Liberty Basic Booster |
| DDS5 | 3033 seconds | Interp | Extended Tiny Basic |
| my-basic | 3302 seconds | ? | |
| MiniBASIC | 5471 seconds | Interp | |
| nuBASIC | 8132 seconds | ? | |
| Basic256 | 18852 seconds | ? | |
| bscript | 21558 seconds | ? | Basic interpreter |
Various Toy implementations - mainly to experiment with speeds of various implementations:| Toy7.c | 7.80 seconds | VM | gcc goto's, TOS, superinstructions |
| Toy6.c | 19.76 seconds | VM | gcc goto's |
| Toy.bas | 33.35 seconds | VM | compiled with FreeBASIC -lang qb -O 3 |
| Toy5.c | 53.48 seconds | VM | Simple stack-based VM, standard C switch |
| Toy.bas Oxygen ver. | 82.34 seconds | VM | console integer version - float version fails |
| Toy - tokenized, no VM | 137 seconds | Interp | Tokenizing interpreter - no VM |
| Toy - pure interpreter | 1154 seconds | Interp | pure-interpreter - re-lexes each token - no VM |
All the rest:| gcc C -O3 or -O2 | 1.00 seconds | Native | |
| FreeBasic | 1.09 seconds | Native | Basic Compiler |
| BCX | 1.12 seconds | Native | Basic to C translator |
| Nim | 1.12 seconds | Native | |
| gcc C -O or -O1 | 1.13 seconds | Native | |
| Java | 1.23 seconds | JIT | |
| C# | 1.34 seconds | JIT | |
| VisualBasic.Net | 1.52 seconds | JIT | |
| FreePascal | 1.92 seconds | Native | |
| C, no options | 1.96 seconds | Native | |
| C -Os | 2.38 seconds | Native | |
| Borland C | 2.80 seconds | Native | |
| TinyC | 3.23 seconds | Native | |
| euc -gcc -con | 3.58 seconds | Native | Euphoria to C translator |
| Oxygen Basic | 3.77 seconds | Native | |
| PowerBasic | 4.03 seconds | Native | |
| Fast Toy.c | 7.80 seconds | VM | gcc goto's, TOS, superinstructions |
| Javascript via node.js | 10.23 seconds | JIT | |
| pe 64-bit | 14.26 seconds | VM | |
| pe 32-bit | 17.96 seconds | VM | |
| Toy6.c | 19.76 seconds | VM | gcc goto's |
| Java -Xint | 21.24 seconds | VM | Java without JIT |
| QB64 | 32.43 seconds | Native | Basic to C++ translator |
| Toy.bas | 33.35 seconds | VM | compiled with FreeBASIC -lang qb -O 3 |
| Euphoria v4.1 beta 2 | 35.66 seconds | VM | |
| Pike | 44.39 seconds | VM | C like interpreter |
| Toy5.c | 53.48 seconds | VM | |
| TinyPas.c | 58.97 seconds | VM | Pascal-S converted to C. |
| Ruby | 60.25 seconds | VM | |
| SAL | 67.22 seconds | VM | |
| vspl | 68.76 seconds | VM | |
| Lua | 80.14 seconds | VM | |
| php | 81.23 seconds | VM | |
| Toy.bas Oxygen ver. | 82.34 seconds | VM | console integer version - float version fails |
| Wren | 84.57 seconds | VM | |
| PL0.c | 97.33 seconds | VM | Wirth's 1976 Tiny Pascal converted to C |
| C4 | 97.80 seconds | VM | C subset interpreter |
| UnderC | 103 seconds | VM | C++ interpreter |
| JWillia basic | 109 seconds | VM | Basic interpreter |
| Toy - tokenized, no VM | 137 seconds | Interp | Tokenizing interpreter - no VM |
| hoc | 140 seconds | VM | Higher Order Calculator |
| Lily | 162 seconds | VM | |
| NaaLaa | 168 seconds | VM | |
| CInt | 201 seconds | VM | C++ interpreter |
| ChipMunkBasic | 216 seconds | Interp | |
| Python | 274 seconds | VM | |
| Yabasic | 278 seconds | VM | |
| SdlBasic | 310 seconds | ? | |
| Hanson-calc | 321 seconds | VM | Hanson's version of hoc |
| SmallBASIC | 354 seconds | ? | |
| SpecBAS | 358 seconds | VM | Basic interpreter |
| BBCBasic | 531 seconds | ? | |
| ThinBasic | 543 seconds | ? | |
| FBSL | 551 seconds | ? | Basic interpreter |
| Ch | 582 seconds | ? | C interpreter |
| Scriba | 618 seconds | ? | Basic interpreter |
| RCBasic | 778 seconds | ? | |
| SI | 1020 seconds | Interp | C subset interpreter |
| Toy - pure interpreter | 1154 seconds | Interp | pure-interpreter - re-lexes each token - no VM |
| LBB | 1502 seconds | ? | Liberty Basic Booster |
| LittleC | 2160 seconds | Interp | C subset interpreter |
| PicoC | 2352 seconds | ? | C interpreter |
| DDS5 | 3033 seconds | Interp | Extended Tiny Basic |
| my-basic | 3302 seconds | ? | |
| MiniBASIC | 5471 seconds | Interp | |
| nuBASIC | 8132 seconds | ? | |
| Basic256 | 18852 seconds | ? | |
| bscript | 21558 seconds | ? | Basic interpreter |
