[Pseudoknot benchmark]Benchmarking Implementations of Functional Languages with ``Pseudoknot'', a Float-Intensive Benchmark
[Hartel, Feeley et al.]
With few exceptions, the compilers take a significant amount of time to compile this program, though most compilers were faster than the current GNU GCC compiler. Compilers that generate C or Lisp are often slower than those that generate native code directly; the cost of compiling the intermediate form is normally a large fraction of the total compilation time.
There is no clear distinction between the runtime performance of eager and lazy implementations when appropriate annotations are used: lazy implementations have clearly come of age when it comes to implementing largely strict applications, such as the Pseudoknot program. The speed of C can be approached by some implementations, but to achieve this performance, special measures such as strictness annotations are required by non-strict implementations.
The benchmark results have to be interpreted with care. Firstly, a benchmark based on a single program cannot cover a wide spectrum of ``typical'' applications. Secondly, the many compilers used are varied in the kind and level of optimisations offered, so the efforts to obtain optimal versions of the program are similarly varied.