Benchmarking Forth


That Forth is a superior programming model is obvious. But how exactly is the economical benefit for using Forth in real applications? That is very hard to guess. Let us first research a case in which Forth makes no advantages and even slow down the computer. If somebody is using his normal x86 computer and runs on top a gforth he will recognize that the algorithm runs slower than with a C-Compiler. A program written in C and compiled to assembly needs 100 seconds, while the same algorithm implemented in gforth takes on the same hardware at least 500 seconds. So it makes no sense to switch to forth only for performance reasons.

Now we repeat the experiment with a setup which is more fair to forth. We construct a dedidcated Forth chip in an FPGA and testing out the performance. According to http://www.richardhaskell.com/files/forthcoremm.pdf the pure Forth version is 6x times faster then the same algorithm written in C and compiled to assembly. That is a lot, if the algorithm takes with a compiler 100 seconds to run, that Forth version would only take 17 seconds. But, it is possible to improve the advantage for Forth further. Because Forth chips are easier to develop than the x86 CPU it is possible to use the increase by the factor 6x only as the starting point and develop a more efficient CPU which uses parallel architecture, optimizer and so forth. This would be a longterm goal, because such adavanced CPU are not available today.

So we have explained the benefit of Forth, but there are also disadvantages. At first, a Forth engine can only calculate the same data as a normal C program can do. Both a turing-complete machines. That means a poor written algorithm in Forth is no improvement compared to the same algorithm in C. The second problem with Forth is, that no good tutorials are available. And if a book describes the inner working of Forth it is hard to understand for beginners. And the third problem is, that programming in Forth is more difficult than programming in C. Around the C programming language there is huge library of ready-to-run programs which were developed over years, for example, operating systems, compilers and user applications. In theory the same is possible with forth, but that is something which has to build from scratch. Efforts from the past to promote Forth for example the “ACM Sigforth newsletter” were not very successful, and todays Forth community like the SVFIG from Silicon Valley is unknown by computer mainstream. There are doubt, that it is really possible to decrease the costs with Forth, perhaps in the longterm future, but not today.

Jupiter Ace

From the raw numbers the Jupiter Ace is amazing, it is technical a superior machine. The reason why the Ace was economical not a success had three reasons: software, software and software. After turn the machine on, only a Forth interpreter was shown. If the user wants to make something useful with it, he have to program the system in Forth. The other option was to buy commercial software written in Forth, which was not available. So it was very usual that nobody want’s to buy the Ace. Other homecomputers from that time, had after turning on a BASIC interpreter which is very easy to program, because it has variables and for spending money other commercial software was available.

Today the problem is the same. From a technical point of view and from computer science perspective systems like the GA144 chip are a dream machine. They are Forth in hardware which runs extreme well. But software is missing too. Commercial software in Forth is not available and programming an operating system from scratch is out of reach for most users.

Let us watch a remarkable hybrid system. The “Java Optimized Processor”. It is a stackmachine designed for running the Java virtual machine. Java is language widely used in mainstream computing, and the JOP CPUs runs the system faster. Surprisingly, the JOP was also a failure, nobody is interested in the chip. The problem is, that apart from Java sourcecode the CPU can not run any other program for example, which was written in C++. A writing a C++ to Java compiler is a hard task. It is the same problem like writing a compiler for a forth CPU. The problem is, that both systems are very different. And a c compiler usually needs lots of registers, which isn’t provided by a stackmachine.

Let us go a step back. What is the problem? After turning a computer on, the user want’s to do with the machine. For example play a game or surfing in the internet. Doing so is only possible with software. A short look to github shows us, that 99% of all programs are written in C/C#/C++/Java and only 1% or less are written in a stackbased programming language. Why is the number of sourcecode in Forth not higher? This has to do with education. The programmers are not familiar with it, it isn’t teached in the university. So the general question is: how to write software in Forth?

Running software which is already written is easy. There are many Forth machines out there and lots of Forth CPUs. But without the sourcecode they are not useful. As far as i know, there is no commercial company like Microsoft out there who is programming operating systems, games and office applications in Forth. And even Forth experts, who can write sourcecode in this language are preferring other language for writing code.

Lets take a look into the details why programming in Forth is hard. The first fact is, that a Forth program has the same lines of code like a c program. Forth is a high-level-language for implementing an algorithm. If a jump’n’run game takes in C++ 1000 lines of code, than the same games takes in Forth also 1000 lines of code. The reason is, that all the routines like gui, keyinput, game-engine must be available in both programs. Reducing the number of lines is in theory possible but the average programmer is not able to do so. Even if Forth is a powerful language it has no magic trick to express with fewer lines of code the same.

On the Internet is a famous Forth game available called “Darkstar” which is very well programmed. If somebody want’s to port it to C the resulting sourcecode has the same length. That means, from the number of codelines Forth and C are equal. And if somebody is fluent in Forth he would take the same amount of time to code the game. And this is perhaps the main reason why most programmers are not using Forth. They have no advantage with the language. And in case of doubt, Forth programming is more complicated because it is different.

The elegance and the simpilcity of Forth is its biggest weakness. Programming in Forth means, program a computer. The most users doesn’t want this. They want to program software, that means for a virtual machine, which is in their imagination. The use the C language for communicating with the machine. Not with a real CPU or a real computer, but with an interface provided by C/C++. This interface is called structured programming or object-oriented programming. Not using it is from a technical point of view possible, but it is harder.

In a previous blogpost I have compared Forth with a fixed gear bike. This is true. A fixie is the superior bicycle, that means it is state of art in mechanics. But, 99% of all people are not interesting in fixed gear. They understand perhaps the difference, but they do not want to ride a bike, they want to drive around. Sometimes ago they have learned to use a bike, and this learning implied that every bike has a gear, which means it is possible to leave the feet on the bike. Somebody may argue, that the have learned the wrong thing, but in reality it is not possible to learn to drive with a fixed gear bike. It is an economical problem and in some countries it is forbidden to use fixies in real traffic.

Forth migration project

Let us talk about a Forth migration project. The Windows 10 operating system has around 40 million lines of code. Mostly written in C and C++. If we want to have the same software in Forth, the same number of codelines are nessary. That results into a Forth project containing 40 million lines of code. The average programmer writes around 10 lines of code per day. This results into 11000 man-years of programming. The average programmer costs 50k US$ per year, which results into 0.55 billion US$ total. This new Forth project wouuldn’t run on normal CPUs very well, so we need new hardware which costs also many amount of US$. And what is the benefit? Right, there is no benefit. Because the new Forth operating system would have the same problems like todays Operating systems. .

I have described above two major bottlenecks. At first, Forth sourcecode needs the same amount of codelines like C sourcecode and secondly, the productivity of a programmer is with 10 Loc/day the same. From my knowledge, there is no way to overcome the problems. Program more lines of code per day is not possible. In reality, programming in Forth would be a bit slower. And secondly, it is not possible to get the same software with less lines of code. It is right, that todays Forth programs are very small, but the reason is not because they are so well written, it is because that their inventor have no time for programming bigger applications with GUIs and similar things.

I see the future of Forth not in the commercial sector but for educational purposes. It makes sense to use Forth for teaching about compilers, stackmachines and turing-machines. Perhaps in the same way, Pascal was used for creating the PL/0 virtual machine. That means, Forth is well suited in a clean environment for leaving out the details. But I’m in doubt that it is possible to use Forth in huge projects for programming games or operating systems.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.