Managing Open Source projects

According to the book “Producing Open Source Software: How to Run a Successful Free Software Project, 2006” an open source project contains of a mailing list, a bug tracker and a version control system. Let us describe an example:

A version control system and a bugtracker can be realized with github. It is only one click away. A mailing list can be created with other tools, or in the simple case a Facebook group is well suited. Now the fictional project has everything what is needed. But something is missing: right, the project is not attractive to anybody. After a month, we as the admin are the only guy who is contributing to the github repository, and apart from us nobody else is posting to the Facebook group. It is not wrong to use these tools, but it seems that something is missing.

Successful Open Source projects like the Linux kernel or the Lyx software for writing scientific papers contains usually of the above cited tools. There is a mailing list, a bugtracker and a version control system. But why have these projects traffic but our example project not? The answer is given in the following diagram:

Additional to the technical infrastructure, the figure has a social component which is called customer. The relationship is now divided. The customer is the outside world, while the group is locked in the project. The relationship between the customer and the open source projects defines in which direction the project will go. In our fictional project with a github repository and a facebook group no customer is there. That means, there is no need for traffic. In the case of the Linux kernel there is a customer, called Red Hat. The relationship between customer and project members is defined by money. Here the details: A company like IBM needs a certain feature in the Linux kernel. They are asking Red Hat for doing so. Red Hat collects there feature-request and prioritize them according to what the customer want to pay. IBM says for example, that the new scheduler is fair paid with 10000 US$. This money is the motivation for the Linux kernel team to have a detailed look into the need of IBM. They will analyze the problem, discuss the issues at the mailing list and write a patch to fix the problem. After the modifications runs, IBM pays the money and the transaction is complete. Understanding Open Source projects is possible with understanding the customer relationship and the amount of money which is transferred.

Let us make a different example. Somebody needs a 100 page long academic paper about a topic. This is without any doubt a project which can be done with group work. The naive approach is, if the students starts a github project, and initiate a mailing list. He believes, that his project is so important for the world, that other people will fork his github project with the LaTeX source and discuss the issue on the mailing. This will never happen. After 2 months, the student will alone on his github project, the group work isn’t there.

Now we modify the situation a bit and focus on a customer-relationship. This time, the student goes to a ghostwriter, pays 1000 US$ and after 2 months he gets the paper. It was written by a group, and this group is using github together with a mailing list. What is different, is that a social aspect is there and that some money was transferred.

It seems, that customer relationship and money are going hand in hand. But perhaps we can modify the chain a bit, but maintaining group working at the same time? The Stackoverflow website has an interesting reputation system, called “featured questions”, https://stackoverflow.com/?tab=featured The idea is the somebody sets a bounty (for example 400 points of his score) and after the question gets answered this score is transfered to other party. The result is amazing: Featured questions gets more attention than normal one. A bounty of 400 points is not so valuable like real money but it comes close to the idea. The reputation system abstracts from a concret problem. It is no longer interesting if somebody has a bug with “react-native” or “Json” instead it is about an issue which is connected to a bounty of 400 points, 200 points or whatever.

What I want to make clear, that groupworking needs a customer from the outside. That is the purpose of forming groups, and that is the motivation why the team is working together. Without a customer, the groupworking isn’t there. The success rate of a group is defined by the relationship to the customer. The group is not intrinsically motivated because a certain problem is so interesting. If this would be true, a student can set up a github project with the title “homework: programming a database in Java”, and after one day, he will gets lots of contributers to his project. But this won’t happen. Because the world has different plans than contributing to a students homework.

Stackoverflow’s reputation system

The Stackoverflow reputation system is interesting because it contains no real money but a fictional score at the website. Each user of the website can accumulate points for contributing content. It is similar to a computergame in which the player has also a score. If somebody sets a bounty for a question he doesn’t provide new content to the website, instead he asking the community for providing content. That means, a user sets a bounty and the other side is providing the answer. Suppose, somebody has an unlimited score. He can put via a random generator the maximum bounty of 400 points to all the question he likes. The result is, that these questions gets a higher traffic and after a short time they get answered by the community. This relationship is not totally predictable, because sometimes a question has a bounty but not answer, but in general there is a connection.

The bounty system of Stackoverflow simulates a customer relationship. The user who provides the bounty is the customer. He can ask for support. And the other side has to provide the work.

Exactly the same principle is used by Open Source projects like the Linux kernel. The only difference is, that the bounty there is real money and that the sum is higher. The principle itself remains the same. Somebody has unlimited amount of money, and can exchange this into sourcecode. Other people are able to program the sourcecode but they need the money. The group working is organized in a way, that the people are working together because this allows them to get a higher bounty from the customer. A single programmer is not able to program the request of IBM, but if this programmers talks on the mailing list with other user, they can combine their force. The reason why they are talking on the mailing list is not because they are interested in programming, but they need the bounty.

But let us go back to the Stackoverflow website, which has the advantage that everything is transparent. Suppose somebody is able to provide a bounty. What will happen? He can ask anything what he want. For example “Hello world, i need a gui written in Java which is connected to a server database”. Then he puts the 400 point bounty on this question and waits. What will happen? At first, the traffic of this question will raise. That means, many people are clicking on the question because they want to help. After a while, comments and answers are there. If the problem is larger the people will work together in the hope that each of them gets a proportion of the bounty. That means, user1 gets 200 points and user2 gets 200 points because both have contributed a bit.

But what is the inner principle? Why generate bounty questions more traffic and more answers? This has to do with the social situation. The stackoverflow website works like a computer game. The winner is the person, who gets the highest score. And answering bounty questions is one way to maximize the own score.

Reputation

The reason why people are contributing to a team is because the teams gets reputation. Every company works with that principle. The company has customer, they pay for a service, and this payment is transferred to the individuals in the company. That means, McDonalds employees are not interested foremost in meat in general, they are interested in selling the meat to the customer and receiving back a salary. The workflow is usually organized as teamwork. And if the team become more reputation which is equal to higher sales, this team is attractive to more employees because the hope, that they will get a fraction from the reputation for their own.

What is not widely known is, that apart from business, also scientific projects, open source projects and research can be organized with that principle. If a customer interacts with a a group, the group will find a result. Bringing Open Source projects forward is very simple. All what the group needs are better customers. That means, if the Linux kernel finds somebody who needs Linux and is able to pay for it, the traffic on the Linux mailing list will grow and the number of programmers too.

Customer for the Lyx software

The Lyx software is apart from the Linux kernel another famous example for a successful open source project. Lyx is using like all projects a mailing list, a bug tracker and a version control system. But the success come from a different direction, called customers and sponsering. On the URL https://www.lyx.org/Donate some past subprojects in the Lyx ecosystem are visible. A while ago a, Continuous spell checking feature was implemented. In my installation of Lyx, this feature works great. It is equal to what is known from MS-Word. The user enters text and wrong written words gets a red underline. The more interesting point is how this feature was implemented. It was sponsered and the customer paid 1000 EUR for that. That means, an individual or a company asked for that feature, sponsered the bounty, and the lyx team programmed the sourcecode.

Advertisements

Is Java dead?

To answer this question we must go back in the 1990s and investigate what the idea behind Java was. The assumption was, that C++ is hard to learn because it supports pointers and multiple inheritance and the assumption was, that C++ is not a cross plattform language. Java was introduced as an answer to both problems. The main feature of Java is, that it doesn’t supports pointers, have no multiple inheritance and that the java compiler is controlled by a single company which was SUN in the 1990s and Oracle in the now.

The question can be reformulated into “Do we need an alternative to C++?”. According to Microsoft the alternative is called C#. According to the Linux community the alternative to C++ is plain C. And here we get the reason why Java is maybe dead. If the millions of Microsoft programmers are preferring C# over Java, if the beginners in computers courses are learning Python and if the Linux users have a need for C/C++ but not for Java, where is the target group of people who should learn Java?

I’m very pessimistic what the future of Java will be. Right now, Java can’t be called a dead language. Because the number of published books about the subject is high, and the Java tag at Stackoverflow has around 1.5 million questions. But from a technical perspective the problem is, that Java not really solve a major problem. The idea that software should run on multiple operating systems was common in the 1990s before the upraising of the Internet. Today, most software is written for the internet as a server application. This is done with PHP, ASP.NET or C++. Sure we can use Java for backend server programming too, and many business applications are doing so. That means, Java is used as a better form of PHP. But the more elegant way is to use either C# or C++.

The main reason why I’m skeptical about Java is, that the Microsoft as the largest software company doesn’t support the language. Instead they have their own virtual machine language which is promoted as the successor to C++. That means, Java is in a weaker position on Microsoft Server systems.

On the other hand, Red Hat is known for a strong Java support. Java is sometimes called an open source version of C#. Because the sources are available as open source. So in theory, it makes sense to use for server programming on RHEL clusters. But there are many reasons who are speaking against Java, especially in the Linux world. At first, Java is not C. Writing a library in Java but not in C doesn’t make much sense. Secondly, Java is slower than C. And compared with C++, Java is weaker, because it is a managed language which is hiding most advanced features from the programmer.

I’m not sure, if Java is dying right now. Many people are supporting the language, and according to the TIOBE index, Java is more often used than C++. So it is hard to guess what the future will bring.

From Polymath to group working – a short lesson in modern science

A Polymath is a single researcher from the renaissance period. He worked outside of groups. Since 200 years, Polymath education is obsolete. That means science is equal with group working. That might be a surprising, because the assumption is, that researchers are not in team play, but the opposite is the case. 100% of all papers at Google scholar are the result of team. In the author field at least 3 names are given, sometimes more. Even phd thesis are the result of group working. That was not always the case. If we are going back to Renaissance, group working was the exception.

But why has the situation changed? A common myth is, that science become so complex that a single person is no longer able to handle all that knowledge. But if a single person is not familiar with Artificial Intelligence, how should become the group invent something important? Right, that is not the real explanation. The correct answer is that group working increases the public attraction. That means, if a group wants to publish a paper and request for money they will be successful. If a single person is trying to publish something he fails. Another advantage of team work is, that the group can make an arrangement before they are going public. That means, they are discussing the theory first internally and only if the group is sure, they are going life and inform the public. This second layer increases the quality of academic work, because the public can trust the scientific community. It is a sign of professionalism if a dedicated peer-review takes place. And only if the peer-review was successful, the public gets informed.

Peer-review makes no sense for a single researcher. A single researcher is according to the definition not in a position to peer-review his own work. Without a group, the quality is lower and this is equal to unscientific work. Team work is mandatory for modern researchers. They doesn’t have to decide, they are forced to work as team players. If somebody leaves a group and doesn’t find a new group, he is no longer a researcher. He can’t publish no more his ideas, he is no longer allowed to speak for the community. Instead he can call himself a retired researcher.

How far Nanotechnology is developed?

The best way to measure the scientific progress is to ask for commercial application. Everything what can be bought in a store is COTS hardware which is equal the cheap price. In case of Nanotechnology a typical COTS hardware is a DLP-Beamer. Or to be more specific, the Digital micromirror device which is an array of microscopic mirrors which are generated by MEMS assembly. The most amazing feature of this array is, that the number of servo motors is hundred of thousands, that the size is very tiny and that it is consumer product which is sold for under 1000 US$.

What can we do with such a mirror array? In a DLP beamer the mirror is used to focus the light for the display. But apart from this usecase, it is some kind of mechanical robotics system. That means, there are tiny servo motors which can be controlled by software. And the price for each motor is very low. In theory, it is possible to arrange the servo motors to any structure. From a fabrication point of view, the technology is available, that means it is possible to build such machines as a consumer product. The open question is only what the purpose is. DLP beamers are an interesting product, but they can’t be called high-tech. But what is, if we use the same MEMS technology to build robots which can walk?

Magnetic controlled robot

The fascinating feature of MEMS devince in a beamer is, that the combination of many thousands of movable mirrors together with a cheap price for the system. In comparison to a toy robot, for example the Lego Mindstorms EV3 kit, the customer gets more actuators for a lower price. In the EV3 kit, only a few servos are included, and it is possible to build exactly one robot. If the aim is to build 12 robots who can play soccer, the customer has to spend 12×300 US$ which is a lot of money for beginners.

How can we reduce the costs? One possibility is to use microrobots which are actuated by magnets. This technique is some kind of simplified nanorobotics, in which the robot is only a magnet which is driven by the environment. The advantage is, that such a device can be built tiny and it is possible to reduce the costs to nearly zero. That means, it is possible to buy 12 mini-magnetic robots for under 1 US$.

A second advantage over classical toy robots like the Lego EV3 kit is, that magnetic mini-robots are scaling very well to complex tasks. It is possible to build smaller robots which can move and it is possible to increase the number into 1000s of them. The costs remain stable. That means it is possible for the customer to be owner of the own robotic swarm without paying much money.

Help, the ship is sinking …

Under the URL https://ai.stackexchange.com/search?q=answers%3A0 a list of open questions is given at the SE.AI website. The problem is, that the number grows. Last month the record was around 390. Now it is 403. The problem is, if nobody answers the open questions, the website will look bad. Questions which have no answer are too hard for the community, that means it looks like the users are overwhelmed. Are the questions hard to answer? Sometimes. Some of the questions are 1 year and older. They are remaining in the database. It is not the fault of the OP, but of the community in SE.AI. The problem is, that the ressources are limited. And asking a question is easier than posting an answer.

To overcome the problem it would be necessary to increase the effort, which means that more AI-experts are needed which are motivated to increase their reputation by answering questions by beginners. The task itself is easy, all what the user needs to know is a bit English skills, a working browser with Google Scholar and the goal to bring AI forward. But i would guess, that it is hard to find such people, so my prediction is, that the number of unanswered question will raise further.

How to fail a Linux project

On the first look, Linux is a great and cheap operating system. If a company is using Linux but not Microsoft, they can save a lot of money. For example, the company needs an Intranet server and pays 1000 US$ for the hardware plus 2000 US$ for the server operating system which is MS Windows. In the case of the Linux, the company would pay 35 US$ for the Raspberry Pi computer and 0 US$ for the Fedora operating system. It is a great deal, isn’t it?

Not for all companies. Let us take a look what happens with failed software projects. The place in which failed it projects are discussed is the court. That means, the company is no longer arguing with the software supplier, but they are using lawyers to fight against the other side. In a typical law case is about many million US$. But where is the place in which failed Linux projects can be discussed? If Open Source software comes without warranty, and the customer didn’t have paid anything, it is not possible to accuse the opposite side in doing something wrong. If a company is using Linux, they are no longer able to go to a court.

This is not equal to that all Linux projects are successful. Instead the place in which failed projects are discussed is different. Usually, failed Linux projects are not discussed with lawyers, but with programmers. They are not talking about million of US$, but about lines of code. There are many companies out there who have no ressources to handle conflicts in such a way. What most companies have learned is how to handle failed software projects in front of a court. They have experts for such a case and they have won many law-cases in the past. But if a company is using it is hard or even impossible to solve a conflict in such a way. Let us go back to the example from the beginning. Suppose, the company has really bought the Raspberry Pi hardware and is using the Fedora version which is for free to drive their infrastructure. From a technical point of view, it is a great decision. The Raspberry Pi computer is well suited for that task, it has an onboard 1000 Mbit ethernet plugin, and the Fedora software boots the system including a fileserver and a webserver. But what will happen, if the company is not happy with that device? What is, if the employees can’t handle it, if booting problems occur and if these problems cost real money?

It is not possible for the company to make a law case against the Raspberry Pi foundation or against Red Hat. Because if a customer didn’t pay anything, he has no obligation to get a compensation back. As far as i know, the Raspberry Pi hardware comes without any technical support, and the same is true for the Fedora software. So, what should the company do? Right, they don’t know. This is the reason, why in a real life situation they wouldn’t use the Linux operating system but the Microsoft product. Which costs more but is predictable for lawyers.

How can a modern company use Open Source software and is able at the same time to handle failed project? The answer is to switch the place in which problems get discussed. In front of a court is not the right place to discuss why the Linux kernel doesn’t boot or the Gnu compiler doesn’t compile. Both are technical issues which have to be discussed on mailing list, at Stackoverflow or with computer experts in real life. Before somebody can switch to Linux, he has to figure out these alternative ways to solve conflicts. That means, he must plan first the case of a malfunction system and must be able to identify stakeholders who can contribute to the problem. Let us make the situation more realistic.

Before a company can use Fedora on a Raspberry Pi mini-server, the company should be aware what kind of online forum is the right place for asking for help and which person in the company can formulate the question. If the company doesn’t know, they will be not happy with Linux. Because the situation, that the project fails is the normal case. According to the latest statistics, 10 out of 10 newly shipped Raspberry Pi computers are difficult to install because the users never had before such a device in use. Most customers are not familiar with the technology. And it is also correct, that 10 out of 10 newly installed Linux systems will make problems in the first week because they doesn’t boot.

Open source software has a different kind of failure management than closed source software. That means, the language used in talking about a failure is different and the place in which failed problems are negotiated are different. Introducing Linux successfully in a company forces the company to adapt to these different kind of discussions.

Migration to Linux is not the best idea

Suppose a company has a Windows Server. The hardware costs are 1000 US$ and the software costs are 2000 US$. Now the company decides to save money and switch from Microsoft Server to a Raspberry Pi system, with costs of 35 US$ overall. Is this decision right? If the company is already familiar with open source and has programmers who are able to ask questions in online forums and can program bash scripts it is the best decision they can made. The company will save around 3000 US$ and the resulting system is more powerful than any Windows server. It is also a great choice in respect to future technology which will be open source only. But what is, if the company is not familiar with Linux, has no programmers who can write bash scripts and doesn’t plan to become familiar with computer technology? This company will run into big trouble. They have a Linux system but they can’t use it. They have the software it has bugs. And if the server doesn’t work, the company doesn’t know how to fix the problem or which person has to be called to solve the issue. At the end, the small Raspberry Pi system will make more problems then the previous Windows Server.

If a company has right now a Windows Server and the system runs stable it is a good idea to stay at Microsoft. It is not possible to transform a company overnight into a Linux-friendly environment in which the employees and the staff is able config-files in the command line or talk with Linux experts in chat to solve specific security issues. If a company defines themself as a low tech company which doesn’t know what Linux is, the best idea is to stay away from Open Source. It is a bad choice to switch to Linux, if the knowledge about compiling software isn’t available inhouse. Learning on the fly, is nothing what a business oriented company can handle. They will fail the task and at the end, they will not have any kind of computer infrastructure.

Switching to Linux is also a bad idea, if the company is more familiar with computers. If only a few employees in the company have experience with the Visual Studio environment but never have used Virtualbox or similar software, the migration to Linux will also fail. If somebody is not able to run Linux in Virtualbox, he is also not able to run it on real hardware. In case of doubt, the best idea is to stay at Microsoft and ignore Open Source software.

The only reason why a company should migrate from Microsoft Server to a Linux server is, if the company defines themself as a tech-company, if 100% of the employees are calling themself advanced programmers, the company needs access to the sourcecode and if the company is interested in working closely with Red Hat for providing kernel changes back to the open source community.

Why Turbo Pascal has failed

In the early 1990s the Borland company was one of the flagship in software development. They have produced lots of compilers and the Turbo Pascal language was the famous one. Many books were published about the subject and every computer magazine had a dedicated Pascal tutorial available. The language was teached in universities and nobody could imagine to use anything else.

Surprisingly, Borland resigned after a while. Today, the company is no longer active, and Turbo Pascal and everything around it is dead. That means, nobody is using the language and the old books are only available in antiquarian bookshop. What happened? The main reason why Turbo pascal failed was because operating systems programmed in Pascal were not powerful enough. And using the pascal language without an OS on bare-metal hardware doesn’t make much sense. Instead, modern operating systems like Linux and Windows were programmed in C/C++. The reason is, that C/C++ was powered by advanced compilers (Watcom, Microsoft, GNU) and the language was able to handle pointers. This makes it a great choice for writing lowlevel hardware drivers and for programming system libraries for graphics output.

An additional problem of Turbo Pascal was, that it doesn’t supported object oriented programming. All these problems together resulted in a abandoned programming language. In the mid 1980s it was not clear for most people who important dedicated operating systems are. In that area, the programmers were focussed on languages like BASIC and Pascal and there were focussed on hardware like the Atari SI or the Commodore 64. They used programming languages to program bare metal for a hardware and nobody thought about a layer in between them. The importance of a dedicated OS came later with the increasing complexity of software package. Today, it is normal to use existing library. If today a database application should print out the report on the gui and use a TCP/IP connection nobody would program the routines from scratch. Instead all the programmers are interested in saving time and they are using existing libraries which are integrated in operating systems.

The computer language which is best for writing operating systems is the king. And that is C/C++. If Pascal was chosen for writing the Windows kernel or the Linux kernel, then probably Pascal would use also for higher applications.

Pascal vs. C++

Turbo Pascal was the dominant language in the early 1990s. It was recognized as a beginners language and as a professional way to become familiar with computers. In the early 1990s nobody thought about C++. In that period, C++ was recognized as slow, because a C++ compiler needs much more RAM than a Pascal compiler, and C++ was not available on 16bit microcomputers. Even on mid-range computers on that period (Atari ST, Amiga series), the dominant languages were Assembly, C and Pascal. Computers from the early 1990s were not shipped with a full blown operating system which took 100s of Megabyte instead the OS-kernel was a simple 3.5” floppy disk or it was stored on the ROM chip in hardware.

The success of C++ came later in the late 1990s with the upraising of the PC desktop as a standard computer in mainstream and with full blown operating systems like WIndows NT and Linux. A operating system which consumes 800 MB on the harddrive an more need at the first hand better hardware, but it provides more gui-libraries for the programmer. These libraries simplify the programming.

For the non-expert is sounds crazy to call C++ an easy to use programming language. Because the specification needs 1000 pages and it takes month until a program runs smoothly. But compared to the technology before C++, it is easy-going.