Shared libraries in Linux: growing pains or fundamental problem?

Introduction: is system DLL problem Windows-specific?

The speed of development of Linux is usually considered a very good thing. Now I shall talk about one aspect of it, namely quick change of software components commonly known as shared libraries, or DLLs (dynamically linked libraries). This is often thought to be serious problem in the Windows world, when some new update which comes with new version of some product will change system-wide libraries, and as a result some other software packages would not run properly (DirectX is particularly notorious for that effect; it is not that surprising given that some parts of it are hardware-specific). However, this problem is also present in Linux world, and in fact it might be even worse there. The idea to write this article has emerged after rather long talk with my friend who was telling me about his RedHat 6.0 experience. You will find some extracts from our talk in the text below.

Upgrading system: from libc 5 to glibc 2.0 to glibc 2.1

The first and rather obvious problem about shared libraries and their versions is that all software you had linked against old library have to be recompiled. This is not always the case; after all, the versions of glibc are not designed to be incompatible with each other; however, little things have crept here and there, and not all programs built with glibc 2.0.x will run with glibc 2.1. But when upgrading from libc5 to glibc, this is a necessity -- programs linked with libc5 will not run with glibc2 and either have to be recompiled or you'll have to keep libc5 libraries around just to be able to run binaries linked with them.

With very complex software packages like GNOME or KDE this becomes real problem: it is often easier just to download recompiled binaries than to rebuild them from the sources or make older binaries work on a new system -- of course, if new binaries are available! The prerequisite list of required libs for monsters like GNOME is quite long (13 pieces), and it is very easy to break the entire package just by installing newer/older version of libpng.

Linux distributions

This binary compatibility effect also has great impact on Linux distributions. Namely, if you don't want to spend the rest of your life recompiling things, you often have to stick to very limited set of distributions. Not all distros include everything, and sticking RPMs from RedHat into Slackware usually only works for more or less simple software. With complex things, lots of library dependencies make using foreign binaries unrealistic. My friend explained to me how he managed to install Russian fonts into Ghostscript on his RedHat. He simply took them from KSI Linux (this is a Russian Linux distribution). I have asked him why did he not try the KSI Linux itself. He told me that he wanted to stay mainstream, just to make sure that everything made for Linux will run on his system without recompile. Essentially, this makes smaller distributions (like KSI) to be pushed off scene: users want to be compatible! My friend was especially concerned about StarOffice and IBM DB2. Since both software packages were not tested on KSI by their manufacturers, will they run on KSI or not is not something he was willing to investigate. This leads us to next major obstacle: running closed-source software.

Proprietary software

Whatever some people might say, proprietary (i.e. closed-source) software will continue to exist in foreseeable future. Recently we had a whole lot of news on this arena: from big database vendors (Oracle, IBM DB2, Informix) to games (Quake, Civilization). Apparently it is quite unlikely that companies which consider their technologies and art top secret are going to release the source any time soon. Therefore we have to put up with binary only releases. With time, the situation might get better, but it is as likely will get worse as new software players will enter growing Linux market. The aforementioned problem is very serious with this kind of software. When you cannot recompile the thing, you'll have to stick to what it will run with -- in other word, you are some sort of locked into having predefined set of shared libraries in your system! Here comes another little story: my friend who has just installed shiny new RedHat 6.0 was very enthusiastic about it, except one thing: Star Office would not run! Software wanted glibc 2.0.7 while RedHat has switched to glibc 2.1. I heard (please don't blame me if it is wrong) that initial version of IBM ViaVoice for Linux was linked against glibc 2.1 (on RedHat 6.0). This essentially means that StarOffice and ViaVoice cannot run at the same time.

The rest of the world

If you will read FreeBSD mailing lists, you'll discover a rather abundant category of questions: how to run some Linux binary-only package on FreeBSD. Since FreeBSD has a Linux emulation layer, it is often not a problem to use Linux-specific software on FreeBSD. But sometimes it is -- when the Linux binary depends on several libraries of particular versions. It is interesting to note that on the FreeBSD itself the library compatibility problem is less important: system core just does not change frequently enough. The major FreeBSD releases happen much less often than Linux ones, and there are no concerns about distributions because there's only one.

The situation with X11 libraries is quite similar to FreeBSD's one. Since major revisions of the X11 take years, the entire set of calls and DLLs remains very compatible between versions. Some people would say however that X needs serious overhaul because it is outdated :-) At the time of the writing, next revision of XFree86 (4.0) is not yet released.

Possible solutions

What possible solutions could be offered for this problem? Honestly, I am not an OS developer, and my point of view is not probably well thought out. I can think of several ways of resolving the difficulty:
  1. use statically linked executables. This approach is generally regarded as undesirable because it kills all advantages of dynamic linking. It causes major bloat; programs tend to consume much more disk space and RAM when linked statically. If the library does contain a bug, it will be linked hard into the executable and cannot be fixed by replacing shared library. Sometimes, however, this is the best and simplest way to achieve binary compatibility. With closed source software, statically linked binaries should be provided as a fallback, together with dynamically linked ones. In most other cases, their use should be discouraged.

  2. switch to finer version numbering. In other words, let to link to or even instead of Apparently pursuing this approach too far is undesirable as libc-2.0.7 is supposed to hold latest bugfixed in libc-2.0.x line; the limit has to be put somewhere. But in ultimate cases, it could make sense to link even with specific versions of a library because really complex software could be quite touchy about library version. Correct me if I am wrong, but in current incarnation everything is supposed to be linked against (with glibc 2.x).

  3. expand the definition of Linux kernel to include at least libc and several other basic libraries (termcap, ncurses, pthreads?) and make those libraries stable. This has many rather obvious drawbacks: licensing issues, does not solve all of the problem as kernel releases are also happen quite often, nobody can hope to include everything (GTK?). Some things just can't be stabilized because they are under active development. Besides, the current trend has exactly opposite direction; in most recent Linux distribution libc5 (which was being viewed as Linux-only) is replaced with glibc2 (which is supposed to be the ultimate libc on Earth).

  4. create some sort of a committee which will make prepackaged version of standard libraries (both in source and binary form), and use of library versions which are not in the packages will be highly discouraged. The version numbers will be assigned to the packages as a whole, and the rate of package major releases (which are not 100% compatible with each other) should be much slower (say, once a year). This is the technique used in FreeBSD distributions, X11, Qt and many others. The drawbacks: lots of political issues, slow rate of new technology adoption. Some people with too much love for creeping featurism might seem this as a loss of their freedom because committee will have to make hard decisions from time to time.
In some cases, the solution will involve a lot of political struggle and debates. But whether the threat of binary incompatibility is real or it is just a growing pains of a young operating system remains to be seen...


I am not Linux core developer. I do write software on Linux and other OSes, but my understanding of some basic issues might be wrong in this writing. I will readily accept any corrections for mistakes. Links to related materials are also welcome.

Further reading: GNU libc-6 announcement.

Copyright (C) 1999 Sergey Ayukov. Parts of this text can be reproduced without permission, with reference to this page or the author.
16 May 1999

return to essays | return to homepage | send comment