Problem running game on older systems: `GLIBC_2.4' not found

[GBGames]

I had a beta tester inform me that my game blew up on his Debian Stable machine: ./killerkittens.bin: /lib/tls/i686/cmov/libc.so.6: version `GLIBC_2.4' not found (required by ./killerkittens.bin)

I found http://www.trilithium.com/johan/2005/06/static-libstdc/ about statically linking to libstdc++. Apparently it silently fails to do so, and if it doesn't statically link to stdc++, it also doesn't statically link to libgcc.

I followed the article, but for some reason my game still dynamically links to libstdc++.

$ ldd killerkittens.bin
linux-gate.so.1 => (0xffffe000)
libkyra.so.0 => not found
libSDL-1.2.so.0 => /usr/lib/libSDL-1.2.so.0 (0xb7e84000)
libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb7e6c000)
libSDL_mixer-1.2.so.0 => /usr/lib/libSDL_mixer-1.2.so.0 (0xb7dfe000)
libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7dd7000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7c96000)
/lib/ld-linux.so.2 (0xb7f22000)
libengine.so.0 => not found
libasound.so.2 => /usr/lib/libasound.so.2 (0xb7bd0000)
libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7bcc000)
libdirectfb-0.9.so.25 => /usr/lib/libdirectfb-0.9.so.25 (0xb7b75000)
libfusion-0.9.so.25 => /usr/lib/libfusion-0.9.so.25 (0xb7b6f000)
libdirect-0.9.so.25 => /usr/lib/libdirect-0.9.so.25 (0xb7b60000)
libvorbisfile.so.3 => /usr/lib/libvorbisfile.so.3 (0xb7b58000)
libvorbis.so.0 => /usr/lib/libvorbis.so.0 (0xb7b30000)
libogg.so.0 => /usr/lib/libogg.so.0 (0xb7b2b000)
libsmpeg-0.4.so.0 => /usr/lib/libsmpeg-0.4.so.0 (0xb7ad3000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb79e9000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb79dc000)


Now if I use LD_LIBRARY_PATH to point it to my custom-built libraries (SDL, SDL_mixer, and others that I built to reduce dependencies), then ldd results in:

linux-gate.so.1 => (0xffffe000)
libkyra.so.0 => ../lib/lib/libkyra.so.0 (0xb7fa6000)
libSDL-1.2.so.0 => ../lib/lib/libSDL-1.2.so.0 (0xb7f15000)
libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb7ef0000)
libSDL_mixer-1.2.so.0 => ../lib/lib/libSDL_mixer-1.2.so.0 (0xb7ee0000)
libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7eb9000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7d78000)
/lib/ld-linux.so.2 (0xb7fc5000)
libengine.so.0 => ../lib/lib/libengine.so.0 (0xb7d1a000)
libpng12.so.0 => /home/gberardi/Projects/wc_KillerKittens/KillerKittens/lib/lib/libpng12.so.0 (0xb7cf6000)
libz.so.1 => /home/gberardi/Projects/wc_KillerKittens/KillerKittens/lib/lib/libz.so.1 (0xb7ce2000)
libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7cde000)
libSDL_image-1.2.so.0 => /home/gberardi/Projects/wc_KillerKittens/KillerKittens/lib/lib/libSDL_image-1.2.so.0 (0xb7cda000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7cce000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7be3000)

My makefile includes:

LIBS := -static-libgcc -L. -lstdc++ -L${LIB_INSTALL_DIR}/lib -lkyra $(shell ${LIB_INSTALL_DIR}/bin/sdl-config --libs) -lSDL_mixer

LIB_INSTALL_DIR is where I have my custom-built libs. libstdc++.a is symlinked in the current directory, as the article mentions.

The only part I am confused about is: "Once again, for this to work reliably you must not use dynamically loaded C++ code, including code loaded with dlopen." How do I determine if I am using such code?
_________________
--
GBGames' Blog: An Indie Game Developer's Somewhat Interesting Thoughts http://www.gbgames.com/blog

Staff Reviewer for Game Tunnel: The Independent Underground http://www.gametunnel.com

[MatzeBraun]

Your error is probably glibc (and not libstdc++). It's probably your glibc. glibc devs invented this strange thing: When they change a function (because of a trivial bugfix or something bigger) then they keep a copy of the old version and add the new changed version. Now if you build something the linker will automatically choose the latest version of the function (which has a "GLIBC_2.4" symbol appended) while binaries linked against the old version will still use the old version.

Of course this makes creating binaries on linux really hard. Some companies have really old linux boxes around so the binaries will be linked against very old glibc versions. Other like the autopackage guy created tools like apgcc which is basically a wrapper around gcc which resets function versions to not use the new ones. The autopackage has lots of interesting information anyway about the (bad) state of binary compatibility on linux:

http://trac.autopackage.org/wiki/LinuxProblems


Another thing: Your libstdc++ dependency might get imported indirectly, ie it's not your application directly that depends on it but 1 of the libraries that you depend on. You can check direct dependencies with "objdump -x your_application | grep NEEDED"

[GBGames]

So if I understand it correctly, older apps will continue to run on future systems because the symbols on the new systems will be backwards compatible, but new apps will not run on older systems because the newly built apps will assume that only newer symbols are available?

As for your last statement about where the libstdc++ dependency gets imported, you were right. It was coming in through a library, specifically Kyra Sprite Engine's libengine.so. It is yet one more reason for me to stop using it, I guess. I wonder if I can hack autotools to give kyra a static compile option for now.
_________________
--
GBGames' Blog: An Indie Game Developer's Somewhat Interesting Thoughts http://www.gbgames.com/blog

Staff Reviewer for Game Tunnel: The Independent Underground http://www.gametunnel.com

[Elanthis]

The guys at the AutoPackage project created a tool called apgcc, which will allow you to compile binaries that use old symbol versions so they will work on older systems. Try using that to create your binaries.

[quote]When they change a function (because of a trivial bugfix or something bigger)[/quote]

They certainly don't do this for trivial bug fixes or other minor function changes. They only do this when the new function is incompatible with the old in same way, such as having different behaviour or a different signature. It would be insane to think that they keep multiple copies of a function for every change. :/
_________________
Sean Middleditch
http://www.awemud.net

[nh]

New versions of gcc have built-in protection against buffer overflows on the stack. However, that support requires GLIBC 2.4 or above.

To get rid of this problem (at the expense of reduced security), add the -fno-stack-protector flag when compiling and linking.

Of course, there may be other reasons for why you have a dependency on GLIBC_2.4 (check the symbols via nm as suggested before), but this is by far the most likely problem, and the one we keep forgetting in Widelands ;)

cu,
Nicolai

[GBGames]

[GBGames]

Strange how I hadn't thought to try this before, but I installed gcc-3.3 and g++-3.3, which pulled down the appropriate stdc++. I specified CC=gcc-3.3 and CXX=g++-3.3, and then I used -fapi-version=1 as the compiler flags. It seems to work just fine. I don't think I need to go back too far in terms of what glibc version is supported.

Before the new compiler/libs were used, objdump -x libengine.so gives:

Version References:
required from libm.so.6:
0x0d696910 0x00 06 GLIBC_2.0
required from libgcc_s.so.1:
0x0b792650 0x00 10 GCC_3.0
0x0d696910 0x00 05 GLIBC_2.0
required from libstdc++.so.6:
0x056bafd3 0x00 07 CXXABI_1.3
0x08922974 0x00 03 GLIBCXX_3.4
required from libc.so.6:
0x0d696914 0x00 09 GLIBC_2.4
0x0d696911 0x00 08 GLIBC_2.1
0x09691f73 0x00 04 GLIBC_2.1.3
0x0d696910 0x00 02 GLIBC_2.0

After:

Version References:
required from libm.so.6:
0x0d696910 0x00 06 GLIBC_2.0
required from libgcc_s.so.1:
0x0b792650 0x00 09 GCC_3.0
0x0d696910 0x00 05 GLIBC_2.0
required from libstdc++.so.5:
0x081a2972 0x00 07 GLIBCPP_3.2
0x056bafd2 0x00 04 CXXABI_1.2
required from libc.so.6:
0x0d696911 0x00 08 GLIBC_2.1
0x09691f73 0x00 03 GLIBC_2.1.3
0x0d696910 0x00 02 GLIBC_2.0

If I can do this with older versions of gcc/g++ installed on my system, why is there a need for apgcc/apg++?
_________________
--
GBGames' Blog: An Indie Game Developer's Somewhat Interesting Thoughts http://www.gbgames.com/blog

Staff Reviewer for Game Tunnel: The Independent Underground http://www.gametunnel.com

[simcop2387]

my understanding is that it allows you to do it without installing another gcc version, which allows you to avoid some bugs that exist in certain gcc versions, and allows you to use newer/better/worse/etc. optimizations existing in newer gcc's

[GBGames]

One side-effect of using the older version of the compiler tools was that my game now had dependencies on libX11 and libXext. Yes, my tar.gz was now 50% smaller, but if it was going to swap one dependency (GLIBC_2.4) for another (MY system's version of the X11 libraries), it's not a real solution.

I still can't figure out why apgcc/apg++ can't build Kyra properly. It seems I have some work cut out for me no matter which way I go.
_________________
--
GBGames' Blog: An Indie Game Developer's Somewhat Interesting Thoughts http://www.gbgames.com/blog

Staff Reviewer for Game Tunnel: The Independent Underground http://www.gametunnel.com

[GBGames]

Also, I noticed that -fabi-version didn't actually do anything. Just using gcc-3.3/g++-3.3 was what produced the non-GLIBC_2.4-depending binaries.
_________________
--
GBGames' Blog: An Indie Game Developer's Somewhat Interesting Thoughts http://www.gbgames.com/blog

Staff Reviewer for Game Tunnel: The Independent Underground http://www.gametunnel.com

[GBGames]

I found out that the reason why X11 wasn't dynamic linking properly was because SDL 1.2.11 introduced a requirement for it.

[GBGames]

I have solved it! Well, mostly.

I have written about my solution here:
http://gbgames.com/blog/2008/04/linux-game-development-glibc_24-errors-solved/

Short version: I compile everything with -fno-stack-protector.

Why only mostly?
http://gbgames.com/blog/2008/04/linux-game-development-frustrations-with-glibc_24-and-ubuntu/

Short version: Ubuntu's libs are using stack protector, so even if my libs and binary are using the flag mentioned above, when I link with Ubuntu's libs, such as libgcc and libstdc++, my game then depends on GLIBC_2.4.

So I am using my Debian system to create release builds for now.
_________________
--
GBGames' Blog: An Indie Game Developer's Somewhat Interesting Thoughts http://www.gbgames.com/blog

Staff Reviewer for Game Tunnel: The Independent Underground http://www.gametunnel.com