Mapfiles and static linking of standard libraries (was: Why do we need both - export maps AND -fvisibility=hidden/__attribute__((visibility("default"))))
Magnus Ihse Bursie
magnus.ihse.bursie at oracle.com
Wed Feb 19 09:22:09 UTC 2014
(Adding in build-dev since we're moving into the jdk build as well now.)
On 2014-02-17 17:39, Volker Simonis wrote:
> On Mon, Feb 17, 2014 at 4:08 PM, Dmitry Samersoff
> <dmitry.samersoff at oracle.com> wrote:
>> It was my bad - missed two related threads.
>> We have two problems related to map files:
>> 1. We have hand-written mapfiles and have to manage it manually.
>> It's better to generate map file automatically or use visibility attributes.
> I would strongly vote against automatically generating the map files
> from sources by parsing for special patterns as proposed by Magnus
> because I think this will just introduce another level of complexity
> and another point of failure.
My priorities is to prefer no map files if we can avoid it, but to
prefer automatically generated over static, checked in, mapfiles if they
cannot be avoided. So I'll try to join you in the fight to get rid of
them altogether, but if that fails, I still want to generate them. :-)
Having static map files are a source of complexity and point of failure
in itself as well.
> >From this discussion so far I learned the following:
> - as long as Oracle insists on static linking libstdc++ and libgcc
> there's no way of getting rid of the map files.
> - using -fvisibility=hidden/__attribute__((visibility("default"))) is
> still desirable because it has positive performance impacts on some
> platforms and because it is the easiest and cleanest solution for the
> future when we can finally get rid of the map files. Moreover it is
> already used anyway.
__attribute__((visibility("default"))) sounds very much like a gcc
extension. Is there a similar construct for solaris studio? Otherwise we
would still need mapfiles on solaris. Also, does
__attribute__((visibility("default"))) work with clang? When I check the
jdk build system, I notice that we do not use mapfiles on macosx. (This
does not stop us from having a "mapfile-macosx" in a library... :-/)
Also, it seems that the same goes for xlc on aix; we will process
mapfiles but in the end ignore them. I have no idea what this means for
the situation on the exported symbols on these platforms.
And then we have the reorder files in jdk, where we extend the mapfiles
with a reorder section which specify a specific order in which to store
functions, based on a (believed, perceived, or once upon a time, real)
Oh my! When I carefully checked the jdk code, it seems that there's a
bug in NativeCompilation, that makes all mapfiles ignored unless there
is also a reorder file. That's only true for 7 libraries, out of the 40
that has mapfiles. The same goes for launchers, we point to an incorrect
directory containing the mapfiles.
So either I'm mis-reading the code, or we have not actually used most of
the mapfiles for possibly a very long time. I'm not sure why this has
not been noticed.
> - This solution would also establish something like a very low level
> contract that the VM will only ever export JVM_*/JNI_* symbols.
> Currently, the following symbols which don't start with JNI_ and JVM_
> and are exported by the
> hotspot/make/linux/makefiles/mapfile-vers-product map file from
> libjvm.so on Linux:
> # miscellaneous functions
> - exported and used in the jdk/ (trough jdk/src/share/javavm/export/jvm.h)
> - could be easily renamed to JVM_*
Unfortunately, it's not that easy. While not nicely designed, those
function that by tradition has been exported by libjvm.so can be
considered part of the already-existing JVM/JDK contract.
> For the shared libraries in the jdk/ repository, the situation is a
> little different.
> Only the following three libraries do statically link libstdc++/libgcc:
Yeah, that's a story in itself, isn't it. I remember being upset about
it when I realized this was the case, but then forgot about it. We
shouldn't mix statically and dynamically linking the standard library,
that's just dumb.
* The benefits of static linking is that we do not depend on a specific
version of the library being available on the customer's computer; we
can run with our own well known, tested and trusted version. But if you
still do dynamic linking, other libraries are susceptible to this problem.
* The benefits of dynamic linking is that we get a smaller footprint,
and do not need to keep track of security patches in the standard
libraries. But if we ship some libraries with static linking, we get a
unneccessary large footprint and we *still* need to keep track of
I can see good points for both approaches (even though I would tend to
prefer dynamic linking), but mixing them just gives us the bad parts
from both worlds.
More information about the build-dev