Of C Shared Libraries and Python
<<< Previous		Next >>>	Blog

4. Creating a Shared Object in C

What is a shared object? Who knows? I don't.

I don't know the 100% pedantically correct definition but basically a shared object is a file that contains code functions, but not a main(). It usually makes most sense to put common functions that several different executables will want to use into one shared object. You then create code which has main()'s from which you can call the functions from the shared object file. The really neat thing is that the executable environment allows several executables to call the shared functions independently without interfering with each other. What's more, the mechanisms responsible for allowing this to occur will load the shared object only once into memory, thus saving space.

So now that you're all excited about shared objects, let's see how we go about making them. Here's the gist of what we're going to do:

Create a shared object that has one function: int MyAdd( int a, int b ). MyAdd() will take two integers, add them, and return the result. Certainly not earth-shattering, but enough for demonstration purposes.
From this source we'll build and install the shared object. It has to be installed in such a way so that that dynamic loader can find it.
We'll then create a C program that will call MyAdd() proving that the mechanism and linkages worked.

So let's get on with the details.

4.1. The Shared Object's Source

/*
 *     AUTHOR: Trevor Woerner
 * START DATE: 25 September 2002 - 02:53:46 PM
 *   MODIFIED: 25 September 2002 - 08:05:28 PM
 *   FILENAME: add.c
 *    PURPOSE: source for shared object
 *
 * $Revision: 1.4 $
 * Copyright (C) 2002  Trevor Woerner
 *
 * This is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public Licence, version 2,
 * as published by the Free Software Foundation.
 *
 * This code is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public Licence for more details.
 *
 * To receive a copy of the GNU General Public Licence write to
 * the Free Software Foundation, 59 Temple Place - Suite 330,
 * Boston, MA 02111-1307, USA.
 */
int MyAdd( int a, int b )
{
    return a+b;
}

4.2. Building and Installing by Hand

I would never think of trying to explain all the steps and provisos required to build shared objects completely from scratch -- there are simply too many platforms and too many variations. To help out, we call on libtool to do the heavy lifting for us. Here are the steps required to build a shared object from our C source:

[trevor]$ libtool gcc -o libadd.lo -c add.c
rm -f .libs/libadd.lo
gcc -c add.c  -fPIC -DPIC -o .libs/libadd.lo
gcc -c add.c -o libadd.o >/dev/null 2>&1
mv -f .libs/libadd.lo libadd.lo
[trevor]$ libtool gcc -o libadd.la libadd.lo -rpath $HOME/lib
rm -fr .libs/libadd.la .libs/libadd.* .libs/libadd.*
gcc -shared  libadd.lo   -Wl,-soname -Wl,libadd.so.0 -o .libs/libadd.so.0.0.0
(cd .libs && rm -f libadd.so.0 && ln -s libadd.so.0.0.0 libadd.so.0)
(cd .libs && rm -f libadd.so && ln -s libadd.so.0.0.0 libadd.so)
ar cru .libs/libadd.a  libadd.o 
ranlib .libs/libadd.a
creating libadd.la
(cd .libs && rm -f libadd.la && ln -s ../libadd.la libadd.la)
[trevor]$ libtool install -c libadd.la $HOME/lib
install -c .libs/libadd.so.0.0.0 /home/trevor/lib/libadd.so.0.0.0
(cd /home/trevor/lib && rm -f libadd.so.0 && ln -s libadd.so.0.0.0 libadd.so.0)
(cd /home/trevor/lib && rm -f libadd.so && ln -s libadd.so.0.0.0 libadd.so)
install -c .libs/libadd.lai /home/trevor/lib/libadd.la
install -c .libs/libadd.a /home/trevor/lib/libadd.a
ranlib /home/trevor/lib/libadd.a
chmod 644 /home/trevor/lib/libadd.a
PATH="$PATH:/sbin" ldconfig -n /home/trevor/lib
----------------------------------------------------------------------
Libraries have been installed in:
   /home/trevor/lib
If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the `LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the `LD_RUN_PATH' environment variable
     during linking
   - use the `-Wl,--rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to `/etc/ld.so.conf'
See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------

Your shared library is now waiting for you in $HOME/lib:

[trevor]$ ls -l $HOME/lib
total 16
-rw-r--r--    1 trevor   trevor        724 Sep 25 20:40 libadd.a
-rwxr-xr-x    1 trevor   trevor        700 Sep 25 20:40 libadd.la
lrwxrwxrwx    1 trevor   trevor         15 Sep 25 20:40 libadd.so -> libadd.so.0.0.0
lrwxrwxrwx    1 trevor   trevor         15 Sep 25 20:40 libadd.so.0 -> libadd.so.0.0.0
-rwxr-xr-x    1 trevor   trevor       4605 Sep 25 20:40 libadd.so.0.0.0

Like the instructions from libtool say, you'll have to take some steps in order for code to be able to see your shared library. If you have superuser priviledges on your system, and you want this library to be available system-wide you could go and install it in any of the directories mentioned in /etc/ld.so.conf. Alternatively you could put it anywhere on the system, but then add the directory where it's found as an entry in /etc/ld.so.conf. Once you've done that you then need to invoke /sbin/ldconfig and everything should be set.

If you don't have superuser priviledges you can still use shared libraries. Simply install it someplace logical that you have access to (like the above-suggested $HOME/lib location). Then you need to do an export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/lib and you're all set. Just make sure that any shell you want to run code linked to your shared library has this environment variable set in this way and you should be fine.

4.3. Creating a Main

Now we move onto the exciting part of creating a main() executable from which we're going to call the MyAdd() function in our shared library.

Here's what some example code might look like:

/*
 *     AUTHOR: Trevor Woerner
 * START DATE: 25 September 2002 - 08:54:44 PM
 *   MODIFIED: 25 September 2002 - 08:55:43 PM
 *   FILENAME: main.c
 *    PURPOSE: main used to call shared library
 *
 * $Revision: 1.4 $
 * Copyright (C) 2002  Trevor Woerner
 *
 * This is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public Licence, version 2,
 * as published by the Free Software Foundation.
 *
 * This code is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public Licence for more details.
 *
 * To receive a copy of the GNU General Public Licence write to
 * the Free Software Foundation, 59 Temple Place - Suite 330,
 * Boston, MA 02111-1307, USA.
 */
#include <stdio.h>
int MyAdd( int, int );
int main( void )
{
    printf( "result: %d\n", MyAdd(70,25) );
    return 0;
}

4.4. Building the Main By Hand

Here are the steps necessary to build an executable:

[trevor]$ gcc -c main.c
[trevor]$ gcc -o main main.o -L$HOME/lib -ladd

Here's what a test run would look like:

[trevor]$ ./main
result: 95

4.5. Building with the help of the Autotools

In case you happen to be interested, there are some tools which can make building the above a little easier. Also, it is better at building on many different hosts and targets, in case things need to be done differently on other machines. These are the autotools: autoconf, automake, and company. libtool is also part of this ensemble.

We start with the following files:

[trevor]$ ls -l
total 16
-rw-rw-r--    1 trevor   trevor       1048 Sep 25 21:14 Makefile.am
-rw-rw-r--    1 trevor   trevor        856 Sep 25 21:13 add.c
-rw-rw-r--    1 trevor   trevor       1145 Sep 25 21:15 configure.in
-rw-rw-r--    1 trevor   trevor        970 Sep 25 21:13 main.c

We've already seen the contents of main.c and add.c. We simply need to examine the contents of the configuration files required to get the autotools to help us.

configure.in:

dnl     AUTHOR: Trevor Woerner
dnl START DATE: 25 September 2002 - 03:37:31 PM
dnl   MODIFIED: 25 September 2002 - 09:15:38 PM
dnl   FILENAME: configure.in
dnl    PURPOSE: 
dnl 
dnl $Revision: 1.4 $
dnl 
dnl Copyright (C) 2002  Trevor Woerner
dnl
dnl This is free software; you can redistribute it and/or modify
dnl it under the terms of the GNU General Public Licence, version 2,
dnl as published by the Free Software Foundation.
dnl
dnl This code is distributed in the hope that it will be useful,
dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
dnl GNU General Public Licence for more details.
dnl
dnl To receive a copy of the GNU General Public Licence write to
dnl the Free Software Foundation, 59 Temple Place - Suite 330,
dnl Boston, MA 02111-1307, USA; or visit
dnl http://www.gnu.org/licenses/licenses.html
AC_INIT(add.c)
AM_INIT_AUTOMAKE(shared-add-lib, 0.0.1)
AC_PROG_CC
AC_PROG_CPP
AC_PROG_MAKE_SET
AC_PROG_INSTALL
AC_PROG_LN_S
AC_PROG_RANLIB
AC_PROG_LIBTOOL
AC_CHECK_PROG(strip, strip, strip, :)
AC_OUTPUT(Makefile)

Makefile.am:

##     AUTHOR: Trevor Woerner
## START DATE: 25 September 2002 - 03:38:47 PM
##   MODIFIED: 25 September 2002 - 09:14:39 PM
##   FILENAME: Makefile.am
##    PURPOSE: 
##
## $Revision: 1.4 $
##
## Copyright (C) 2002  Trevor Woerner
## This is free software; you can redistribute it and/or modify
## it under the terms of the GNU General Public Licence, version 2,
## as published by the Free Software Foundation.
##
## This code is distributed in the hope that it will be useful,
## but WITHOUT ANY WARRANTY; without even the implied warranty of
## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
## GNU General Public Licence for more details.
##
## To receive a copy of the GNU General Public Licence write to
## the Free Software Foundation, 59 Temple Place - Suite 330,
## Boston, MA 02111-1307, USA; or visit
## http://www.gnu.org/licenses/licenses.html
SUBDIRS = .
## src/Makefile.am
bin_PROGRAMS = main
main_SOURCES = main.c
main_LDADD = -L${HOME}/lib -ladd
# shared lib
lib_LTLIBRARIES = libadd.la
libadd_la_SOURCES = add.c

We now perform the following steps:

[trevor]$ aclocal
[trevor]$ autoconf
[trevor]$ automake --add-missing --foreign
[trevor]$ ./configure --prefix=$HOME
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for gawk... gawk
checking whether make sets ${MAKE}... yes
checking for gcc... gcc
checking for C compiler default output... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables... 
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking how to run the C preprocessor... gcc -E
checking whether make sets ${MAKE}... (cached) yes
checking for a BSD-compatible install... /usr/bin/install -c
checking whether ln -s works... yes
checking for ranlib... ranlib
checking build system type... i686-pc-linux-gnu
checking host system type... i686-pc-linux-gnu
checking for ld used by GCC... /usr/i686-pc-linux-gnu/bin/ld
checking if the linker (/usr/i686-pc-linux-gnu/bin/ld) is GNU ld... yes
checking for /usr/i686-pc-linux-gnu/bin/ld option to reload object files... -r
checking for BSD-compatible nm... /usr/bin/nm -B
checking how to recognise dependant libraries... pass_all
checking command to parse /usr/bin/nm -B output... ok
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking dlfcn.h usability... yes
checking dlfcn.h presence... yes
checking for dlfcn.h... yes
checking for ranlib... (cached) ranlib
checking for strip... strip
checking for objdir... .libs
checking for gcc option to produce PIC... -fPIC
checking if gcc PIC flag -fPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.lo... yes
checking if gcc supports -fno-rtti -fno-exceptions... yes
checking whether the linker (/usr/i686-pc-linux-gnu/bin/ld) supports shared libraries... yes
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking whether -lc should be explicitly linked in... no
creating libtool
checking for strip... strip
configure: creating ./config.status
config.status: creating Makefile
config.status: executing depfiles commands
[trevor]$ make
Making all in .
make[1]: Entering directory `/home/trevor/code/doodles/python/clibs'
source='add.c' object='add.lo' libtool=yes \
depfile='.deps/add.Plo' tmpdepfile='.deps/add.TPlo' \
depmode=gcc3 /bin/sh ./../depcomp \
/bin/sh ./libtool --mode=compile gcc -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE=\"shared-add-lib\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1  -I. -I.     -g -O2 -c -o add.lo `test -f 'add.c' || echo './'`add.c
mkdir .libs
gcc -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE=\"shared-add-lib\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -I. -I. -g -O2 -c add.c -MT add.lo -MD -MP -MF .deps/add.TPlo  -fPIC -DPIC -o .libs/add.lo
gcc -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE=\"shared-add-lib\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -I. -I. -g -O2 -c add.c -MT add.lo -MD -MP -MF .deps/add.TPlo -o add.o >/dev/null 2>&1
mv -f .libs/add.lo add.lo
/bin/sh ./libtool --mode=link gcc  -g -O2   -o libadd.la -rpath /home/trevor/lib  add.lo  
rm -fr .libs/libadd.la .libs/libadd.* .libs/libadd.*
gcc -shared  add.lo   -Wl,-soname -Wl,libadd.so.0 -o .libs/libadd.so.0.0.0
(cd .libs && rm -f libadd.so.0 && ln -s libadd.so.0.0.0 libadd.so.0)
(cd .libs && rm -f libadd.so && ln -s libadd.so.0.0.0 libadd.so)
ar cru .libs/libadd.a  add.o 
ranlib .libs/libadd.a
creating libadd.la
(cd .libs && rm -f libadd.la && ln -s ../libadd.la libadd.la)
source='main.c' object='main.o' libtool=no \
depfile='.deps/main.Po' tmpdepfile='.deps/main.TPo' \
depmode=gcc3 /bin/sh ./../depcomp \
gcc -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE=\"shared-add-lib\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1  -I. -I.     -g -O2 -c `test -f 'main.c' || echo './'`main.c
/bin/sh ./libtool --mode=link gcc  -g -O2   -o main  main.o -L/home/trevor/lib -ladd 
gcc -g -O2 -o main main.o  -L/home/trevor/lib /home/trevor/lib/libadd.so -Wl,--rpath -Wl,/home/trevor/lib -Wl,--rpath -Wl,/home/trevor/lib
make[1]: Leaving directory `/home/trevor/code/doodles/python/clibs'
[trevor]$ make install
Making install in .
make[1]: Entering directory `/home/trevor/code/doodles/python/clibs'
make[2]: Entering directory `/home/trevor/code/doodles/python/clibs'
/bin/sh ./../mkinstalldirs /home/trevor/lib
 /bin/sh ./libtool --mode=install /usr/bin/install -c  libadd.la /home/trevor/lib/libadd.la
/usr/bin/install -c .libs/libadd.so.0.0.0 /home/trevor/lib/libadd.so.0.0.0
(cd /home/trevor/lib && rm -f libadd.so.0 && ln -s libadd.so.0.0.0 libadd.so.0)
(cd /home/trevor/lib && rm -f libadd.so && ln -s libadd.so.0.0.0 libadd.so)
/usr/bin/install -c .libs/libadd.lai /home/trevor/lib/libadd.la
/usr/bin/install -c .libs/libadd.a /home/trevor/lib/libadd.a
ranlib /home/trevor/lib/libadd.a
chmod 644 /home/trevor/lib/libadd.a
PATH="$PATH:/sbin" ldconfig -n /home/trevor/lib
----------------------------------------------------------------------
Libraries have been installed in:
   /home/trevor/lib
If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the `LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the `LD_RUN_PATH' environment variable
     during linking
   - use the `-Wl,--rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to `/etc/ld.so.conf'
See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
/bin/sh ./../mkinstalldirs /home/trevor/bin
  /bin/sh ./libtool --mode=install /usr/bin/install -c main /home/trevor/bin/main
/usr/bin/install -c main /home/trevor/bin/main
make[2]: Nothing to be done for `install-data-am'.
make[2]: Leaving directory `/home/trevor/code/doodles/python/clibs'
make[1]: Leaving directory `/home/trevor/code/doodles/python/clibs'

And now to run the executable:

[trevor]$ $HOME/bin/main
result: 95

Furthur checking reveals:

[trevor]$ ldd $HOME/bin/main
        libadd.so.0 => /home/trevor/lib/libadd.so.0 (0x40014000)
        libc.so.6 => /lib/i686/libc.so.6 (0x42000000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)

which indicates that all is well, just as we expected.

<<< Previous	Home	Next >>>
Tools		Calling Shared Objects from Python