Table of Contents

Dynamic Loading Basics

Since the original XLISP-STAT release most major operating systems have introduced shared libraries along with some form of interface to allow user code to dynamically load shared libraries. While there are differences, the mechanism are fairly similar and can for the most part be made to emulate each other. In reviewing XLISP-STAT's dynamic loading, I propose to take a subset of the UNIX standard SystemVR4 ELF approach as the basis and emulate it on Macintosh, Windows, and other UNIX platforms.

One item I will not rely on is library startup and termination code. All systems claim to allow this, and most probably do (early UNIX versions didn't seem to) but the problem is that there is no control (as far as I know) over when these routines are called. This could create some headaches, especially if termination code is called after resources it depends on might have been released. For now, I prefer to use explicit calls along with my finalization mechanism.

This note outlines a basic implementation of the dlfcn interface on a range of platforms and describes some issues in using the interface I have identified. I'm sure there are more little traps buried here; I'll update this document as i learn more. If you read this and find errors or things that should be added, please let me know.

The code described here is available. It works in my setup; your mileage may vary.

Interface

The interface consists of four functions and two constants,

<generic version of dlfcn.h>=
#define RTLD_LAZY 1
#define RTLD_NOW  2

void *dlopen(const char *, int);
void *dlsym(void *, const char *);
int dlclose(void *);
char *dlerror(void);
Defines dlclose, dlerror, dlopen, dlsym, RTLD_LAZY, RTLD_NOW (links are to index).

The constants may have other values on some systems.

dlopen opens a shared library specified by the file name path in its first argument. How relative path names are resolved is system-dependent. The second argument specifies whether undefined references in the library must be resolved at load time (RTLD_NOW) or can be resolved later (RTLD_LAZY). Deferred resolution sounds more attractive but it is risky---failure usually results in an unrecoverable error. In contrast, failure to resolve at load time produces a NULL return value and (hopefully) a meaningful message from dlerror. I will only use RTLD_NOW and assume all references in the loaded library are either into the executable or resolved when the shared library was linked. dlopen returns a library handle, or NULL on failure.

dlsym returns the address of a symbol in the library handle, or NULL on failure. Depending on the system, this symbol may or may not be guaranteed to be a function pointer. ``Real'' dlsym and the Mac version will return addresses of globals in the shared library; the windows variant probably will not (**** check this out).

dlclose takes a library handle and closes the library. A reference count should be maintained to make sure the library is only really closed after the last handle for it has been closed. (**** need to check that this is really true).

dlerror returns a string with an error message for the most recent error. I'm not sure if some versions don't return NULL if there was no error.

Implementations

AIX

The XLISP-STAT distribution includes an emulation library written by Jens-Uwe Mager. A newer version may be available.

AIX is one of the few UNIX systems that seems to require explicit exporting of symbols via an exports file or perhaps command line options. When I looked into this some time ago I did not find a way to make all external symbols as exported. I don't currently have access to an AIX machine, so I can't do any further testing.

HPUX

HPUX has its own shared library system. To map the dlfcn interface to the HPUX base, use the header file

<HPUX dlfcn.h>=
#include <dl.h>

#define RTLD_LAZY (BIND_DEFERRED | BIND_NONFATAL)
#define RTLD_NOW BIND_IMMEDIATE

void *dlopen(const char *, int);
void *dlsym(void *, const char *);
int dlclose(void *);
char *dlerror(void);
Defines dlclose, dlerror, dlopen, dlsym, RTLD_LAZY, RTLD_NOW (links are to index).

The implementation is

<HPUX dlfcn.c>=
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <dlfcn.h>

/*
 * This is a minimal implementation of the ELF dlopen, dlclose, dlsym
 * and dlerror routines based on HP's shl_load, shl_unload and
 * shl_findsym. */

/*
 * Reference Counting.
 *
 * Empirically it looks like the HP routines do not maintain a
 * reference count, so I maintain one here.
 */

typedef struct lib_entry {
  shl_t handle;
  int count;
  struct lib_entry *next;
} *LibEntry;

#define lib_entry_handle(e) ((e)->handle)
#define lib_entry_count(e) ((e)->count)
#define lib_entry_next(e) ((e)->next)
#define set_lib_entry_handle(e,v) ((e)->handle = (v))
#define set_lib_entry_count(e,v) ((e)->count = (v))
#define set_lib_entry_next(e,v) ((e)->next = (v))
#define increment_lib_entry_count(e) ((e)->count++)
#define decrement_lib_entry_count(e) ((e)->count--)

static LibEntry Entries = NULL;

static LibEntry find_lib_entry(shl_t handle)
{
  LibEntry entry;

  for (entry = Entries; entry != NULL; entry = lib_entry_next(entry))
    if (lib_entry_handle(entry) == handle)
      return entry;
  return NULL;
}

static LibEntry new_lib_entry(shl_t handle)
{
  LibEntry entry;

  if ((entry = (LibEntry) malloc(sizeof(struct lib_entry))) != NULL) {
    set_lib_entry_handle(entry, handle);
    set_lib_entry_count(entry, 1);
    set_lib_entry_next(entry, Entries);
    Entries = entry;
  }
  return entry;
}

static void free_lib_entry(LibEntry entry)
{
  if (entry == Entries)
    Entries = lib_entry_next(entry);
  else {
    LibEntry last, next;
    for (last = Entries, next = lib_entry_next(last);
         next != NULL;
         last = next, next = lib_entry_next(last)) {
      if (entry == next) {
        set_lib_entry_next(last, lib_entry_next(entry));
        break;
      }
    }
  }
  free(entry);
}


/*
 * Error Handling.
 */

#define ERRBUFSIZE 1000

static char errbuf[ERRBUFSIZE];
static int dlerrno = 0;

char *dlerror(void)
{
  return dlerrno ? errbuf : NULL;
}


/*
 * Opening and Closing Liraries.
 */

void *dlopen(const char *fname, int mode)
{
  shl_t handle;
  LibEntry entry = NULL;
  
  dlerrno = 0;
  if (fname == NULL)
    handle = PROG_HANDLE;
  else {
    handle = shl_load(fname, mode, 0L);
    if (handle != NULL) {
      if ((entry = find_lib_entry(handle)) == NULL) {
        if ((entry = new_lib_entry(handle)) == NULL) {
          shl_unload(handle);
          handle = NULL;
        }
      }
      else
        increment_lib_entry_count(entry);
    }
    if (handle == NULL) {
      char *errstr;
      dlerrno = errno;
      errstr = strerror(errno);
      if (errno == NULL) errstr = "???";
      sprintf(errbuf, "can't open %s: %s", fname, errstr);
    }
  }
#ifdef DEBUG
  printf("opening library %s, handle = %x, count = %d\n",
         fname, handle, entry ? lib_entry_count(entry) : -1);
  if (dlerrno) printf("%s\n", dlerror());
#endif
  return (void *) handle;
}

int dlclose(void *handle)
{
  LibEntry entry;
#ifdef DEBUG
  entry = find_lib_entry(handle);
  printf("closing library handle = %x, count = %d\n",
         handle, entry ? lib_entry_count(entry) : -1);
#endif

  dlerrno = 0;
  if ((shl_t) handle == PROG_HANDLE)
    return 0; /* ignore attempts to close main program */
  else {

    if ((entry = find_lib_entry((shl_t) handle)) != NULL) {
      decrement_lib_entry_count(entry);
      if (lib_entry_count(entry) > 0)
        return 0;
      else {
        /* unload once reference count reaches zero */
        free_lib_entry(entry);
        if (shl_unload((shl_t) handle) == 0)
          return 0;
      }
    }
    /* if you get to here, an error has occurred */
    dlerrno = 1;
    sprintf(errbuf, "attempt to close library failed");
#ifdef DEBUG
    printf("%s\n", dlerror());
#endif
    return -1;
  }
}


/*
 * Symbol Lookup.
 */

void *dlsym(void *handle, const char *name)
{
  void *f;
  shl_t myhandle;

  dlerrno = 0;
  myhandle = (handle == NULL) ? PROG_HANDLE : (shl_t) handle;

  if (shl_findsym(&myhandle, name, TYPE_PROCEDURE, &f) != 0) {
    dlerrno = 1;
    sprintf(errbuf, "symbol %s not found", name);
    f = NULL;
  }

  return(f);
}
Defines dlclose, dlerror, dlopen, dlsym (links are to index).

The makefile for the HPUX dlfcn emulation library is

<HPUX Makefile for dlfcn library>=
CFLAGS = -Aa +z -I.

libdl.sl: dlfcn.o
        ld -b -o libdl.sl dlfcn.o -ldld

Macintosh

The Macintosh implementation uses the generic header file. It is based on Bob Stine's code with Modifications by Steve Majewski and perhaps a few modifications derived from the Tcl 8.0 sources. Tcl 8.0 does a more elaborate search for the code fragment by looking for a cfrg resource---maybe this is worth adding****.

<Macintosh dlfcn.c>=
#include <stddef.h>
#include <string.h>
#include <dlfcn.h>
#include <stdio.h>

#include <CodeFragments.h>
#include "macutils.h"

static char errbuf[512];

/* Minimal emulation of SysVR4-ELF dynamic loading routines for the Macintosh.
 * Based on code by Bob Stine as Modified by Steve Majewski. */
void *dlopen(const char *name, int mode)
{
  FSSpec fileSpec;
  Str255 errName, libName;
  OSErr err;
  Ptr mainAddr;
  CFragConnectionID connID;

  /* Build a file spec record for GetDiskFragment */
  if (strlen(name) < 254)
    strcpy((char *) libName, name);
  else {
    sprintf(errbuf, "library name too long");
    return NULL;
  }
  CtoPstr((char *) libName);
  err = FSMakeFSSpecFromPath((ConstStr255Param) libName, &fileSpec);
  if (err != noErr) {
    sprintf(errbuf, "error code %d creating file spec for library %s",
            err, name);
    return NULL;
  }

  /* Open the fragment (will not add another copy if loaded, though gives
     new ID) */
  err = GetDiskFragment(&fileSpec, 0, kCFragGoesToEOF, 0, kLoadCFrag,
                        &connID, &mainAddr, errName);
  if (err == noErr)
    return (void *) connID;
  else {
    PtoCstr(errName);
    sprintf(errbuf, "error code %d getting disk fragment %s for library %s",
            err, errName, name);
    return NULL;
  }
}

/* This version does not handle NULL as the library for looking in the
   executable. It also does not check the symbol class. */
void *dlsym(void *lib, const char *name)
{
  CFragConnectionID connID = (CFragConnectionID) lib;
  OSErr err;
  Ptr symAddr;
  CFragSymbolClass symClass;
  Str255 symName;
  
  if (strlen(name) < 254)
    strcpy((char *) symName, name);
  else {
    sprintf(errbuf, "symbol name too long");
    return NULL;
  }
  CtoPstr((char *) symName);
  err = FindSymbol(connID, symName, &symAddr, &symClass);
  if (err == noErr)
    return (void *) symAddr;
  else {
    sprintf(errbuf, "error code %d looking up symbol %s", err, name);
    return NULL;
  }
}

int dlclose(void *lib)
{
  CFragConnectionID connID = (CFragConnectionID) lib;
  OSErr err;
  err = CloseConnection(&connID);
  if (err == noErr)
    return 0;
  else {
    sprintf(errbuf, "error code %d closing library", err);
    return -1;
  }
}

char *dlerror()
{
  return errbuf;
}
Defines dlclose, dlerror, dlopen, dlsym (links are to index).

This needs a support routine that is based on some routines from the MoreFiles package and is derived from a routine in the Tcl 8.0 distribution. This routine is used to make sure that any aliases along the path for a shared library are handled properly. At some point I may add more of the stuff in MoreFiles, and perhaps just use the whole package. The header file for these utilities is

<macutils.h>=
OSErr GetDirectoryID(short vRefNum, long, StringPtr, long *, Boolean *);
OSErr FSpGetDirectoryID(const FSSpec *, long *, Boolean *);
OSErr FSMakeFSSpecFromPath(ConstStr255Param, FSSpecPtr);
Defines FSMakeFSSpecFromPath, FSpGetDirectoryID, GetDirectoryID (links are to index).

The implementation is

<macutils.c>=
#include "macutils.h"

static void CopyNamePart(StringPtr Name, ConstStr255Param fileName, start)
{
  int end = fileName[0] + 1, nlen, pos;
  for (nlen = 0, pos = start; pos < end && fileName[pos] == ':'; nlen++, pos++)
    Name[nlen + 1] = ':';
  for (; pos < end && fileName[pos] != ':'; nlen++, pos++)
    Name[nlen + 1] = fileName[pos];
  Name[0] = nlen;
}

/* This function is an adaptation of the function FSpLocationFromPath in
   tclMacUtils.c in the Tcl 8.0 distribution */
OSErr FSMakeFSSpecFromPath(ConstStr255Param fileName, FSSpecPtr spec)
{
  Boolean isDir, wasAlias;
  int pos, end;
  OSErr err;
  Str255 Name;
  short vRefNum;
  long dirID;
  
  /* get the initial directory information and set up first path component */
  CopyNamePart(Name, fileName, 1);
  if (Name[0] < fileName[0] && Name[1] != ':') { /* absolute path */
    Name[0]++;
    Name[Name[0]] = ':';
    if ((err = FSMakeFSSpec(0, 0, Name, spec)) != noErr)
      return err;
    if ((err = FSpGetDirectoryID(spec, &dirID, &isDir)) != noErr)
      return err;
    if (! isDir)
      return dirNFErr;
    vRefNum = spec->vRefNum;
    pos = Name[0] + 1;
    CopyNamePart(Name, fileName, pos);
  }
  else {
    dirID = 0;
    vRefNum = 0;
    pos = 1;
    isDir = true;
  }
  
  /* process remaining path parts */
  end = fileName[0] + 1;
  while (true) {
    if ((err = FSMakeFSSpec(vRefNum, dirID, Name[0] == 0 ? NULL : Name,
                            spec)) != noErr ||
        (err = ResolveAliasFile(spec, true, &isDir, &wasAlias)) != noErr)
      return err;
    pos += Name[0];
    if (pos < end) {
      if ((err = FSpGetDirectoryID(spec, &dirID, &isDir)) != noErr)
        return err;
      if (! isDir)
        return dirNFErr;
      vRefNum = spec->vRefNum;
      CopyNamePart(Name, fileName, pos);
    }
    else
      return noErr;
  }
}


/*
 * The following functions are taken from MoreFiles. For now these are all I
 * use; if I end up using lots more I'll include the whole MoreFiles library.
 */

OSErr GetDirectoryID(short vRefNum, long dirID,
                     StringPtr name, long *theDirID, Boolean *isDirectory)
{
  CInfoPBRec pb;
  Str31 tempName;
  OSErr error;

  /* Protection against File Sharing problem */
  if ( (name == NULL) || (name[0] == 0) ) {
    tempName[0] = 0;
    pb.hFileInfo.ioNamePtr = tempName;
    pb.hFileInfo.ioFDirIndex = -1;      /* use ioDirID */
  }
  else {
    pb.hFileInfo.ioNamePtr = name;
    pb.hFileInfo.ioFDirIndex = 0;       /* use ioNamePtr and ioDirID */
  }
  pb.hFileInfo.ioVRefNum = vRefNum;
  pb.hFileInfo.ioDirID = dirID;
  error = PBGetCatInfoSync(&pb);
  *isDirectory = (pb.hFileInfo.ioFlAttrib & ioDirMask) != 0;
  *theDirID = (*isDirectory) ? pb.dirInfo.ioDrDirID : pb.hFileInfo.ioFlParID;
  return error;
}

OSErr FSpGetDirectoryID(const FSSpec *spec, long *theDirID,
                        Boolean *isDirectory)
{
  return GetDirectoryID(spec->vRefNum, spec->parID,
                       (StringPtr)spec->name, theDirID, isDirectory);
}
Defines CopyNamePart, FSMakeFSSpecFromPath, FSpGetDirectoryID, GetDirectoryID (links are to index).

Mac header files are assumed to come in from a standard precompiled header.

MS Windows

Windows uses the generic header file. The implementation is simple. dlsym only returns function pointers not data addresses. For Win32 more detailed error messages could be obtained using FormatMessage. Currently a NULL argument to dlopen does not open a reference to the executable; it should not be too hard to add this feature using GetModuleHandle. I'm not sure if this works only in Win32 or also in Win16.

<Windows dlfcn.c>=
#include <windows.h>
#include <stdio.h>
#include <dlfcn.h>

static char errbuf[512];

void *dlopen(const char *name, int mode)
{
  HINSTANCE hdll;

  hdll = LoadLibrary(name);
#ifdef _WIN32
  if (! hdll) {
    sprintf(errbuf, "error code %d loading library %s", GetLastError(), name);
    return NULL;
  }
#else
  if ((UINT) hdll < 32) {
    sprintf(errbuf, "error code %d loading library %s", (UINT) hdll, name);
    return NULL;
  }
#endif
  return (void *) hdll;
}

void *dlsym(void *lib, const char *name)
{
  HMODULE hdll = (HMODULE) lib;
  void *symAddr;
  symAddr = (void *) GetProcAddress(hdll, name);
  if (symAddr == NULL)
    sprintf(errbuf, "can't find symbol %s", name);
  return symAddr;
}

int dlclose(void *lib)
{
  HMODULE hdll = (HMODULE) lib;

#ifdef _WIN32
  if (FreeLibrary(hdll))
    return 0;
  else {
    sprintf(errbuf, "error code %d closing library", GetLastError());
    return -1;
  }
#else
  FreeLibrary(hdll);
  return 0;
#endif
}

char *dlerror()
{
  return errbuf;
}
Defines dlclose, dlerror, dlopen, dlsym (links are to index).

Some Simple Test Code

This code exercises calling a routine in a shared library, allowing a routine in the executable to be called by a shared library, and allowing a global in the executable to be accessed and modified by the shared library. I use two shared libraries to test whether library closing in the same order as allocation will work on windows---long ago I think that used to blow up. The main program is

<dltest.c>=
#include <stdlib.h>
#include <stdio.h>
#include <dlfcn.h>

#if defined(__WATCOMC__)
#  define foosym "foo_"
#  define barsym "bar_"
#elif defined(__BORLANDC__)
#  define foosym "_foo"
#  define barsym "_bar"
#else
#  define foosym "foo"
#  define barsym "bar"
#endif

#define DLERRCHECK(x) \
  do { \
    if (!(x)) { \
      fprintf(stderr, "Error: %s\n", dlerror()); \
      exit(1); \
    } \
  } while(0)

int zip = 0;

void main(int argc, char *argv[])
{
  void *liba, *libb;
  void (*foo)(void), (*bar)(void);

  DLERRCHECK((liba = dlopen("a.dll", RTLD_NOW)) != NULL);
  DLERRCHECK((libb = dlopen("b.dll", RTLD_NOW)) != NULL);
  DLERRCHECK((foo = (void (*)(void)) dlsym(liba, foosym)) != NULL);
  DLERRCHECK((bar = (void (*)(void)) dlsym(libb, barsym)) != NULL);
  foo();
  bar();
  DLERRCHECK(dlclose(liba) != -1);
  DLERRCHECK(dlclose(libb) != -1);
}

void baz(void);

void baz()
{
  printf("baz called; zip = %d\n", zip);
}
Defines baz, DLERRCHECK, main, zip (links are to index).

The first library, a.dll, just provides one routine, foo, to call.

<a.c>=
#include <stdio.h>

#if defined(_WINDOWS) || defined(_WIN32)
#  define EXPORT __declspec(dllexport)
#else
#  define EXPORT
#endif

EXPORT void foo(void);

EXPORT void foo()
{
  printf("foo called\n");
}
Defines foo (links are to index).

The second library, b.dll, provides the function bar. In addition, it calls the function baz in the main program and accesses the main program's global variable zip.

<b.c>=
#include <stdio.h>

#if defined(_WINDOWS) || defined(_WIN32)
#  define EXPORT __declspec(dllexport)
#  define IMPORT __declspec(dllimport)
#else
#  define EXPORT
#  define IMPORT extern
#endif

IMPORT void baz(void);
IMPORT int zip;

EXPORT void bar(void);

EXPORT void bar()
{
  printf("calling baz\n");
  zip++;
  baz();
  printf("done with bar\n");
}
Defines bar (links are to index).

UNIX Issues

Building on most UNIX systems is fairly straightforward. The Makefile for building the tests on HPUX is

<HPUX Makefile for tests>=
CFLAGS = -Aa +z -I../HP
LDFLAGS = -L../HP -Wl,-E

all: dltest a.dll b.dll

a.dll: a.o
        ld -b a.o -o a.dll

b.dll: b.o
        ld -b b.o -o b.dll

dltest: dltest.o
        cc $(LDFLAGS) -o dltest dltest.o -ldl

clean:
        rm -f *.o dltest a.dll b.dll

Makefiles for other flavors of UNIX need a bit of diddling to get the right flags for generating PIC code and the right commands for creating shared libraries. The Tcl 8.0 configure files and the GNU libtool stuff should allow this to be automated, but I haven't done that yet****.

Some UNIX flavors, like AIX, seem to require explicit exporting of symbols.

Some UNIX flavors may add leading or trailing underscores to external symbol names.

It may be possible to support older UNIX systems that don;t have a shared library mechanism by making a dlfcn emulation based on GNU dld or the COFF ldfcn code used for MIPS architectures. I should be able to test the COFF stuff on our SGI, but I won't bother unless there is lots of demand (and then I'll look for ``volunteers'').

Macintosh Issues

On the Mac I use CodeWarrior Pro 1. I have only tested things with PPC, but in theory everything should work with CFM68K as well, but I probably won't worry about that for now.

The Mac does not seem to allow a shared library to have unresolved symbols, but you can handle references into an executable by linking the library against the executable. You also need to take some action to export external symbols; fortunately this is easy. I have not yet explored what happens when files are moved around, search pat issues, and the like ****.

To allow all external symbols in a program or library to be referenced, choose All globals form the Export Symbols menu in the project PPC PEF entry of the Linker section of the Target Settings Panel. To allow a shared library to reference globals in an executable, add the executable's file to the project and in the File Mappings entry of the Target section of the Target Settings Panel make the ``Compiler'' for APPL (or whatever th appropriate type in under CFM68K) be PEF Import PPC. It's only three or four mouse clicks---of course it takes hours to find out which three or four mouse clicks.

Windows Issues

The Windows version also seems to need a stub for the DllMain function (for Win32---it needs to be a little different for Win16).

<dllstub.c>=
#include <windows.h>

int APIENTRY DllMain(HANDLE hdll, DWORD  reason, LPVOID reserved )
{
  switch( reason ) {
  case DLL_THREAD_ATTACH: break;
  case DLL_THREAD_DETACH: break;
  case DLL_PROCESS_ATTACH: break;
  case DLL_PROCESS_DETACH: break;
  }
  return( 1 );
}
Defines DllMain (links are to index).

Now the fun part. There are lots of compilers available on Windows, each with its own little idiosyncrasies. Windows on the whole seems less than thrilled about resolving symbol references at runtime with no compile time hints. Watcom seems able to handle that, but I haven't been able to figure out how to do it on other compilers. The approach I have used instead is to produce a .lib file for the main executable, analogous to th e import library for a DLL, and to link that file with DLL's that need to do callbacks.

Some issues that need to be considered:

With these issues in mind, I will probably stick with Borland as the compiler for XLISP-STAT since it will use __cdecl. I will need to resolve how to handle the exported symbol names for callback. They can either be renamed to MS conventions (no underscore) in the .def file, or a DLL header file can handle the renaming. THe header is needed in any case to put the __declspec(dllimport) in front of variables.

Here are some details on making (or trying to make) the test code work with Metrowerks, Watcom, Borland, and VC. I have not yet tried Cygnus gcc or lcc.

Metrowerks

I tried CW Pro 1 but was not able to get that to work; maybe I'll try CW Pro 2 which I now have.

CW Pro 1 doesn't seem to use .def files, and I can't seem to make a .lib from a .exe. It doesn't like what Borland's implib makes

Watcom

I don't seem to be able to get Watcom to pay attention to a .def file for exporting symbols from .exe (it may work somehow) but an EXPORT linker directive works.

C functions compiled with default calling conventions (nonstandard register-based) get a trailing underscore; data get a leading underscore. Using wlib dltest.lib +dltest.exe creates the .lib file. I'm not sure if Watcom includes a utility for finding external symbols automatically; maybe pedump or the Tcl 8.0 dumpexts can be used.

Here is the Watcom Makefile for the tests.

<Watcom Makefile>=
SRC = ..
DLFCNDIR = ..\..\WIN

INCLUDES = F:\h;F:\h\nt;$(DLFCNDIR)

DEBUG_CFLAGS = -w4 -e25 -zq -od -d2
CODEGEN_CFLAGS = -5r -bt=nt -mf
DLL_CFLAGS = -i=$(INCLUDES) $(DEBUG_CFLAGS) $(CODEGEN_CFLAGS) -bd
EXE_CFLAGS = -i=$(INCLUDES) $(DEBUG_CFLAGS) $(CODEGEN_CFLAGS)

DEBUG_LDFLAGS = d all op inc op m op maxe=25 op q op symf
DLL_LDFLAGS = $(DEBUG_LDFLAGS) SYS nt_dll
EXE_LDFLAGS = $(DEBUG_LDFLAGS) SYS nt

all: dltest.exe a.dll b.dll

a.obj : $(SRC)\a.c .AUTODEPEND
  *wcc386 $(SRC)\a.c $(DLL_CFLAGS)

dllstub.obj : $(SRC)\dllstub.c .AUTODEPEND
  *wcc386 $(SRC)\dllstub.c $(DLL_CFLAGS)

a.dll : a.obj dllstub.obj .AUTODEPEND
 @%write a.lk1 FIL a.obj,dllstub.obj
 *wlink name a $(DLL_LDFLAGS) @a.lk1
 wlib -n -b a.lib +a.dll

dlfcn.obj : $(DLFCNDIR)\dlfcn.c .AUTODEPEND
 *wcc386 $(DLFCNDIR)\dlfcn.c $(EXE_CFLAGS)

dltest.obj : $(SRC)\dltest.c .AUTODEPEND
 *wcc386 $(SRC)\dltest.c $(EXE_CFLAGS)

dltest.exe : dlfcn.obj dltest.obj .AUTODEPEND
 @%write dltest.lk1 FIL dlfcn.obj,dltest.obj
 @%append dltest.lk1 
 *wlink name dltest $(EXP_LDFLAGS) exp baz_,_zip @dltest.lk1

dltest.lib : dltest.exe
 wlib dltest.lib +dltest.exe

b.obj : $(SRC)\b.c .AUTODEPEND
 *wcc386 $(SRC)\b.c $(DLL_CFLAGS)

b.dll : b.obj dllstub.obj dltest.lib .AUTODEPEND
 @%write b.lk1 FIL b.obj,dllstub.obj
 @%append b.lk1 LIBR dltest.lib
 *wlink name b $(DLL_LDFLAGS) @b.lk1
 wlib -n -b b.lib +b.dll

clean:
        -del *.exe
        -del *.dll
        -del *.lib
        -del *.obj
        -del *.lk1
        -del *.map
        -del *.ilk
        -del *.sym

Borland

By default, __cdecl conventions are used and symbols have a leading underscore; I think this can be turned off with a flag. implib can be used to make .lib file for .exe. tdump can be used to get a listing of external symbols.

<Borland Makefile>=
.AUTODEPEND


#
# Borland C++ tools
#
IMPLIB  = $(TOOLBIN)\IMPLIB
BCC32   = $(TOOLBIN)\BCC32
TLINK32 = $(TOOLBIN)\TLINK32


#
# Directories
#
TOOLS   = F:\BC5
TOOLBIN = $(TOOLS)\BIN
LIBDIRS = $(TOOLS)\LIB
INCDIRS = $(TOOLS)\INCLUDE;$(DLFCNSRC)
SRC     = ..
DLFCNSRC = ..\..\WIN


#
# Runtime Options
#
# This is for using the Borland runtime DLL (single threaded)
# Drop the _RTLDLL and use cw32.lib for static runtime
RTDEFINES = -D_RTLDLL
RTLIB = cw32i.lib

#
# Options
#
DEFINES = $(RTDEFINES) -DSTRICT
EXE_CFLAGS = -w -v -H=dltest.csm -WC -I$(INCDIRS) $(DEFINES)
DLL_CFLAGS = -w -v -H=dltest.csm -WD -I$(INCDIRS) $(DEFINES)
DLL_LDOPTS = -L$(LIBDIRS) -Tpd -aa -c $(TOOLS)\LIB\c0d32.obj
EXE_LDOPTS = -L$(LIBDIRS) -Tpe -ap -c $(TOOLS)\LIB\c0x32.obj


#
# Dependency List
#
all: dltest.exe a.dll b.dll

dltest.lib : dltest.exe
  $(IMPLIB) dltest.lib dltest.exe

dltest.exe : dlfcn.obj dltest.def dltest.obj
  $(TLINK32) @&&|
 /v $(EXE_LDOPTS) dlfcn.obj dltest.obj
$<,$*
import32.lib $(RTLIB)
dltest.def
|

a.lib : a.dll
  $(IMPLIB) $@ a.dll

a.dll : dllstub.obj a.obj
  $(TLINK32) @&&|
 /v $(DLL_LDOPTS) dllstub.obj a.obj
$<,$*
import32.lib $(RTLIB)
|

b.lib : b.dll
  $(IMPLIB) $@ b.dll

b.dll : dllstub.obj dltest.lib b.obj
  $(TLINK32) @&&|
 /v $(DLL_LDOPTS) dllstub.obj b.obj
$<,$*
dltest.lib import32.lib $(RTLIB)
|

dlfcn.obj dltest.obj : cfgexe.cfg
dllstub.obj a.obj b.obj : cfgdll.cfg

dlfcn.obj :  $(DLFCNSRC)\dlfcn.c
  $(BCC32) +cfgexe.cfg -c -o$@ $(DLFCNSRC)\dlfcn.c

dltest.obj :  $(SRC)/dltest.c
  $(BCC32) +cfgexe.cfg -c -o$@ $(SRC)/dltest.c

dllstub.obj : $(SRC)\dllstub.c
  $(BCC32) +cfgdll.cfg -c -o$@ $(SRC)\dllstub.c

a.obj :  $(SRC)\a.c
  $(BCC32) +cfgdll.cfg -c -o$@ $(SRC)\a.c

b.obj :  $(SRC)\b.c
  $(BCC32) +cfgdll.cfg -c -o$@ $(SRC)\b.c


# Compiler configuration files
cfgexe.cfg : 
   Copy &&|
$(EXE_CFLAGS)
| $@

cfgdll.cfg : 
   Copy &&|
$(DLL_CFLAGS)
| $@


# Remove all generated files
clean:
        -@erase *.exe
        -@erase *.lib
        -@erase *.dll
        -@erase *.obj
        -@erase *.cfg
        -@erase *.map

*

<Borland dltest.def>=
EXPORTS _baz
        _zip

Microsoft VC++

The C compiler cl.exe seems to make a .lib for a .exe automatically if there are exports. dumpbin produces information on external symbols. I think the default calling convention is __cdecl.

<Microsoft VC++ Makefile>=
TOOLS = f:\devstudio\vc
CC=$(TOOLS)\bin\cl.exe
LINK32=link.exe

DLL_CFLAGS=/nologo /MT /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D "_WINDOWS" /FD
EXE_CFLAGS=/nologo /ML /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /FD\
 /D "_MBCS" -I..\..\win
DLL_LDFLAGS=$(STDLIBS) /nologo /subsystem:windows /dll /incremental:no\
 /machine:I386
EXE_LDFLAGS=$(STDLIBS) /nologo /subsystem:console /incremental:no\
 /machine:I386

STDLIBS=kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib\
 advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib\
 odbccp32.lib

SRC = ..

all: dltest.exe a.dll b.dll

a.obj : $(SRC)\a.c
        $(CC) $(DLL_CFLAGS) -c $(SRC)\a.c

b.obj : $(SRC)\b.c
        $(CC) $(DLL_CFLAGS) -c $(SRC)\b.c

dllstub.obj : $(SRC)\dllstub.c
        $(CC) $(DLL_CFLAGS) -c $(SRC)\dllstub.c

dlfcn.obj : ..\..\win\dlfcn.c
        $(CPP) $(EXE_CFLAGS) -c ..\..\win\dlfcn.c

dltest.obj : $(SRC)\dltest.c
        $(CC) $(EXE_CFLAGS) -c $(SRC)\dltest.c

a.dll : a.obj dllstub.obj
    $(LINK32) @<<
  $(DLL_LDFLAGS) /out:a.dll a.obj dllstub.obj
<<

b.dll : b.obj dllstub.obj dltest.lib
    $(LINK32) @<<
  $(DLL_LDFLAGS) /out:b.dll b.obj dllstub.obj dltest.lib
<<

dltest.exe dltest.lib : dltest.def dlfcn.obj dltest.obj
    $(LINK32) @<<
  $(EXE_LDFLAGS) /def:dltest.def /out:dltest.exe dlfcn.obj dltest.obj
<<

clean :
        -@erase *.obj
        -@erase *.dll
        -@erase *.exp
        -@erase *.lib
        -@erase *.exe
        -@erase *.idb

*

<Microsoft VC++ dltest.def>=
EXPORTS baz
        zip

More on Watcom vs MS Calling Conventions

I recently came looked at Venables and Ripley's new complements for Splus 4.0 where they describe some changes in Splus support for DLL's. It looks like using VC or Watcom should now be possible. For Watcom, old DLL's need to be recompiled with stack calling conventions. It isn't clear exactly what is going on, but my best guess (which may not be very good) is tht StatSci has changed to using the -3s, -4s, or -5s calling convention option, which uses stack based calls. In any care, V&R seem to indicate that DLL's with Watcom stack calling and MS standard (i.e. __cdecl I think) can be used interchangeably, and that callback to things like unif_rand works. I'm a little concerned and confused about this. Here is what Watcom's dosumentation says about their stack-based convention used when compiled with the 3{r|s} option set to s:
If the s suffix is specified, the following machine-level code strategy is employed. The s conventions are similar to those used by the MetaWare High C 386 compiler.
About the calling convention that goes with __cdecl they write:
Watcom C/C++ supports the __cdecl keyword to describe C functions that are called using a special convention.

Notes:

  1. All symbols are preceded by an underscore character.
  2. Arguments are pushed on the stack from right to left. That is, the last argument is pushed first. The calling routine will remove the arguments from the stack.
  3. Floating-point values are returned in the same way as structures. When a structure is returned, the called routine allocates space for the return value and returns a pointer to the return value in register EAX
  4. For the 16-bit compiler, registers AX, BX, CX and DX, and segment register ES are not saved and restored when a call is made.
  5. For the 32-bit compiler, registers EAX, ECX and EDX are not saved and restored when a call is made.
Watcom C/C++ predefines the macros cdecl, _cdecl, _Cdecl and SOMLINK (16-bit only) to be equivalent to the __cdecl keyword.
This doesn't say who is responsible for cleaning up the stack for the Watcom conventions; let's assume it is also the caller. The phrasing about certain refisters not being preserved across calls means these registers may be modified freely by callees.

Now if Splus (i.e. Watcom s convention I assume) calls a __cdecl function with void return (as in a .C or .Fortran) then everything should work fine -- the registers the callee modifies without saving are a subset of those the caller knows it preserves. Return values are not an issue if all calling is of void functions. Integer-like return values may work fine -- I'll need to check this. So using ``simple'' DLL's compiled with __cdecl might work.

On the other hand, suppose the __cdecl DLL calls back into the Watcom s program. First there is a concern about the two registers the callee might mess with but the caller thinks are safe, GS and FS. These are segment registers, so maybe this isn't an issue with 32-bit code, I don't know. But if you re calling unif_rand, or some other function returning double, I don't see how this will work: the two conventions return their results in completely different places. Probably there is something I don't understand here.

Indices

Chunk Index

Identifier Index