Creating an Open-source Program in C With Autotools (Part 1 of 2)

« More entries

In this post I will start building a command-line utility in C, using some interesting tools and distributing my package properly. This is not a tutorial on Git, Autotools, C or GDB, but rather on using all the tools together to make an open-source project. This first part will focus on setting up the construction system with Autotools.

The program I’m going to develop is a command-line utility for tagging files systemwide. I’ll call it dfym (Don’t Forget Your Music). It will allow us to tag files or directories and find the files and directories matching a tag, as well as manage the database. It will use SQLite as a database (it won’t work at the filesystem level, as this is not an exercise on userspace filesystems). Actually, using a database will have its benefits as well as it drawbacks.

1. Setting up the project structure and repository

The first step will be setting up a Git repository:

Creating the project’s Git repository
mkdir dfym && cd dfym
git init

Once ready, I will add the basic structure and files required by the GNU standard for an open-source program distributed with Autotools. But before that, I think is best to add a .gitignore file like this one, to avoid tracking the tons of autotools-generated files:

Recommended .gitignore template
/autom4te.cache/
/config/
Makefile
Makefile.in
aclocal.m4
app.info
compile
config.h
config.h.in
config.log
config.status
configure
configure.scan
coverage_report
depcomp
install-sh
libtool
m4
missing
stamp-h1
*~
*.html
*.log
*.o
*.la
*.so*
*.a
.deps
*.tar*
*.zip
*.lo
*.gcno
*.gcda

I should now also add src/bin/dfym so the executable is not included in the repository.

2. Setting up a minimal Autotools configuration

The next step is creating the necessary files for GNU’s Autotools to work, along with the directory structure for the source files.

Creating the project’s files and structure
touch NEWS README AUTHORS ChangeLog
mkdir -p src/bin src/lib

Autotools needs NEWS, README, AUTHORS and ChangeLog for Automake to work. I guess this is a way to enforce GNU’s standards of source code distribution and packaging. If I don’t create this files, Automake won’t work. The next step will be creating a minimal C program, as a stub for growing organically later on, which I can also use to test the build system. Once I’ve done this, I commit these changes with Git.

src/bin/main.c
#include <stdio.h>

int main() {
  printf("Welcome to DFYM\n");
}

2.1 Configuring the binary

Compiling it with Autotools requires a configure.ac configuration file (which takes care of building the ./configure script for me) and one or more Makefile.am files, one in the root folder and one in each source directory (these will serve as the base for building the makefiles when invoking ./configure). First off, I will create the Makefile.am files, which at the moment only require pointing at the source files. The first Makefile.am will be placed at the root directory, and contain information about where the other Makefile.am files are.

Makefile.am
SUBDIRS = \
          src/bin

The other Makefile.am file is needed right at the directory I pointed to in the previous Makefile.am. This one will contain the instructions of what source files are used to build the binary.

src/bin/Makefile.am
AM_CFLAGS = -I$(top_srcdir)/src/bin

bin_PROGRAMS = dfym
dfym_SOURCES = main.c

Now, the more tedious part: writing the configure.ac. Fortunately, once the Makefile.am files are prepared, I can get some help partially generating this configuration file by using autoscan:

Invoking autoscan
autoscan

…which will produce a configure.scan file, looking like this:

configure.scan
#                                               -*- Autoconf -*-
# Process this file with autoconf to produce a configure script.

AC_PREREQ([2.69])
AC_INIT([FULL-PACKAGE-NAME], [VERSION], [BUG-REPORT-ADDRESS])
AC_CONFIG_SRCDIR([src/bin/main.c])
AC_CONFIG_HEADERS([config.h])

# Checks for programs.
AC_PROG_CC

# Checks for libraries.

# Checks for header files.

# Checks for typedefs, structures, and compiler characteristics.

# Checks for library functions.

AC_CONFIG_FILES([Makefile
                 src/bin/Makefile])
                 AC_OUTPUT

This will serve as a template for the configure.ac, but I will edit it manually. First, I’ll rename configure.scan to configure.ac. Then follow these steps:

  1. In AC_INIT: subsitute FULL-PACKAGE-NAME, VERSION and BUG-REPORT-ADDRESS with “dfym”, “0.1” (or the current software version) and the contact address respectively.
  2. Add AM_INIT_AUTOMAKE below AC_CONFIG_HEADERS. This is mandatory.
  3. Add AC_PROG_CPP to the Checks for programs section. This will check for the C preprocessor (not C++, as it might seem).
  4. Add AC_LANG([C]) before the next checks, so it uses the C language for them.
  5. Add AC_HEADER_STDC and AC_CHECK_HEADERS([stdio.h]) for checking the standard C headers and the pressence of stdio.h (I’m using it in the stub program).
  6. Add AC_TYPE_SIZE_T for checking size_t type in the Checks for typedefs, structures, and compiler characteristics section.
  7. Optionally, change AC_PROG_CC to AC_PROG_CC_C99 if I want to use C99.

The result should look like this:

Edited configure.ac (copied from configure.scan)
AC_PREREQ([2.69])
AC_INIT([dfym], [0.1], [a@fourthbit.com])
AC_CONFIG_SRCDIR([src/bin/main.c])
AC_CONFIG_HEADERS([config.h])
AM_INIT_AUTOMAKE

# Checks for programs.
AC_PROG_CC_C99
AC_PROG_CPP

# Use the C language and compiler for the following checks
AC_LANG([C])

# Checks for libraries.

# Checks for header files.
AC_HEADER_STDC
AC_CHECK_HEADERS([stdio.h])

# Checks for typedefs, structures, and compiler characteristics.
AC_TYPE_SIZE_T

# Checks for library functions.

AC_CONFIG_FILES([Makefile
                 src/bin/Makefile])
AC_OUTPUT

This is now ready to use Autotools for generating the makefiles and the ./configure script for us. It will need just one command to put all the machinery to work, creating the missing files and all the scripts that will take care of checking the system, building the code and installing it: autoreconf. It’s important to use this command after every update to configure.ac or Makefile.am files.

Invoking autoreconf with flags
autoreconf -iv

Finally, I can call ./configure, make , and make install in case I want to install it in my system. As a side note, Autotools and the compilation process creates tons of garbage. A trick to removing all these files, takes advantage of Git’s ignored files to remove all that non-tracked files:

Removing all generated files
git clean -f -d -x

2.2 Configuring a static library

The project will be composed of a main binary and a static library, which can be used later on for designing the program. At this point, I will add a stub for the static libray, in order to test the build system with Autotools. It consists of just a function returning 0, placed in src/lib under the name libdfym-base.c and its header libdfym-base.h. First, I will create the directory src/lib and then create the following files:

Create the src/lib directory
mkdir src/lib
Stub: src/lib/libdfym-base.h
int dfym_fun();
Stub: src/lib/libdfym-base.c
#include "dfym_base.h"

int dfym_fun() {
  return 0;
}

The main concern at the moment is the build system, so I will focus on updating the configure.ac and creating/updating the necessary Makefile.am files, without developing any real functionality in C. The new configure.ac will follow the next updates:

  1. Add AC_PROG_RANLIB to the Checks for programs section.
  2. Add src/lib/Makefile to the AC_CONFIG_FILES macro.

It should now look like this:

Add the static library to configure.ac
AC_PREREQ([2.69])
AC_INIT([dfym], [0.1], [a@fourthbit.com])
AC_CONFIG_SRCDIR([src/bin/main.c])
AC_CONFIG_HEADERS([config.h])
AM_INIT_AUTOMAKE

# Checks for programs. These may set default variables, such as CFLAGS
AC_PROG_CPP
AC_PROG_CC_C99
AC_PROG_RANLIB

# Use the C language and compiler for the following checks
AC_LANG([C])

# Checks for libraries.

# Checks for header files.
AC_HEADER_STDC
AC_CHECK_HEADERS([stdio.h])

# Checks for typedefs, structures, and compiler characteristics.
AC_TYPE_SIZE_T

# Checks for library functions.

AC_CONFIG_FILES([Makefile
                 src/lib/Makefile
                 src/bin/Makefile])
AC_OUTPUT

Now, I’ll add src/lib to the root Makefile.am. The order is of importance, since it will need lib built before building bin.

Add the static library subdirectory to the root Makefile.am
SUBDIRS = \
          src/lib \
          src/bin

The last step in setting up the static library compilation is the src/lib/Makefile.am file, which will become the Makefile for the static library, after Autotools’ magic. I will write this by hand, following the next steps:

  1. First, describe that I intend to build a library target. I don’t want to install it in the lib directory of the system (it is used only as an internal library for the program), thus a noinst, and it will use a LIBRARIES primary (the boilerplate generation script for libraries that don’t use Libtool). To understand this better, I’d check the Automake manual; take a look at the amhello’s Makefile.am example explanation. Some other examples of targets would be: check_PROGRAMS (programs to build for testing), include_HEADERS (headers to install in the system-wide include directory).
  2. Then, set a noinst_HEADERS list of headers that I want to distribute but not to include in the system (they are necessary for the user to be able to compile the program). If I want these headers to be installed in the include directory of the system, use include_LIBRARIES instead.
  3. As a final step, set the SOURCES variable associated with this target: libdfym_base_a_SOURCES (the sources that will be build the library, also distributed with the package). Other variables would be libdfym_base_adir (the headers installation directory) and libdfym_base_a_HEADERS (the headers for this library that must be installed in the system). However, I don’t need them, since the library is not meant to be installed in the system. An important note to take notice of is that the names of the targets must be rewritten as follows: Non-alphanumeric characters of the variable name are changed to underscore ‘’ (e.g. libdfym-base.a to libdfym_base_a_SOURCES_).
Add the static library subdirectory to the root Makefile.am
# The libraries to build
noinst_LIBRARIES = libdfym-base.a
noinst_HEADERS = dfym_base.h

# The files to add to the library and to the source distribution
libdfym_base_a_SOURCES = \
                         $(libdfym_base_a_HEADERS) \
                         dfym_base.c

2.3 Adding debug support

In this section, I will add debug support via the —enable-debug to the ./configure script.

Add the static library to configure.ac
# Add debug support
AC_ARG_ENABLE(debug,
  AS_HELP_STRING(
    [--enable-debug],
    [enable debugging, default: no]),
    [case "${enableval}" in
      yes) debug=true ;;
      no)  debug=false ;;
      *)   AC_MSG_ERROR([bad value ${enableval} for --enable-debug]) ;;
    esac],
    [debug=false])
AM_CONDITIONAL(DEBUG, test x"$debug" = x"true")
AM_COND_IF(DEBUG,
    AC_DEFINE(DEBUG, 1, [Define to 0 if this is a release build]),
    AC_DEFINE(DEBUG, 0, [Define to 1 or higher if this is a debug build]))

This will add #define DEBUG 0 if this is a release compilation, and #define DEBUG 1 if is a debug compilation. It will also create the DEBUG conditional, which can be used by Makefile.am to set the proper compilation flags:

Makefile.am:

Add conditional compiler flags to src/lib/makefile.am
if DEBUG
  AM_CFLAGS =-I$(top_srcdir)/src/lib -Wall -g -O3
else
  AM_CFLAGS =-I$(top_srcdir)/src/lib -Wall
endif
Add conditional compiler flags to src/bin/makefile.am
if DEBUG
  AM_CFLAGS =-I$(top_srcdir)/src/bin -I$(top_srcdir)/src/lib -Wall -g -O3
else
  AM_CFLAGS =-I$(top_srcdir)/src/bin -I$(top_srcdir)/src/lib -Wall -O3
endif

However, there is a strange quirk in Autotools by which there are some different CFLAGS by default depending on the system. These CFLAGS get appended everytime, unless the user manually sets CFLAGS=“” when invoking ./configure. This is due to AC_PROG_CC (or AC_PROG_CC_C99) invokation, which sets these defaults. I need a way to handle this, and it is achieved by forcing CFLAGS to always become the user defined ones (so if the user doesn’t define any, I just get an empty string). Thanks to this workaround, I still get user-definable flags, which will override any others, but I can still use my own in case the user doesn’t supply any, without being overriden by the defaults set by the AC_PROG_CC/C99 macros.

Workaround for default CFLAGS overriding custom ones configure.ac
# Remember externally set CFLAGS
EXTERNAL_CFLAGS="$CFLAGS"

# Checks for programs. These may set default variables, such as CFLAGS
AC_PROG_CPP
AC_PROG_CC_C99
AC_PROG_RANLIB

# Reset the externally set CFLAGS after calling AC_PROG*
CFLAGS="$EXTERNAL_CFLAGS"

2.4 Linking an external library

For this program, I know in advance (or highly suspect) that I will be using two external libraries: SQLite3 and GLib. This final step will add these libraries to the linking process. This is very easy, and can be done for any other library that I’d like to include.

First, in configure.ac, I need to add two lines for each library:

  1. PKG_CHECK_MODULES: will use pkg-config to check for a library existence.
  2. AM_CONDITIONAL: will use the result of the previous test to define a conditional variable, available in Makefile.am files. I’m not actually using it in this case, but in case is necessary, I make it available in Makefile.am via this macro.

These are the lines I will add for GLib and SQLite3:

Lines to add to configure.ac for library checks
# Checks for libraries.
PKG_CHECK_MODULES([GLIB], [glib-2.0], [have_libglib=yes], [have_libglib=no])
AM_CONDITIONAL([GLIB],  [test "$have_libglib" = "yes"])

PKG_CHECK_MODULES([SQLITE3], [sqlite3], [have_libsqlite3=yes], [have_libsqlite3=no])
AM_CONDITIONAL([LIB_SQLITE3],  [test "$have_libsqlite3" = "yes"])

So the final configure.ac looks like this:

Final configure.ac, including external libraries checks
AC_PREREQ([2.69])
AC_INIT([dfym], [0.1], [a@fourthbit.com])
AC_CONFIG_SRCDIR([src/bin/main.c])
AC_CONFIG_HEADERS([config.h])
AM_INIT_AUTOMAKE
PKG_PROG_PKG_CONFIG

# Remember externally set CFLAGS
EXTERNAL_CFLAGS="$CFLAGS"

# Checks for programs. These may set default variables, such as CFLAGS
AC_PROG_CPP
AC_PROG_CC_C99
AC_PROG_RANLIB

# Reset the externally set CFLAGS after calling AC_PROG*
CFLAGS="$EXTERNAL_CFLAGS"

# Use the C language and compiler for the following checks
AC_LANG([C])

# Checks for libraries.
PKG_CHECK_MODULES([GLIB], [glib-2.0], [have_libglib=yes], [have_libglib=no])
AM_CONDITIONAL([GLIB],  [test "$have_libglib" = "yes"])

PKG_CHECK_MODULES([SQLITE3], [sqlite3], [have_libsqlite3=yes], [have_libsqlite3=no])
AM_CONDITIONAL([LIB_SQLITE3],  [test "$have_libsqlite3" = "yes"])

# Checks for header files.
AC_HEADER_STDC
AC_CHECK_HEADERS([stdio.h])

# Checks for typedefs, structures, and compiler characteristics.
AC_TYPE_SIZE_T

# Add debug support
AC_ARG_ENABLE(debug,
  AS_HELP_STRING(
    [--enable-debug],
    [enable debugging, default: no]),
    [case "${enableval}" in
      yes) debug=true ;;
      no)  debug=false ;;
      *)   AC_MSG_ERROR([bad value ${enableval} for --enable-debug]) ;;
    esac],
    [debug=false])
AM_CONDITIONAL(DEBUG, test x"$debug" = x"true")
AM_COND_IF(DEBUG,
    AC_DEFINE(DEBUG, 1, [Define to 0 if this is a release build]),
    AC_DEFINE(DEBUG, 0, [Define to 1 or higher if this is a debug build]))


# Checks for library functions.

AC_CONFIG_FILES([Makefile
                 src/lib/Makefile
                 src/bin/Makefile])
AC_OUTPUT

Finally, I need to add GLib’s and SQLite3’s CFLAGS and LDFLAGS, which get defined automatically for us in GLIB_CFLAGS, GLIB_LIBS and SQLITE3_CFLAGS, SQLITE3_LIBS respectively (the names are generated from the first argument in the PKG_CHECK_MODULES macro in configure.ac). A clean way to add these CFLAGS and LDFLAGS would be appending them like in the final src/bin/Makefile.am:

Final src/bin/Makefile.am, including external libraries linking
if DEBUG
  AM_CFLAGS =-I$(top_srcdir)/src/bin -I$(top_srcdir)/src/lib -Wall -g -O3
else
  AM_CFLAGS =-I$(top_srcdir)/src/bin -I$(top_srcdir)/src/lib -Wall -O3
endif

AM_CFLAGS += $(GLIB_CFLAGS)
AM_CFLAGS += $(SQLITE3_CFLAGS)

bin_PROGRAMS = dfym
dfym_SOURCES = main.c

dfym_LDADD = $(top_builddir)/src/lib/libdfym-base.a $(AM_LDFLAGS)
dfym_LDADD += $(GLIB_LIBS)
dfym_LDADD += $(SQLITE3_LIBS)

I keep a template for projects alike. This template is also hosted in Github.

Part II will focus on the process of development and using tools that will aid us during the task of creating a simple open-source project following GNU’s standards of distribution.


Did you find it useful? Please share!

Comments