Find Every Header File in Your Ubuntu Repositories

Last night I was following a thread on Penguin Pete's blog where Pete uses Ubuntu's dropping the Dillo browser from the distribution as a launching pad for a discussion about — heck, I can't describe what it's about, read it yourself.

Anyway, in the course of reading the discussion — if that's the word, Pete gets worked up about certain things — I was possessed by the moral imperative to install Dillo from source.

This would seem to be straightforward. You go to the Dillo home page, click Download, and follow the instructions.

Except (there's always an except) — the first install you have to make is the FLTK2 (Fast Light Toolkit) library.

Now Ubuntu offers FLTK1.1, which might well work, but hey, we need the latest and the greatest, right? So I downloaded the tarball, followed the instructions, and the make command crashed and burned with:

fatal error: X11/extensions/XInput.h

I looked on the web, found that XInput.h is in the package libxi-dev, installed that, reran make,

fatal error: GL/glu.h

and so on. It took about an hour to find every single header file needed. And that was just to install FLTK2, after which I still had to install Dillo.

So what's going on here? Well, first we have to know what a header file is. Basically, it's a file that defines a bunch of variables and statements that can be used by multiple routines. See Wikipedia for a better explanation. In particular, header files tell a program's source code what variables are used in a particular library. Thus the header XInput.h defines variables available in some X11 libraries.

OK, suppose I'm compiling a program and I find that I need a header file GrantsTomb/Buried.h. All I have to do is find the package that has that header, install it with apt-get, and I'm good to go. Right?

Well, without prior knowledge you're pretty much doomed here. I was able to find XInput.h and glu.h by web search, but who's going to tell you where is Buried.h in GrantsTomb?

Yet there has to be a solution, because:

• I know that when I use synaptic or aptitude to install packages I can see every package that is, or can be, installed on my system from my selected repositories.
• After searching around a bit, I find that apt and its children list the available files in /var/lib/apt/lists, each repository has its own file, and each file name is on a line starting with Package.
• I know about the program apt-file, in particular that the command
  apt-file list foobar
  will give me a list of all of the files in the foobar package.
• Header files are traditionally located in packages ending with the string -dev.

Given all of that, I should be able to write a script which will list every header file available in every Repository I've accessed. I can then save it in a file, and the next time a program says I need bojangles/tambourine.h I can just search the list, find that the header file is in the NittyGritty-dev package, load that up, and I'm ready to compile:

• First, install apt-file, if you don't have it already:
  sudo apt-get install apt-file
• Then run this script:
  

  #! /bin/bash
  
  # First make sure you've got all your repositories updated:
  sudo apt-get update
  
  # Next sync apt-file's database.  Note that if you do this without
  #  the sudo you'll write the data into your home directory, which is
  #  probably OK
  
  sudo apt-file update
  
  # Now search through every repository database, looking for lines that
  #  start with "Package", and end with "-dev".
  # Pull out the package name, look through each package, and print out
  #  the header files:  those that end in "-h"
  
  #! /bin/bash
  
  # No matter how it wraps on your screen, this next line starts with
  # "for" and ends with "xargs`"
  
  for hfile in `find /var/lib/apt/lists -type f -exec grep '^Package: ' {} ';' | awk '{print $2}' | grep -e '-dev' | xargs`
  do
  apt-file list $hfile | grep "\.h$"
  done
  

Be warned, this takes a long time, since every call to apt-file goes out over the network. On my system, with FIOS broadband, it took over an hour. So you'll only want to run this file once in a while, probably only when you update your repository list. You may also get an error message like
grep: /var/lib/apt/lists/lock: Permission denied
Don't worry about that.

This script should work pretty well for Debian. There are also equivalent commands for RPM-based distributions. And, of course, you can modify it to list any subset of files in any subset of repositories you want. I'll leave all of that as an exercise for the reader.