As a systems administrator, I maintain a department full of computers for many users. But as with anything else, people’s needs and tastes in software are all different. In some cases, I can install multiple types of programs for anyone who asks, but some might be more difficult (relying on newer – or older – versions of software which needs to be installed as-is for the system to run properly). Because of that, and because of my own laziness in not wanting to always install every software package in the same /usr/local dumping ground (which quickly becomes polluted) I came up with a method which works well and is usable on just about any UNIX-type system for anything. If you run such a system, and are interested, read on for a lesson in local installs.
Why Not /usr/local?
I used to install everything in /usr/local – at least everything which wasn’t available as a system package. But this quickly became a mess. With every install writing stuff in bin, lib, etc, share, include and wherever else, it becomes a nightmare to clean up. What if you want to remove something? What if a newer version of a program overwrites some, but not all, of the previous version’s files? You end up with a directory of old cruft, some of which is outdated and possibly problematic, some is just lost to time. And what happens if you upgrade something, and find out that it doesn’t do what you wanted and you want to downgrade? The only hope is to compile the old version again and reinstall, on top of everything yet again. The path becomes a mess, and sooner or later it’s easier to take a quick inventory of what you’ve installed, erase the whole thing and reinstall again. But that only puts off the problem, doesn’t it?
This kind of problem is why package managers were created. They allow a simple way to install a software package and remove it later. But that only works in some cases – for one, your software may not have a package available. It might also be compiled in such a way that it must be installed in a system location (such as /usr/bin and /usr/lib) and can’t be moved without recompiling. It might also interfere with a system-installed and required package, either directly or by some dependency relationship. A package manager is a wonderful thing – no matter if you prefer debs, RPMs or whatever else. But it’s still not going to solve all problems, especially if you’re not root and want to install something!
~/Installs To The Rescue
The first thing I needed to do was figure out a place to put things. But I didn’t want to replicate the /usr/local debacle in my home directory; I also didn’t want to clutter my ~/dls/ directory with tarballs and compile directories, so I started by calling it ~/Installs. This keeps everything separated out nicely, though if you want a shorter name you might do something like ~/I (you’ll see why a short name can be useful later, when it comes time to telling your shell about these directories). As an example, let’s look at some of the directories in the Installs directory on my work desktop:acroread/ curl/ elinks/ firefox/ gems@ gpa/ john/ jxplorer/ old/ rtorrent/ rubygems/ screen/ thunderbird/
The ‘old’ directory is where I move something I’m about to upgrade, so I can keep it around just in case. But the rest are each a directory for a single package – or in the case of rtorrent, for two packages which are closely related. ‘gems’ is a symlink to rubygems/gems, but that whole package is a bit more than the introduction needs to explain. The rest are pretty straightforward – either a program which isn’t/wasn’t available as a system package, or one that I needed a newer version than what is installed to work (or otherwise wanted a version compiled differently). Since rtorrent is the one I just upgraded today, I’ll step through how that all worked as an example.
Your First Local Install
After creating the Installs directory, make a directory called ‘rtorrent’. In here you will install the things which rtorrent needs to function (namely libtorrent and rtorrent itself). Download them from http://libtorrent.rakshasa.no/ to this rtorrent directory, then uncompress each one. You’ll now have directories named for each version of libtorrent and rtorrent. Since libtorrent is required first, ‘cd‘ into its directory and prepare to compile.
Normally you’d run a standard ‘./configure‘ and then make. However, here’s where things diverge: you want to specify a prefix. You might have done this previously – for example when compiling software for /usr/local – but we want a much more specific prefix. Since my username is huston, the command I ran was
./configure --prefix=/home/huston/Installs/rtorrent
When that finished, ‘make && make install‘ ran normally to install everything. However, all of the files are now installed in the Installs/rtorrent directory. Instead of cluttering up system locations, the entire libtorrent install is contained within a specific directory. Now when you’re finished, you’ll want to ‘cd‘ back to the rtorrent-<version> directory to compile it. But wait, it needs to know where libtorrent is installed in order to compile and link properly. Fortunately, libtorrent uses pkg-config, so this is a simple thing to do. In a single command:
export PKG_CONFIG_PATH=/home/huston/Installs/rtorrent/lib/pkgconfig ; ./configure --prefix=/home/huston/Installs/rtorrent && make && make install
The PKG_CONFIG_PATH variable tells pkg-config where to find the libtorrent library, and again we set the prefix for our rtorrent package directory. When this finishes, you’ll have a working rtorrent binary in ~/Installs/rtorrent/bin/ which you can run directly to start the program.
Hey, What About libcurl?
In my case, on the machines I compiled rtorrent I needed a newer libcurl installed than the system version. This is a perfect example of not wanting to disturb the installed package in order to have a newer one available for some use. I did as above, creating a ‘curl’ directory and compiling with the appropriate –prefix. When finished, I then edited the PKG_CONFIG_PATH above to include the curl/lib/pkgconfig directory which was now created, and pkg-config properly picked the newer package to compile against. In this way, I can keep the newer curl around if it’s required for something – like libtorrent – but it’s not interfering with the files in /usr/bin or /usr/lib, upon which many other installed packages depend. This eliminates a lot of the “dependency hell” that people complain about with package managers, because I just install the newer library in a place that the newly compiled program can use it, but it’s out of the way for everything else.
Easy PATHing
So far, you see how this can keep software installs neat and tidy, but there’s some major drawbacks. Since the programs aren’t in system locations, typing ‘rtorrent’ at a command prompt won’t run it – your shell doesn’t know where to find the binary (you’d have to use ‘~/Installs/rtorrent/bin/rtorrent’ which can get cumbersome after a couple executions). Likewise, if you type ‘man rtorrent’ you’ll probably get “No manual entry for rtorrent” as a reply because ‘man’ doesn’t know where to look for the proper manpage. This, too, is simple to fix with a little scripting. You could add these paths by hand to your shell startup files, but a quick script can make light work of it – and you never have to think about them again! Add this code block to your .bashrc file (apologies for how poorly this is formatted, but I’m not inclined to dig through making WordPress format bash script properly; if you really must see it in a nicer format, look here instead):
for D in $HOME/Installs/* ; do
if [ -d $D/bin ]; then
[[ "$PATH" =~ "(^|:)$D/bin($|:)" ]] || \
export PATH=$D/bin${PATH:+:$PATH}
fi
if [ -d $D/share/man ]; then
[[ "$MANPATH" =~ "(^|:)$D/share/man($|:)" ]] || \
export MANPATH=$D/share/man:${MANPATH:+$MANPATH}
fi
if [ -d $D/man ]; then
[[ "$MANPATH" =~ "(^|:)$D/man($|:)" ]] || \
export MANPATH=$D/man:${MANPATH:+$MANPATH}
fi
done
So what does this do? For each directory in your Installs directory, it looks for a bin/ directory and adds it to your PATH. It also looks for share/man/ and man/ to add to your MANPATH. The end result is, if you have this bit of code in your .bashrc and add a new install it will automatically get added to the proper paths next time you open a shell (at least one which sources .bashrc, but more on that some other time). Now when I want to run rtorrent, I just type ‘rtorrent‘. And ‘man rtorrent‘ shows me the manpage installed in ~/Installs/rtorrent/share/man/man1/rtorrent.1 without having to remember where it is. As an added bonus, the Installs paths are added to the beginning of your PATH and MANPATH, so if you install a newer version of something it will be found before the system versions, ensuring your hand-installed packages are the ones you get when you run a command.
That Was Easy
I’ve been doing this for a couple years now on my own machines, and recently started advocating it for others. It makes for a simple way to install programs without asking root@ to put them on the system, or when system installs aren’t desirable for some reason. At the same time, it prevents things from getting crufty since any one package is a single ‘rm -rf <directory>‘ away from being removed entirely. It also helps when you upgrade a system, or move your home directory, since you can easily see a list of programs that you installed without having to hunt through /usr/local and hope you catch them all (usually forgetting a dependency or two in the process). I use this on my work desktop, my home server, and my Mac laptop and desktop as well, and haven’t had a /usr/local or otherwise non-package-manager-managed path setup since.