Saturday, June 09, 2007

Re-Installing Intel(ligence?)

If all Linux distribution were the same, there wouldn't be a point in having so many of them. But that, of course means that different distributions are different. OK, there are certain categories you can use to pigeonhole them, e.g., tarball/rpm/deb packages, GNOME/KDE/whatever Window Manager, graphical/text installation, etc., but there are also more subtle differences.

Case in point, for the standard /bin/sh shell required of every Unix-like computer, Ubuntu (and, I suppose, Debian) uses the Debian Almquist shell (dash), using a symbolic link from /bin/dash to /bin/sh. Most other distributions link to the heftier Bourne-again shell (bash). You can see which shell you are linked to with the command:

$ ls -l /bin/sh
lrwxrwxrwx 1 root root 9 2007-05-28 12:26 /bin/sh -> /bin/dash

which shows that when I call /bin/sh, I'm really running /bin/dash.

And that's fine, in most cases. If some program in the Ubuntu distribution requires the capabilities of bash, rather than dash, to run its scripts, the package maintainer can alter every invocation of /bin/sh to read /bin/bash, and all is right with the world.

Unless, of course, you have a third-party program that doesn't allow a distribution maintainer to alter the code in any way, assuming that the distribution is allowed to put the code in its repositories in the first place.

And therein lies today's sermon:

A long, long, time ago, four weeks, I guess, we upgraded Hal to the Feisty Fawn edition of Ubuntu Linux. Back then I said something about eventually re-installing the free-for-noncommercial-use Intel® Fortran Compiler for Linux and corresponding Intel® Math Kernel Library (MKL).

Today was the day. (Amazing what you'll do to keep from balancing the checkbook.) I got the newest versions of the compiler and the library, filled out the yes – I'm – only – going – to – use – this – myself – and – yes – we – bought – a – copy – for – work form, and went to town.

I've installed various versions of the Fortran Compiler before, and the procedure listed in that link always worked, with suitable modifications of the version numbers. Not this time, though. I got the compiler installed, but when I tried to run it, an error message told me I'd performed an illegal operation. The problem turned out to be in Intel's ifort script, which actually calls the compiler. It's written using the generic shell /bin/sh, which Intel expected to be linked to /bin/bash, but which Ubuntu links to /bin/dash. Since dash is a lot simpler than bash, the script fails. The solution is to simply edit the ifort file, which on my system is at /opt/intel/fc/9.1.043/bin/ifort, so that the first line reads

#!/bin/bash

For more on this, see Intel's help pages and the Ubuntu Forums.

That's half the battle. As I've noted before, for scientific calculations it is useful to have certain operations optimized (in assembly) for your CPU. When using the Intel compilers, the preferred vehicle for this is the Intel Math Kernel Library (MKL), which is distributed under the same license as the compilers.

Naturally that library expects /bin/sh to also mean /bin/bash. Worse, the installation script for the libraries fails because of the bash/dash problem, and just changing /bin/sh to /bin/bash everywhere doesn't seem to work..

There's a solution at the bottom of this Ubuntu Forums thread, but it ain't pretty:

In a standard Ubuntu installation, /bin/sh is soft-linked to /bin/dash:

$ ls -l /bin/sh
lrwxrwxrwx 1 root root 9 2007-05-28 12:26 /bin/sh -> /bin/dash

We're going to change that:

$ sudo ln -sf /bin/bash /bin/sh

Warning: if you leave out the -s option, you will mess up your system beyond belief. At the very least, you're liable to have bash and dash point to the same file, with uncertain consequences.

If the ln command is successful, bash is now your default shell. Run the MKL install. You shouldn't have to edit any scripts after this.

Finally, if you want to restore dash as the default shell, go back and run

$ sudo ln -sf /bin/dash /bin/sh

Again, do not forget the -s!!!!!

So far, all my codes compile correctly with ifort, and the MKL libraries all link properly, so I believe everything works.

But it sure would have been easier if Ubuntu developers had access to the package.

3 comments:

Anonymous said...

"If all Linux distribution were the same, there wouldn't be a point in having so many of them."

yah! I dig it! I think we could stand to see more of this attitude. No need to flatter them all into the same pancake.

I never knew before that Debian uses it's own shell. That's one pet peeve of mine with Debian distros, they change something to their own brand, then softlink it to the same name so you have no clue - until your old scripts break. They should have big warning signs: "We've replaced Bash with a subtly different shell to drive you a little crazy. If something breaks, look here first!"

grml is the only one I know that uses zsh. Which had me sweating every time I used it, but lately it's been growing on me.

-Penguin Pete

Anonymous said...

s/flatter/flatten/

s/uses it's own/uses its own/

oy, I'm typing with 13 thumbs today...

rcjhawk said...

Well, yeah, it's sort of Debian's fault, but then again if you are going to write scripts that depend on explicit properties of bash, meaning things not in the original /etc/sh, then does it kill you to head your scripts with

#! /bin/bash ?

Two characters. And so much time would be saved.

Which is why I like open source over closed-but-free (as in beer), but in this case there's not a lot of choice if you want to do real physics calculations on a desktop computer that's only moderately more powerful than a Cray XMP.