Saturday, September 25, 2010

Bum-Bum-Bum-Bum-BUM-Bum

I liked Boomer better than I like Kona.

Which is a short way of saying that if you're watching the re-make Hawaii Five-0 in order to see more of Grace Park, you're better off getting all the Battlestar Galactica disks from Netflix and watching them. Boomer/Sharon/Athena/Eight was interesting and sexy. Kona looks like a school girl. A painfully thin school girl.

To put it mildly, I'm unimpressed by the whole Five-0 experience. At the start, they do keep the same theme song, how could they not, but they make the images flash way too fast, and cut it short as well.

Then there's McGarrett. Not my idea of a NCIS/Seal/Cop. Well, maybe a Don Johnson-style cop, since he always needs a shave. But Alex O'Loughlin is no Jack Lord. Or Mark Harmon, for that matter.

I did like Danno. The first thing I thought was He reminds me of James Caan. Then I look up Scott Cann, and find that he's James Caan's son. This is one improvement over the old show, the old Danno was a wimp.

And I love the makeup work they did on Jean Smart, who plays the governor. She's my age, but they managed to make her look old.

OK, this is a meander, and not a review, but lets see — what was good and bad about the episode itself:

Good: There's an underlying mystery here, something only hinted at in the first episode. Sort of like in last year's The Good Wife, where we learn that there's some deep dark secret in Chicago politics (who knew?)

Bad: Jean Smart's roll is apparently going to be along the lines of:

McGarrett: You gave me unlimited authority!

Governor: And you exceeded it!

Implausible: McGarrett and Danno attack what is identified as a Chinese cargo ship. While there, they are attacked by a variety of bad guys, including the episode's main villain. They win easily, of course. In the entire scene we see not one person of Asian extraction.

I'll probably watch a few more episodes, as time permits, either on Hulu or with FIOS On Demand, but this isn't going to be one of those shows that's automatically DVR'd.

Friday, September 24, 2010

Ubuntu and Flash

I've been running the 64-bit version of Ubuntu 10.04 for about five months, and have had few problems. Most of them you've heard about. Lately, though, I've had a major problem — Ubuntu's unbranded version of Google Chrome, aka chromium-browser, wouldn't run Flash.

If I tried to visit a flash site, I'd get something like this:

Annoying.

The apparent cause is a corrupted profile. Exactly what's corrupted, I couldn't say. Basically, to get Flash running again you need to wipe out all traces of chromium and start over. I did this, but I wanted to at least save my bookmarks. Here's how. Open a terminal and type in everything to the left of the # signs:

cd # Go to your home directory
mv .config/chromium chromium_old # save your old configuration, at least for now
chromium-browser # start the browser up.  Exit immediately.
cp chromium_old/Default/Bookmarks* .config/chromium/Default # restore you old bookmarks
chromium-browser # surf normally.

Of course, you've lost all your saved passwords, surfing history, etc. Think of it as a way of weeding out the chaff. A project, for someone who has the interest and time, would be to go through each file in the old profile and determine which one got corrupted. If you do that, feel free to let us know what you find.

Saturday, September 18, 2010

Back to Blogging

Well, not really, just another placeholder/bookmark entry. It was a busy summer, and now I'm coming up on Fiscal New Year's Day.

From our Department of Procrastination: I owe y'all a follow-up to Finding Your External IP Address, where I show you how to use the thing, i.e., log onto any computer in your house from anyplace else in the Universe. This is half-written, so maybe in a week or two it will actually get posted.

Book Reviews: I really want to mention the greatest history book ever written, Lies My Teacher Told Me by James Loewen. I lost the first edition long ago, but a second, substantially rewritten, edition is now out. Find it, buy it, read it. You won't agree with everything, but you'll agree that it's relevant.

Another one: Human Smoke, by Nicholson Baker. It's just a collection of quotations and stories from the end of World War I to the end of 1941. No one, and I mean no one, comes off well here. None of your heroes, and certainly none of your villains. You'll have to read it slowly, because it's just too damn depressing to read all at once. But read it. It covers that part of history that Loewen complains is left out of history books.

Tips and tricks: A recent update to my Chromium browser (Google Chrome, if you're using a branded version) left me without a bookmark toolbar. Annoying. Reading through this entire comment, and noting that I don't seem to have an option for this under the wrench, I found that Ctrl-Shift-B toggles the toolbar.

We'll get back to real live articles Real Soon Now. I hope.

Sunday, September 05, 2010

Virtual Windows

Or virtual anything else: I've been playing around with the open source edition of VirtualBox, a utility from Oracle (boo!) that lets you run another operating system in a Window in your current operating system.

Any host operating system: Linux, Windows, OS X. Any target OS: Linux, Windows, OS X — well, maybe you'd better ask Steve about that. Tell him I said it was OK.

The open source version of VirtualBox is included in the Ubuntu repositories, so install it with:

sudo apt-get install virtualbox-ose

On your Gnome menu you'll find and entry for VirtualBox OSE under Applications => Accessories

Click New and follow the directions. I was able to install Windows XP without the usual hassles. VB provided all the needed drivers.

OK, one problem. The virtual machine wouldn't boot. The problem turned out to be that VB was looking for a floppy disk drive, which Hal doesn't have. I had to click Settings => System, and then unclick the Floppy Drive label under Boot Order. After that, everything worked fine.

I was even able to stream Netflix movies in a browser under Windows, once I installed Silverlight.

Useful for the very few times I need Windows.

Thursday, July 29, 2010

Updating the un-updatable

In my post on the relative easy of updating Linux I used Ubuntu's graphical Update Manager to do the work.

Of course, in Linux/Unix you can always do this from the command line. In Debian based systems the relative commands are:

sudo apt-get update
sudo apt-get upgrade

The first line compares the packages available online with those installed on your machine, and the second line upgrades the packages that aren't current. Easy, huh?

Except that sometimes you get messages like this:

The following packages have been kept back:
  linux-generic linux-headers-generic linux-image-generic

This is more or less a are you sure you want to do this now? message. You'll note that the kernel is involved here, so you're going to have to reboot the machine once the packages are installed.

All right, I'll reboot, so what do I have to do? Well, it turns out the the packages can be explicitly installed:

sudo apt-get install linux-generic linux-headers-generic linux-image-generic

This will take care of everything. Just remember to reboot the computer to get the new kernel up and running.

Monday, July 26, 2010

This Week's Sign of the Apocalypse

Since time began, classes at the University of Kansas have ended with the sound of a steam whistle which tooted for five seconds at 50 minutes past the hour.

As a budget cutting measure, the steam whistle, Tootie-Toot (never heard that name when I was there), now only sounds for 2.5 seconds.

The 2.5 second version is at the end of this clip:

Saturday, July 24, 2010

The Update Log

I've spent most of July out of town/state/country, so I haven't been very diligent about updating all the computers. Today, though, I took a stab at it:

I. Windows Vista

We have a Windows Vista desktop. It's slated to be upgraded to Linux after Child2 moves files off of it to the new Macbook. For now, though, it runs several programs that said child uses, e.g. Photoshop, and has a full set of browsers (IE, Firefox, Chrome, Safari) that I use to judge how new web pages look to the masses.

OK, here's everything I needed to do to update that machine:

  • Update Virus Definitions: We have McAfee installed, primarily because we bought this machine before I knew about Microsoft's Windows Security Essentials. It's not smart enough to know that it should autoupdate virus definitions if the computer's been off a long time, it just gives you a Dire Warning of Impending Doom.
  • Update Firefox
  • Update Thunderbird
  • Update Google Chrome
  • Update Safari: This one popped up on its own, offering updates to both Safari and iTunes.
  • Check Flash Version: Up to date
  • Perform Microsoft Updates: At least these were done in the background, all I had to do was reboot.
  • Reboot: Note that three (3) programs asked for permission to reboot: Windows itself, Apple's Safari/iTunes, and McAfee
  • Run Virus Scan: otherwise it wouldn't do it until late tonight.

II. Mac OS X

This is my work laptop.

  • Run Software Update
  • Run Microsoft Autoupdate: to check if any Office components needed to be updated.
  • Update Firefox
  • Update Thunderbird
  • Update Google Chrome
  • Check Flash Version
  • Check for updates to Aqua Emacs
  • Run sudo port upgrade: to check for updates to Macports programs.
  • Virus check: I'm required to run a virus checker by the powers that be, never mind that the next Mac virus in the wild will be the first I'm aware of. Fortunately, on the Mac McAfee is smart enough to do all its work in the background, without the warnings.
  • Reboot to take care of the system updates

III. Linux machines

  • System > Administration > Update Manager > Check > Install Updates: then wait a bit
  • Reboot: to activate the new kernel

Now. Will some one please remind me how much easier it is to run Windows or Mac OS X as opposed to Linux? Because sometimes I forget.

Wednesday, July 21, 2010

Why People Don't Read History

It's just not believable.

Stolen from a comment in Contrary Brin.

I've been on vacation. More actual Linux stuff this weekend, I hope.

Saturday, June 26, 2010

Finding Your External IP Address

For various reasons you might want to find your public IP address, that is, the 32-bit number, expressed in pseudo-decimal form as something like 72.14.204.99, that tells the world how to reach your computer.

For most businesses and government offices this is trivial: your computer is assigned a static IP address, one that never changes with time. For home users, though, it's a little more complex.

Start with the IP address itself. The largest possible IP address is 255.255.255.255. That means that you can have no more than 4,294,967,296 unique IP addresses, and some of those are reserved for local use, as we'll see. We're coming up on 7 billion people in the world. Not everyone has a computer (yet, and remember iPhones, iPads, Kindle, and all 'Droids count), but many people have more than one (see iPhones, iPads, …). So there aren't enough IP addresses to go around. Eventually, everyone will switch over to the IPv6 address scheme, which will give us 340,282,366,920,938,463,463,374,607,431,768,211,456 (2128, or 3.4×1038) possible addresses, which is probably enough for every non-hydrogen nucleus in the observable universe to have its own address.

But we're not there yet, so something has to be done to shrink the address space. Your local ISP solves this problem by 1) charging you extra to get a static IP address; 2) giving you only one IP address per gateway (your cable, FIOS, DSL, or dial-up modem, or some other router or hub that connects you to the internet); and 3) taking your IP address away from you when you're not using it.

That last one is called dynamic IP addressing, and it's part of what we're going to talk about today. If you have a dynamic IP address, then when you turn your gateway connection (usually some type of modem or router) on, your ISP assigns you an IP address from its bank of addresses. As long as your connection is turned on, you'll have an IP address, but your ISP can change it at any time. It will almost certainly change it if you turn off your connection and turn it on again. So if you know your IP address right now, there's no reason to expect it will be the same tomorrow, or even five minutes from now — though to be fair, Verizon FIOS has kept our house on the same IP address for weeks.

And there is worse to come: Suppose your ISP has given you the address 244.117.16.25. The computer you're reading this on doesn't have a clue as to what that address is. Go back to 2), above. Your house has only one IP address, and it belongs to your gateway. Every other computer in the house belongs to your home network. When you want to download your email, your computer politely requests the gateway to go fetch it:

Please, sir, may I have my email?

Oh, all right. Just this once. But don't tell any of the other computers what we're doing.

Thank you, sir! I'll be eternally grateful

You'd better be, or no Internet for you.

Let's see how this works. ping is a command that will poke a stick at a computer and get information about it. Let's try it with a well known site:

ping google.com
PING google.com (72.14.204.99) 56(84) bytes of data.
64 bytes from iad04s01-in-f99.1e100.net (72.14.204.99): icmp_seq=1 ttl=53 time=21.8 ms
64 bytes from iad04s01-in-f99.1e100.net (72.14.204.99): icmp_seq=2 ttl=53 time=20.2 ms
64 bytes from iad04s01-in-f99.1e100.net (72.14.204.99): icmp_seq=3 ttl=53 time=23.4 ms
^C

(you'll need to hit ctrl-C to stop this)

So we see that google.com has an IP address of 72.14.204.99 — that's actually one of many, but don't worry about that. Now let's try it with a local machine:

ping george
PING george.home (192.168.1.2) 56(84) bytes of data.
64 bytes from george.home (192.168.1.2): icmp_seq=1 ttl=64 time=9.77 ms
64 bytes from george.home (192.168.1.2): icmp_seq=2 ttl=64 time=2.70 ms
64 bytes from george.home (192.168.1.2): icmp_seq=3 ttl=64 time=0.961 ms
^C

So far has the home network is concerned, george lives at 192.168.1.2, which is one of those reserved addresses we mentioned above. The range 192.168.0.0 to 192.168.255.255 is reserved for internal networks. As we read this, there are no doubt hundreds of thousands of computers with the local IP address of 192.168.1.2. Every one of them, however, has a different public IP address.

Great. But sometimes I need to know my external IP address, and it's apparent that Hal, here, hasn't got a clue as to what it is. How can I find it?

The easiest way is to go to another website. Every website you connect with gets your public IP address automatically, unless you use some kind of anonymous service. Some websites are set up to tell you what your IP address is. One example is What Is My IP?. They even provide a way to get your IP address with a simple one-line terminal command:

wget www.whatismyip.com/automation/n09230945.asp -O - -q ; echo

If you read your Gmail via Thunderbird, Evolution, Outlook, etc., and keep your web client on all day, gmail will tell you the address of your computer. Just log on to your gmail account and look for the line that says Last account activity: …. The IP address there is the last computer to download email. If you're using more than one computer from different locations, say you left your office machine on over the weekend and now you're using a home computer, click on the Details link and you'll find all of the activity for this account.

There's a company DynDNS, which will not only find your IP address, but give you a domain name that constantly updates to your current numerical address. Of course, they do this by having your computer constantly tell them the public IP address.

And, of course, your gateway knows its public IP address. Just log onto it through your browser (something like http://192.168.0.1, http://192.168.1.1, or http://192.168.100.1, depending on your system), and your IP address will be up there someplace.

But all of these systems have flaws: Say you need your IP address in a script. You can get it via your browser from your gmail or your gateway machine, but it's hard to parse a browser page inside a script. The What's My IP? one-liner works in a script, but what if their site goes down, or, worse, they go out of business? Same for DynDNS. We want a solution that works any time your gateway is on and your computer has an internet connect, and we want to get it from the command line.

Any other way just ain't elegant.

If you search the Internet you find surprisingly few ways (i.e., none) to do this. Partly I think it's because there isn't a consistent way to find your gateway from your current machine. For example, in the days when we used Comcast, we had a cable modem as the gateway. On the house side of the modem we connected a Linksys router, which was the hub of our home network. All the computers in the house were connected through the router to the modem — a two step-process. Now we have Verizon FIOS, with a Actiontec MI424WR router that serves as gateway/modem and router — a one-step process.

By stumbling around, and with a little help from nixCraft, I was able to come up with a solution that works for me. It's a bash script. Let me put it here (finally, I hear you say), and we'll dissect it below:

#! /bin/bash

# Determine the public IP address for your DHCP system by bootstrapping

#  calls to your gateway/router

# Requires "route," available in Ubuntu package net-tools, and
# nslookup, available in Ubuntu package dnstools

# Disclaimers:
# 1) This works with Verizon FIOS using the Actiontec MI424WR Router
#    If you have another system YMMV.
#
# 2) And, as you'll no doubt note, this is set up for a Debian system,
#    specifically Ubuntu 10.04 .  Package names and installation
#    procedures for other systems WILL vary.
#

###
# Start Here:
###

#
# Get Router internal Gateway IP
# Information found on
# http://www.cyberciti.biz/faq/how-to-find-gateway-ip-address/
# If route is not installed, run
# sudo apt-get install net-tools
# and make sure /sbin is in your $PATH
# Note "U" = up, and "G" = gateway

internalip=`route -n | grep UG | awk '{print $2}'`

echo Your Gateway\'s Internal IP Address is: $internalip

# Now we need to get the name of the router.  If you don't
#  have nslookup installed, run
# sudo apt-get install dnsutils

echo -n "Your Gateways's name is: "

routername=`nslookup $internalip | grep "name =" | awk '{print $4}'`

# On the Actiontec, at least, the name has an annoying "." at the end.
# Remove it:

routername=${routername%\.}

echo $routername

echo -n "Your Public IP Address is: "

nslookup $routername | grep Address | tail -1 | awk '{print $2}'

OK, what's going on? First of all script only works if you have the route and nslookup commands installed on your system. If they aren't there, in Ubuntu, and presumably and Debian system, you can install them using

sudo apt-get install net-tools dnsutils

The first call, route -n, will look at the Kernel's IP routing table:

$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.1.0     0.0.0.0         255.255.255.0   U     1      0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 eth0
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth0

which on this machine tells us that we have several local connections. The U under Flags means that the connection is up, and G means it's the gateway. From this we see that the numerical address of the gateway is 192.168.1.1. Let's see what nslookup can tell us about that. Note that the script has to parse all of the output to get this far.

$ nslookup 192.168.1.1
Server:  192.168.1.1
Address: 192.168.1.1#53

1.1.168.192.in-addr.arpa name = Wireless_Broadband_Router.home.

Here's something new, our gateway's got a name. Wireless_Broadband_Router.home. Again, the script has to do a little work to tease that name out of nslookup's output, but we'll just do it by hand here:

$ nslookup Wireless_Broadband_Router.home
Server:  192.168.1.1
Address: 192.168.1.1#53

Name: Wireless_Broadband_Router.home
Address: 192.168.1.1
Name: Wireless_Broadband_Router.home
Address: xxx.yyy.zzz.aaa

The first address, 192.168.1.1, is just the router's internal IP address. The last address (Of course I'm not giving you my IP address. I'm not daft, you know.) is the one we want. xxx.yyy.zzz.aaa is the external address of this machine.

Of course there are many, many, problems with this script.

  • So far it only works with the Verzion/Actiontec setup. A friend, running Ubuntu, wasn't able to pick out his gateway machine with the route -n command.
  • Some machines can have multiple gateways.
  • The Mac's route command doesn't work in the same way as under Linux, so you can't parse out the gateway, even if you have none.
  • You might be able to tease the gateway IP address out of the /etc/resolv.conf file. I don't know what happens if you have multiple gateways here. Under Ubuntu, the one numerical gateway is 192.168.1.1, on the Apple it gives me 192.168.1.1 and xxx.yyy.zzz.aaa, my public IP address.

If you try the script, leave a comment below. Let me know your internet setup (gateway/router), if it works, and any tweaks you did to make it work. Maybe we can come up with a universal script.

Now, by all that's geekly, why did we do all of this? That's the subject of an upcoming post, boys and girls …

Saturday, June 19, 2010

Smart ssh tricks

I want my computers to talk to one another.

More specifically, I want to be able to remotely log in to one of my computers from another one, and I want to be able to easily copy files from one computer to another. The easiest, most secure, way to do that these days is to use the Secure Shell protocol, aka ssh.

Examples: I have an account on a computer called grumpy (sleepy and doc were already taken). To log onto grumpy from hal, I open up a terminal window and type:

$ ssh rcjhawk@grumpy

The program then logs me onto grumpy, where I can access the system just as I can for any desktop machine.

Or, say, I have a file at /home/rcjhawk/Documents/strasburg_pictures.ppt on grumpy. To copy it over to Hal, I use the secure copy program, scp:

$ scp rcjhawk@grumpy:Documents/strasburg_pictures.ppt .

which, obviously, works pretty much like the regular copy program except that I have to specify the machine and account name right up front.

And, finally, I can send files the other way:

$ scp The_Hard_Times_of_Zach_Greinke.txt rcjhawk@grumpy:Documents

which sends that regrettable file into the Documents folder on grumpy.

Except:

You will probably find that this doesn't work out of the box in your Ubuntu setup. You'll get a message that looks something like this:

ssh: connect to host grumpy port 22: Connection refused

That's because Ubuntu ships ssh in two parts:

  • openssh-client: this lets you use ssh and scp from your machine to log onto another machine. This is installed with the standard Ubuntu installation, and is all you need if, for example, you want to log onto your work computer from home.
  • openssh-server, on the other hand, allows other machines to log onto your personal desktop machine. This is not installed by default in Ubuntu because it's an obvious security hole unless you know what you're doing. If you do know, or assume that you will learn, install the package with

    $ sudo apt-get install openssh-client

    Once you do that, sshd, the daemon that controls remote access, will automatically start up, and will start up every time you reboot the machine. Then you're good to go.

In the example above, grumpy needs to have openssh-server installed and running, but hal does not.

Note for Mac Users: You get the equivalent of openssh-server by going to System Preferences => Sharing and clicking Remote Login.

Security

ssh encrypts the transmissions between machines, but it also has other security features. The first time you try to log onto a machine, for example, you'll get a message that looks something like this:

$ ssh rcjhawk@grumpy
The authenticity of host 'grumpy (192.168.54.42)' can't be established.
RSA key fingerprint is xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx.
Are you sure you want to continue connecting (yes/no)?

If you answer yes, then you'll get the message:

Warning: Permanently added 'grumpy,192.168.54.42' (RSA) to the list of known hosts.

After that you'll be able to log onto grumpy without the warning message. If you want to start over, i.e. not recognize any computer as being previously visited, erase the file $HOME/.ssh/known_hosts

You can also limit access to the machine by editing your /etc/hosts.deny and /etc/hosts.allow files. As I've noted before, if you don't want ever Tom, Dick, and Susie to have a chance to access your machine, your /etc/hosts.deny file should read:

ALL: ALL

after which only addresses listed in your /etc/hosts.allow file will have access to the machine. In my /etc/hosts.allow file I currently have

ALL: 192.168.1.0/255.255.255.0 ,

which allows all machines on our local network to access grumpy.

OK, everything is good, right? Well, not quite. As things are set up right now, every time you ssh or scp to grumpy you must enter a password. This isn't a big annoyance with ssh, because presumably you will keep the remote login running for a relatively long time. However, it is an annoyance with scp, if you want to copy a large number of files from different places, requiring a large number of scp commands.

The solution is to use ssh-keygen to generate a a key. Here's how it works:

Start the program:

$ssh-keygen -t dsa Generating public/private dsa key pair.
Enter file in which to save the key (/home/rcjhawk/.ssh/id_dsa):


We'll use the default directory location, so just hit return here.

Enter passphrase (empty for no passphrase):

Frankly, I don't know anyone who does use a passphrase. It does protect your key against someone who's gotten access to your account, but at that point you're probably hosed anyway. So for now we'll just hit return.

Enter same passphrase again:
Your identification has been saved in /home/rcjhawk/.ssh/id_dsa.
Your public key has been saved in /home/rcjhawk/.ssh/id_dsa.pub.

Now let's take a look at the .ssh directory:

$ ls -la $HOME.ssh
total 16
drwx------  2 rcjhawk rcjhawk  4096 2010-06-19 13:04 .
drwxr-xr-x 34 rcjhawk rcjhawk  4096 2010-06-19 13:01 ..
-rw-------  1 rcjhawk rcjhawk   672 2010-06-19 13:04 id_dsa
-rw-r--r--  1 rcjhawk rcjhawk   599 2010-06-19 13:04 id_dsa.pub

Here you'll notice two files. One, id_dsa, is your private key. The second, id_dsa.pub, is your public key. Key the private key private. Don't let anyone have access to it. Don't copy it over into the .ssh folder of another account, or to another computer. You'll note that both the .ssh directory and id_dsa are set so you are the only person who can read those files. Keep it that way.

The public key, now, that's another matter. If you place id_dsa.pub into the file $HOME/.ssh/authorized_keys on another computer, you'll be able to access that computer from your home computer. Here's one way to to it (I'm leaving out all the password prompts):

$ ssh rcjhawk@grumpy
Welcome to grumpy!
rcjhawk:~ $ cd .ssh
rcjhawk:~/.ssh $ scp rcjhawk@hal:.ssh/id_dsa.pub tempid
rcjhawk:~/.ssh $ cat tempid >> authorized_keys
rcjhawk:~/.ssh $ rm tempid
rcjhawk:~/.ssh $ exit

What did we do? We logged into grumpy, copied hal's public key over to grumpy's .ssh directory, and added it to the end of the authorized_keys file. I renamed the file during the copy so as not to write over grumpy's own id_dsa.pub file, if one was there. Notice that scp was able to get that file even though Hal's .ssh directory is copy protected, because you gave the system the right password.

The next time you try to ssh or scp from hal to grumpy, ssh will find the public key on grumpy, compare it to the private key on hal, and conclude that you are, indeed, allowed to log in or transfer files.

The discerning reader will note that I'm only applying this to my local network, which all sits behind a router and firewall. How, you may ask, can I use ssh to log onto my local machine from outside my home?

Well, I wish I could tell you, but at the moment I can't. It's something to be researched in the future. If you're reading this, and you know how to do it, post a link to the procedure in the comments.

Friday, June 11, 2010

A Sign From Above

Tonight coming home from work I learned that there was a backup on US-50 East between I-97 and the Severn River Bridge in Annapolis.

I knew this, because I was given a Sign. This sign, as a matter of fact, which today had an actual message.
The sign in question is about 15 miles from the start of the backup. Not to worry, I had plenty of time to read it because I was in a five-mile backup caused by people slowing down to read the sign.

Obscene comments about Maryland drivers omitted.

Friday, June 04, 2010

Stupid Bash Tricks

I grew up scripting with the C shell (csh), and I'm still more comfortable with it than any other shells. However, for really complicated scripts I find that BASH (the Bourne-again shell) is more flexible. The problem is that I don't really know how to use it all that well.

I could buy a book, I suppose, but that costs money. And anyway, much of what I want is already on the web. For example, we have

I've used all of these, but the only way to really remember something is to build an example, and refer back to it as needed. So here, presented for your consideration, are a couple of examples.

The first one just counts. It's basically here to show how to do math with variables in BASH:

#! /bin/bash

typeset -i count

count=$1
echo $count

while (( $count < $2 ))
do
 count=$count+1
 echo $count
done

Actually, that one is adapted from the Korn shell (ksh), but it works in bash. A purer bash form would be

#! /bin/bash

let count=$1
echo $count

while (( $count < $2 ))
do
 let count=$count+1
 echo $count
done

Either way, if we call the file count, then it counts from the first argument up to the second:

./count 5 10
5
6
7
8
9
10

On to another program. I often have programs that recompute a certain quantity repeatedly, say the total energy in a Density Functional Theory run. For example,

$ grep "TOTAL ENERGY=" INFO
                          TOTAL ENERGY=      -551.5802151120
                          TOTAL ENERGY=      -551.5798689965
                          TOTAL ENERGY=      -551.5810124810
                          TOTAL ENERGY=      -551.5809217653
                          TOTAL ENERGY=      -551.5796453493
                          TOTAL ENERGY=      -551.5797608209
                          TOTAL ENERGY=      -551.5806860076
                          TOTAL ENERGY=      -551.5804690663
                          TOTAL ENERGY=      -551.5793575483
                          TOTAL ENERGY=      -551.5794583810

where I only want the last line. For a single file, I could get that with

$ grep "TOTAL ENERGY=" INFO | tail -1
                          TOTAL ENERGY=      -551.5794583810

but I want to look at multiple files, make a note of the file I'm looking at, and extract the last entry. So I wrote this little script, which I call lastgrep. Among other things, it shows you how to use the shift command to delete elements from the calling string.

#! /bin/bash

# usage

# lastgrep string  file1 file2 file3 file4

# finds the last occurrence of string in each of the files
#  listed in the argument list.  If string contains white space
#  it should be quoted

string=$1

# Pop the first argument (string) off the stack.  The
#  remaining arguments should be files

shift

for file in $@
do
    echo -n $file

#    Put quotes around $string to keep white space in place

    grep "$string" $file | tail -1
done 

So if I have a set of files INFO in directories a100, a200, a300, I might do something like this:

$ lastgrep "TOTAL ENERGY=" a*/INFO
a100/INFO  TOTAL ENERGY=      -551.7810244761
a200/INFO  TOTAL ENERGY=      -551.6774784178
a300/INFO  TOTAL ENERGY=      -551.6192819148

What this doesn't have is a way of passing arguments to grep. That's for another time.

Tuesday, June 01, 2010

CUPS and a Wireless Printer

Eric Raymond once wrote a famous essay about the difficulty of using Linux's Common UNIX Printing System (CUPS) to set up a printer over a network. Thus, when we got ChildII's new HP C4780 Wireless Printer/Scanner I was less than optimistic about getting it Hal to access it. I mean, we're talking about a wireless printer which is talking to a Verizon FIOS wireless router, and through that to a Linux computer? No way.

I should note that I wouldn't go out and buy an inkjet printer on its own, anymore, particularly an HP. But we just got ChildII a MacBook Pro (new college uses Macs, and Mac Photoshop, for art projects), and they threw in an iPod Touch (which I get to glance at longingly) and the HP (which it's my duty to set up, operate, and maintain). So we got the thing, we might as well use it. Anyway, having a wireless or networked printer for a MacBook is an essential, given that the thing only has two USB and one Firewire port.

Setting up the HP required that I use the Mac or a PC, connected by USB to the printer, to enter the network login information. Given the connection between open source software and printers this left me a little chagrined, but I'm not RMS so I let it slide. Anyway, I didn't expect the thing to work with Linux, remember?

It took about a half-hour to install the software and drivers onto a Mac, and set up the printer. We tested the printer and scanner via wireless, and it all worked fine, though the scanner was pretty slow, much slower than my USB Canon. It all worked, though.

Then it took another half hour or so to install everything on a Windows PC. Again, it all worked flawlessly.

OK, let's try our luck. I left the printer on, went downstairs to Hal, clicked on System => Administration => Printing and watched the CUPS interface come up. Hit Add, Printer, Find Network Printer. Blinked. There it was, an HP C4700 type printer. Unlike the Canon, all the drivers were already available, I didn't have to play guess-the-correct-version. Within five minutes of clicking that Printing button I was printing a test page from Hal back to the printer.

Now all is not sweetness and light. While CUPS had no problem finding the printer, SANE, the corresponding program for scanners, couldn't find the HP. Not surprising, I guess, but it would have been nice if everything worked out. Of course I'm not particularly sure what good a remote scanner does anyone, especially if, like, me, they use it primarily to scan old pictures onto a disk.

But to summarize, things have much improved since the days of ESR's essay.

And I've got to look into getting a networked color Laser printer.

Monday, May 31, 2010

Backing Up Your Blog

After the discussion about creating disk backups I realized that I haven't been terribly diligent about backing up this blog. After all, what if Google should fail? Thousands of words of my deathless prose might be lost. (I know I've written some unforgettable things. Just last week I got three emails telling me how my posts had affected them. Only two mentioned nightmares.)

So how do you back up a Blogger blog?

One way to do it is to download the monthly archive HTML. The advantage to this is that everything in in HTML, and easy to read. But you have to remember to do it. I did, for a while, but haven't done it for some years. Sloth, I know.

But there is a way to download the whole blog:

  1. Log onto your Blogger account.
  2. Go to the Dashboard, if you're not there already.
  3. Click Settings.
  4. Click Export Blog.
  5. Click the orange button.
  6. A big file, ending in .xml, will appear in your download folder.

Note that the xml file is hard to read, but if you're moving your blog from one account to another it's the easy way to go. Also note that this does not save pictures or comments, those linked (the HTML archives don't save the pictures, either). So you'll have to find that stuff yourself.

This doesn't save your blogger template, either. To do that, go back to the dashboard, click Layout, then Edit HTML, and Download Full Template. Important if you have witty sayings embedded in you blog header.

Also note that either the HTML or XML backups are useful for searching your blog, because that Google search box in the upper left doesn't always find everything you've written.

Sunday, May 30, 2010

Return With Us Now to the Thrilling Days of Yesteryear

I'm not very good at doing disk backups.

OK, let's be honest: I'm terrible at doing disk backups. Oh, back in the day, every once in a while I'd copy as much of the disk as I could onto CDs or DVDs. But that takes a lot of time, and, as disks become larger while DVDs don't, requires a lot of picking and choosing to figure out what goes where.

To be fair, I haven't needed to. In all the years I've run computers here at home, I've never had a disk crash that lost data, even after our most serious disk mishap. So not having a backup hasn't been that big a problem. (Insert your favorite ominous foreboding music here.) Anyway, backups are a pain in the neck.

My view of backups has changed, though, primarily because they're so much easier now. Easiest, of course, is Apple's Time Machine. Plug a second disk (USB, Firewire, internal, if you've got such a thing) into your Mac, let it format to Apple's HFS format (or not), and start Time Machine. Tell it which directories you want to back up, and it will do that, every few minutes. Then you can actually scroll back through time (Tardis sounds not included) and pick out a file you might have deleted a year ago — assuming you were running Time Machine back then. It's an extremely neat and easy-to-use utility, and takes the pain out of doing backups, as long as you remember to leave the external disk plugged in, if that's what you've got.

Then, last week, Office Depot ran a sale where you could get a Terabyte Verbatim USB disk for $79.99 + tax. I bought three, one for Hal, one each for the college students.

Now the disk comes with Nero backup software for Windows (taking care of student I), and Apple's got Time Machine (taking care of student II). But what about poor Hal, stuck in Linux?

I called up the Synaptic package manager and typed backup in the search box. That brought up, among other things, a package called backintime, which is a front-end which uses the utilities cron (to schedule backups), diff (to find which files are changed), and rsync (to copy files that have changed to the backup disk). Perfect. I installed the program — actually both backintime-common, the guts, and backintime-gnome, the graphical front end, then went to get the disk.

Problem: Verbatim formatted the disk to the lowest common denominator, FAT32. This is not suitable for low-volume backups, because it does not allow multiple hard links to the same file. What backintime does, you see, is create a copy of your chosen disk directories every time you schedule a backup. The trick is that unchanged files are hard-linked from one backup to the next, taking a minimum of space. Only the changed files are stored in multiple copies. Since the FAT format doesn't do multiple hard links, it can't do the trick, and so we need to reformat the disk to a better file system.

So I installed gparted from Synaptic, launched it (using sudo gparted), and formatted the Verbatim disk to ext3 format.

Second problem: If you just plug a USB disk into a Ubuntu box, it auto-mounts, but is usually only accessible to the user who did the mounting (to be fair, I haven't tried this with a USB ext3 disk). That's unacceptable in a backup system that needs to be accessible to everyone. So we have to set up /etc/fstab. To reflect this. Ideally, we'd add a line which looks something like this:

/dev/sdd /backup ext3 errors=remount-ro 0 2

(Note added after the fact: The last entry on this line should indeed be a 2, not a 1 as previously listed. This this tells fsck to check this file system after any file system with a 1 in the last column. The only file system with 1 there should be the boot partition. Everything else can be a 2, unless you don't want fsck to look at the file, in which case you put in a 0. Got that?)

create a directory entry /backup, and then doing

mount /backup

would load up the disk drive.

However, since you usually have multiple USB ports on your system, there's no guarantee that this disk is going to always get /dev/sdd as its port. The solution is to use the ext3's UUID, which was created when gparted did its thing. You find the UUID with the command:

sudo blkid
[sudo] password for hal:************
/dev/sda1: LABEL="PQSERVICE" UUID="0A72323D72322DB7" TYPE="ntfs" 
/dev/sda2: LABEL="SYSTEM RESERVED" UUID="CCA811C2A811AC48" TYPE="ntfs" 
/dev/sda3: LABEL="Gateway" UUID="7E143E7F143E3A8B" TYPE="ntfs" 
/dev/sda5: UUID="2eec3fff-73d2-419c-8a3c-b92733d46da2" TYPE="swap" 
/dev/sda6: UUID="f3781533-d891-46e1-b9f0-839e8a538d35" TYPE="ext4" 
/dev/sdb1: LABEL="Verbatim" UUID="9deee42d-539b-45c5-8683-e95f889c1792" TYPE="ext3"

Verbtim is how I labeled the disk partition of the USB drive, so that's the UUID we want. Fill out the line in /etc/fstab as:

UUID=9deee42d-539b-45c5-8683-e95f889c1792 /backup ext3 errors=remount-ro 0 1

and we're ready to go. Note that this will automount so that everyone can read it every time the disk drive is plugged in and turned on.

That settled, setting up backintime is easy. You can specify the directories you want to back up, and the file types (say, those ending in ~ or .o) and directories you want to exclude. You can then select a backup time interval: minutes, hours, days, weeks ... Once you have a set of backups, you can use the graphical interface to look through your files of any snapshot, and restore a file that was deleted or moved on your main system, or copy it to another location if you like. It's very nice and easy to use.

You can see your backup schedule using crontab:

crontab -l
@daily nice -n 19 /usr/bin/backintime --backup-job >/dev/null 2>&1

Note the @daily there. That means the backup will start every night at midnight, and it's the only choice backintime gives you if you select a daily backup. This is unsatisfactory for me, as someone is frequently using the computer on and after midnight. So I used crontab -e to edit this entry, changing it to
0 3 * * * nice -n 19 /usr/bin/backintime --backup-job >/dev/null 2>&1
which launches the backup at 3am. backintime accepts this, but note that every time you change your setup it will reset the time to @daily. Just watch it.

Anyway, now I have a disk backup. And since I've never had a major loss of disk data from one disk, having two disks copying the data should mean that it can never, ever, ever happen, right? (Insert even more ominous and foreboding musing here.)