Thursday, December 30, 2010

Happy New Year

This blog started just over seven years ago, and sometime tonight or tomorrow it will get its forty thousandth recorded hit. Thanks to all of you for listening.

Thanks to a Penguin Pete rant (see Sitemaps). I've discovered Google's webmaster tools and Analytics. My New Year's Blogging Resolution is to learn how to use these next year. If I come up with any neat tricks I'll let you know.

In the meantime, Happy New Year, everyone.

My primary resolution is to still have a job next year.

Monday, December 27, 2010

Should We Call It gnarcissism?

Linux Journal, or, more properly, the janitor at Linux Journal since everyone else is off this week, is running a pole asking if you Google® yourself.

The answer I'd choose isn't there:

No, because there are at least a half-dozen people* with the same name who are at least as famous as I am, and that doesn't count the 1,300 or so in China.

* Yes, I know, but only for the purpose of this post.§

Working from the old if you're one in a million line.

§That's my story, and I'm sticking to it.

Saturday, December 25, 2010

The Christmas Toy

So here I am, Christmas Evening, waiting for BBC America to broadcast the Dr. Who Christmas special and playing with my new toy: a Logitech Harmony 650 Universal remote. I bought the thing at Radio Shack because: a) It was cheap ($69.95), and b) I'm sick and tired of having to manipulate three remotes to change from FIOS to BluRay to USB/Harddisk when watching shows on the bigscreen TV.

The neat thing about the Harmony series of remotes is that you program it on-line. The bad part is that there isn't a native Linux program to do it. There is something called Concordance which is supposed to work, but I'll try that out when I'm prepared to trash the remote if the software fraks out, OK? It's not worth the struggle (forgive me, Saint Richard).

So we boot Hal up in Windows 7 mode, and I'm blogging what's going on using a laptop.

  • First: gather up the model numbers of the gadgets to be used. That's fairly easy. I've got a Samsung LCD 46 inch TV, and the model numbers on the side. A Samsung BluRay BD-P360. The hardest part is the FIOS DVR. It's by Motorola, and looking at the pictures on the FIOS site it appears to be a 6416. We'll see.
  • Launch IE8 on Hal (NNNNOOOOO!!!!!!!). Go to http:www.logitech.com/myharmony and create an account. Click on My Account. No registration box. Click on the safest looking link, Go to My Stuff There's a login screen and a registration option there. For some idiot reason they want a daytime phone number. Tempted to give them the one for our security, but refrain.
  • Click on Product Support. Says it will detect USB corded product. The unit comes with a USB 2 cable, but of course the connector on the remote side isn't standard USB2. Annoying, especially if I loose the cord, but I sort of forfeited the moral outrage option when I booted into Windows. So plug it in, let Windows find the driver software, and click the Detect your devices button. Of course Logitech has an IE add-on that has to be installed. It seems to work. It says I've got something called an N/N: N-10003 Harmony 650 remote. Well, I know the Harmony and 650 part are right.
  • It's not particularly obvious what to do next. Click Download Software and get version 7.7.0-WIN-x86 version of LogitechHarmonyRemote, I guess. Install that. You know, I'm kind of glad I'm doing all of this on the Windows side. Installation involves the usual number of clicks to accept the license. Eventually an Icon shows up on my desktop and the thing launches. First thing it wants to know is if it should check for updates. Surprisingly, there are none.
  • Log in. You'd think the account I just set up with Harmony would work, but no. So set up another account here. It's another complete registration, folks.
  • Now things start. It finds my remote (again). Keep hitting Next until we get to the Add Device page. Adding devices seems straightforward, so long as you have the right device numbers. Fortunately they give you examples of how they want the information entered in.
  • Once I finally got the right models in, the software found my devices. The only question I had to ask is if I wanted to turn the cable box off when it wasn't in use. I said yes. We'll see if that's a good idea or not. (I worry what's going to happen when I switch devices. Is the TV going to be out of sync with the cable box?)
  • Specify that FIOS uses HDMI1 on the TV, and the BluRay uses HDMI2.
  • And somehow we lost the signal, so I HAVE TO LOG IN AGAIN???? And it lost the connection to the remote. But now it has, indeed, started downloading the settings to the remote.
  • Looks done. I'm told to disconnect the remote and go try it out.
  • Preliminary tests done. It mostly works. There are several glitches, though:
    1. Both the TV remote and the FIOS remote have options for resizing the picture on the screen. It's especially handy when you are watching a standard definition channel, but they're broadcasting something in letterbox format (as in the above-mentioned Dr. Who special). Resize the picture, and it fits the whole screen. Not high-def, but in this case bigger is better. The 650 doesn't have such a button.
    2. The FIOS remote has a standalone DVR button, which bypasses the main menu and takes you to the stuff you use most. The 650 doesn't, you have to go through the main Menu (though the Menu, Info, and Guide buttons work exactly the way they should, as does the Favorites option on the 650's screen).
    3. The worst problem: If I want to stream Netflix using the BluRay, I press a button labeled B on the BluRay remote. One of four buttons, Red/A, Green/B, Yellow/C, Green/D. The 650 has unlabeled Red/Blue/Yellow/Green buttons, but they don't do anything to the BluRay. Since I stream Netflix as much as watch DVDs, if not more, this is a serious difficulty, and a deal-breaker.
  • Back to Win-Hal. Click the I need to make some adjustments box. Log in again. At least this time it remembered my password.
  • There's a Customize Buttons section for the remotes. For the BluRay, set up Blue to be Netflix, Yellow Pandora, and Green for YouTube. A/Red on the BluRay is Blockbuster, which we don't use. For the FIOS box, set A on the screen (not to be confused with A/Red below) to Aspect, which should set the screen size properly. While we're at it, set B up there to Close Caption, something we use a lot since we're old. Do that for the BluRay as well. I may have to tweak that later. Can't find a DVR button option, but I may have a way around that. Let me test out the new setup and get back to you in a minute.
  • OK, those changes work. Nice touch: the buttons attached to the screen are labeled A/B/C..., but you can change those to Aspect/Closed Caption/On Demand, etc.
  • Now let's tackle that DVR button problem. On the setup screen for the FIOS remote (labeled PVR in 650's language) there is a button labeled Learn Command. Click that, and go down until you find Learn a New Command. Type DVR Menu in the box and click the Learn a New Command button. Put the front of the FIOS remote to the back of the 650, press the DVR button on the FIOS remote. Wait. Go back to the Additional Buttons setup screen, and assign the button you want the new DVR Menu command.

And, for now, that's enough. If I encounter more difficulties I'll let you know. In particular, we'll see if the family can use it easily or not.

Preliminary bottom line: This isn't all that easy to program, especially if you have an exotic system. But it does seem to be straightforward to program, and you can add special keys as needed. We'll see how it all works in practice.

Sunday, November 28, 2010

Mounting Remote Computer Files with FUSE

I'm sitting here, upstairs, typing on my laptop. There's a file I want on my office machine downstairs. So I should go downstairs and look for the file, right?

Wrong. The TV up here is on NFL Red Zone ( (sigh) Red Zone). I'm addicted, OK? It's hard enough to break away for a bathroom trip.

Several options exist, of course. I could ssh to the downstairs machine, search for the file, and use scp to copy it upstairs. But the file is a picture, and I while I remember what it looks like, I don't remember what name I used for it. So ideally I'd like to use an image viewer, say gthumb or F-Spot to look through the directory and find the file. But I can't, because this computer doesn't mount the downstairs computer's disk.

Ah, but it can. The classic way to do this is to use the Network File System protocol, aka NFS. But that requires work to set up. Besides, some days I might want to search my computer from work, or from a hotel. It's tricky to get NFS to work that way.

Fear not, there is a solution. It uses File System in Userspace (FUSE), a complicated name for using ssh and scp to fool my laptop into thinking that the downstairs computer's files are on the laptop's local drive.

In Ubuntu the way to do this is to install a package called sshfs. Then you just follow the directions on the linked page. Let's say I want to be able to access all the files in /home/rcjhawk on hal, my downstairs computer, from my laptop, majel:

  • First make sure I have both hal and majel set up to use ssh communications. If I'm away from home, I'll also need to know hal's ip address, and make sure I can ssh to hal from outside my router.
  • Next, install sshfs on majel. Just use
    $ sudo apt-get install sshfs
    from a terminal window on majel. This doesn't need to be done on hal. Hal's just going to see a series of ssh requests.
  • Add myself to the FUSE group on majel:
    $ sudo gpasswd -a $USER fuse
    from the same command window.
  • Log off my account on majel, log back on. Yes, it's annoying, but you're not officially added to the FUSE group until you do.
  • Make a directory on majel. Doesn't matter what the name is:
    $ mkdir ~/hal
  • Now we're going to mount all of hal's directory tree starting from /home/rcjhawk in majel's ~/hal directory:
    $ sshfs -o idmap=user rcjhawk@hal:/home/rcjhawk ~/hal
  • If this works, then running the command
    $ ls ~/hal
    from a terminal window on majel lists hal's directory tree.
  • To unmount hal, just run:
    $ fusermount -u ~/hal

To make things easier on myself, I created two files. The first, mthal, contains the mounting command, and the second, umthal, the unmount command. Make these executable, put them in your path, and it's easy to mount and unmount.

A caveat: I was getting an error remote host has disconnected. That occurred because my .cshrc file, which used a variable that's undefined when you are running ssh commands. You can find out how to diagnose that here.

Does it work? We'll, I started typing this on the laptop, and finished it up on the big machine downstairs. So, yes, it does work.

There are also versions of FUSE for BSD, OpenSolaris, and Mac OS X. In fact, I first learned about this trick using my Mac at work. With FUSE you can mount just about anything anywhere. There's even GmailFS, which transforms your Gmail account into backup storage.

Saturday, November 27, 2010

Finding Your Header Files — A Better Way

I should really read my own posts. Especially since that's the original purpose of this blog.

The other day I was looking through old posts with the Ubuntu label, and found this comparison of dpkg/apt-get and rpm/yum commands. The relevant one is

dpkg -l '*'

which lists all of the packages available in your repository, installed or not.

Duh

This makes my header file lister a lot simpler to write, towit:

#! /bin/bash

# First make sure you've got all your repositories updated:
sudo apt-get update

# Now search through every repository database, looking for lines that
#  start with "Package", and end with "-dev".
# Pull out the package name, look through each package, and print out
#  the header files:  those that end in "-h"

# No matter how it wraps on your screen, this next line starts with
# "for" and ends with "awk '{print $2}'`"

for hfile in `dpkg -l '*' | grep dev | awk '{print $2}'`
do
# echo $hfile
apt-file list $hfile | grep "\.h$"
done

It's still going to take a long time to go through every package, but it's a lot neater now.

Thursday, November 25, 2010

Look Familiar?

I saw this watching the Star Trek reboot on the bigscreen with our Epix OnDemand account. It's during the Old Spock/New Kirk mind-meld, at the part of the story where Spock is transported back in time. The blue box is suggestive, don't you think?

Since Epix lets you stream its movies to a Linux box, unlike Netflix, it was relatively easy to find the shot and save it. No, the resolution's not that great, but, frankly, this movie suffers even on a 48" high-def screen. To do it justice you really need to watch it in a theater.

Saturday, November 20, 2010

Undercutting Yourself

Maybe I just don't understand…

The other day Borders sent me an email which gave me ten dollars in Borders' Bucks. A great promotion, to be sure. And I had a book in mind I wanted to buy, but not spend a lot of money on: Jim Butcher's collection of Harry Dresden short stories, Side Jobs.

First check it out online. We have:

List Price $25.95
Online Discount -$11.22
Borders' Bucks -$10.00
Shipping $3.98
MD Tax $0.29

Total $9.00

Not bad, but the shipping price sucks, it will take days to get here, and anyway I want to get out of the house for awhile, so let's go to a nearby Borders and buy it there.

I get to the Borders, and find Side Jobs. The sticker on it says 20% off. Do a little bit of math: 80% of $25.95 = $20.76 - $10 Borders' Bucks = $10.76, + 6% tax ( $0.65) = $11.41. A bit of a rip, compared to online, but I'm here, I've already spent a buck in gas, and I'll walk out with the book.

Except: I then notice that the sticker also says Borders Rewards Plus Members. I'm just an ordinary Borders Rewards member. I checked with one of the sales people, and no, I'm not entitled to the discount. That makes the cost $16.91, Borders' Bucks and tax included. I made a point of telling the guy his store's website was undercutting the store by selling the book for $5 less, including shipping, put the book down, and walked out.

I suspect some marketing guru has it all figured out, but to me this just seems plain old-fashioned stupid. If Borders gave me the online price at the store, or even got close to it, I would have gotten the book then and there. Since I was in the store, I probably would have had a coffee as well. Maybe a cookie. I would have looked at a Linux magazine while I was sipping my coffee, and possibly would have purchase that, too. I would most likely have spent the $5 in the store, maybe more, and I wouldn't have been pissed.

Instead, I went home and ordered the Dresden book online. Even with the $2 gas I just wasted I still saved $3 over buying it in the store. And, instead of waiting for more coupons/bucks to come my way, I went over to Amazon and ordered two books I would have otherwise eventually gotten at Borders (along with more coffee, cookies, magazines, etc.): Lois McMaster Bujold's newest Vorkosigan novel, Cryoburn, and Terry Pratchett's new Tiffany Aching/Discworld novel, I Shall Wear Midnight. Add in a book ChildII wanted, we made the supersaver price, and got free shipping.

In summary: by charging full price for a book in the store, and a deep discount online, Borders lost the sale of at least two books, two-three cups of coffee, a couple of cookies, and maybe a magazine.

It looks like Borders would rather you didn't come to their store. OK, stores cost money to keep up. Employees have to be paid, not to mention heating, lights, insurance, and all of that. But if I come into the store, I'm going to browse. I'll look around, and might well find something else I'd like to buy. Online, I'm going to get the books I want, hit the send button, and go on with my life.

Like I said, some genius with an MBA probably has figured out that this approach makes more sense for Borders, but how that could be is a mystery to me.

The trip wasn't wasted, I drove a mile to the local wine emporium, tasted a couple of wines, and bought a bottle.

Thursday, November 11, 2010

Unintended Consequences

Article 1, Section 2, of the United States Constitution says, in part, The Number of Representatives shall not exceed one for every thirty Thousand, but each State shall have at Least one Representative. The very first amendment proposed (you can look it up) tried to change this:

After the first enumeration required by the first article of the Constitution, there shall be one Representative for every thirty thousand, until the number shall amount to one hundred, after which the proportion shall be so regulated by Congress, that there shall be not less than one hundred Representatives, nor less than one Representative for every forty thousand persons, until the number of Representatives shall amount to two hundred; after which the proportion shall be so regulated by Congress, that there shall not be less than two hundred Representatives, nor more than one Representative for every fifty thousand persons.

but this was never ratified. So we can be pretty sure that the Founders of this Republic wanted us to have one Representative for every 30,000 Americans.

Currently, however, we have 435 Congressmen, thanks to something called Public Law 62-5. I guess this was supposed to keep the number of Congressmen down to a reasonable number. Otherwise, with the current U.S. population around 300,000,000, we'd have 10,000 Congressmen running around. Think of it! Mark Russell once said something on like every night 500,000 people in the District of Columbia go to bed without a Congressman. But if we had 10,000 Congressmen, you give yours (DC residents excepted) a booty call just about every night.

The current limit of 435 Congressmen also gives rise to some startling inequities. For example, according to the 2000 Census, the total population of the United States was 281,421,906. If we exclude the District of Columbia (population 572,059) since it has no Congressional representation, then there are 280,849,847 Americans and 435 representatives, so each member of Congress should represent 645,632 people. However, Wyoming (493,782) and Montana (902,195) each have only one representative. To put it another way, Cynthia Lummis only needs to attract half as many votes as Denny Rehberg to get elected, but each gets one vote in the House. So by moving from Wyola, Montana to Parkman, Wyoming (16 miles) you can effectively double the strength of your vote for Congress.

There is a lot more information about the inequities of the current system, and the sorry history of Amendment the First, at thirty-thousand.org. I'd recommend downloading their pamphlet, Taking Back Our Republic, for more information.

One thing they didn't cover, though: the effect of all of these changes on the Electoral College. Each state gets N+2 electoral votes, where N is the number of representatives, plus the two members of the Senate allowed each state. That +2, along with Public Law 62-5, weights the Electoral College heavily toward the smaller states. Since D.C. does have 3 votes in the Electoral College, each elector theoretically represents 281,421,906/538 = 523,098 voters. However, each state has a minimum of three electoral votes. So in Wyoming each elector represents 164,594 people. In Montana, it's 300,732 people, still substantially below the national average. It's likely (I haven't checked in detail) that the state with the most people/elector is California, where there are 33,871,648 people and 55 electors, or 615,848 people/elector. Put it this way: an elector in California represents four times as many people as one in Wyoming, but each has the same say in who's elected President.

Now, as anyone can tell you, most of the smaller states, population-wise, are in the west, and most of those states vote Republican, while many of the larger states tend to vote Democratic. Another wrinkle in the system is that most states (Maine and Nebraska excepted) cast their electoral votes in a block.

Couple these two facts with the 30,000 people/representative rule and you get some interesting results. Very interesting, in some cases.

In particular, let's consider the 2000 Presidential Election. Very close, as we all recall. Al Gore got 50,999,897 votes, while George Bush got 50,456,002, but because most states cast their electoral votes as a block, Bush won in the Electoral College, where it counts, 271-266.

What if we'd apportioned Congress the way the Founders intended? And kept the current winner-take-all (mostly) electoral system? The table below shows the results. I constructed it using the following rules:

  • Winner take all in each state, as now. Maine (42 votes under these rules) went to Gore, and Nebraska (54) went to Bush. Changing all of Maine's votes to Bush or all of Nebraska's to Gore isn't going to change the final result, so it doesn't matter if several congressional districts shifted from Red to Blue or vice versa.
  • To get the number of representatives, I took the states' populations from the 1990 census, which governed apportionment at the time, divided by 30,000, and rounded down. Rounding to the nearest integer, or rounding up, can't change the electoral vote by more than 51, and so won't affect the result, either.
  • If the District of Columbia was a state, it would have 22 electoral votes in this scheme. However, by the Twenty-Third Amendment, D.C. gets 17 electoral votes, equal to the state with the fewest, Wyoming. In the real 2000 election one D.C. elector abstained. Even assuming all 17 of the new electors abstained it wouldn't change the bottom line here, so I assume that they all voted — for Gore, naturally.

The result: Since each state has more or less the same representation per capita in the electoral college, and since most of the larger states went for Gore, he wins the election, 4,314–4,047.

Think about it. Al Gore, Forty-Third President of the United States. Somehow, I don't think that's what those currently saying we should go back to the Principles of the Founders had in mind.

State Population Congress Electors Bush Gore Bush Gore
Popular Popular Electoral Electoral
Alabama 4040587 134 136 941173 692611 136 0
Alaska 550043 18 20 167398 79004 20 0
Arizona 3665228 122 124 781652 685341 124 0
Arkansas 2350725 78 80 472940 422768 80 0
California 29760021 992 994 4567429 5861203 0 994
Colorado 3294394 109 111 883748 738227 111 0
Connecticut 3287116 109 111 561094 816015 0 111
Delaware 666168 22 24 137288 180068 0 24
DC 606900 0 17 18073 171923 0 17
Florida 12937926 431 433 2912790 2912253 433 0
Georgia 6478216 215 217 1419720 1116230 217 0
Hawaii 1108229 36 38 137845 205286 0 38
Idaho 1006749 33 35 336937 138637 35 0
Illinois 11430602 381 383 2019421 2589026 0 383
Indiana 5544159 184 186 1245836 901980 186 0
Iowa 2776755 92 94 634373 638517 0 94
Kansas 2477574 82 84 622332 399276 84 0
Kentucky 3685296 122 124 872492 638898 124 0
Louisiana 4219973 140 142 927871 792344 142 0
Maine 1227928 40 42 286616 319951 0 42
Maryland 4781468 159 161 813797 1140782 0 161
Massachusetts 6016425 200 202 878502 1616487 0 202
Michigan 9295297 309 311 1953139 2170418 0 311
Minnesota 4375099 145 147 1109659 1168266 0 147
Mississippi 2573216 85 87 572844 404614 87 0
Missouri 5117073 170 172 1189924 1111138 172 0
Montana 799065 26 28 240178 137126 28 0
Nebraska 1578385 52 54 433862 231780 54 0
Nevada 1201833 40 42 301575 279978 42 0
New Hampshire 1109252 36 38 273559 266348 38 0
New Jersey 7730188 257 259 1284173 1788850 0 259
New Mexico 1515069 50 52 286417 286783 0 52
New York 17990455 599 601 2403374 4107697 0 601
North Carolina 6628637 220 222 1631163 1257692 222 0
North Dakota 638800 21 23 174852 95284 23 0
Ohio 10847115 361 363 2350363 2183628 363 0
Oklahoma 3145585 104 106 744337 474276 106 0
Oregon 2842321 94 96 713577 720342 0 96
Pennsylvania 11881643 396 398 2281127 2485967 0 398
Rhode Island 1003464 33 35 130555 249508 0 35
South Carolina 3486703 116 118 785937 565561 118 0
South Dakota 696004 23 25 190700 118804 25 0
Tennessee 4877185 162 164 1061949 981720 164 0
Texas 16986510 566 568 3799639 2433746 568 0
Utah 1722850 57 59 515096 203053 59 0
Vermont 562758 18 20 119775 149022 0 20
Virginia 6187358 206 208 1437490 1217290 208 0
Washington 4866692 162 164 1108864 1247652 0 164
West Virginia 1793477 59 61 336475 295497 61 0
Wisconsin 4891769 163 165 1237279 1242987 0 165
Wyoming 453588 15 17 147947 60481 17 0
Totals     8361 50455156 50992335 4047 4314

Sorry about the lack of commas in the table above. I used a very primitive spreadsheet to generate the data.

1990 Census data is from http://lwd.dol.state.nj.us/labor/lpa/census/1990/poptrd1.htm. Presidential vote totals are from http://www.presidency.ucsb.edu/showelection.php?year=2000.

Monday, November 08, 2010

Find Every Header File in Your Ubuntu Repositories

Last night I was following a thread on Penguin Pete's blog where Pete uses Ubuntu's dropping the Dillo browser from the distribution as a launching pad for a discussion about — heck, I can't describe what it's about, read it yourself.

Anyway, in the course of reading the discussion — if that's the word, Pete gets worked up about certain things — I was possessed by the moral imperative to install Dillo from source.

This would seem to be straightforward. You go to the Dillo home page, click Download, and follow the instructions.

Except (there's always an except) — the first install you have to make is the FLTK2 (Fast Light Toolkit) library.

Now Ubuntu offers FLTK1.1, which might well work, but hey, we need the latest and the greatest, right? So I downloaded the tarball, followed the instructions, and the make command crashed and burned with:

fatal error: X11/extensions/XInput.h

I looked on the web, found that XInput.h is in the package libxi-dev, installed that, reran make,

fatal error: GL/glu.h

and so on. It took about an hour to find every single header file needed. And that was just to install FLTK2, after which I still had to install Dillo.

So what's going on here? Well, first we have to know what a header file is. Basically, it's a file that defines a bunch of variables and statements that can be used by multiple routines. See Wikipedia for a better explanation. In particular, header files tell a program's source code what variables are used in a particular library. Thus the header XInput.h defines variables available in some X11 libraries.

OK, suppose I'm compiling a program and I find that I need a header file GrantsTomb/Buried.h. All I have to do is find the package that has that header, install it with apt-get, and I'm good to go. Right?

Well, without prior knowledge you're pretty much doomed here. I was able to find XInput.h and glu.h by web search, but who's going to tell you where is Buried.h in GrantsTomb?

Yet there has to be a solution, because:

  • I know that when I use synaptic or aptitude to install packages I can see every package that is, or can be, installed on my system from my selected repositories.
  • After searching around a bit, I find that apt and its children list the available files in /var/lib/apt/lists, each repository has its own file, and each file name is on a line starting with Package.
  • I know about the program apt-file, in particular that the command
    apt-file list foobar
    will give me a list of all of the files in the foobar package.
  • Header files are traditionally located in packages ending with the string -dev.

Given all of that, I should be able to write a script which will list every header file available in every Repository I've accessed. I can then save it in a file, and the next time a program says I need bojangles/tambourine.h I can just search the list, find that the header file is in the NittyGritty-dev package, load that up, and I'm ready to compile:

  • First, install apt-file, if you don't have it already:
    sudo apt-get install apt-file
  • Then run this script:
    #! /bin/bash
    
    # First make sure you've got all your repositories updated:
    sudo apt-get update
    
    # Next sync apt-file's database.  Note that if you do this without
    #  the sudo you'll write the data into your home directory, which is
    #  probably OK
    
    sudo apt-file update
    
    # Now search through every repository database, looking for lines that
    #  start with "Package", and end with "-dev".
    # Pull out the package name, look through each package, and print out
    #  the header files:  those that end in "-h"
    
    #! /bin/bash
    
    # No matter how it wraps on your screen, this next line starts with
    # "for" and ends with "xargs`"
    
    for hfile in `find /var/lib/apt/lists -type f -exec grep '^Package: ' {} ';' | awk '{print $2}' | grep -e '-dev' | xargs`
    do
    apt-file list $hfile | grep "\.h$"
    done
    

Be warned, this takes a long time, since every call to apt-file goes out over the network. On my system, with FIOS broadband, it took over an hour. So you'll only want to run this file once in a while, probably only when you update your repository list. You may also get an error message like
grep: /var/lib/apt/lists/lock: Permission denied
Don't worry about that.

This script should work pretty well for Debian. There are also equivalent commands for RPM-based distributions. And, of course, you can modify it to list any subset of files in any subset of repositories you want. I'll leave all of that as an exercise for the reader.

Thursday, October 28, 2010

Nature Publishes Science Fiction

As in The Greatest Science Fiction Story Ever Told, by Eric James Stone.

Nature 467 , 1146 (28 October 2010) doi:10.1038/4671146a

Saturday, October 23, 2010

An Epix Decision

We've had Verizon FIOS for nearly three years and have been pretty happy with it. So last week we finally decided to drop our final link to twentieth-century audio-visual technology, the copper-wire phone line, and rely on the FIOS line and cell phones.

Since this meant we were going to be using FIOS for a while, I decided to call up Verizon and see what kind of a deal they could give me if we locked in for a couple of more years. Primarily, I was looking to save money and add Showtime, so that we could watch Dexter, Weeds, and Nurse Jackie without waiting for the DVDs to get to Netflix. (Yes, these are all shows about, shall we say, socially challenged individuals. Deal with it.)

Well, I didn't save any money — well, I will if we drop HBO, but otherwise not. But such a deal!. In addition to Showtime, we get all of the HD sports channels turned on, the regional Fox sports channels, and some other HD movie channels.

In particular, we now have something called EpixHD. This is a joint venture by Paramount, MGM, and Lionsgate to make a few more dollars off of their movies before Netflix buys about a million DVDs and starts sending them out essentially for free.

So here's the deal. Epix streams movies such as Star Trek, Iron Man, etc., starting about 90 days before Netflix gets streaming rights. In addition, Epix streams to a Linux computer, unlike Netflix.

Furthermore, after a little bit of searching I found that HBO also streams its shows, even to a Linux box, although I had to do the sign-up from Windows.

And, finally, even Showtime is coming around to streaming its shows, although right now it's apparently only available to Comcast customers through something called Fancast.

So here's the dilemma: I can watch movies and shows from HBO, Showtime, and Epix using Verizon's OnDemand service. I can also stream HBO and Epix to my Linux, Mac, or Windows boxes. So do I really need to keep Netflix?

Look, I love the fact that Netflix lets me watch ancient episodes of Dr. Who and things like Doctor Horrible's Sing-Along Blog, but I really don't watch it all that much. It's useful that I can get a copy of KU's 1988 NCAA championship DVD when I want it, but, really, how often is that? Is it worth another $10/month?

And, really, I'm upset with Netflix, which won't consider streaming to Linux. They say it's a DRM issue, but if the studios that make up Epix can solve that to their satisfaction, why can't Netflix?

For now, I guess, we'll keep Netflix, but we'll see for how long.

Saturday, October 02, 2010

Remote Login to the Command Line

The other day I was away from home when my wife called up and asked me where I'd put the papers I'd promised to print out.

I let out a very soft damn, because of course I'd forgotten to print out anything.

But all was not lost. I simply logged onto my home computer, found the appropriate file, and printed it, on my home printer. She was suitably impressed.

This, of course, is the whole point of a long-ago post, where we figured out how to determine a router's external IP address. What we want to do today is to use that information to log onto one of your home computers from afar. We'll assume you have figured out your current dynamic IP address, perhaps by looking at the bottom of your gmail page, or using the software I assembled in the previous post.

So let's say that you've found that your router currently has the dynamic IP address aaa.bbb.ccc.ddd . Then, in principle, all you have to do to log onto your account at home would be to open up a terminal and type

ssh rcjhawk@aaa.bbb.ccc.ddd

where you'd replace rcjhawk by your own account, of course. Then, if you have ssh set up properly, you'll be logged to a CLI on your home machine.

In the usual home setup, with many computers connected to a router with only one external IP address, this will never work out of the box. First, the router doesn't (usually) do ssh on its own, and if it did there wouldn't be any useful information for you there, unless you wanted to trash fix your own network. Second, the router doesn't know which machine you want to talk to. And third, even if it did, that machine might not be accepting ssh requests (and won't, if you've followed my previous recommendation).

So what do we need to do? Let's see: First, we need to tell the router which computer gets the ssh session. Then, we need to set that computer up to receive ssh commands. Finally, if we want to be able to contact multiple computers, all behind the same router, we have to figure out how to do that, as well.

Setting up the router

First things first: how does the router pick a computer for ssh? Well, though Wikipedia tries to explain it in technical detail, basically your computer connections to the outside world are controlled by a set of 65536 (or 216) ports. If you like, you can consider each port as a connection in an old-style telephone switchboard. When you call up a specific service on a computer, you ask to be hooked up to a specific phone number.

Some of these phone numbers, or ports, are customarily assigned to specific services. For example, port 80 is usually reserved for a web server. It doesn't have to be that way, though, you could start your web server running on port 31526, if you liked. The problem is that everyone's browser would be looking for your server on port 80 of your computer, and you wouldn't get many hits. (If that's what you desire, go for it.)

ssh, it turns out, is typically assigned port 22. So when I run

ssh rcjhawk@aaa.bbb.ccc.ddd

the computer I'm on asks the operator (my home router), to connect it to port 22 so that it can talk to whoever is there. If that contact speaks ssh, then we can communicate. If, however, the contact is speaking telnet, or ftp, we're going to have problems.

The router, as we said above, knows nothing about ssh, or at least doesn't want to talk to the unwashed masses of the Internet. It has to pass the call on to a computer that I've designated to receive ssh requests. This works by what is known as Port Forwarding. This tells that router that whenever someone comes calling at port 22 it is to pass that signal along to a specific computer.

How do you do port forwarding? Well, that depends on your router, your firewall, if any, and maybe on your modem. I can't give general directions, though PortForward.com might be able to help. There is also a YouTube instructional video, and you ought to be able to find specific instructions for your own router/firewall.

Since I'm on Verizon FIOS here at the house, I have a Verizon MI424WR modem/router. There is an old set of instructions for port forwarding online, but the software seems to have changed. That did provide a starting point, however, and I've figured how how to set up forwarding on my current setup:

  1. Point your Browser at http://192.168.1.1/
  2. Click Firewall at the top of the page.
  3. Click Port Forwarding on the left.
  4. You see a line that says Create new port forwarding rule: and below that a box that says IP Address forward to or select from menu. Click on that, and then click on the machine you want to pipe your ssh requests to.
  5. Click on the box labeled Application to forward … and select SSH. When things settle down, click apply.

Setting up your home machine

This is fairly straightforward. Let's assume that we've instructed the router to forward port 22 to hal, here. On hal, we've followed all the instructions to get the ssh daemon running, so that we know what to do with incoming ssh requests.

Currently, however, we have the line
ALL: ALL
in our /etc/hosts.deny file, and
ALL: 192.168.1.0/255.255.255.0
in /etc/hosts.allow, which means that only computers on the local network can ssh to hal. One way to fix this would be to add
sshd: ALL
to /etc/hosts.allow. This allows hal to receive ssh requests from any machine, anywhere. Another way to do it would be to specify those external machines that can access hal by host. For example, if I worked for Google, the line might read:
sshd: 72.14.0.0/255.255.0.0
or some-such, letting me log in from any computer with an ip number 72.14.xxx.yyy.

Once we have your home computer's /etc/hosts.allow file properly set up, then running
ssh youraccount@aaa.bbb.ccc.ddd
from an allowed machine, where youraccount is the account name on your home machine, and aaa.bbb.ccc.ddd is your router's current IP address, should get you in, at least after you enter the appropriate password.

Other machines on your network

The problem with all of that is that I can only access hal by this method. Suppose I want to access another computer, fred, instead. I could, of course, get into my router and change port 22 forwarding from hal to fred, but that's difficult to do from the road, unless I allowed my router to be administered remotely.

However, the port/service assignment is more of what you'd call a guideline. We can, in fact, use any port in a storm for ssh services. In the example above the Verizon router had presets for ssh using port 22, but we could have used any other port instead.

Again, the exact way of doing this will vary from router to router, but to set up forwarding of an arbitrary port to a machine with the Verizon router,

  1. Point your Browser at http://192.168.1.1/
  2. Click Firewall at the top of the page.
  3. Click Port Forwarding on the left.
  4. Under IP Address … click on your target machine.
  5. Under Application click Custom Ports. When things settle down you'll have a bunch of boxes to click.
  6. Under Protocol select TCP.
  7. Under Source Ports select Any.
  8. Under Destination Ports select Specify, and then put in your selected port. In this case we'll use 1022. We don't have to use 1022, but it's simple to remember, once we know that ssh ordinarily uses port 22.
  9. Click on Apply and you're done here.

Now go back to your target machine. Open up a terminal window. Then,

  1. Run
    sudo vi /etc/hosts.allow
    and edit the file to allow sshd access from your favorite machines, just as you did with the first machine.
  2. Run
    sudo vi /etc/ssh/sshd_config
    Look for a line that says Port 22
  3. Below that line add the line
    Port 1022
    or whatever other number you've chosen.
  4. Stop and restart the ssh daemon by running the command:
    sudo /etc/init.d/ssh restart
  5. Now go over to your remote machine. Run
    ssh youraccount@aaa.bbb.ccc.ddd -p 1022
    if 1022 is the port you've chosen. You should be logged on to the second machine in your network.

That concludes the tutorial on accessing your home machines' command lines. Ubuntu also has a remote desktop feature, which I've yet to use. When I get around to trying it out we'll put that online here as well.

Saturday, September 25, 2010

Bum-Bum-Bum-Bum-BUM-Bum

I liked Boomer better than I like Kona.

Which is a short way of saying that if you're watching the re-make Hawaii Five-0 in order to see more of Grace Park, you're better off getting all the Battlestar Galactica disks from Netflix and watching them. Boomer/Sharon/Athena/Eight was interesting and sexy. Kona looks like a school girl. A painfully thin school girl.

To put it mildly, I'm unimpressed by the whole Five-0 experience. At the start, they do keep the same theme song, how could they not, but they make the images flash way too fast, and cut it short as well.

Then there's McGarrett. Not my idea of a NCIS/Seal/Cop. Well, maybe a Don Johnson-style cop, since he always needs a shave. But Alex O'Loughlin is no Jack Lord. Or Mark Harmon, for that matter.

I did like Danno. The first thing I thought was He reminds me of James Caan. Then I look up Scott Cann, and find that he's James Caan's son. This is one improvement over the old show, the old Danno was a wimp.

And I love the makeup work they did on Jean Smart, who plays the governor. She's my age, but they managed to make her look old.

OK, this is a meander, and not a review, but lets see — what was good and bad about the episode itself:

Good: There's an underlying mystery here, something only hinted at in the first episode. Sort of like in last year's The Good Wife, where we learn that there's some deep dark secret in Chicago politics (who knew?)

Bad: Jean Smart's roll is apparently going to be along the lines of:

McGarrett: You gave me unlimited authority!

Governor: And you exceeded it!

Implausible: McGarrett and Danno attack what is identified as a Chinese cargo ship. While there, they are attacked by a variety of bad guys, including the episode's main villain. They win easily, of course. In the entire scene we see not one person of Asian extraction.

I'll probably watch a few more episodes, as time permits, either on Hulu or with FIOS On Demand, but this isn't going to be one of those shows that's automatically DVR'd.

Friday, September 24, 2010

Ubuntu and Flash

I've been running the 64-bit version of Ubuntu 10.04 for about five months, and have had few problems. Most of them you've heard about. Lately, though, I've had a major problem — Ubuntu's unbranded version of Google Chrome, aka chromium-browser, wouldn't run Flash.

If I tried to visit a flash site, I'd get something like this:

Annoying.

The apparent cause is a corrupted profile. Exactly what's corrupted, I couldn't say. Basically, to get Flash running again you need to wipe out all traces of chromium and start over. I did this, but I wanted to at least save my bookmarks. Here's how. Open a terminal and type in everything to the left of the # signs:

cd # Go to your home directory
mv .config/chromium chromium_old # save your old configuration, at least for now
chromium-browser # start the browser up.  Exit immediately.
cp chromium_old/Default/Bookmarks* .config/chromium/Default # restore you old bookmarks
chromium-browser # surf normally.

Of course, you've lost all your saved passwords, surfing history, etc. Think of it as a way of weeding out the chaff. A project, for someone who has the interest and time, would be to go through each file in the old profile and determine which one got corrupted. If you do that, feel free to let us know what you find.

Saturday, September 18, 2010

Back to Blogging

Well, not really, just another placeholder/bookmark entry. It was a busy summer, and now I'm coming up on Fiscal New Year's Day.

From our Department of Procrastination: I owe y'all a follow-up to Finding Your External IP Address, where I show you how to use the thing, i.e., log onto any computer in your house from anyplace else in the Universe. This is half-written, so maybe in a week or two it will actually get posted.

Book Reviews: I really want to mention the greatest history book ever written, Lies My Teacher Told Me by James Loewen. I lost the first edition long ago, but a second, substantially rewritten, edition is now out. Find it, buy it, read it. You won't agree with everything, but you'll agree that it's relevant.

Another one: Human Smoke, by Nicholson Baker. It's just a collection of quotations and stories from the end of World War I to the end of 1941. No one, and I mean no one, comes off well here. None of your heroes, and certainly none of your villains. You'll have to read it slowly, because it's just too damn depressing to read all at once. But read it. It covers that part of history that Loewen complains is left out of history books.

Tips and tricks: A recent update to my Chromium browser (Google Chrome, if you're using a branded version) left me without a bookmark toolbar. Annoying. Reading through this entire comment, and noting that I don't seem to have an option for this under the wrench, I found that Ctrl-Shift-B toggles the toolbar.

We'll get back to real live articles Real Soon Now. I hope.

Sunday, September 05, 2010

Virtual Windows

Or virtual anything else: I've been playing around with the open source edition of VirtualBox, a utility from Oracle (boo!) that lets you run another operating system in a Window in your current operating system.

Any host operating system: Linux, Windows, OS X. Any target OS: Linux, Windows, OS X — well, maybe you'd better ask Steve about that. Tell him I said it was OK.

The open source version of VirtualBox is included in the Ubuntu repositories, so install it with:

sudo apt-get install virtualbox-ose

On your Gnome menu you'll find and entry for VirtualBox OSE under Applications => Accessories

Click New and follow the directions. I was able to install Windows XP without the usual hassles. VB provided all the needed drivers.

OK, one problem. The virtual machine wouldn't boot. The problem turned out to be that VB was looking for a floppy disk drive, which Hal doesn't have. I had to click Settings => System, and then unclick the Floppy Drive label under Boot Order. After that, everything worked fine.

I was even able to stream Netflix movies in a browser under Windows, once I installed Silverlight.

Useful for the very few times I need Windows.

Thursday, July 29, 2010

Updating the un-updatable

In my post on the relative easy of updating Linux I used Ubuntu's graphical Update Manager to do the work.

Of course, in Linux/Unix you can always do this from the command line. In Debian based systems the relative commands are:

sudo apt-get update
sudo apt-get upgrade

The first line compares the packages available online with those installed on your machine, and the second line upgrades the packages that aren't current. Easy, huh?

Except that sometimes you get messages like this:

The following packages have been kept back:
  linux-generic linux-headers-generic linux-image-generic

This is more or less a are you sure you want to do this now? message. You'll note that the kernel is involved here, so you're going to have to reboot the machine once the packages are installed.

All right, I'll reboot, so what do I have to do? Well, it turns out the the packages can be explicitly installed:

sudo apt-get install linux-generic linux-headers-generic linux-image-generic

This will take care of everything. Just remember to reboot the computer to get the new kernel up and running.

Monday, July 26, 2010

This Week's Sign of the Apocalypse

Since time began, classes at the University of Kansas have ended with the sound of a steam whistle which tooted for five seconds at 50 minutes past the hour.

As a budget cutting measure, the steam whistle, Tootie-Toot (never heard that name when I was there), now only sounds for 2.5 seconds.

The 2.5 second version is at the end of this clip:

Saturday, July 24, 2010

The Update Log

I've spent most of July out of town/state/country, so I haven't been very diligent about updating all the computers. Today, though, I took a stab at it:

I. Windows Vista

We have a Windows Vista desktop. It's slated to be upgraded to Linux after Child2 moves files off of it to the new Macbook. For now, though, it runs several programs that said child uses, e.g. Photoshop, and has a full set of browsers (IE, Firefox, Chrome, Safari) that I use to judge how new web pages look to the masses.

OK, here's everything I needed to do to update that machine:

  • Update Virus Definitions: We have McAfee installed, primarily because we bought this machine before I knew about Microsoft's Windows Security Essentials. It's not smart enough to know that it should autoupdate virus definitions if the computer's been off a long time, it just gives you a Dire Warning of Impending Doom.
  • Update Firefox
  • Update Thunderbird
  • Update Google Chrome
  • Update Safari: This one popped up on its own, offering updates to both Safari and iTunes.
  • Check Flash Version: Up to date
  • Perform Microsoft Updates: At least these were done in the background, all I had to do was reboot.
  • Reboot: Note that three (3) programs asked for permission to reboot: Windows itself, Apple's Safari/iTunes, and McAfee
  • Run Virus Scan: otherwise it wouldn't do it until late tonight.

II. Mac OS X

This is my work laptop.

  • Run Software Update
  • Run Microsoft Autoupdate: to check if any Office components needed to be updated.
  • Update Firefox
  • Update Thunderbird
  • Update Google Chrome
  • Check Flash Version
  • Check for updates to Aqua Emacs
  • Run sudo port upgrade: to check for updates to Macports programs.
  • Virus check: I'm required to run a virus checker by the powers that be, never mind that the next Mac virus in the wild will be the first I'm aware of. Fortunately, on the Mac McAfee is smart enough to do all its work in the background, without the warnings.
  • Reboot to take care of the system updates

III. Linux machines

  • System > Administration > Update Manager > Check > Install Updates: then wait a bit
  • Reboot: to activate the new kernel

Now. Will some one please remind me how much easier it is to run Windows or Mac OS X as opposed to Linux? Because sometimes I forget.

Wednesday, July 21, 2010

Why People Don't Read History

It's just not believable.

Stolen from a comment in Contrary Brin.

I've been on vacation. More actual Linux stuff this weekend, I hope.

Saturday, June 26, 2010

Finding Your External IP Address

For various reasons you might want to find your public IP address, that is, the 32-bit number, expressed in pseudo-decimal form as something like 72.14.204.99, that tells the world how to reach your computer.

For most businesses and government offices this is trivial: your computer is assigned a static IP address, one that never changes with time. For home users, though, it's a little more complex.

Start with the IP address itself. The largest possible IP address is 255.255.255.255. That means that you can have no more than 4,294,967,296 unique IP addresses, and some of those are reserved for local use, as we'll see. We're coming up on 7 billion people in the world. Not everyone has a computer (yet, and remember iPhones, iPads, Kindle, and all 'Droids count), but many people have more than one (see iPhones, iPads, …). So there aren't enough IP addresses to go around. Eventually, everyone will switch over to the IPv6 address scheme, which will give us 340,282,366,920,938,463,463,374,607,431,768,211,456 (2128, or 3.4×1038) possible addresses, which is probably enough for every non-hydrogen nucleus in the observable universe to have its own address.

But we're not there yet, so something has to be done to shrink the address space. Your local ISP solves this problem by 1) charging you extra to get a static IP address; 2) giving you only one IP address per gateway (your cable, FIOS, DSL, or dial-up modem, or some other router or hub that connects you to the internet); and 3) taking your IP address away from you when you're not using it.

That last one is called dynamic IP addressing, and it's part of what we're going to talk about today. If you have a dynamic IP address, then when you turn your gateway connection (usually some type of modem or router) on, your ISP assigns you an IP address from its bank of addresses. As long as your connection is turned on, you'll have an IP address, but your ISP can change it at any time. It will almost certainly change it if you turn off your connection and turn it on again. So if you know your IP address right now, there's no reason to expect it will be the same tomorrow, or even five minutes from now — though to be fair, Verizon FIOS has kept our house on the same IP address for weeks.

And there is worse to come: Suppose your ISP has given you the address 244.117.16.25. The computer you're reading this on doesn't have a clue as to what that address is. Go back to 2), above. Your house has only one IP address, and it belongs to your gateway. Every other computer in the house belongs to your home network. When you want to download your email, your computer politely requests the gateway to go fetch it:

Please, sir, may I have my email?

Oh, all right. Just this once. But don't tell any of the other computers what we're doing.

Thank you, sir! I'll be eternally grateful

You'd better be, or no Internet for you.

Let's see how this works. ping is a command that will poke a stick at a computer and get information about it. Let's try it with a well known site:

ping google.com
PING google.com (72.14.204.99) 56(84) bytes of data.
64 bytes from iad04s01-in-f99.1e100.net (72.14.204.99): icmp_seq=1 ttl=53 time=21.8 ms
64 bytes from iad04s01-in-f99.1e100.net (72.14.204.99): icmp_seq=2 ttl=53 time=20.2 ms
64 bytes from iad04s01-in-f99.1e100.net (72.14.204.99): icmp_seq=3 ttl=53 time=23.4 ms
^C

(you'll need to hit ctrl-C to stop this)

So we see that google.com has an IP address of 72.14.204.99 — that's actually one of many, but don't worry about that. Now let's try it with a local machine:

ping george
PING george.home (192.168.1.2) 56(84) bytes of data.
64 bytes from george.home (192.168.1.2): icmp_seq=1 ttl=64 time=9.77 ms
64 bytes from george.home (192.168.1.2): icmp_seq=2 ttl=64 time=2.70 ms
64 bytes from george.home (192.168.1.2): icmp_seq=3 ttl=64 time=0.961 ms
^C

So far has the home network is concerned, george lives at 192.168.1.2, which is one of those reserved addresses we mentioned above. The range 192.168.0.0 to 192.168.255.255 is reserved for internal networks. As we read this, there are no doubt hundreds of thousands of computers with the local IP address of 192.168.1.2. Every one of them, however, has a different public IP address.

Great. But sometimes I need to know my external IP address, and it's apparent that Hal, here, hasn't got a clue as to what it is. How can I find it?

The easiest way is to go to another website. Every website you connect with gets your public IP address automatically, unless you use some kind of anonymous service. Some websites are set up to tell you what your IP address is. One example is What Is My IP?. They even provide a way to get your IP address with a simple one-line terminal command:

wget www.whatismyip.com/automation/n09230945.asp -O - -q ; echo

If you read your Gmail via Thunderbird, Evolution, Outlook, etc., and keep your web client on all day, gmail will tell you the address of your computer. Just log on to your gmail account and look for the line that says Last account activity: …. The IP address there is the last computer to download email. If you're using more than one computer from different locations, say you left your office machine on over the weekend and now you're using a home computer, click on the Details link and you'll find all of the activity for this account.

There's a company DynDNS, which will not only find your IP address, but give you a domain name that constantly updates to your current numerical address. Of course, they do this by having your computer constantly tell them the public IP address.

And, of course, your gateway knows its public IP address. Just log onto it through your browser (something like http://192.168.0.1, http://192.168.1.1, or http://192.168.100.1, depending on your system), and your IP address will be up there someplace.

But all of these systems have flaws: Say you need your IP address in a script. You can get it via your browser from your gmail or your gateway machine, but it's hard to parse a browser page inside a script. The What's My IP? one-liner works in a script, but what if their site goes down, or, worse, they go out of business? Same for DynDNS. We want a solution that works any time your gateway is on and your computer has an internet connect, and we want to get it from the command line.

Any other way just ain't elegant.

If you search the Internet you find surprisingly few ways (i.e., none) to do this. Partly I think it's because there isn't a consistent way to find your gateway from your current machine. For example, in the days when we used Comcast, we had a cable modem as the gateway. On the house side of the modem we connected a Linksys router, which was the hub of our home network. All the computers in the house were connected through the router to the modem — a two step-process. Now we have Verizon FIOS, with a Actiontec MI424WR router that serves as gateway/modem and router — a one-step process.

By stumbling around, and with a little help from nixCraft, I was able to come up with a solution that works for me. It's a bash script. Let me put it here (finally, I hear you say), and we'll dissect it below:

#! /bin/bash

# Determine the public IP address for your DHCP system by bootstrapping

#  calls to your gateway/router

# Requires "route," available in Ubuntu package net-tools, and
# nslookup, available in Ubuntu package dnstools

# Disclaimers:
# 1) This works with Verizon FIOS using the Actiontec MI424WR Router
#    If you have another system YMMV.
#
# 2) And, as you'll no doubt note, this is set up for a Debian system,
#    specifically Ubuntu 10.04 .  Package names and installation
#    procedures for other systems WILL vary.
#

###
# Start Here:
###

#
# Get Router internal Gateway IP
# Information found on
# http://www.cyberciti.biz/faq/how-to-find-gateway-ip-address/
# If route is not installed, run
# sudo apt-get install net-tools
# and make sure /sbin is in your $PATH
# Note "U" = up, and "G" = gateway

internalip=`route -n | grep UG | awk '{print $2}'`

echo Your Gateway\'s Internal IP Address is: $internalip

# Now we need to get the name of the router.  If you don't
#  have nslookup installed, run
# sudo apt-get install dnsutils

echo -n "Your Gateways's name is: "

routername=`nslookup $internalip | grep "name =" | awk '{print $4}'`

# On the Actiontec, at least, the name has an annoying "." at the end.
# Remove it:

routername=${routername%\.}

echo $routername

echo -n "Your Public IP Address is: "

nslookup $routername | grep Address | tail -1 | awk '{print $2}'

OK, what's going on? First of all script only works if you have the route and nslookup commands installed on your system. If they aren't there, in Ubuntu, and presumably and Debian system, you can install them using

sudo apt-get install net-tools dnsutils

The first call, route -n, will look at the Kernel's IP routing table:

$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.1.0     0.0.0.0         255.255.255.0   U     1      0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 eth0
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth0

which on this machine tells us that we have several local connections. The U under Flags means that the connection is up, and G means it's the gateway. From this we see that the numerical address of the gateway is 192.168.1.1. Let's see what nslookup can tell us about that. Note that the script has to parse all of the output to get this far.

$ nslookup 192.168.1.1
Server:  192.168.1.1
Address: 192.168.1.1#53

1.1.168.192.in-addr.arpa name = Wireless_Broadband_Router.home.

Here's something new, our gateway's got a name. Wireless_Broadband_Router.home. Again, the script has to do a little work to tease that name out of nslookup's output, but we'll just do it by hand here:

$ nslookup Wireless_Broadband_Router.home
Server:  192.168.1.1
Address: 192.168.1.1#53

Name: Wireless_Broadband_Router.home
Address: 192.168.1.1
Name: Wireless_Broadband_Router.home
Address: xxx.yyy.zzz.aaa

The first address, 192.168.1.1, is just the router's internal IP address. The last address (Of course I'm not giving you my IP address. I'm not daft, you know.) is the one we want. xxx.yyy.zzz.aaa is the external address of this machine.

Of course there are many, many, problems with this script.

  • So far it only works with the Verzion/Actiontec setup. A friend, running Ubuntu, wasn't able to pick out his gateway machine with the route -n command.
  • Some machines can have multiple gateways.
  • The Mac's route command doesn't work in the same way as under Linux, so you can't parse out the gateway, even if you have none.
  • You might be able to tease the gateway IP address out of the /etc/resolv.conf file. I don't know what happens if you have multiple gateways here. Under Ubuntu, the one numerical gateway is 192.168.1.1, on the Apple it gives me 192.168.1.1 and xxx.yyy.zzz.aaa, my public IP address.

If you try the script, leave a comment below. Let me know your internet setup (gateway/router), if it works, and any tweaks you did to make it work. Maybe we can come up with a universal script.

Now, by all that's geekly, why did we do all of this? That's the subject of an upcoming post, boys and girls …

Saturday, June 19, 2010

Smart ssh tricks

I want my computers to talk to one another.

More specifically, I want to be able to remotely log in to one of my computers from another one, and I want to be able to easily copy files from one computer to another. The easiest, most secure, way to do that these days is to use the Secure Shell protocol, aka ssh.

Examples: I have an account on a computer called grumpy (sleepy and doc were already taken). To log onto grumpy from hal, I open up a terminal window and type:

$ ssh rcjhawk@grumpy

The program then logs me onto grumpy, where I can access the system just as I can for any desktop machine.

Or, say, I have a file at /home/rcjhawk/Documents/strasburg_pictures.ppt on grumpy. To copy it over to Hal, I use the secure copy program, scp:

$ scp rcjhawk@grumpy:Documents/strasburg_pictures.ppt .

which, obviously, works pretty much like the regular copy program except that I have to specify the machine and account name right up front.

And, finally, I can send files the other way:

$ scp The_Hard_Times_of_Zach_Greinke.txt rcjhawk@grumpy:Documents

which sends that regrettable file into the Documents folder on grumpy.

Except:

You will probably find that this doesn't work out of the box in your Ubuntu setup. You'll get a message that looks something like this:

ssh: connect to host grumpy port 22: Connection refused

That's because Ubuntu ships ssh in two parts:

  • openssh-client: this lets you use ssh and scp from your machine to log onto another machine. This is installed with the standard Ubuntu installation, and is all you need if, for example, you want to log onto your work computer from home.
  • openssh-server, on the other hand, allows other machines to log onto your personal desktop machine. This is not installed by default in Ubuntu because it's an obvious security hole unless you know what you're doing. If you do know, or assume that you will learn, install the package with

    $ sudo apt-get install openssh-client

    Once you do that, sshd, the daemon that controls remote access, will automatically start up, and will start up every time you reboot the machine. Then you're good to go.

In the example above, grumpy needs to have openssh-server installed and running, but hal does not.

Note for Mac Users: You get the equivalent of openssh-server by going to System Preferences => Sharing and clicking Remote Login.

Security

ssh encrypts the transmissions between machines, but it also has other security features. The first time you try to log onto a machine, for example, you'll get a message that looks something like this:

$ ssh rcjhawk@grumpy
The authenticity of host 'grumpy (192.168.54.42)' can't be established.
RSA key fingerprint is xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx.
Are you sure you want to continue connecting (yes/no)?

If you answer yes, then you'll get the message:

Warning: Permanently added 'grumpy,192.168.54.42' (RSA) to the list of known hosts.

After that you'll be able to log onto grumpy without the warning message. If you want to start over, i.e. not recognize any computer as being previously visited, erase the file $HOME/.ssh/known_hosts

You can also limit access to the machine by editing your /etc/hosts.deny and /etc/hosts.allow files. As I've noted before, if you don't want ever Tom, Dick, and Susie to have a chance to access your machine, your /etc/hosts.deny file should read:

ALL: ALL

after which only addresses listed in your /etc/hosts.allow file will have access to the machine. In my /etc/hosts.allow file I currently have

ALL: 192.168.1.0/255.255.255.0 ,

which allows all machines on our local network to access grumpy.

OK, everything is good, right? Well, not quite. As things are set up right now, every time you ssh or scp to grumpy you must enter a password. This isn't a big annoyance with ssh, because presumably you will keep the remote login running for a relatively long time. However, it is an annoyance with scp, if you want to copy a large number of files from different places, requiring a large number of scp commands.

The solution is to use ssh-keygen to generate a a key. Here's how it works:

Start the program:

$ssh-keygen -t dsa Generating public/private dsa key pair.
Enter file in which to save the key (/home/rcjhawk/.ssh/id_dsa):


We'll use the default directory location, so just hit return here.

Enter passphrase (empty for no passphrase):

Frankly, I don't know anyone who does use a passphrase. It does protect your key against someone who's gotten access to your account, but at that point you're probably hosed anyway. So for now we'll just hit return.

Enter same passphrase again:
Your identification has been saved in /home/rcjhawk/.ssh/id_dsa.
Your public key has been saved in /home/rcjhawk/.ssh/id_dsa.pub.

Now let's take a look at the .ssh directory:

$ ls -la $HOME.ssh
total 16
drwx------  2 rcjhawk rcjhawk  4096 2010-06-19 13:04 .
drwxr-xr-x 34 rcjhawk rcjhawk  4096 2010-06-19 13:01 ..
-rw-------  1 rcjhawk rcjhawk   672 2010-06-19 13:04 id_dsa
-rw-r--r--  1 rcjhawk rcjhawk   599 2010-06-19 13:04 id_dsa.pub

Here you'll notice two files. One, id_dsa, is your private key. The second, id_dsa.pub, is your public key. Key the private key private. Don't let anyone have access to it. Don't copy it over into the .ssh folder of another account, or to another computer. You'll note that both the .ssh directory and id_dsa are set so you are the only person who can read those files. Keep it that way.

The public key, now, that's another matter. If you place id_dsa.pub into the file $HOME/.ssh/authorized_keys on another computer, you'll be able to access that computer from your home computer. Here's one way to to it (I'm leaving out all the password prompts):

$ ssh rcjhawk@grumpy
Welcome to grumpy!
rcjhawk:~ $ cd .ssh
rcjhawk:~/.ssh $ scp rcjhawk@hal:.ssh/id_dsa.pub tempid
rcjhawk:~/.ssh $ cat tempid >> authorized_keys
rcjhawk:~/.ssh $ rm tempid
rcjhawk:~/.ssh $ exit

What did we do? We logged into grumpy, copied hal's public key over to grumpy's .ssh directory, and added it to the end of the authorized_keys file. I renamed the file during the copy so as not to write over grumpy's own id_dsa.pub file, if one was there. Notice that scp was able to get that file even though Hal's .ssh directory is copy protected, because you gave the system the right password.

The next time you try to ssh or scp from hal to grumpy, ssh will find the public key on grumpy, compare it to the private key on hal, and conclude that you are, indeed, allowed to log in or transfer files.

The discerning reader will note that I'm only applying this to my local network, which all sits behind a router and firewall. How, you may ask, can I use ssh to log onto my local machine from outside my home?

Well, I wish I could tell you, but at the moment I can't. It's something to be researched in the future. If you're reading this, and you know how to do it, post a link to the procedure in the comments.

Friday, June 11, 2010

A Sign From Above

Tonight coming home from work I learned that there was a backup on US-50 East between I-97 and the Severn River Bridge in Annapolis.

I knew this, because I was given a Sign. This sign, as a matter of fact, which today had an actual message.
The sign in question is about 15 miles from the start of the backup. Not to worry, I had plenty of time to read it because I was in a five-mile backup caused by people slowing down to read the sign.

Obscene comments about Maryland drivers omitted.

Friday, June 04, 2010

Stupid Bash Tricks

I grew up scripting with the C shell (csh), and I'm still more comfortable with it than any other shells. However, for really complicated scripts I find that BASH (the Bourne-again shell) is more flexible. The problem is that I don't really know how to use it all that well.

I could buy a book, I suppose, but that costs money. And anyway, much of what I want is already on the web. For example, we have

I've used all of these, but the only way to really remember something is to build an example, and refer back to it as needed. So here, presented for your consideration, are a couple of examples.

The first one just counts. It's basically here to show how to do math with variables in BASH:

#! /bin/bash

typeset -i count

count=$1
echo $count

while (( $count < $2 ))
do
 count=$count+1
 echo $count
done

Actually, that one is adapted from the Korn shell (ksh), but it works in bash. A purer bash form would be

#! /bin/bash

let count=$1
echo $count

while (( $count < $2 ))
do
 let count=$count+1
 echo $count
done

Either way, if we call the file count, then it counts from the first argument up to the second:

./count 5 10
5
6
7
8
9
10

On to another program. I often have programs that recompute a certain quantity repeatedly, say the total energy in a Density Functional Theory run. For example,

$ grep "TOTAL ENERGY=" INFO
                          TOTAL ENERGY=      -551.5802151120
                          TOTAL ENERGY=      -551.5798689965
                          TOTAL ENERGY=      -551.5810124810
                          TOTAL ENERGY=      -551.5809217653
                          TOTAL ENERGY=      -551.5796453493
                          TOTAL ENERGY=      -551.5797608209
                          TOTAL ENERGY=      -551.5806860076
                          TOTAL ENERGY=      -551.5804690663
                          TOTAL ENERGY=      -551.5793575483
                          TOTAL ENERGY=      -551.5794583810

where I only want the last line. For a single file, I could get that with

$ grep "TOTAL ENERGY=" INFO | tail -1
                          TOTAL ENERGY=      -551.5794583810

but I want to look at multiple files, make a note of the file I'm looking at, and extract the last entry. So I wrote this little script, which I call lastgrep. Among other things, it shows you how to use the shift command to delete elements from the calling string.

#! /bin/bash

# usage

# lastgrep string  file1 file2 file3 file4

# finds the last occurrence of string in each of the files
#  listed in the argument list.  If string contains white space
#  it should be quoted

string=$1

# Pop the first argument (string) off the stack.  The
#  remaining arguments should be files

shift

for file in $@
do
    echo -n $file

#    Put quotes around $string to keep white space in place

    grep "$string" $file | tail -1
done 

So if I have a set of files INFO in directories a100, a200, a300, I might do something like this:

$ lastgrep "TOTAL ENERGY=" a*/INFO
a100/INFO  TOTAL ENERGY=      -551.7810244761
a200/INFO  TOTAL ENERGY=      -551.6774784178
a300/INFO  TOTAL ENERGY=      -551.6192819148

What this doesn't have is a way of passing arguments to grep. That's for another time.

Tuesday, June 01, 2010

CUPS and a Wireless Printer

Eric Raymond once wrote a famous essay about the difficulty of using Linux's Common UNIX Printing System (CUPS) to set up a printer over a network. Thus, when we got ChildII's new HP C4780 Wireless Printer/Scanner I was less than optimistic about getting it Hal to access it. I mean, we're talking about a wireless printer which is talking to a Verizon FIOS wireless router, and through that to a Linux computer? No way.

I should note that I wouldn't go out and buy an inkjet printer on its own, anymore, particularly an HP. But we just got ChildII a MacBook Pro (new college uses Macs, and Mac Photoshop, for art projects), and they threw in an iPod Touch (which I get to glance at longingly) and the HP (which it's my duty to set up, operate, and maintain). So we got the thing, we might as well use it. Anyway, having a wireless or networked printer for a MacBook is an essential, given that the thing only has two USB and one Firewire port.

Setting up the HP required that I use the Mac or a PC, connected by USB to the printer, to enter the network login information. Given the connection between open source software and printers this left me a little chagrined, but I'm not RMS so I let it slide. Anyway, I didn't expect the thing to work with Linux, remember?

It took about a half-hour to install the software and drivers onto a Mac, and set up the printer. We tested the printer and scanner via wireless, and it all worked fine, though the scanner was pretty slow, much slower than my USB Canon. It all worked, though.

Then it took another half hour or so to install everything on a Windows PC. Again, it all worked flawlessly.

OK, let's try our luck. I left the printer on, went downstairs to Hal, clicked on System => Administration => Printing and watched the CUPS interface come up. Hit Add, Printer, Find Network Printer. Blinked. There it was, an HP C4700 type printer. Unlike the Canon, all the drivers were already available, I didn't have to play guess-the-correct-version. Within five minutes of clicking that Printing button I was printing a test page from Hal back to the printer.

Now all is not sweetness and light. While CUPS had no problem finding the printer, SANE, the corresponding program for scanners, couldn't find the HP. Not surprising, I guess, but it would have been nice if everything worked out. Of course I'm not particularly sure what good a remote scanner does anyone, especially if, like, me, they use it primarily to scan old pictures onto a disk.

But to summarize, things have much improved since the days of ESR's essay.

And I've got to look into getting a networked color Laser printer.

Monday, May 31, 2010

Backing Up Your Blog

After the discussion about creating disk backups I realized that I haven't been terribly diligent about backing up this blog. After all, what if Google should fail? Thousands of words of my deathless prose might be lost. (I know I've written some unforgettable things. Just last week I got three emails telling me how my posts had affected them. Only two mentioned nightmares.)

So how do you back up a Blogger blog?

One way to do it is to download the monthly archive HTML. The advantage to this is that everything in in HTML, and easy to read. But you have to remember to do it. I did, for a while, but haven't done it for some years. Sloth, I know.

But there is a way to download the whole blog:

  1. Log onto your Blogger account.
  2. Go to the Dashboard, if you're not there already.
  3. Click Settings.
  4. Click Export Blog.
  5. Click the orange button.
  6. A big file, ending in .xml, will appear in your download folder.

Note that the xml file is hard to read, but if you're moving your blog from one account to another it's the easy way to go. Also note that this does not save pictures or comments, those linked (the HTML archives don't save the pictures, either). So you'll have to find that stuff yourself.

This doesn't save your blogger template, either. To do that, go back to the dashboard, click Layout, then Edit HTML, and Download Full Template. Important if you have witty sayings embedded in you blog header.

Also note that either the HTML or XML backups are useful for searching your blog, because that Google search box in the upper left doesn't always find everything you've written.