Mounting Remote Computer Files with FUSE

I'm sitting here, upstairs, typing on my laptop. There's a file I want on my office machine downstairs. So I should go downstairs and look for the file, right?

Wrong. The TV up here is on NFL Red Zone ( (sigh) Red Zone). I'm addicted, OK? It's hard enough to break away for a bathroom trip.

Several options exist, of course. I could ssh to the downstairs machine, search for the file, and use scp to copy it upstairs. But the file is a picture, and I while I remember what it looks like, I don't remember what name I used for it. So ideally I'd like to use an image viewer, say gthumb or F-Spot to look through the directory and find the file. But I can't, because this computer doesn't mount the downstairs computer's disk.

Ah, but it can. The classic way to do this is to use the Network File System protocol, aka NFS. But that requires work to set up. Besides, some days I might want to search my computer from work, or from a hotel. It's tricky to get NFS to work that way.

Fear not, there is a solution. It uses File System in Userspace (FUSE), a complicated name for using ssh and scp to fool my laptop into thinking that the downstairs computer's files are on the laptop's local drive.

In Ubuntu the way to do this is to install a package called sshfs. Then you just follow the directions on the linked page. Let's say I want to be able to access all the files in /home/rcjhawk on hal, my downstairs computer, from my laptop, majel:

• First make sure I have both hal and majel set up to use ssh communications. If I'm away from home, I'll also need to know hal's ip address, and make sure I can ssh to hal from outside my router.
• Next, install sshfs on majel. Just use
  $ sudo apt-get install sshfs
  from a terminal window on majel. This doesn't need to be done on hal. Hal's just going to see a series of ssh requests.
• Add myself to the FUSE group on majel:
  $ sudo gpasswd -a $USER fuse
  from the same command window.
• Log off my account on majel, log back on. Yes, it's annoying, but you're not officially added to the FUSE group until you do.
• Make a directory on majel. Doesn't matter what the name is:
  $ mkdir ~/hal
• Now we're going to mount all of hal's directory tree starting from /home/rcjhawk in majel's ~/hal directory:
  $ sshfs -o idmap=user rcjhawk@hal:/home/rcjhawk ~/hal
• If this works, then running the command
  $ ls ~/hal
  from a terminal window on majel lists hal's directory tree.
• To unmount hal, just run:
  $ fusermount -u ~/hal

To make things easier on myself, I created two files. The first, mthal, contains the mounting command, and the second, umthal, the unmount command. Make these executable, put them in your path, and it's easy to mount and unmount.

A caveat: I was getting an error remote host has disconnected. That occurred because my .cshrc file, which used a variable that's undefined when you are running ssh commands. You can find out how to diagnose that here.

Does it work? We'll, I started typing this on the laptop, and finished it up on the big machine downstairs. So, yes, it does work.

There are also versions of FUSE for BSD, OpenSolaris, and Mac OS X. In fact, I first learned about this trick using my Mac at work. With FUSE you can mount just about anything anywhere. There's even GmailFS, which transforms your Gmail account into backup storage.

Finding Your Header Files — A Better Way

I should really read my own posts. Especially since that's the original purpose of this blog.

The other day I was looking through old posts with the Ubuntu label, and found this comparison of dpkg/apt-get and rpm/yum commands. The relevant one is

dpkg -l '*'

which lists all of the packages available in your repository, installed or not.

Duh

This makes my header file lister a lot simpler to write, towit:

#! /bin/bash

# First make sure you've got all your repositories updated:
sudo apt-get update

# Now search through every repository database, looking for lines that
#  start with "Package", and end with "-dev".
# Pull out the package name, look through each package, and print out
#  the header files:  those that end in "-h"

# No matter how it wraps on your screen, this next line starts with
# "for" and ends with "awk '{print $2}'`"

for hfile in `dpkg -l '*' | grep dev | awk '{print $2}'`
do
# echo $hfile
apt-file list $hfile | grep "\.h$"
done

It's still going to take a long time to go through every package, but it's a lot neater now.

Look Familiar?

I saw this watching the Star Trek reboot on the bigscreen with our Epix OnDemand account. It's during the Old Spock/New Kirk mind-meld, at the part of the story where Spock is transported back in time. The blue box is suggestive, don't you think?

Since Epix lets you stream its movies to a Linux box, unlike Netflix, it was relatively easy to find the shot and save it. No, the resolution's not that great, but, frankly, this movie suffers even on a 48" high-def screen. To do it justice you really need to watch it in a theater.

Undercutting Yourself

Maybe I just don't understand…

The other day Borders sent me an email which gave me ten dollars in Borders' Bucks. A great promotion, to be sure. And I had a book in mind I wanted to buy, but not spend a lot of money on: Jim Butcher's collection of Harry Dresden short stories, Side Jobs.

First check it out online. We have:

List Price	$25.95
Online Discount	-$11.22
Borders' Bucks	-$10.00
Shipping	$3.98
MD Tax	$0.29
Total	$9.00

Not bad, but the shipping price sucks, it will take days to get here, and anyway I want to get out of the house for awhile, so let's go to a nearby Borders and buy it there.

I get to the Borders, and find Side Jobs. The sticker on it says 20% off. Do a little bit of math: 80% of $25.95 = $20.76 - $10 Borders' Bucks = $10.76, + 6% tax ( $0.65) = $11.41. A bit of a rip, compared to online, but I'm here, I've already spent a buck in gas, and I'll walk out with the book.

Except: I then notice that the sticker also says Borders Rewards Plus Members. I'm just an ordinary Borders Rewards member. I checked with one of the sales people, and no, I'm not entitled to the discount. That makes the cost $16.91, Borders' Bucks and tax included. I made a point of telling the guy his store's website was undercutting the store by selling the book for $5 less, including shipping, put the book down, and walked out.

I suspect some marketing guru has it all figured out, but to me this just seems plain old-fashioned stupid. If Borders gave me the online price at the store, or even got close to it, I would have gotten the book then and there. Since I was in the store, I probably would have had a coffee as well. Maybe a cookie. I would have looked at a Linux magazine while I was sipping my coffee, and possibly would have purchase that, too. I would most likely have spent the $5 in the store, maybe more, and I wouldn't have been pissed.

Instead, I went home and ordered the Dresden book online. Even with the $2 gas^† I just wasted I still saved $3 over buying it in the store. And, instead of waiting for more coupons/bucks to come my way, I went over to Amazon and ordered two books I would have otherwise eventually gotten at Borders (along with more coffee, cookies, magazines, etc.): Lois McMaster Bujold's newest Vorkosigan novel, Cryoburn, and Terry Pratchett's new Tiffany Aching/Discworld novel, I Shall Wear Midnight. Add in a book Child_II wanted, we made the supersaver price, and got free shipping.

In summary: by charging full price for a book in the store, and a deep discount online, Borders lost the sale of at least two books, two-three cups of coffee, a couple of cookies, and maybe a magazine.

It looks like Borders would rather you didn't come to their store. OK, stores cost money to keep up. Employees have to be paid, not to mention heating, lights, insurance, and all of that. But if I come into the store, I'm going to browse. I'll look around, and might well find something else I'd like to buy. Online, I'm going to get the books I want, hit the send button, and go on with my life.

Like I said, some genius with an MBA probably has figured out that this approach makes more sense for Borders, but how that could be is a mystery to me.

^† The trip wasn't wasted, I drove a mile to the local wine emporium, tasted a couple of wines, and bought a bottle.

Unintended Consequences

Article 1, Section 2, of the United States Constitution says, in part, The Number of Representatives shall not exceed one for every thirty Thousand, but each State shall have at Least one Representative. The very first amendment proposed (you can look it up) tried to change this:

After the first enumeration required by the first article of the Constitution, there shall be one Representative for every thirty thousand, until the number shall amount to one hundred, after which the proportion shall be so regulated by Congress, that there shall be not less than one hundred Representatives, nor less than one Representative for every forty thousand persons, until the number of Representatives shall amount to two hundred; after which the proportion shall be so regulated by Congress, that there shall not be less than two hundred Representatives, nor more than one Representative for every fifty thousand persons.

but this was never ratified. So we can be pretty sure that the Founders of this Republic wanted us to have one Representative for every 30,000 Americans.

Currently, however, we have 435 Congressmen, thanks to something called Public Law 62-5. I guess this was supposed to keep the number of Congressmen down to a reasonable number. Otherwise, with the current U.S. population around 300,000,000, we'd have 10,000 Congressmen running around. Think of it! Mark Russell once said something on like every night 500,000 people in the District of Columbia go to bed without a Congressman. But if we had 10,000 Congressmen, you give yours (DC residents excepted) a booty call just about every night.

The current limit of 435 Congressmen also gives rise to some startling inequities. For example, according to the 2000 Census, the total population of the United States was 281,421,906. If we exclude the District of Columbia (population 572,059) since it has no Congressional representation, then there are 280,849,847 Americans and 435 representatives, so each member of Congress should represent 645,632 people. However, Wyoming (493,782) and Montana (902,195) each have only one representative. To put it another way, Cynthia Lummis only needs to attract half as many votes as Denny Rehberg to get elected, but each gets one vote in the House. So by moving from Wyola, Montana to Parkman, Wyoming (16 miles) you can effectively double the strength of your vote for Congress.

There is a lot more information about the inequities of the current system, and the sorry history of Amendment the First, at thirty-thousand.org. I'd recommend downloading their pamphlet, Taking Back Our Republic, for more information.

One thing they didn't cover, though: the effect of all of these changes on the Electoral College. Each state gets N+2 electoral votes, where N is the number of representatives, plus the two members of the Senate allowed each state. That +2, along with Public Law 62-5, weights the Electoral College heavily toward the smaller states. Since D.C. does have 3 votes in the Electoral College, each elector theoretically represents 281,421,906/538 = 523,098 voters. However, each state has a minimum of three electoral votes. So in Wyoming each elector represents 164,594 people. In Montana, it's 300,732 people, still substantially below the national average. It's likely (I haven't checked in detail) that the state with the most people/elector is California, where there are 33,871,648 people and 55 electors, or 615,848 people/elector. Put it this way: an elector in California represents four times as many people as one in Wyoming, but each has the same say in who's elected President.

Now, as anyone can tell you, most of the smaller states, population-wise, are in the west, and most of those states vote Republican, while many of the larger states tend to vote Democratic. Another wrinkle in the system is that most states (Maine and Nebraska excepted) cast their electoral votes in a block.

Couple these two facts with the 30,000 people/representative rule and you get some interesting results. Very interesting, in some cases.

In particular, let's consider the 2000 Presidential Election. Very close, as we all recall. Al Gore got 50,999,897 votes, while George Bush got 50,456,002, but because most states cast their electoral votes as a block, Bush won in the Electoral College, where it counts, 271-266.

What if we'd apportioned Congress the way the Founders intended? And kept the current winner-take-all (mostly) electoral system? The table below shows the results. I constructed it using the following rules:

• Winner take all in each state, as now. Maine (42 votes under these rules) went to Gore, and Nebraska (54) went to Bush. Changing all of Maine's votes to Bush or all of Nebraska's to Gore isn't going to change the final result, so it doesn't matter if several congressional districts shifted from Red to Blue or vice versa.
• To get the number of representatives, I took the states' populations from the 1990 census, which governed apportionment at the time, divided by 30,000, and rounded down. Rounding to the nearest integer, or rounding up, can't change the electoral vote by more than 51, and so won't affect the result, either.
• If the District of Columbia was a state, it would have 22 electoral votes in this scheme. However, by the Twenty-Third Amendment, D.C. gets 17 electoral votes, equal to the state with the fewest, Wyoming. In the real 2000 election one D.C. elector abstained. Even assuming all 17 of the new electors abstained it wouldn't change the bottom line here, so I assume that they all voted — for Gore, naturally.

The result: Since each state has more or less the same representation per capita in the electoral college, and since most of the larger states went for Gore, he wins the election, 4,314–4,047.

Think about it. Al Gore, Forty-Third President of the United States. Somehow, I don't think that's what those currently saying we should go back to the Principles of the Founders had in mind.

State	Population	Congress	Electors	Bush	Gore	Bush	Gore
				Popular	Popular	Electoral	Electoral
Alabama	4040587	134	136	941173	692611	136	0
Alaska	550043	18	20	167398	79004	20	0
Arizona	3665228	122	124	781652	685341	124	0
Arkansas	2350725	78	80	472940	422768	80	0
California	29760021	992	994	4567429	5861203	0	994
Colorado	3294394	109	111	883748	738227	111	0
Connecticut	3287116	109	111	561094	816015	0	111
Delaware	666168	22	24	137288	180068	0	24
DC	606900	0	17	18073	171923	0	17
Florida	12937926	431	433	2912790	2912253	433	0
Georgia	6478216	215	217	1419720	1116230	217	0
Hawaii	1108229	36	38	137845	205286	0	38
Idaho	1006749	33	35	336937	138637	35	0
Illinois	11430602	381	383	2019421	2589026	0	383
Indiana	5544159	184	186	1245836	901980	186	0
Iowa	2776755	92	94	634373	638517	0	94
Kansas	2477574	82	84	622332	399276	84	0
Kentucky	3685296	122	124	872492	638898	124	0
Louisiana	4219973	140	142	927871	792344	142	0
Maine	1227928	40	42	286616	319951	0	42
Maryland	4781468	159	161	813797	1140782	0	161
Massachusetts	6016425	200	202	878502	1616487	0	202
Michigan	9295297	309	311	1953139	2170418	0	311
Minnesota	4375099	145	147	1109659	1168266	0	147
Mississippi	2573216	85	87	572844	404614	87	0
Missouri	5117073	170	172	1189924	1111138	172	0
Montana	799065	26	28	240178	137126	28	0
Nebraska	1578385	52	54	433862	231780	54	0
Nevada	1201833	40	42	301575	279978	42	0
New Hampshire	1109252	36	38	273559	266348	38	0
New Jersey	7730188	257	259	1284173	1788850	0	259
New Mexico	1515069	50	52	286417	286783	0	52
New York	17990455	599	601	2403374	4107697	0	601
North Carolina	6628637	220	222	1631163	1257692	222	0
North Dakota	638800	21	23	174852	95284	23	0
Ohio	10847115	361	363	2350363	2183628	363	0
Oklahoma	3145585	104	106	744337	474276	106	0
Oregon	2842321	94	96	713577	720342	0	96
Pennsylvania	11881643	396	398	2281127	2485967	0	398
Rhode Island	1003464	33	35	130555	249508	0	35
South Carolina	3486703	116	118	785937	565561	118	0
South Dakota	696004	23	25	190700	118804	25	0
Tennessee	4877185	162	164	1061949	981720	164	0
Texas	16986510	566	568	3799639	2433746	568	0
Utah	1722850	57	59	515096	203053	59	0
Vermont	562758	18	20	119775	149022	0	20
Virginia	6187358	206	208	1437490	1217290	208	0
Washington	4866692	162	164	1108864	1247652	0	164
West Virginia	1793477	59	61	336475	295497	61	0
Wisconsin	4891769	163	165	1237279	1242987	0	165
Wyoming	453588	15	17	147947	60481	17	0
Totals			8361	50455156	50992335	4047	4314

Sorry about the lack of commas in the table above. I used a very primitive spreadsheet to generate the data.

1990 Census data is from http://lwd.dol.state.nj.us/labor/lpa/census/1990/poptrd1.htm. Presidential vote totals are from http://www.presidency.ucsb.edu/showelection.php?year=2000.

Find Every Header File in Your Ubuntu Repositories

Last night I was following a thread on Penguin Pete's blog where Pete uses Ubuntu's dropping the Dillo browser from the distribution as a launching pad for a discussion about — heck, I can't describe what it's about, read it yourself.

Anyway, in the course of reading the discussion — if that's the word, Pete gets worked up about certain things — I was possessed by the moral imperative to install Dillo from source.

This would seem to be straightforward. You go to the Dillo home page, click Download, and follow the instructions.

Except (there's always an except) — the first install you have to make is the FLTK2 (Fast Light Toolkit) library.

Now Ubuntu offers FLTK1.1, which might well work, but hey, we need the latest and the greatest, right? So I downloaded the tarball, followed the instructions, and the make command crashed and burned with:

fatal error: X11/extensions/XInput.h

I looked on the web, found that XInput.h is in the package libxi-dev, installed that, reran make,

fatal error: GL/glu.h

and so on. It took about an hour to find every single header file needed. And that was just to install FLTK2, after which I still had to install Dillo.

So what's going on here? Well, first we have to know what a header file is. Basically, it's a file that defines a bunch of variables and statements that can be used by multiple routines. See Wikipedia for a better explanation. In particular, header files tell a program's source code what variables are used in a particular library. Thus the header XInput.h defines variables available in some X11 libraries.

OK, suppose I'm compiling a program and I find that I need a header file GrantsTomb/Buried.h. All I have to do is find the package that has that header, install it with apt-get, and I'm good to go. Right?

Well, without prior knowledge you're pretty much doomed here. I was able to find XInput.h and glu.h by web search, but who's going to tell you where is Buried.h in GrantsTomb?

Yet there has to be a solution, because:

• I know that when I use synaptic or aptitude to install packages I can see every package that is, or can be, installed on my system from my selected repositories.
• After searching around a bit, I find that apt and its children list the available files in /var/lib/apt/lists, each repository has its own file, and each file name is on a line starting with Package.
• I know about the program apt-file, in particular that the command
  apt-file list foobar
  will give me a list of all of the files in the foobar package.
• Header files are traditionally located in packages ending with the string -dev.

Given all of that, I should be able to write a script which will list every header file available in every Repository I've accessed. I can then save it in a file, and the next time a program says I need bojangles/tambourine.h I can just search the list, find that the header file is in the NittyGritty-dev package, load that up, and I'm ready to compile:

• First, install apt-file, if you don't have it already:
  sudo apt-get install apt-file
• Then run this script:
  

  #! /bin/bash
  
  # First make sure you've got all your repositories updated:
  sudo apt-get update
  
  # Next sync apt-file's database.  Note that if you do this without
  #  the sudo you'll write the data into your home directory, which is
  #  probably OK
  
  sudo apt-file update
  
  # Now search through every repository database, looking for lines that
  #  start with "Package", and end with "-dev".
  # Pull out the package name, look through each package, and print out
  #  the header files:  those that end in "-h"
  
  #! /bin/bash
  
  # No matter how it wraps on your screen, this next line starts with
  # "for" and ends with "xargs`"
  
  for hfile in `find /var/lib/apt/lists -type f -exec grep '^Package: ' {} ';' | awk '{print $2}' | grep -e '-dev' | xargs`
  do
  apt-file list $hfile | grep "\.h$"
  done
  

Be warned, this takes a long time, since every call to apt-file goes out over the network. On my system, with FIOS broadband, it took over an hour. So you'll only want to run this file once in a while, probably only when you update your repository list. You may also get an error message like
grep: /var/lib/apt/lists/lock: Permission denied
Don't worry about that.

This script should work pretty well for Debian. There are also equivalent commands for RPM-based distributions. And, of course, you can modify it to list any subset of files in any subset of repositories you want. I'll leave all of that as an exercise for the reader.