And Still More Changes

So Google, not content to have killed off my beloved Google Notebook, has now decided to kill off iGoogle.

Now I know it wasn't the most popular of services, but it was useful to me. My iGoogle page has the news feeds from the Washington Post, New York Times, Time, a Weather feed, and a score of other things, including some of my favorite bloggers, like Cletis, Pete, and Fran. I can see what's new at a glance. And now it's going away in a little over a year.

On Google+ I found a post recommending an alternative, Protopage. It looks — well, it looks like iGoogle, really. It even has the ability to add sticky notes, which function pretty much like entries in Google Notebook — unfortunately, the only way to add a new note is to choose Add a sticky note from the Add Widgets tab, but maybe that can be change.

Thanks to our benevolent masters, we have over a year to try out alternatives. Let's see how this one works.

Finding Your Header Files — A Better Way

I should really read my own posts. Especially since that's the original purpose of this blog.

The other day I was looking through old posts with the Ubuntu label, and found this comparison of dpkg/apt-get and rpm/yum commands. The relevant one is

dpkg -l '*'

which lists all of the packages available in your repository, installed or not.

Duh

This makes my header file lister a lot simpler to write, towit:

#! /bin/bash

# First make sure you've got all your repositories updated:
sudo apt-get update

# Now search through every repository database, looking for lines that
#  start with "Package", and end with "-dev".
# Pull out the package name, look through each package, and print out
#  the header files:  those that end in "-h"

# No matter how it wraps on your screen, this next line starts with
# "for" and ends with "awk '{print $2}'`"

for hfile in `dpkg -l '*' | grep dev | awk '{print $2}'`
do
# echo $hfile
apt-file list $hfile | grep "\.h$"
done

It's still going to take a long time to go through every package, but it's a lot neater now.

Find Every Header File in Your Ubuntu Repositories

Last night I was following a thread on Penguin Pete's blog where Pete uses Ubuntu's dropping the Dillo browser from the distribution as a launching pad for a discussion about — heck, I can't describe what it's about, read it yourself.

Anyway, in the course of reading the discussion — if that's the word, Pete gets worked up about certain things — I was possessed by the moral imperative to install Dillo from source.

This would seem to be straightforward. You go to the Dillo home page, click Download, and follow the instructions.

Except (there's always an except) — the first install you have to make is the FLTK2 (Fast Light Toolkit) library.

Now Ubuntu offers FLTK1.1, which might well work, but hey, we need the latest and the greatest, right? So I downloaded the tarball, followed the instructions, and the make command crashed and burned with:

fatal error: X11/extensions/XInput.h

I looked on the web, found that XInput.h is in the package libxi-dev, installed that, reran make,

fatal error: GL/glu.h

and so on. It took about an hour to find every single header file needed. And that was just to install FLTK2, after which I still had to install Dillo.

So what's going on here? Well, first we have to know what a header file is. Basically, it's a file that defines a bunch of variables and statements that can be used by multiple routines. See Wikipedia for a better explanation. In particular, header files tell a program's source code what variables are used in a particular library. Thus the header XInput.h defines variables available in some X11 libraries.

OK, suppose I'm compiling a program and I find that I need a header file GrantsTomb/Buried.h. All I have to do is find the package that has that header, install it with apt-get, and I'm good to go. Right?

Well, without prior knowledge you're pretty much doomed here. I was able to find XInput.h and glu.h by web search, but who's going to tell you where is Buried.h in GrantsTomb?

Yet there has to be a solution, because:

• I know that when I use synaptic or aptitude to install packages I can see every package that is, or can be, installed on my system from my selected repositories.
• After searching around a bit, I find that apt and its children list the available files in /var/lib/apt/lists, each repository has its own file, and each file name is on a line starting with Package.
• I know about the program apt-file, in particular that the command
  apt-file list foobar
  will give me a list of all of the files in the foobar package.
• Header files are traditionally located in packages ending with the string -dev.

Given all of that, I should be able to write a script which will list every header file available in every Repository I've accessed. I can then save it in a file, and the next time a program says I need bojangles/tambourine.h I can just search the list, find that the header file is in the NittyGritty-dev package, load that up, and I'm ready to compile:

• First, install apt-file, if you don't have it already:
  sudo apt-get install apt-file
• Then run this script:
  

  #! /bin/bash
  
  # First make sure you've got all your repositories updated:
  sudo apt-get update
  
  # Next sync apt-file's database.  Note that if you do this without
  #  the sudo you'll write the data into your home directory, which is
  #  probably OK
  
  sudo apt-file update
  
  # Now search through every repository database, looking for lines that
  #  start with "Package", and end with "-dev".
  # Pull out the package name, look through each package, and print out
  #  the header files:  those that end in "-h"
  
  #! /bin/bash
  
  # No matter how it wraps on your screen, this next line starts with
  # "for" and ends with "xargs`"
  
  for hfile in `find /var/lib/apt/lists -type f -exec grep '^Package: ' {} ';' | awk '{print $2}' | grep -e '-dev' | xargs`
  do
  apt-file list $hfile | grep "\.h$"
  done
  

Be warned, this takes a long time, since every call to apt-file goes out over the network. On my system, with FIOS broadband, it took over an hour. So you'll only want to run this file once in a while, probably only when you update your repository list. You may also get an error message like
grep: /var/lib/apt/lists/lock: Permission denied
Don't worry about that.

This script should work pretty well for Debian. There are also equivalent commands for RPM-based distributions. And, of course, you can modify it to list any subset of files in any subset of repositories you want. I'll leave all of that as an exercise for the reader.

What Google Knows About You

No, this isn't the revelation of any big conspiracy. It's just to point out (i.e., make myself a bookmark) that the page

http://www.google.com/dashboard/

lets you get information about most of your Google accounts, including your privacy settings.

Skip This Post

L&T's thanks go to the Washington Post's Rob Pegoraro for this link to a site which addresses one of the Web's biggest problems:

The pointless opening page.

Preserving My Web History

Back at the dawn of the Internet I had a GeoCities Web Page. For its time, it was marvelous. All you had to do to keep up with the terms of service was to make sure each page had a link to the GeoCities home page, and upload files through a somewhat clunky web interface.

I haven't touched the thing since about 1997, but it's still up on the web.

Not for long, unfortunately. Yahoo, which bought the place in 1999, is boarding up the site sometime this fall.

So, for posterity, I've downloaded my contribution to early web culture and uploaded it to my current free web site. That I was able to do this tells you something about why GeoCities is about to go the way of Thylacinus cynocephalus: AwardSpace gives me free web space so long as I register my domain with them, with minimal restrictions, and I can manage it with standard ftp. All I have to do is remember to keep my domain registration active.

Looking back on the thing, the only part that might still be relevant are my book reviews, most of which I did as a paid-for-connect-time science advisor for GEnie. They aren't particularly dated, but I obviously needed an editor to go over them. I've also mis-remembered some of the reviews. For example, I was certain that my review of The Curse of the Bambino started with the line Red Sox fans whine a lot, but that turns out not to be the case. (And, the curse having been obliterated, this is the most dated of the reviews.)

Anyway, it's a new home for old web pages, at least until AwardSpace disappears, hopefully in the far, far future.

The Science Hodgepodge Archival Edition

Obsfucating email

One of the annoying things about some jobs is that random people need to be able to email you, meaning that your email needs to be out on the web — No, not me, but you'll note I have my email address on the sidebar anyway.

This almost automatically generates a lot of spam, as harvester programs search your HTML source and look for the mailto: links.

There are ways to hide email addresses from programs, as a look at the source code for this page will show. Depending on how secure you want to try to be, I've found three levels code that might help reduce spam to your email address. Of course, it doesn't protect you from spam generated because a virus or spyware code got your email from someone's address book, but it should help keep the spam down a bit. Note that all require Javascript:

• Simple (From Tim Williams)
• Better (Also from Tim Williams)
• Still Better (Andrew Moulden's modification of the above code

Note that I've not labeled any of these best, and I don't claim they will work against every possible harvester, but it should help. You'll probably also be better off starting with a brand-new email address. And, of course, making sure that everyone who puts your email in his address book uses Linux.

The Strange Search

Occasionally I look at the SiteMeter logs for this blog. Other than the fact that readership has been going down lately, probably because I've concentrated on “Things” rather than “Linux,” this lets me know what people are interested in. (Mainly in making transparent PNGs, it seems. It hasn't changed, people, unlike much of the GIMP.)

Anyway, we get sometimes get strange search requests, like this one:

images of hawk girl, courtesy of Prodigy (that's still around?)

Note that they had to match hawk out of hawknotes to get anything at all.

While Google® found:

Hawkgirl pictures

Immediately.

Trust our Guaranteed Non-Evil Masters®, fellow websurfers.

Motivate Yourself

Augh! I should be in bed, and instead I'm doing this:

Remember those Star Trek Inspirational Posters? Well, I was downloading a few more tonight for use as wallpaper when it hit me, “gee, Dave, there really ought to be some StarGate Inspirational Posters on the web somewhere.” And, of course, there are.

This led me directly to the Motivator: Creator site, which lets you put in your own pictures and sayings. Neat, huh?

So, in honor of last weekend's NCAA Midnight Madness, and in keeping with the theme of this blog, let me get rid of any illusions you may have about the coming season:

Click & Clack in a Pod

Unlike Prairie Home Companion, NPR's Car Talk now makes it easy to download an MP3 version of the latest show: Just go to the Car Talk Podcast page, search for the MP3 button, and download the file.

A nice, simple way to do it, Garrison.

Oh, NPR has a directory of all its podcasts that's worth checking out.

Taking Garrison Keillor to the Gym

I'm a fan of Garrison Keillor and A Prairie Home Companion. Kansas German Lutherans aren't quite as shy as Minnesota Norwegian Lutherans, but it's a close race, and there are a lot of parallels between Lake Wobegon and Holyrood.

So I like to listen, but never get to hear an entire show.

Problem: I just can't take two hours out of every Saturday night to listen to the show. (I'm a busy (ex-)Kansas German (ex-)Lutheran.)

However, I spend 3-4 hours a week at the gym, and I have an Samsung YP-U1 MP3 player (inherited from Youngest Child after the purchase of an iFollowtheCrowdPod). So can we somehow get the show onto the MP3 player?

Yes! PHC has an archive where you can find old shows.

Problem: The archives play streaming Real Media, suitable for listening with RealPlayer or Helix Player (untested). Not a format supported by my Samsung.

Solution: Download the streaming media, convert it MP3 or OGG (the Samsung plays both).

But how? With mplayer. However, first we have to find the stream, and then capture it.

It's not as simple as all that, as the PHC site hides the file pretty well. Here's the procedure for last Saturday's (April 21) show, which just went on line:

1. Go to the archive page, and select the month and year of the program you want to hear. Click GO
2. Scroll down to the show you want, click on the date.
3. On the next page, right click on the “Listen to the whole show” line, and chose the “copy link location” option.
4. Open a terminal window, change to the directory where you want to save your files, write “wget” on the command line and then middle-click. This will produce the command:
   wget http://www.publicradio.org/tools/media/player/phc/2007/04/21_phc
   which will return a file named 21_phc.ram. (Obviously the numbers will change with the date of the show.)
5. $ wget `21_phc.ram`
   (Note that those are backquotes.)
6. This yields a file named 21_phc.smil. If you cat that file, you'll find a string like
   rtsp://archivemedia.publicradio.org/5559/phc/2007/04/21_phc.rm
   That's the audio stream.
7. From here it's simple, if you read the reference article:
   mplayer -vc null -vo null -ao pcm:file=phc_070421.wav rtsp://archivemedia.publicradio.org/5559/phc/2007/04/21_phc.rm
8. This produces a WAV file. You can produce MP3s directly with this method, but they are terribly compressed.
9. After the two hours are up (this is streaming media, remember?), go back to the command line and compress the file. Since the Samsung reads Ogg format, I use:
   sox phc_070421.wav phc_070421.ogg
   which produces a file that I can store on the Samsung and listen to in the gym.

Of course, now that you know the trick it's trivial to write a script to do the whole thing every week, but I wanted to note how you find all of this out, in case PHC changes the format of the files involved.

Advice for IE Users

Most readers of this blog use something other than IE to surf the web. (Currently only 19% use IE here, down to 2% over on Working With Fedora.)

However, the Linux & Things staff does not discriminate against IE users. All are welcome here.

Other sites, however, are not so tolerant.

Fixing Opera

turned out not to be too hard. A search of the Unbuntu Forums turned up an Opera HOWTO page. Basically, you need to make sure that the Motif/Qt libraries are loaded:

sudo apt-get install libmotif3 lesstif1 lesstif2 motif-clients

There are also some java fixes that you can learn about on this page.

Now you can go to the Ubuntu download page, select the Ubuntu version of 8.54 for download, and click. Then open a terminal window and

$ cd ~/downloads
$ sudo dpkg -i opera_8.54-20060330.6-shared-qt_en_etch_i386.deb

and you have an up-to-date version of Opera.

Making It Never Exist

Part two of that Slashdot article I mentioned before:

For some reason, this happens most frequently when I go to washingtonpost.com, but it happens on other sites, too. I click on an article, and get the message

Waiting for doubleclick.net

and then the web page hangs for awhile.

What's happening is that the Post is going to doubleclick.net and waiting for advertisements to load up. Of course, the net doesn't just go to doubleclick, it goes elsewhere as well. And many of those ``elsewheres'' are nothing but annoying ads.

Wouldn't it be great to get rid of those ads? OK, we can use something like Adblock, and I recommend that, but wouldn't it be wonderful to make your computer believe that places like doubleclick never existed? (And, incidentally, decrease worries about your privacy.) There is a way.

Let's go back to the early days of the Internet. There was nothing like the current Domain Name Service (DNS). Oh, each web site had an IP address, but there were no central directories to tell you which IP address belonged to Google (not that Google existed back then).

So how would you have found Google? In order to find things on the Internet, each computer had a file called /etc/hosts. (OK, each Unix-like computer had such a file. I have no idea what other OS called it.) In the /etc/hosts file was a line like:

216.239.41.104 www.google.com

which told your computer the IP address of Google. Of course, this file had to be upgraded periodically by downloading a new file from somewhere, and eventually it got so big that the whole thing had to be scrapped and DNS was adopted. But, the /etc/hosts file still exists, and it's read before your computer goes out to your chosen name server!

This is really quite useful. If you use Google a lot, for example, adding the line

216.239.41.104 www.google.com

to /etc/hosts saves you from having go to the DNS every time you want to do a search. And there's another use:

Near the top of /etc/hosts is a line like

127.0.0.1      localhost.localdomain localhost

This tells your computer where your local computer lives on your local network (even if you don't have a local network). If you try to go to localhost from your web browser, you'll get a message like

The connection was refused when attempting to contact localhost

(Unless you are running your own web server, in which case you'll get your home page.) In this case, localhost and localhost.localdomain function as aliases for the address 127.0.0.1. Any request to go to localhost takes you to 127.0.0.1, and does whatever you tell it to.

Now suppose the webpages you visit regularly connect you to a scuzzy site we'll call reallystupidads.com. You're tired of these ads and never want to see them again. What to do? Remember that /etc/hosts is read first, before the request goes out to the DNS. So add a line

127.0.0.1  reallystupidads.com

to /etc/hosts. Now, when that page you are interested in asks reallystupidads.com send some advertisements, your computer looks up the address, finds that it is 127.0.0.1, and does whatever you've set it up to do. If you don't run your own web server, it does nothing. It's as if that site never existed.

So, what you need is a list of really annoying sites that should be banned from your computer. Of course, this being the Internet, there are many such lists. The one I'm using is Mike Skallas' Ad Blocking Hosts file. Basically, you add it to /etc/hosts. And, frabjous day, there are instructions for doing the same thing in Windows 2000 and XP.

And, of course, once you know the trick, you can add your own annoying sites to the list. Parents can add sites that they want to keep their children out of, e.g.

Hopefully no one will add

127.0.0.1 hawknotes.blogspot.com

Anonymous Registration

Just an update to the temporary email address article. If all you want to do is anonymously register for a website, you don't have to do all that much work. The site bugmenot.com collects username/passwords for site registration, and will give them to you for free. If they don't have a login for the site you want, they'll create one for you.

Thanks to The Internet Tourbus for this tip. If you don't subscribe, take a look at the site. It's primarily directed towards Windows users (e.g., how to do the impossible -- use IE safely), but they often have sites of general interest.

P.S. Hope spelling hasn't gone completely to heck. With Blogger's new design, spell check doesn't work here anymore, and I'm not going to cut-and-paste everything into emacs just to do a global spell-check.

Essential Googling

Well, maybe not essential, but http://dmiessler.com/study/google/index.html has a lot of useful tricks for using Google.

For example, did you know that the string (fay | scow) thomas will tell you about sites which mention the only professional baseball player born in Holyrood, Kansas, whether they refer to him by his given name or his nickname?

Mail Temps

OK, you want to see some web site's one and only picture of Ann Coulter voting in a Democratic primary. But, the site requires you register. So you give it the relevant information. They want an email address. But, having seen some of what this site offers, you really don't want them to have your email address. So what do you do?

It so happens that there are two (count 'em, two) choices:

1. mailinator: Tell the site that your email address is, say, sample@mailinator.com. Then, after you submit your information, go to http://mailinator.com/ enter sample in the box, and press go. Any message sent to sample@mailinator.com will show up. You can click any confirmation links they have, and then go look at Ann.
   Two warnings:
   • Anyone who knows email is being sent to sample@mailinator.com can read the email. If you want to keep things secret, use a less obvious name. Mailinator will suggest random email addresses, e.g. puyuvnzgeved@mailinator.com.
   • The mail is only kept for a few hours. This is a Good Thing.
   
   For more information about Mailinator, read the FAQ.



2. spamgourmet: I haven't tried this one. It's a little different than Mailinator. You have to register a username and a forwarding email address. Say your username is samiam. Then you set your email for the above example as, e.g., annspicts.4.samiam@spamgourmet.com. The registration bot sends information to this address, and it's forwarded to you. The "4" is means that only 4 emails can come into this account before it's closed. This would seem like a good way to track who's spaming you, then cut if off before the spam gets to heavy.

The Cookie Machine

When surfing the web, cookies are little blips of information that a web site you've visited stores on your computer. Cookies are good in that they let a site you visit often store your preferences, making it easier to work with. For example, many sites which require registration will save your registration information in a cookie. Cookies are bad because they leave traces on your computer of where you've been. E.g., not that this applies to anyone I know, if you search for some steamy books/videos online, any site which can access that cookie knows what you've been looking for.

Firefox, like all Mozilla (and I suppose all other) browsers, lets you manage your cookies. In Ff > 0.8 you do this by going to Edit => Preferences => Cookies. Here you can examine the cookies on you have stored, delete the ones (or all) you don't like, designate sites which can always leave cookies, and forbid certain sites to leave cookies on your machine.

That's all well and good, but I'd like a slightly better option: There are a list of sites with cookies I'd like to keep, e.g., the Double-Click Opt-Out, various newspaper registrations, SABR, etc. Everything else I'd like to delete just by pressing a button.

And so we have the Firefox/Mozilla extension CookieCuller. It loads just like a regular extension. In Firefox, you then right-click on a blank spot of the navigation tool bar and click Customize. In the box with the icons will appear a large cookie. (It's really too large for the Pinball theme I'm currently using, but never mind.) Drag the cookie onto the navigation tool bar. Now, when you click on the cookies, you'll see list of those on your browser. To the left is a column labeled "Protected". You can toggle the status of a cookie from protected to unprotected. And clicking "Remove All Cookies" does just that -- it removes all cookies you haven't protected. Exactly what I wanted.

Slight Changes to the Blog

1. Removed the Internet Storm Center tool as boring. You people are being too good about keeping your machines updated -- we haven't had a big virus outbreak since I put the thing up on May 5.
2. Moved the Google (TM) search bar from the bottom of the blog to the top. Makes searching much easier.

Forgot to put this in the last searchbar note:
The source code for the toolbar is available from Google as Google Free web search with site search. Note that when it says replace "YOUR DOMAIN NAME" with your sites, they mean without the http:// and any "/" at the end of the address. Also, it only works on the domain level, you can't restrict the search to a given file.

And oh, yes, the link to the search bar came from Blogger Help

Macromedia Flash Update for Linux

Macromedia has released version 7 of its Flash Player for Linux. The official download site is
here at Macromedia,
but there are RPMs available from Rutgers (the State University) at
http://ruslug.rutgers.edu/macromedia/site_ru.html.
Note that this is not a Shockwave Player for Linux, which still does not exist AFAIK.