Broken wing: on upgrading to Ubuntu 8.04 Hardy Heron

This past weekend I upgraded from Ubuntu 7.10 (Gutsy Gibbon) to 8.04 (Hardy Heron). Right off the bat I encountered an unexpected hitch. Previously I had upgraded by using the Alternate ISO. This serves two purposes: 1) Normally on the days immediately following an Ubuntu release, the package servers are choked by users doing automatic upgrades, but the ISOs are available from bittorrents, which thrive in this type of situation. 2) I have multiple machines to upgrade, so it makes more sense to download once and upgrade many.

Unfortunately, my attempt to upgrade from the CD, and only the CD, failed on a package related to nvidia-glx, according to the logs, though according to the output presented directly to the user, the failure occurred because of "obsolete or locally installed packages, unofficial repositories, or [something else]". I had already moved all my third-party repositories out of sources.list.d. I then decided to allow the upgrade process to access the latest packages on the net, only to discover to my horror that I would still wind up downloading 500 MB of packages from thu Ubuntu repositories, essentially negating having downloaded the 600 MB ISO. The download actually occurred fairly quickly, even over my DSL connection, but the installation and configuration took well over an hour.

After rebooting, I was greeted with the login screen, on which my laptop's trackpad caused the mouse pointer to go crazy any time I attempted to move it. (Note, this is probably due to my Xorg configuration which is set up for both one and two monitors, but it pisses me off because it didn't do this before.) After logging in, compiz started fine and the desktop seemed to be peachy. The actual process of logging in seems to take just as long as it did in 7.10, and that blasted trackerd still crushes system performance immediately upon login by riding the hard drive harder than a bucking bronco as it indexes the hard drive.

I did what I normally do upon finally getting control of my system: firing up Firefox. I was presented with a Firefox 3 prompt informing me that I had about a half dozen plugins incompatible with Firefox 3 beta 5, and if I would like to proceed, I would need to click the button that would disable them all. "No problem," I thought, "I'll just go back to Firefox 2." I did play around in Firefox 3 for a bit and tried it out to see if it was as fast and slick as I had heard. Generally, I had to agree it was a good browsing experience, but utterly worthless with no support for critical addons like del.icio.us bookmarks, CookieSafe, and Zotero. When I had my fun, I used aptitude to install Firefox 2 and fired it up. Well, I quickly discovered that about the only thing left of Firefox 2 was its bookmarks. All of my cookie settings were gone, and no plugins were running on it at all. When I went to the Addons menu, I discovered I was unable to re-enable any of the plugins. After digging around the net and many searches, I came up empty handed with a working solution and wound up having to delete my Firefox profile and starting afresh with Firefox 2.

To skip ahead, I later learned once Firefox 3 is launched, it will render a profile backwards-incompatible with Firefox 2. At least, this is the case for Ubuntu 8.04. According to Paulo Nuin, this is not the case for Windows. (Which probably grates on me more.) So for Ubuntu users running into this problem, back up what Firefox data you can (bookmarks, etc.), rm -rf your profile, and start again in Firefox 2. Then make sure to set Firefox 2 as your web browser in the Preferred Applications menu, and only launch from launchers connected to Firefox 2. In fact, you may want to apt-get remove Firefox 3 all together.

After that, there was the problem that my fonts looked like trash in the gnome-terminal, and also for certain pages browsing with Firefox 2 (presumably those calling on system fonts because their font was not explicitly set). I double-checked that I had "Subpixel Smoothing" checked in the GNOME fonts dialogue in the Appearance application in the Preferences menu. Then I had to do some digging around the Ubuntu forums to find other people who had this issue. I came across some threads that suggested setting the DPI in the about:config of Firefox to 96, but this wasn't much of a help. I finally came upon this post that gave directions for reconfiguring the fonts using dpkg-reconfigure. Restarting Xorg and firing up the gnome-terminal, I was relieved to see that the fonts were finally rendering with antialiasing. I felt the terminal font still looked ugly--much taller than in 7.10--so I wound up setting it to Lucida Sans Typewriter.

Firefox still rendered terribly for some pages (really poor kerning), particularly on Twitter. The only other obvious thing to try would be to set the fonts. I almost without fail use defaults so I hated this option, but I set all the fonts to Bitstream fonts, restarted Firefox, and finally the troublesome pages were readable again. I breathed a sigh of relief.

A few peeves remained. For one, only one program could make use of the soundcard at a time--no good! I went into the sound and disabled ESD, but that still didn't do it, so I had to set all the sound devices from Autodetect to ALSA, which finally allowed for multiple sounds at once. Of course, not being content, I decided to fiddle around more, and instead re-activated ESD and set the devices to use ESD. This led to a horrible out of control spiral as gnome-sound-preferences proceeded to consume every bit of the 2 GB of RAM in my laptop and crawled out into the swap space, sending my system thrashing horribly until I could kill the process. I have no idea what that was about but it was terrible. I immediately reverted everything to ALSA. Lastly, my compiz settings weren't entirely preserved; particularly, the hot corners that I had for the Scale plugin weren't available anymore, and it took me a while to actually figure out how to set them again, completely missing the fact that the little screen icons were indicating that's how I was supposed to set hot corners. In 7.10, there was just a simple dialogue box that said in words "Screen" or something like this. I'm sure this was changed to a picture of a screen for usability, but it had the opposite effect on me.

So this is where I stand with Hardy Heron today. I still consider Gutsy Gibbon the greatest release of Ubuntu, and Hardy Heron as a significant backslide. In fact, I can't believe Hardy is going to be a long-term support release, particularly in light of the Firefox 3 fiascos. How do they plan to support a beta version of a web browser for three years? Plugins such as Zotero already require newer versions of Firefox 3 because fixes have been implemented. What other plugins are waiting for Firefox 3 to solidify before converting? I dunno, it just seems a poor choice to have that as their default browser, given that the web browser is now the most critical piece of user software.

I'm skipping this release for all my other machines and holding out for what I hope will be a much better release in 8.10. Best of luck to you other brave souls moving to this lame bird that is 8.04.

On the appropriate blogging software

As a meta-post following my reply to Thomas Upton, I'm thinking about how inappropriate a medium this blog is for publishing code. Hence, I posted the code to a pastebin, though I'm not sure how long it will live there. I think it goes to show that at some point, I'm moving this blog off of Blogger and onto some software on my site. Things that I must have in the blogging software are colorized syntax highlighting, particularly for Python and C. Also, I need the ability to do LaTeX markup, for a lot of the research I do requires mathematical notation, and I'd like to be able to get on the open research train and whore out for ideas on how to improve my research.

I know Wordpress must have syntax plugins, and I know it has a LaTeX plugin. But it's Wordpress, which seems to be an exploitfest unless you update it every five hours. I've dabbled a bit in Django and have thought about coding up a blog in it, but then I get to the wheel reinvention issue, and I have to balance the sheer enjoyment of coding something with the pragmatic point of view of doing something productive. Of course, there must be ready-to-launch Django blog applications, and writing my own module to take a LaTeX formatted string and convert it to a PNG via dvipng must not be hard. That looks like the most favorable option. I dunno. Thoughts?

The good news is that I have my blog RSS hooked up via FeedBurner, so (if and) when I do move the blog, nobody will have to change the address in their RSS reader--their syndication shall continue unbroken (much to their chagrin, I'm sure). I'm glad I found out about that service, and I recommend it to all bloggers as a Good Idea™.

Weather powered by Python

No, I haven't gone delusional in my ever-continuing worship of Python and exaggerated its capabilities. My friend and fellow hacker Thomas Upton posted a clever script to obtain weather forecasts from Yahoo! RSS feeds, specifically for the use in Geek Tool.

I wanted to point out to Thomas a couple of areas of improvement on his blog, but unfortunately, he doesn't seem to have commenting enabled on his Wordpress blog because he's paranoid that his site will get pwned by pr0n ads. (Rightly so, it's PHP software after all.) Hence, I'll post my response here.

I made a few revisions to Thomas's code. Namely, I ported it from the getopt Python standard library module, which is somewhat deprecated, to another handy stdlib module, optparse. I also removed the option for the two-day forecast and replaced it with an argument for the number of days to forecast. I may have changed an option name or two, as well.

You can see the expressiveness of optparse compared to getopt. The boilerplate is standard for optparse and prevents you from having to roll your own for a lot, including help documentation, usage statements, and most helpfully, in parsing the CLI for errors.

The final thing of note is the movement to string formatting as opposed to lots of string concatenation. String formatting is considered more Pythonic. Of course string formatting is going to be a lot easier and prettier in Python 3.0; thus this script will definitely have to get overhauled for that (though the 2to3 tool should take care of this). You'd have to do it anyways, though, because print will no longer be a statement but a function in Python 3.0.

Without further ado, you can have a look at my modifications at http://rafb.net/p/YWiyo548.html.

You can also apply a patch if you would like, which I supply here:

=== modified file 'weather.py'
--- weather.py  2008-04-23 20:11:38 +0000
+++ weather.py  2008-04-23 21:48:58 +0000
@@ -1,45 +1,67 @@
-#! /usr/bin/python
+#!/usr/bin/env python
+
+"""Fetches weather reports from Yahoo!"""

import sys
import urllib
-import getopt
+from optparse import OptionParser
from xml.dom.minidom import parse

+# Yahoo!'s limit on the number of days they will forecast for
+DAYS_LIMIT = 2
WEATHER_URL = 'http://xml.weather.yahoo.com/forecastrss?p=%s'
WEATHER_NS = 'http://xml.weather.yahoo.com/ns/rss/1.0'

-def get_weather(zip_code):
+def get_weather(zip_code, days):
+    """
+    Fetches weather report from Yahoo!
+
+    :Parameters:
+    -`zip_code`: A five digit US zip code.
+    -`days`: number of days to obtain forecasts for
+
+    :Returns:
+    -`weather_data`: a dictionary of weather data
+
+    """
+
# Get the correct weather url.
url = WEATHER_URL % zip_code

# Parse the XML feed.
dom = parse(urllib.urlopen(url))
-
+
# Get the units of the current feed.
yunits = dom.getElementsByTagNameNS(WEATHER_NS, 'units')[0]
-
+
# Get the location of the specified zip code.
ylocation = dom.getElementsByTagNameNS(WEATHER_NS, 'location')[0]
-
+
# Get the currrent conditions.
ycondition = dom.getElementsByTagNameNS(WEATHER_NS, 'condition')[0]

# Hold the forecast in a hash.
forecasts = []
-
+
# Walk the DOM in order to find the forecast nodes.
-    for node in dom.getElementsByTagNameNS(WEATHER_NS,'forecast'):
-  
-        # Insert the forecast into the forcast hash.
-        forecasts.append ({
-            'date': node.getAttribute('date'),
-            'low': node.getAttribute('low'),
-            'high': node.getAttribute('high'),
-            'condition': node.getAttribute('text')
-        })
-
-    # Return a hash of the weather that we just parsed.
-    return {
+    for i, node in enumerate(
+            dom.getElementsByTagNameNS(WEATHER_NS,'forecast')):
+        # stop if obtained forecasts for number of requested days
+        if i + 1 > days:
+            break
+        else:
+            # Insert the forecast into the forcast dictionary.
+            forecasts.append (
+                {
+                    'date': node.getAttribute('date'),
+                    'low': node.getAttribute('low'),
+                    'high': node.getAttribute('high'),
+                    'condition': node.getAttribute('text')
+                }
+            )
+
+    # Return a dictionary of the weather that we just parsed.
+    weather_data = {
   'current_condition': ycondition.getAttribute('text'),
   'current_temp': ycondition.getAttribute('temp'),
   'forecasts': forecasts,
@@ -47,87 +69,117 @@
   'city': ylocation.getAttribute('city'),
   'region': ylocation.getAttribute('region'),
}
-
-def usage():
-    print "Usage: weather.py [-cflv] zip-code\n"
-    print "-c\tSuppress the current weather.\n"
-    print "-f\tPrint the next two days' forecast.\n"
-    print "-l\tPrint the location of the weather.\n"
-    print "-v\tPrint headers for each weather section.\n"
-
-def main():
-    try:
-        # Attempt to get the command line arguments and options.
-        opts, args = getopt.getopt(sys.argv[1:], "cflv")
-
-    except GetoptError, err:
-
-        # Print the getopt() error.
-        print str(err)
-  
-        # Show the user the proper usage.
-        usage()
-  
-        # Exit with error code 2.
-        sys.exit(2)
-
+    return weather_data
+
+
+def create_report(weather_data, options):
+    """
+    Constructs a weather report as a string.
+
+    :Parameters:
+    -`weather_data`: a dictionary of weather data
+    -`options`: options to determine output selections
+
+    :Returns:
+    -`report_str`: a formatted string reporting weather
+
+    """
+
+    report = []
+    if options.location:
+        if options.verbose:
+            # Add location header
+            report.append("Location:")
+
+        # Add the location
+        locstr = "%(city)s, %(region)s\n" % weather_data
+        report.append(locstr)
+
+    if (not options.nocurr):
+        if options.verbose:
+            # Add current conditions header
+            report.append("Current conditions:")
+
+        # Add the current weather.
+        currstr = "%(current_temp)s%(units)s | %(current_condition)s \n" % weather_data
+        report.append(currstr)
+
+    if options.verbose:
+        # Add the forecast header.
+        report.append("Forecast:")
+
+    # Add the forecasts.
+    for forecast in weather_data['forecasts']:
+        forecast['units'] = weather_data['units']
+        forecast_str = """\
+%(date)s
+  Low: %(low)s%(units)s
+  High: %(high)s%(units)s
+  Condition: %(condition)s
+""" % forecast
+        report.append(forecast_str)
+
+    report_str = "\n".join(report)
+    return report_str
+
+
+def create_cli_parser():
+    """Creates command line interface parser."""
+
+    usage = (
+        "python %prog [OPTIONS] ZIPCODE DAYS",
+        __doc__,
+        """\
+Arguments:
+    ZIPCODE: The ZIP code for the region of interest.
+    DAYS: Days to forecast for (0 if only current conditions desired).
+"""
+    )
+    usage = "\n\n".join(usage)
+    cli_parser = OptionParser(usage)
+    # add CLI options
+    cli_parser.add_option('-c', '--nocurr', action='store_true',
+        help="Suppress reporting the current weather conditions"
+    )
+    cli_parser.add_option('-l', '--location', action='store_true',
+        help="Give the location of the weather"
+    )
+    cli_parser.add_option('-v', '--verbose', action='store_true',
+        help="Print the weather section headers"
+    )
+
+    return cli_parser
+
+
+def main(argv):
+
+    cli_parser = create_cli_parser()
+    opts, args = cli_parser.parse_args(argv)
+
# Check that an argument was passed.
-    if not args:
-  
-        # Show the user the proper usage.
-        usage()
-  
-        # Exit with error code 1.
-        sys.exit(1)
-
-    # Check that the first argument exists.
-    if args[0]:
-
-        # Get the weather.
-        weather = get_weather(args[0])
-
-        # Set the option flags
-        c = True
-        f = False
-        l = False
-        v = False
-
-        # Check the options
-        for o, a in opts:
-            if o == "-c":
-                c = False
-            elif o == "-f":
-                f = True
-            elif o == "-l":
-                l = True
-            elif o == "-v":
-                v = True
-      
-        if l and v:
-            # Print the location header.
-            print "Location:"
-
-        if l:
-            # Print the location
-            print weather['city'] + ", " + weather['region'] + "\n"
-  
-        if c and v:
-            # Print the current conditions header.
-            print "Current conditions:"
-
-        if c:
-            # Print the current weather.
-            print weather['current_temp'] + weather['units'] + " | " + weather['current_condition'] + "\n"
-
-        if f and v:
-            # Print the forecast header.
-            print "Forecast:"
-
-        if f:
-            # Print the forecast.
-            for forecast in weather['forecasts']:
-                print forecast['date'] + "\n  Low: " + forecast['low'] + weather['units'] + "\n  High: " + forecast['high'] + weather['units'] + "\n  Condition: " + forecast['condition'] + "\n"
-          
+    if len(args) != 2:
+        cli_parser.error("Not enough arguments supplied.")
+
+    zip_code = args[0]
+    if len(zip_code) != 5 or not zip_code.isdigit():
+        cli_parser.error("ZIP code must be 5 digits")
+
+    days = args[1]
+    if not days.isdigit():
+        cli_parser.error("Days to forecast must be an integer.")
+    days = int(days)
+    if days <> DAYS_LIMIT:
+        cli_parser.error("Days to forecast must be within 0 to %d" %
+                DAYS_LIMIT)
+
+    # Get the weather forecast.
+    weather = get_weather(zip_code, days)
+
+    # Create the report.
+    report = create_report(weather, opts)
+
+    print report
+
+
if __name__ == "__main__":
-    main()
-
\ No newline at end of file
+    main(sys.argv[1:])

I'm going to hope Thomas doesn't sue me for breaking copyright or something, and suck money out of my massive salary that I receive as a grad student.

EDIT: One final comment. Thomas, your code looked very clean and you did a fantastic job adhering to PEP 8!