How tags compress semantics

I just had a simple, yet (personally) powerful revelation—a moment of grokness, if you will. While searching through my Delicious account for a bookmark to a TED talk to link to in another blog post, I came face to face with a predicament that made me really stop and think.

I began my search by using the tag "ted", with which I've tagged all TED talks I've bookmarked. I have 78 TED talks bookmarked. The bookmark entries for these posts have 280 distinct tags, 1024 words total in their bookmark title fields, and 1668 words total in the comment fields. The predicament is, do I look through 78 posts to find the one of interest, or do I instead look through the 280 tags?

Or is it? If we rephrase the question, we see I'm really asking, "Can I find what I'm looking for faster using 280 words, or 2692 words?" See, those 280 tag words actually represent a compression of the semantics (the meanings) of the 2692 descriptive words. I can quickly scan 280 tags to identify the closest to my concept, giving me a significantly more manageable subset of posts to scan in more detail.

Tags seemed very straightforward and powerful before, for example, reading Clay Shirky's article on the power of tagging, but it took this moment to really understand the power behind them, much like the "A ha!" moment of seeing a binary search when you've always thought of search as linear.

Two side notes:

• I'd like to thank the developers of pydelicious for providing me the software to extract those statistics about my Delicious tags.
• It turns out the video I was looking for had the clip of interest removed due to copyright permissions, and so the real answer to the question was to Google it. Still, it was worth it for the thought.

Update of delicous.com bulk for tagger

While waiting for BLAST results this afternoon, I pushed an update to a script that lets you bulk for-tag Delicious bookmarks to friends in your network. You can obtain it from the link above, or with Subversion.

svn export https://gotgenes.com/svn/toolbox/trunk/delicious-bulk-for.py

This version brings in a little more privacy. Your account information should now be stored in a profile file (~/.dbf_profile by default) rather than be explicitly entered on the command line, which exposed your password to anyone looking over your shoulder. It was also a chance for me to get a little more acquainted with the Python standard library's ConfigParser. Additionally, it has an honest-to-goodness CLI parser now, powered by one of my favorite Python stdlib modules, optparse.

If you happen to use it, let me know if you have any problems. I have written zero unit tests for the script, so I consider the code fragile, but it improves as more and more people ask me for it. So far, though, the new version works for me.

(As a side note, I really miss the old del.icio.us name. Foo on delicious.com.)