Friday, January 31, 2014

So you're attending your first Python users group meeting / meetup

Full HouseCongratulations! You have decided to attend your local Python users group (PUG) meeting (or "meetup" in the Meetup parlance), putting yourself on a path for success in mastering Python, and programming in general. Here are a couple of tips to help you arrive prepared and ready to participate.

Attend

The most important thing to bring to a PUG meeting is yourself and your enthusiasm. If you're in a rush and don't have time to set up your laptop or anything else, don't worry, just attend! You will be welcome regardless of your technical and personal background. The global Python community expects an atmosphere of respect and welcoming at each event, especially for newcomers.

Bring a laptop

Each PUG will arrange its meetings differently. DCPython, for example, tended towards mostly presentations while I was a member. In contrast, OCPython has had less formal meetings where there's more interaction and audience participation. Be prepared. Bring a laptop just in case.

Prepare your laptop for Python

Everything in this section should be considered optional but very helpful. Try to get your laptop set up before the meeting. Don't fret if you can't or don't have time, though!. Wi-Fi access can't be guaranteed at a lot of venues, so it will be extremely helpful if you have all your software downloaded, and, if possible installed and configured, before the meeting.

If you get stuck with any of the tasks below, don't fret, and do attend anyway! You will find other attendees willing to troubleshoot and get you un-stuck, I promise. The Python community is among the friendliest and eager to help.

Know how to open your terminal emulator

Most Python programs are invoked via the command line (typically by issuing a command that looks like "python <some_file.py>"). Learn how to access the terminal emulator/command line for your laptop's respective operating system, such as PowerShell for Windows, Terminal for OS X, or GNOME Terminal for Linux.

Make it easy to get to the terminal emulator by putting a shortcut to it on your desktop, dock, or launcher. Oftentimes the presenter will assume you already have your terminal open and at the ready.

Configure your terminal emulator to your liking ahead of time. For example, I find the default font size in OS X's Terminal too small and the colors too pastel to the point of being illegible. Finding the right places to change these settings to suit your style (and eyesight) can take some time, so get them fixed ahead of time. You won't want to be fiddling with these settings while the presenter's nuggets of advice go flying by.

If you're unfamiliar with working on the command line, see Zed Shaw's Command Line Crash Course.

Have a working Python installation

If you have a Mac or a Linux laptop, the good news is you already have Python installed; proceed onward.

If you're a Windows user, you have a little work ahead of you. First, download a Python installer from the Python website. Just grab the latest version (Python 3.4 at time of publication), run the installer (double click the installer file), and go through the dialogs (you can trust the defaults).

The final step is to put Python on the PATH so that you can run Python from your command line. Here is a howto for Windows 7 users. (Note: instead of using "Python27" use a number matching your download, for example "Python34" if you downloaded and installed Python3.4)

You can verify you put Python on your path by opening PowerShell, typing in "python", and pressing the Enter key. If you see a prompt that looks like ">>>" you're good to go. If instead you get an error that "python" isn't found, try going back through the steps of setting your path. If you get stuck here, ask for assistance at your meeting, show your helper this blog post, and they should be able to get you back on track.

Have a text editor or IDE installed and configured

Python programs are plain text files with .py extensions. While you could use Microsoft Word (heaven forbid) to open and edit the files, you should use a specialized tool for editing code. If you're new to programming, you'll probably do fine starting with a good text editor that provides syntax highlighting and help with formatting your code correctly. If you're on Linux, you probably already have a decent editor like Gedit installed. If you're on Mac OS X try something like Sublime Text. If you're on Windows, try Notepad++.

If you have some experience in programming in other languages or have the interest in investing more time upfront, you can install and get started using an integrated development environment (IDE). If you don't yet have a favorite IDE for Python development, try PyCharm or Ninja IDE. IDEs provide not just a text editor but also a debugger, a test runner, an interactive console, and some form of project management, among other tools. Again, if your a newbie, just stick to getting a good text editor set up before attending the meeting.

Have pip installed

pip allows you to easily download and install other Python software libraries (called "Python packages") from the Internet. All you have to do is open your terminal, type in "pip install <PACKAGE_NAME>", and press Enter, and pip will fetch the package from the Internet and install it for you. pip can come in quite handy if you would like to try out any packages mentioned or shown in tutorials during the meeting.

If you installed Python 3.4 or newer, you already have pip installed. Proceed onward!

If you have Python 3.3 or older (including Python 2.7), you will not have pip installed by default (unless you used an alternative Python distribution like Anaconda). You can check whether or not you have pip installed by typing in "pip" on your command line and hitting the Enter key. If you get an error that the pip command cannot be found, you will need to install pip.

The pip documentation describes how to install pip. The easiest method uses the get-pip.py script. If this fails for you, you may need to try one of the alternative install options.

This may be a more difficult step for Windows users. First, make sure that Python is on your PATH (see "Have a working Python installation" above). Next, download the get-pip.py script by right-clicking the link, selecting "Save As", and designating a location to save it, such as your "Downloads" directory. Next, open PowerShell and use the cd command to move to your download directory (e.g., "cd $HOME\Downloads". Finally, in PowerShell, run python get-pip.py.

After the installation of pip completes, you should be able to access pip from the command line by typing in pip and hitting Enter. Again, if you get stuck, show this blog post to someone at the meeting and he or she should be able to help you.

Bring some extra equipment (optional)

In an ideal PUG event, organizers will have time and resources to set up the event area with ample access to power, Wi-Fi, and the like. If your local PUG is new or just getting started, though, the organizers may have a hard enough time just finding a location to meet, let alone have time to prep the area for lots of tech. Here's some equipment that you can bring along that could help everybody have a better experience.

  • A wireless hotspot device
  • Display dongles, adapters, and cables
  • Power strips
  • Business cards

Have a good time; let others have a good time

PUG meetings are social events, not formal meetings. Some may run on tight schedules because they pack a lot of content in, however, don't confuse that structure with formality. Everyone attends PUG meetings first and foremost to share in the joy of the Python programming language. Unless you're presenting (and usually even if you are), you don't need to dress up; just come in something comfortable and inoffensive.

Do conduct yourself appropriately at the meeting, and err on the side of professionalism. The PyCon Code of Conduct provides a good set of guidelines. If another attendee behaves inappropriately towards you or makes you feel uncomfortable or unwelcome, do feel free to confront the attendee on her or his behavior directly or otherwise raise the issue with the PUG's organizers immediately.

On behalf of your local PUG, we look forward to seeing you soon!

Thursday, December 6, 2012

Putting syntax-highlighted code into presentation slides or documents

Brogramming with Tom

Want to include syntax-highlighted code in your presentation? A project called Pygments provides a very helpful tool for this. First, you need to install Pygments:

  • If you're on OS X and use MacPorts, you can fetch it with
    sudo port install py27-pygments
  • If you're on Ubuntu/Debian, you can get it with apt-get with
    sudo apt-get install python-pygments
  • Or you can fetch it on any platform using pip* with
    sudo pip install pygments

Installing Pygments will also install a command line utility called pygmentize**. We can use this tool to help us format code for use in a presentation or document with the following steps

  1. Open the terminal and do
    pygmentize -f rtf <PATH_TO_CODE_FILE> | pbcopy
    if you're on OS X, replacing <PATH_TO_CODE_FILE> with the actual path to your file of interest (use xsel -b instead of pbcopy if you're on Linux, or clip if you're on Windows). This will copy a colorized markup of your code to your clipboard.
  2. Paste the contents of the clipboard to your document or slide
    • This should be as simple as using Edit → Paste (COMMAND + V on OS X, CTRL + V on anything else)
    • If you're using PowerPoint, you instead need to use Edit → Paste Special (CTRL + COMMAND + V shortcut on Mac Office) and select "Formatted Text (RTF)". Also, you may need to create a new text box first, as the default text box will unhelpfully try to insert bullet points for you. Alternatively, you can just remove the bullet points by highlighting all the code and clicking the bullet point button (sometimes having to do this multiple times...)

That's it! pygmentize can parse files in a wide number of programming languages, as well as output in many different formats; for example, if you use LaTeX/Beamer, you can get TeX output by using -f tex. You can look at the Pygments documentation on lexers and formatters to see the full list of languages and output formats Pygments and pygmentize support.

* Don't have pip? Go get it!
** If you installed with MacPorts, you'll probably have to use pygmentize-2.7 instead of pygmentize.

Saturday, September 10, 2011

In remembrance of September 11, 2001

American Flag picture - photo of the American Flag

Today, I set aside some time from preparing for my defense to read through the New York Times' tribute to September 11, 2001, and reflect upon what that day and the events which have followed mean to me. I did not expect to feel so profoundly moved as I read through the stories, and in particular, I could not help but feel struck afresh with anguish and cry as I carefully paged through the moving slideshow of the rise and fall of the towers of the World Trade Center.

Still, other articles reminded me of my core belief in our country — in the people of our country. In spite of the willful erosion of personal privacy and civil liberties and civil tongues, in spite of the tragic sacrifice of human lives both domestic and foreign, in spite of ongoing anti-intellectualism, in spite of continuing sexual, religious, and racial intolerance, in spite of a bitterly polarized political climate, in spite of our continued mismanagement of our environment — in spite of all this, I still believe that the story of the United States of America is one of hope. If ever there were a country to break pre-conceived notions, to defy intolerance, to unite for a greater good, to show that change can be for the better, to overcome adversity, then it must be ours.

Ten years ago, I stood with friends in an undergraduate dorm room and watched the World Trade Center towers collapse and the Pentagon smolder. Now, here I stand to defend my Ph.D., and I can not help but feel grateful for all the opportunities I've had thanks to having a life here in the USA. I am not always proud of our country's actions, but I am proud of what our country stands for: truth, liberty, and justice for all. Our story is marked by tragedy and marred by missteps, but it is, indeed, the story of hope. I will always remember.

Tuesday, August 9, 2011

The bog of eternal singlehood: college towns beyond college

Life in a college town

As I apply for jobs, many of which, for better or worse, are at academic institutions, I keep having a nagging feeling tugging at the back of my mind, like the tantrum-throwing three year old desperate for that Yo Gabba Gabba doll tears at her parent's arm in aisle 14 of the local Target. This pressing thought which brings me so much strife: I'm just not sure if I can take living in yet another land-grant college town.

Don't get me wrong—there are many great things about living in a college town. Life is generally quite pleasant and quiet, save football weekends. The cost of living is usually fantastic. They also tend to be family friendly, with quaint little farmers' markets and little local restaurants and shops. They also tend to be fairly progressive and open-minded, and support culture and art to a greater extent than you'd expect from such a small population.

Yes, for many people, a college town is a rather idyllic place. There is a specific subpopulation in these college towns, however, for whom the experience becomes utterly hopeless. This subpopulation: those who move to college towns, are not college-aged, and arrive without a significant other. Meet those requirements, and you're basically hosed until you escape. It is the bog of eternal singlehood.

I mean, let's take an honest look at the candidates in the dating pool in a college town for those who already hold one or more higher education degrees:

  • College kids: I'm sorry, did you not see the word "kids" there?
  • Grad students: Emotionally unstable semi-adults who incorrectly concluded that the panacea to their life problems was to get yet another degree.
  • Postdocs: Does the sound of frantic typing as they try to finish their latest lit review during the act of love-making turn you on?
  • Junior faculty: Ah, the less youthful, less healthy, more stressed versions of postdocs. Yes, I'm sure you had a good reason behind that choice...
  • Staff: They probably arrived there because of a significant other; if they are single at this point, they're looking for an opportunity to flee, not to stay.
  • Hipster/Hippie Townies: It's okay, so long as their friends never find out they're sleeping with you. Oh wait, it's a small college town...
  • Folk in the surrounding countryside: don't be surprised if you're viewed an over-educated, heathen, pinko socialist who never learned how to do anything actually useful (all of which could be accurate assessments)
  • People in the nearest city... five hours away: They're already pairing up with equally smart, young, attractive, better-paid competition that had the foresight to not force the issue of a long-distance relationship on the first date.

As a consolation, you will find great friends, for whom your sad, lonely, single self will serve as a reminder of why they need to stay committed to their own relationships.

With complete seriousness, I've found a tremendous amount of personal growth in the college towns I've inhabited for the past twelve years, and certainly, the quality of friends I've found in them has been unsurpassed. I admit that location is really only one part of the whole romantic equation.

Anyway, we'll see what the future brings. Maybe I'll finally join the young guns in a big ol' city, myself. Or maybe I'll find the the one who breaks the mold. Or maybe it'll just be the status quo, but hey, there are far worse bogs out there!

Friday, April 29, 2011

Let's talk: designing inter-cellular circuits through synthetic biology

True phone

Thursday, GenBioOrg brought in Prof. Ron Weiss to speak about his work in designing biological circuits, and this crazy, sometimes hyped field of synthetic biology. Prior to Prof. Weiss's talk, while I could appreciate the idea of synthetic biology, I mostly regarded it as a somewhat foolish pursuit, on account of the amount of fundamental biology we still just do not know. In many ways, my field of computational systems biology relies on building and testing models from "Swiss cheese knowledge", where gaps prevail (e.g., protein-protein interaction networks built from yeast-two-hybrid studies with high false-positive rates, or microarray analysis suffering from from the high-dimensionality, low-sample conundrum). Thus, whatever decries the prematurity of systems biology goes doubly so for synthetic biology, for which systems biology provides a central strut. The rationale is, if you don't understand it, how can you manipulate it? Of course, as I've learned repeatedly (but have failed to generalize), "You don't need to understand the internal combustion engine to drive a car."

Well, Thursday afternoon, Prof. Weiss deftly reminded me of this reality through his combination of humility-tempered optimism, his impressive collection of proofs-of-concept, and his insight for possible applications. He presented a number of intriguing biological circuits in his talk, but I felt most excited by his work on pattern formation through synthetic inter-cellular signaling networks (behind a Nature paywall, sorry). In this work, Weiss and his colleagues created a population of "receiver" bacteria cells, which had a genetic circuit that would cause cells to fluoresce (light up green) at a moderate concentration of a molecule called acyl-homoserine lactone (AHL).

To make the receivers fluoresce within a specific concentration of AHL, Weiss and colleagues actually made the receiver cells fluoresce (i.e. "be on") by default. They then created two AHL-detection circuits with very different input thresholds: a high-detection circuit which activates in the presence of large amounts of AHL, and a low-detection circuit which activates in low amounts or in the absence of AHL. Weiss and colleagues wired both detectors to the same output: when activated, they repressed ("turned off") fluorescence. If you're familiar with electronics, you'll see that Weiss and colleagues constructed a NOR gate, where the inputs are "high AHL" and "low/no AHL". If you're a programmer, you might think of the condition for fluorescence as

if not (ahl_level > high_threshold) and not (ahl_level < low_threshold):
    cells.fluoresce()

Weiss and colleagues then developed "sender" cells containing a circuit that caused synthesis and secretion AHL when exposed to tetracycline. When a colony of sender cells was placed in the middle of a "lawn" of receiver cells and exposed to tetracycline, the sender cells emitted AHL, which then diffused as a radial gradient from the colony, resulting in a concentric ring of fluorescence around the sender colony, but not immediately touching it, like a bullseye. That is right by the sender colony, the AHL was highest, and so the high-detection AHL circuit shut off fluorescence and left those cells dark. A little further out, the levels of AHL that diffused from the senders was at a more moderate amount, so the high-detection and the low-detection circuits remained off, allowing those cells to fluoresce. Beyond those cells, the levels of AHL were too low, and though the high-detection circuit remained off, the low-detection circuit turned on and repressed the fluorescence, again.

Colony of sender cells, fluorescing red, placed in a lawn of receiver cells. The sender colonies secrete signaling molecule AHL, which diffuses through the media. Receiver cells a sufficient distance from the colonies will receive enough AHL to fluoresce green, while those too near or too far will receive too much or to little AHL, respectively, remaining dark. [Image obtained from Ron Weiss with permission, modified by CDL to include labels.]

While this makes for pretty pictures, taxpayers rest assured: glowing cells are only the proof of concept. This research has major implications for practical applications, for example, in stem cell research, tissue engineering, and bioengineering.

As a high school student, I felt incredibly excited to learn the answer to the question, "How can a ball of indistinguishable cells turn into a brain, limbs, skin, etc.?" The answer, as those of you with some developmental biology background know, is "Through protein gradients," and more specifically through transcription factors and their co-activators and co-repressors. Beginning with your mother's egg cell, there already existed protein gradients which pre-determined the regions that formed your head, or your feet, or your inner organs, and as your zygotic cells divided, these protein gradients begot even more protein gradients, in a beautiful choreography perfected through billions of years of evolution. This research by Prof. Weiss and his colleagues demonstrates that synthetic biology may provide a means to not only guiding stem cells (either derived from an embryo or returned to their embryo-like stage) through the difficult process of differentiating into other cell types when cued by specific protein concentrations, but also the means to create colonies of cells capable of producing protein gradients. Through a successful combination of these sender-recipient circuits, we could achieve multiple types of differentiated cells, and maybe even self-organizing tissues, all from the same culture of stem cells.

Likewise, this research has important implications in mixed cell cultures. For example, the liver is primarily composed of cells called hepatocytes, which perform most of the functions of the liver, such as detoxification, lipid homeostasis, and blood plasma production. However, by culturing hepatocytes together with another cell type found in the liver, called liver sinusoidal endothelial cells (LSECs), the hepatocytes maintain their "liver-ness" far better than when cultured alone. Weiss's research implies that we may some day be able to develop synthetic "surrogate" cells to support cells that are characteristically difficult to maintain ex vivo by providing important intercellular signals.

In terms of bioengineering applications, such as biodiesel or pharmaceutical production, a major stumbling block has been the difficulty in engineering biological systems with the biochemical capacities necessary to carry out each step necessary to manufacture a complex molecule. Weiss's research suggests growing practicality in molecule manufacturing by designing chains biological pathways that exist in separate organisms, much as the case for deep-sea vents ecosystems.

Two other profound discoveries that Weiss presented were completely counterintuitive to me: adding complexity to a biological circuit tends to 1) bring about more digital (on/off) behavior rather than analog (continuous gradient from low to high) behavior, and that coupling components tends to reduce noisiness in the circuit rather than increase it. Although I do not have time to recapitulate Prof. Weiss's demonstrations of these emergent behaviors, I encourage you to browse through his publications yourself.

The last two points I'd like to note from Prof. Weiss's talk are the following quips, which I found particularly encouraging (paraphrasing). First:

Computational simulation is absolutely central to synthetic biology. We are beyond the point where we can design biological circuits through intuition alone. —Prof. Ron Weiss
This statement makes me feel validated for pursuing a background in computational biology. Second:
We've been working on a project for eight years now that we still haven't published results from. We're very close, though. It will be just another year or so. At least, that's what I tell my graduate student. And the graduate student that takes the project after she graduates. And the one after she graduates. —Prof. Ron Weiss
Researchers with careers as illustrious as Prof. Weiss's can come in and dazzle us grad students with tales of field-changing success, and I think this gives unreasonable and unwarranted expectations of how our own research paths should go. Certainly in my case I've felt that because I've struggled, I must not be successful, because it rarely seems the successful people struggled. It's refreshing to see an admirable figure in his field open up and show vulnerability by admitting that, even to this day, he has his struggles.

To summarize, here are my takeaway thoughts from Prof. Weiss's talk:

  • The time for synthetic biology research is now.
  • Researchers can engineer cell-cell communication, beginning the era of human-designed mixed cell cultures.
  • Even excellent researchers struggle.

Monday, March 28, 2011

Driven by the pursuit of proficiency

velocity

I am looking for a job, and as this is only the third time in my life job hunting, I have sought advice anywhere I can get it. Like most universities, Virginia Tech has a Career Services office, so I consulted their website to help get things rolling. They suggest beginning with a self-assessment, and the very first item of this self assessment bluntly asks, "What do you want to achieve in your work?" While this question is frustratingly broad, it is fair game; one could expect such a question in an interview, and one certainly must have an answer ready.

I have never felt guided by some vision of how my life should be. I mean, sure, when I was 8, I wanted to fly an F-14 Tomcat and shoot down commie MiGs because that made you a hero, and when I was 13, I wanted to be a Marine Biologist because of National Geographic and NOVA PBS shows, and when I was 18 I wanted to be a physician because that's what all biology majors intend to be. Each one of those were fantasies—spurious projections of the possible me, based on the immature and incomplete value system I held at the moment. When I gave up on the gauntlet of medical school my junior year at UGA, I also gave up pretending I could calculate long term career trajectories. Despite blowing off this central tenant of many a cookie-cutter career book and commencement address, I've thus far avoided becoming a complete and utter catastrophe of a human being, so I continue to make without.

Now I'm a grad student in computational biology—the result of a few simple ingredients: 1) I've enjoyed computer programming since high school, and 2) I've found biology fascinating since the days of reading Zoobooks at the dinner table. I have also loved video games since grade school, but I didn't take the career path of a video game programmer because I didn't feel like I would make a substantial contribution to humankind. In contrast, I quit trying to become a physician—a career in which I would have had a direct and tangible impact on other people's lives—because I saw the competition was better than I was at jumping through the med school application hoops. (I also love playing guitar, but let's be realistic—although I've re-evaluated that option and there are worse things.)

According to motivational speaker Dan Pink, motivation boils down to three needs: mastery, autonomy, and purpose (video below if you are unfamiliar with Pink's theory). From that standpoint, I didn't feel a sense of mastery in my quest to become a physician, and I didn't feel a sense of purpose in my pursuit of game programming, but with a decent grasp of biology and a propensity for programming, computational biology seemed a good fit. Now the question stands, has it been?

From the standpoint of fulfilling the need for purposeful work, I have to say I have certainly experienced a boost in motivation after switching research groups, due in large part to shifting the biological subject from bacteria to in vitro liver tissue culture systems, which has more immediate implications for human health—a subject which still motivates me.

Considering proficiency, though, I feel very uncertain about my path in research. My RSS feeds continue to fill up with table-of-contents from journals faster than I can screen them for interesting abstracts. Also, although I don't reading through literature in the field as much as before, I still just dislike doing it. I think this indicates a major obstacle to a career in research because it breaks the virtuous cycle of positive feedback: what we like, we do more of, so we get better at it, which makes us like it more and do more of it, which makes us better at it, and so on.

It's not clear I've grown much as a presenter, either, though it's not for lack of opportunities. I've given at least one presentation a month, sometimes several, mostly to my two research groups, but with some conference and departmental talks, as well. While I've gotten better at recognizing the work pattern that goes into preparing a presentation, I don't feel I've been able to reduce the time it takes to prepare them, and while I feel I've improved in delivery technique, I feel disappointed at how little I've improved given the amount of time I've invested. This said, I have discovered I enjoy delivering a presentation for which I've prepared adequately, which I attribute to the performance aspect.

If we take a look at the most important currency in academia, publications, I'm far from flush, with one co-authorship on a book chapter, one second-authorship on a collaboration paper, and one first-authorship on an original research article (Open Access, yay!). I'm working on another paper currently, and should begin another one prior to defending in June. It's not a sparse record, but it's unremarkable. If I have learned anything from The Dip, it's that I want to do remarkable work.

I really want to become proficient, but after nine years of working in academic research from undergrad, to research tech, to grad student, I feel it's escaping me in this pursuit. My research experience feels like long periods of slogging, largely devoid of any feedback, let alone positive feedback (which is rare and fleeting). I want a research experience that breaks that mold, but I'm willing to accept I might not find one, and I'm becoming more enthusiastic about switching tracks to a career where I can make a genuine success of myself. I want that virtuous cycle of positive feedback. I want to get excited and make things!

So, to the future interviewer who asks, "What do you want to achieve in your work?" I answer this: "I want to achieve remarkable proficiency." Why settle for less? Life is short; let's find a way to become awesome while we still can.

Friday, January 7, 2011

Common Good: Adding a Creative Commons License button to your Blogspot (Blogger) blog

2500 Creative Commons Licenses

I have intended to place the contents of this blog under a Creative Commons (CC) license for a long while, especially given that all the attractive photos I love to use in these blog entries come from Creative Commons-licensed content on Flickr. For those unfamiliar with Creative Commons licenses, they explicitly permit re-use of creative works a priori. Provided you follow the criteria of the particular CC license of the work (usually simply attributing the original creator), you may simply use, or even modify the work, without the need to contact the original creator for direct permission to do so. Read the Creative Commons' website for more detail.

I had let this task linger far too long, so, spurred on by a recent email exchange with Mark Hahnel of Science 3.0, I finally felt the inspiration to get this done. Unfortunately, I didn't find the top-ranked pages in Google searches for placing a CC license button on Blogger/Blogspot blogs very helpful, so I decided to just figure it out. It turned out to be a simple process, so I documented it and present it here, in step-by-step format (all under the CC-BY license, of course):

  1. Go to the Creative Commons website and choose a license
  2. Copy the HTML that CC presents you after you've selected your license
  3. Go to your blog's page, and click the "Design" link in the navigation bar at the top. Alternatively, go to your Blogger author page and click the appropriate "Design" link for your blog there.
  4. Click the "Add a Gadget" link in the design editor (should be one at the bottom of the area).
  5. Click the link to add an "HTML/JavaScript" gadget.
  6. Add a title, like "CC License", paste the HTML of your license button that you copied from the CC website into the contents box, and click "Save".
  7. Optional: You'll be back at the design editor; double-click the new CC License gadget and move it below your Blog Posts gadget (or some other fitting area).

That's it! Your shiny new CC license button should appear where you placed it.

Tuesday, August 17, 2010

Cell on Wheels: Famous Scientist Roller Derby Names

Valtron 3000

This past Sunday I had an exciting first Roller Derby experience when I went out to support the NRV Rollergirls in their bout of against the Mason Dixon Roller Vixens. For those unfamiliar with modern roller derby, it is a contact, point-based sport in which the players of two teams skate in a circuit, trying to help their point-scorer, designated a "jammer", pass the other team, whilst simultaneously using any blunt part of their bodies above the knees to prevent the other team's jammer from passing them. The rest of the details you can pick up as you watch.

Today, roller derby is largely an all-women's sport, where men play supporting roles as coaches and referees, which is sort of an interesting role-reversal. My favorite part of the roller derby culture is that all participants don noms de guerre, which usually involve clever (or even tacky) wordplay, including the use of homophones, oronyms, and portmanteaus, to spin references to pop culture, history, or anything otherwise generally familiar, with a violent, aggressive bent. For example, my favorites for the NRV Rollergirls are Huck Finish Her and Eleanor Blows B. Dealt, but other good examples include Baby Ruthless and Bloody Holly from the enjoyable film "Whip It", or Hyper Lynx, Auntie Christ, Beyonsláy, and Nina Millimeter who have been appeared in various articles in the New York Times.

As I lay awake Monday, unable to sleep with anxieties about upcoming presentations, needing to develop an entirely different computational approaches for research, and general insecurities about my place in life, I started thinking about how amusing derby names are, and then tried inventing some of my own. Then I had a revelation that it would be hilarious if there were derby names based off of (relatively) famous scientists. Once I got a few, I started jotting them down. Here's a list of ones I've come up with, so far:

I would like to point out the obvious that these are all plays ot men's names, which is ironic given that roller derby is played predominantly by women. This is disheartening for three reasons:

  1. I could only come up with four "famous" female scientists offhand: Marie Curie, Jane Goodall, Rosalind Franklin, and Lynn Margulis.
  2. I couldn't come up with a clever spin on any of them.
  3. Did I mention I could come up with only four famous scientists who are women? This reflects poorly on me, but I think also on the inequality that exists in scientific education and scientific research, both of yesteryear but also today. This is another issue for another blog post.

If you have any suggestions for scientists I've missed (particularly famous women who are or were scientists), or better suggestions for the ones I've attempted to spin, I encourage you to post them in the comments, or put them in your own blog and post the link below.

Update 2011-05-17: Randall Munroe published a relevant comic on the final points on female scientists:

Thursday, January 21, 2010

Interactive sandboxes: using IPython with virtualenv

sandbox baby

A very helpful blog post on IPython and virtualenv by Pedro Algarvio inspired this one. The advice found there takes you 90% to where you want. I'll recap on that 90% but explain and give the extra 10%. I am indebted to Pedro for laying down all the hard work.

First of all, if you are unfamiliar with Ian Bicking's virtualenv package, you should know two things about it:

  1. virtualenv allows you to develop in sane, aseptic, "sandbox" development environments, switch between them seamlessly, and maintain harmonious order in your Python universe.
  2. virtualenv is certifiably awesome. Proceed directly to installing it (especially in combination with pip)! Do not pass Go! Do not collect $200!

Arthur Koziel already wrote a really good tutorial on using virtualenv, and, in fact, you'll probably find working with Doug Hellman's excellent virtualenvwrapper more convenient; in this case, Doug already wrote an excellent virtualenvwrapper tutorial, too. I've mentioned IPython in a previous blog post, so I won't cover that here, either. Instead, let's cut to the chase and get IPython and virtualenv playing well together.

Ordinarily, IPython, commonly installed system-wide by your preferred package management system, remains oblivious of an activated virtualenv environment, and will just mill about importing packages and modules from the system, rather than the sandbox. This gives two obvious solutions: either 1) configure the system installation of IPython to work with virtualenv, or 2) install IPython in each virtualenv environment. Doug Hellman wrote a nice tutorial on doing the latter approach; here, we'll focus on the former, which I prefer, since it means having to only install IPython once.

IPython (being a Python program) can read and execute Python scripts during launch; we'll use this mechanism to modify IPython's launch to hook into the virtualenv environment we're currently in. First, we'll tell IPython that we want to execute some code in a at startup. If we go to the $HOME/.ipython/ directory, we'll find a file called ipy_user_conf.py. Open the file in your editor of choice, locate the function main(), and at the within that function (I suggest at the end), insert the following line:

execf('~/.ipython/virtualenv.py')

Next, we need to create this file. Still in the $HOME/.ipython/ directory, create a new file called virtualenv.py and open it with your editor. Next, add these contents to this file:

import site
from os import environ
from os.path import join
import sys

if 'VIRTUAL_ENV' in environ:
    virtual_env = join(environ.get('VIRTUAL_ENV'),
                       'lib',
                       'python%d.%d' % sys.version_info[:2],
                       'site-packages')

    # Remember original sys.path.
    prev_sys_path = list(sys.path)
    site.addsitedir(virtual_env)

    # Reorder sys.path so new directories at the front.
    new_sys_path = []
    for item in list(sys.path):
        if item not in prev_sys_path:
            new_sys_path.append(item)
            sys.path.remove(item)
    sys.path[1:1] = new_sys_path

    print 'VIRTUAL_ENV ->', virtual_env
    del virtual_env

del site, environ, join, sys

If you took a look at Pedro's version of virtualenv.py, you'll recognize most of his code here. The important difference lies in the trickery we play with sys.path in lines 12 through 22. These lines were inspired by a solution to a problem presented by using site.addsitedir(), which adds new paths only to the end of sys.path.

Adding paths to the end of sys.path has, for our purposes, the undesirable side-effect of allowing system-wide packages and modules to preempt locally installed ones, since Python searches through sys.path for modules and packages in first-to-last order. I have filed a feature request for site.addsitedir() to allow inserting new paths at the beginning of sys.path; in the meantime, we'll use this hack inspired by the modwsgi programmers, which keeps track of the paths before and after the call to site.addsitedir(), then swaps the position of the new paths from the end, to just after the first element, '', which represents the current working directory (which should preempt every other path).

IPython will have access to the contents of the virtualenv sandbox in which you're currently working. For example, if I activate my networkx virtual environment, which has the latest development version of the NetworkX graph library, then fire up IPython, I get the following result (note the line that begins with VIRTUALENV indicating I'm accessing the virtualenv sandbox):

(networkx)$ ipython
VIRTUAL_ENV -> /home/lasher/.virtualenvs/networkx/lib/python2.6/site-packages
Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) 
Type "copyright", "credits" or "license" for more information.

IPython 0.9.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: import networkx

In [2]: networkx.__version__
Out[2]: '1.1.dev1518'

When I leave the sandbox (e.g., by using virtualenvwapper's deactivate command), I return to accessing the system-wide default install of NetworkX:

$ ipython
/var/lib/python-support/python2.6/IPython/Magic.py:38: DeprecationWarning: the sets module is deprecated
  from sets import Set
Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) 
Type "copyright", "credits" or "license" for more information.

IPython 0.9.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: import networkx

In [2]: networkx.__version__
Out[2]: '0.36'

So there you have it: one IPython to rule all your virtualenv sandboxes!

Friday, November 13, 2009

Time out: deterring brute force SSH attacks with iptables

Brute Force

These are some simple iptables rules I keep around on my firewall to deter brute force SSH attacks. The original idea came from Dominik Borkowski, a sysadmin at VBI.

If the attacker attempts more than 4 connections within a minute, these rules temporarily blacklist them for the next minute—or as I like to say, "put them in time-out". Their packets will be dropped; to them, it will seem that the machine simply disappeared from the intarwebs. The rules will also log such violators to your syslog. I've found them very effective. Most scripts that these crackers run will drop off after one iteration and look for lower hanging fruit.

Of course, if you forget your password, or have a habit of making a couple of simultaneous connections to your computer, the door will shut on you, too, but the good news is that you'll only be blocked for a minute. More draconian methods that append to actual blacklists have a habit of locking their owners out. (Not that I'm speaking from personal experience at all.) The rules escape this pitfall but will prove just as effective.

## Below includes very successful deterrents for SSH brute force
## that allows a maximum of 4 connection attempts within a minute.
iptables -A INPUT -p tcp -m state --state NEW --dport 22 -m recent --name sshattack --set
iptables -A INPUT -m recent --name sshattack --rcheck --seconds 60 --hitcount 4 -m limit --limit 4/minute -j LOG --log-prefix 'SSH attack: '
iptables -A INPUT -m recent --name sshattack --rcheck --seconds 60 --hitcount 4 -j DROP
iptables -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT

I keep this in a firewall (shell) script that controls iptables rules and executes on bootup. If there's sufficient demand, I can make the entire script available.

Wednesday, July 1, 2009

"Who arrre you?" Getting the hostname back in the Jaunty GDM greeter

Remendos - Patchwork

The latest Ubuntu release, 9.04, codename "Jaunty Jackelope", has turned out to be one of the best, maybe even on par with the "Gutsy Gibbon" release. The aesthetics definitely got some love; for example, if you're not running the "Dust" theme, you're missing out. [Hint: go to Preferences -> Appearance -> Theme and select "Dust"] The GDM greeter login screen looks the best of any Ubuntu release.

Unfortunately, a little bit of usability got lost along the way; most notably, the hostname no longer appears anywhere on the graphical login. This probably bothers only a minority of people, but our lab, for example, just updated all its machines to Jaunty, and we couldn't tell from the greeters which machine belonged to which hostname without logging in. I sat down this afternoon for a few minutes to figure out how the GDM themes work. It turns out they're just coded as fairly simple XML, and looking at other themes, I eventually figured out what to tweak. This patch will bring back the beloved hostname to the GDM login.

To use this patch, just do

sudo patch -p0 < /path/to/hostname_patch_for_Human.xml.patch

Now you'll no longer have to look at login screens and wonder, "Who arrre you?"

Update (16:17): Apparently Blogger's software won't allow XML in their pre tags, so I just hosted the patch on my server instead. All the more reason why I need to host my own blog with Wordpress or something soon...

Sunday, June 28, 2009

Is it in one's Nature?

Bonsai Moon

Today is Sunday, a day of rest to some, but to heathens with Monday meetings like myself, a day of catching up and doing all the things we thought we'd get done earlier. Unfortunately for me, our LDAP server that gives us access to the network is down... again... for the third weekend in a row, preventing access to our workstations, data, and worst for me, my research notebook, which I keep on our group's wiki. I admittedly felt a strong temptation to get out, enjoy the sunshine, and play a little guitar, but here I sit, in the cold, gray, fluorescent-lit cube. I'm here because I'm trying to be less incompetent as a scientific researcher.

One of the things that particularly makes me feel incompetent is my lack of knowledge of scientific literature, and (to a greater extent?) my lack of enthusiasm for reading it. I don't know why, and I give myself grief for this, but I often find reading scientific papers just plain boring. The funny thing is, I really appreciate science, by which I mean the technique of elucidating one's knowledge of the world through rigorous, reproducible means, and keeping a skeptical mindset, especially when it comes to one's own work. Likewise, I will never cease to find biology or computational technology among the most satisfactory pursuits for the very limited time and energy I have here on this good Earth. Yes, science, itself, is awesome, but the excitement of it gets stripped away in a lot of formal education environments, and for me, in the way scientists present it in their formal literature.

I have to qualify that last statement as pertaining to myself because I have colleagues who clearly find the literature still stimulating; a good example is Arjun Krishnan. At any given point, Arjun can tell you a few relevant papers he's read on seemingly any subject, he can give you solid summaries, and he turns it into good research questions, some of which he's following up on. He's a paragon of the Good Graduate Student; I have no doubts Arjun is going to be a superstar scientist in whatever field he ends up in, if not in general. I am certainly no Arjun, however, so I have to focus on humbler goals.

One of our tasks as students in the Murali group is to canvas over a dozen of the journals in bioinformatics and computational biology and scout for pertinent articles. I decided to use my "downtime" to have at the growing stack of journal headlines in my RSS feeds, and since I needed a place to start, I thought I'd tackle my Nature stack, which I'd neglected since the end of May. This meant a back log of over two hundred articles. I scanned through each headline, pausing at ones that had life sciences subjects, opening up a few that had keywords that caught my attention, taking a genuine look at a few of those, and skipping over the rest. At the end of the process, I felt really disappointed.

Of the several hundred articles, I only wound up reading three research highlights, the abstract of one letter, the abstract and some of the figures in another, and the abstract and some of the methods in another, and none of these proved at all pertinent to research I am supposed to be doing now.

Worth pointing out more, at no time did I read the title of a full-fledged research article and think, "Wow, I should read that," or even, "Gee, that sounds interesting." The vast majority of the titles just struck me as extremely esoteric, and this confuses me the most. Aren't Nature, Science, and PNAS supposed to have articles that are of interest not just to a specific field, but to the entire scientific community? But you know, I'm not interested that "GOLPH3 modulates mTOR signalling and rapamycin sensitivity in cancer", or that "Histone H4 lysine 16 acetylation regulates cellular lifespan", or in "A newly discovered protein export machine in malaria parasites". I fail to feel these discoveries shaking my perception of the world around me, of giving me a new topic to explore, or helping me make my own discoveries.

Nature is a journal that can make tenure, a journal where scientists experience great thrills for getting in and great envy when their colleagues do, a journal that says, "I publish like a boss!" It's a journal where my boss says, "You should be reading it anyway." So obviously, like so many things in scientific research, I just don't get it. And now, after my attempt to gain a little face today, I'm right back to where I started. Go ahead, just say it—I'm the worst scientist in the world. I'm a cotton-headed ninny muggins.

Thursday, April 30, 2009

How symbolic: on removing symlinks in Bazaar VCS

the weakest link

I have a strong affinity for distributed revision control systems, and my favorite has been Bazaar VCS (a.k.a., bzr). Like any piece of software, bzr has its quirks and shortcomings. Tonight, I encountered its rather tricky behavior when it comes to symbolic links (symlinks).

I keep my configurations under revision control, which gives me the benefits of rolling back changes when I inevitably break things, and of setting up home on a new system, even a remote one, very quickly and easily. All was well, but I discovered that when I naively placed my .vim/ directory under the repository, I added a ton of symlinks to files in /usr/share/vim/addons. These symlinks were present because I used Ubuntu's vim-scripts and vim-addon-manager packages to install these addons to my Vim profile, which essentially just sets up symlinks to the addons, stored in /usr/share. It's a pretty reasonable system, actually, but it doesn't make sense to have these symbolic links stored in my branch. I can't guarantee that each system I work on will have the files the symlinks point to, therefore, I thought it best to remove them. Therein I encountered a sticky issue with bzr: you really can't remove symlinks from its revision tracking easily.

I thought I could be clever and write a simple one-liner in Bash to remove all the symlinks presently tracked by bzr from further tracking, but still leave them on the file system (I still need the symlinks there, after all, or my Vim goodness will break).

for file in `bzr ls -V`; do          # use bzr ls -VR in later versions
    if [ -h $file ]; then            # see if the file is a symlink
        echo "Removing $file";
        bzr rm --keep $file;         # remove from tracking, not the FS
    fi;
done

Okay, so I reformatted it for annotation, but trust me, it fits on one line. Anyway, I immediately encountered problems, getting this as output:

.vim/compiler/tex.vim
bzr: ERROR: Not a branch: "/usr/share/vim/addons/compiler/tex.vim/".
.vim/doc/NERD_commenter.txt
bzr: ERROR: Not a branch: "/usr/share/vim-scripts/doc/NERD_commenter.txt/".
.vim/doc/bufexplorer.txt
bzr: ERROR: Not a branch: "/usr/share/vim-scripts/doc/bufexplorer.txt/".
.vim/doc/imaps.txt.gz
bzr: ERROR: Not a branch: "/usr/share/vim/addons/doc/imaps.txt.gz/".
.vim/doc/latex-suite-quickstart.txt.gz
...

WTF? "Not a branch!?"

Okay, so, what happens here is that Bazaar de-references the symlink before attempting to remove it, which is not at all what I had in mind. Poking around Launchpad, you can find several bug reports regarding the way Bazaar deals with symlinks. The workaround solutions proposed in those—remove the symlink using rm—wouldn't work for me, because I needed to retain the actual symlinks on the filesystem.

At this point I had solicited the attention of Robert Collins, a.k.a. lifeless in #bzr on Freenode. When I told him the workaround wouldn't work for me, and that I'd need to write a script, he suggested I use WorkingTree.unversion() from bzrlib. Despite being a Python fanatic [understatement] and bzr's codebase being in Python, when I said "script", I meant "Bash script". It never occurred to me to actually write a Python script until he mentioned that. By the completion of the thought, though, I was digging into the codebase of bzrlib to figure out what to do.

My initial approach plan included using os.walk() to move through the filesystem, os.path.islink() to identify the symbolic links, and then WorkingTree.unversion() to mark the files for removal from tracking. I ran into a problem, however, in that unversion() only accepts a list of file IDs, as specified by the bzr metadata. Robert pointed me towards a method called path2ids(), but I had trouble figuring out how I was going to give it the proper paths. os.walk will let me construct absolute paths to files, but I really needed relative paths to the files, truncated at a certain point past the root (e.g., .vim/compiler/tex.vim instead of /home/chris/shell-configs/.vim/compiler/tex.vim). I could see it was getting a little hairy, so I decided to dig a little further into WorkingTree code and see if there was anything else I could use.

What I discovered was the jackpot in the form of WorknigTree.walktree()—a method written precisely for what I needed: traversing the filesystem, identifying the filetypes (especially symlinks), and providing file IDs. Within a few minutes, I banged out a script that did exactly what I needed it to do, presented below.

#!/usr/bin/env python
# -*- coding: UTF-8 -*-

# Copyright (c) 2009 Chris Lasher
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

"""
A simple script to go through a Bazaar repository and ruthlessly
remove all symbolic links (symlinks) from further tracking.

It's important to note that this will not actually remove the symlinks
from the physical filesystem. This is left to the user, if so desired.

"""

__author__ = 'Chris Lasher'
__email__ = 'chris DOT lasher AT gmail DOT com'

import bzrlib.workingtree
import os

tree = bzrlib.workingtree.WorkingTree.open(os.getcwd())
try:
    # use protection -- one-on-one action only
    tree.lock_write()
    symlink_ids = []
    for dir, file_list in tree.walkdirs():
        # dir[1] (the file_id) will be None if it's not under revision
        # control, so this will skip it if it's not
        if dir[1]:
            for file_data in file_list:
                # file_data[2] is the file type, and file_data[4] is the
                # file_id, the necessary specifier for removing the file
                # from revision tracking
                if file_data[2] == 'symlink' and file_data[4]:
                    print "Removing %s" % file_data[0]
                    symlink_ids.append(file_data[4])

    tree.unversion(symlink_ids)

finally:
    # okay, all yours
    tree.unlock()

Hopefully someone else will find this little script useful. It's under the Apache version 2 license; make whatever use of it you can for your particular predicament.

So what were the lessons learned here:

  1. Exercise a little restraint and consideration about what you put under revision control in the first place.
  2. It's awesome to be able to have direct contact with developers of your tools.
  3. It's even more awesome to be able to dig right into their code and help yourself.
  4. Just like in Murali's brutal Theory of Algorithms course, in real life, when facing difficulty solving a problem one way, don't be afraid to step back and try an approach from another (the opposite) direction. Trust your gut—if it feels like the hard way of doing something, it probably is; find the lazy (smart) way.

A special thanks to Robert for his guidance and help.

Saturday, March 21, 2009

Do you choose the research, or does the research choose you?

Note: I give advanced warning to my non-biologist readers that the next few paragraphs below contain a good dose of biology. While I have attempted to keep it conversational, if you feel your eyes glazing over, skip down a few paragraphs for the real meat.

to the air

Yesterday (Friday) I attended a talk by Susan Gottesman about small non-coding RNAs (sRNAs) and how they are involved in protein degradation. At this point in her esteemed career, Gottesman's best known work revolves around a particular Escherichia coli sigma factor—a protein responsible for transcription—called RpoS. RpoS facilitates translation of messenger RNAs (mRNAs) into proteins at low temperature levels. Now, RpoS only appears in E. coli cells during low temperature conditions, but mysteriously (or so it was), the gene that encodes RpoS gets expressed even when the cells are growing at a comfortable temperature.

As Gottesman's lab discovered, the mRNA for RpoS can actually bend back around and stick to itself such that ribosomes aren't able to bind to the mRNA and translate it into the RpoS protein. An sRNA called DsrA, however, which is expressed in low temperature conditions, binds to part of the RpoS mRNA, preventing the mRNA from folding back on itself, and giving ribosomes access to the transcript to translate it into RpoS protein. Why is this important?

Well, previously, sRNAs had only been thought to inhibit translation and prevent proteins from appearing. That is, we say that sRNAs usually inhibit the expression of a protein, so if you found an sRNA, you would bet that its target wouldn't appear when it appeared. Add sRNA and the protein won't be found in the cell; take the sRNA away, and the protein re-appears. The Gottesman lab, however, demonstrated a case where the sRNA actually is responsible for making the proteins appear. That is, when the sRNA DsrA appears, its target, RpoS, appears too; and if you take away DsrA, the protein goes away, too! Craziness! In Biology, we call this a paradigm shift. Paradigm shifts are "big deals", because Biology is all about figuring out the rules, and then identifying the exceptions so we have to re-write the rules. Biology is the science of exceptions.

The story continues, but I'll leave it to the reader to check out Gottesman's publications for more, because as much as I liked the story of her research, what I found most interesting about the talk was this side comment that she made towards the end, which I paraphrase here:

We published this work with RpoS, but then we wanted to work in other directions. We'd try something, then discover we couldn't go in that direction because we needed to know something else about RpoS. Then we'd attempt something else, but again, it would always come back to RpoS. Finally we just said, "Forget it! Fine! We'll just study RpoS. Clearly there's enough here to work on for a while."
I don't know if the fellow grad students in the audience caught the subtle significance of this statement, or if perhaps I was the only person who found this significant. What Gottesman said, in more words, is that she didn't really choose her research; her research chose her. Yet, in spite of spending her career in an area she never intended to stay in, once she identified that she was mired in it, she made the best of it, leading to great scientific contributions and earning her accolades and prestige that even the most jaded of us junior researchers catch ourselves fantasizing about from time to time.

I find this significant because, also from time to time, I wonder how the researchers, and even my peers, that I have come to admire wound up doing the research that they're doing. In my earlier days, I often thought they must possess great foresight and wisdom. While I don't doubt they're clever people, the longer my tenure in research and the more people I harass to tell me about their own careers, the more I've begun to think that a lot of it just comes by chance rather than deliberate choice. We find ourselves in a particular unique positions, somewhat stuck, and somewhat stumped, and we throw up our hands and say, "Aw, Hell! I guess I might as well dig around while I'm here." We do have to make choices about where we dig, but we seem to get to choose our own particular holes about as well as seeds scattered by the winds. (Though, from time to time, we can try to ride the winds to another hole.)

I suppose that I just find it amusing that life is stochastic from the molecular level all the way up to our own grand plans. Like each of our cells, we may as well just deal with the cards we're dealt as best we can. For everything else... well, "Cast Your Fate to the Wind".

Monday, March 16, 2009

If a tree falls in a random forest: a summary of Chen and Jeong, 2009

Trees in fog w shadows

I had to write a summary for a paper, "Sequence-based prediction of protein interaction sites with an integrative method" by Xue-wen Chen and Jong Cheol Jeong[1], for my Problem Solving in Bioinformatics course. I thought I'd share the review here on my blog, in case anybody finds it remotely useful. I doubt anyone will, but it's my blog, so there. Be forewarned, this is my, "Hey, buddy, I'm just a biologist" interpretation of their paper. If you spot any specious, misleading, or just plain incorrect statements, please, by all means, offer corrections.


Chen and Jeong have essentially found a method to apply a machine learning technique called random forests to predict specific binding sites on proteins given only the amino acid sequence with greater accuracy than previously existing methods. Identification of binding sites in proteins remains an important task for both basic and applied life sciences research, for these sites make possible the protein-protein and protein-ligand interactions from which phenotypes, and indeed, the properties of life emerge. These sites also serve as important drug targets for pharmaceutical research.

Traditionally, researchers have identified binding sites from in vivo or in vitro studies involving point mutations that affect phenotypes, as well as through analysis of protein structures as identified through protein crystallography. With the advent and continuous improvement of DNA sequencing technology, however, researchers contribute ever more knowledge in the form of amino acid sequence, rather than structures. Sequencing has rapidly outpaced crystallography, necessitating prediction of proteins' functional characteristics based solely on their amino acid sequence, which Chen and Jeong cite as the motivation behind research presented in this paper.

Previous efforts to infer binding sites purely from amino acid sequence used a different machine learning method called Support Vector Machine (SVM). I'm not entirely certain how SVMs operate, but like random forests, they require a training set of known binding sites and sites not involved in binding. One of the confounding factors about amino acid sequences when applied to machine learning methods like SVMs is that the residues are unevenly distributed between the two categories; in other words, few amino acids in a sequence (1 in 9 in the dataset used by Chen and Jeong) will sit at the interface of the protein and its ligand. Chen and Jeong chose to use random forests because they are robust against this bias in the data. This has to do with the way that random forests are constructed.

For constructing random forests, one must have a set of data. In Chen and Jeong's study, the set is comprised of amino acids belonging to 99 polypeptide chains—or chunks of proteins—culled from a protein-protein interaction set used in previous studies. One must also have a set of features, or measures, about each item in and the data set. In this study, there were 1050 features (as stored in vectors) for each amino acid, which fall into one of three categories: those measuring physical or chemical characteristics (e.g., hydrophobicity, isoelectric point, propensity—which is a fancy word for saying whether an amino acid is likely to be on the surface of a protein or buried deep within it), those measuring the amino acid's minimum distance to any other given amino acid along the sequence, and the position specific score matrix (PSSM), which has to do with how likely certain amino acid substitutions are likely to be at that point.

With this data set and features in hand, one feeds it to the random forest generator. To construct one random decision tree, follow a process like this:

  1. Count the total number of known interface sites (we'll call these positives), and call this number N.
  2. Count the number of features available, and call this number M.
  3. Randomly select a subset of N sites out of the entire set with—and this is important—replacement. This solves the problem of the unbalanced data set. If I recall my statistics correctly (I don't) this has to do with each site now having equal chance at influencing the training.
  4. Now we build the tree. We randomly select m features from the total M features, where m is a lot smaller than M. Then, of those m features, we choose the one which best splits the subset of sites. We continue to do this recursively until all sites have been "classified".
  5. We repeat steps 1-4 to construct the number of desired trees (100 in this study), which gives us our "forest" of randomly generated trees.

With the random forest constructed, essentially you feed in an amino acid site into the random forest, then the site trickles down each tree, and each tree then "votes" as to whether or not it classified the site as an interaction site or not. A simple majority can be used to categorize the site, or more stringent criteria, such as "at least 5 votes are necessary to categorize the site as an interface site". Increasing the votes required improves the confidence at which one claims a site is an interaction site (specificity), but decreases the probability of detecting interaction sites (sensitivity).

Using these measures of sensitivity and specificity in conjunction with leave-one-out studies (one polypeptide sequence is used as the test case, and the other 98 are used as training data), Chen and Jeong demonstrated that their random forests approach performed significantly better than the SVM approach used by the earlier studies. They attribute this improved performance to two things: random forests are more robust to unbalanced data sets, and their approach considered many more features than the previous studies'. When they used only the features used in the previous studies, they found decreased performance, albeit still significantly better than the previous methods'. Chen and Jeong note that a major feature of random forests is that their accuracy increases, rather than decreases, when the number of features increases, due to the random sampling.

Chen and Jeong finished their study with a prediction of binding sites on the DnaK (or Hsp70 in eukaryotes) chaperone system. Their results corroborated with several in vivo studies of mutants where mutations near the sites they predicted yielded changes in phenotypes for both prokaryotic and eukaryotic forms. Their visualization of predicted interaction sites using 3d molecular modeling software provided additional support.

  1. Xue-wen Chen and Jong Cheol Jeong, "Sequence-based prediction of protein interaction sites with an integrative method," Bioinformatics 25, no. 5 (March 1, 2009): 585-591, doi:10.1093/bioinformatics/btp039.

Sunday, March 15, 2009

Why Biopython needs to move to GitHub or Launchpad

Air hosting?

Paulo Nuin wrote a spot on post about the ridiculousness that is Biopython still using CVS as its revision control system (a.k.a. source code management, or SCM), when we code in an era of arguably superior tools in the form of distributed SCMs (DSCMs). Please read his post if you haven't yet. Do not pass go. Do not collect $200. This post will be here for you when you get back.

I'll continue along the thread that Paulo started, in which one of the hangups that the Biopython community must overcome is: "Supposing we do switch to a DSCM, where do we host the code?" Until the Biopython project can decide on an answer to this question, the project won't move to anything.

Peter Cock seems sincerely determined that the code be hosted on the Open Bioinformatics Foundation (OBF) servers at Biopython.org. If I understand Peter's rationale correctly, the notion stems from the desire to maintain control of the code hosting. The alternative to self-hosting the code is to use one of the big players. I'm particularly referring to GitHub and Launchpad. GitHub and Launchpad host repositories of for the DSCMs Git and Bazaar, respectively, and provide a set of tools around these repositories to facilitate collaboration and interactions between the developers and their communities. Launchpad has the backing of Canonical, best known for managing the Ubuntu GNU/Linux distribution, and GitHub has the backing of the only group more rabid than the Python community—the Ruby community; hence, I refer to them as "the big players".

I respect Peter's legitimate concerns. I also really respect Peter, who is much more of a Biopythonista than I'll ever be, and I recognize it will take his blessing for the transition to Git or Bazaar to succeed. I dedicate this blog post to changing Peter's opinion and convincing him that hosting on GitHub or Launchpad is the best option available to us at the time.* Hopefully I'll convince a few other Biopython (or Bio-anything) Devs along the way, too. :-)

The following are my top five reasons for hosting Biopython on GitHub/Launchpad:

  1. It's free. Yeah, okay, only "as in beer"**, but the Biopython source will, itself, remain open. The hosting is generously on someone else's dime, and that's all we need.
  2. It already exists. I do not have technical experience nor interest in running my own webserver-based interface to either Bazaar or Git. From the recent discussions on the Biopython mailing list, I will guess nobody on the Biopython Dev team does or has the time to learn how to, either. Since the OBF staff are volunteers, helping us set these up won't be high on their priority list. Bazaar and Git don't even exist on the servers, yet. Launchpad and GitHub already have the tools in place. The amount of time the Biopython community has to spend setting up the projects here is pretty minimal and painless. In fact, it's already been done. Launchpad and GitHub are clearly very good at what they do. They have the experts, the redundancy, and the robustness to manage hosting code in a public space, and all the headaches that come with it, so that we don't have to.
  3. They have established social networks. I'm already on GitHub and Launchpad. A lot of us are already on these sites, working on our own and other open source projects. These places let other people discover our work, and allow serendipitous connections to occur. "Hmm, this gal works on Biopython. What's that?" This doesn't occur at Biopython.org—people only go there when they know what they're looking for (and not many people are looking for "bioinformatics python"). Additionally, potential employers, co-workers, and employees are on these sites; not all of us will be (un)fortunate or content enough to stay in bioinformatics and computational biology forever.
  4. Everybody else is doing it. Sure, right now, GitHub only hosts very minor, niche projects like Ruby on Rails, Cappuccino, and BioRuby (like that will ever go anywhere), and Launchpad has some lesser known ones like MySQL, Zope, and something called Ubuntu, but I hear that some major players will join these sites really soon! They do seem to be gaining in popularity very rapidly. ;-)
  5. Vendor lock-in is just not an issue. There's some concern that using a third-party site such as GitHub or Launchpad will make the Biopython project vulnerable to possibly unreasonable whims of the owners of the sites. Terms and conditions could change unfavorably (e.g., "You have to pay to continue using our service."), or the service will go under. However, the OBF provides no more protection than Launchpad or GitHub, particularly for the latter scenario. When I think about who's least likely to run out of operating funding—the OBF, Launchpad, or GitHub—I'm not betting on OBF. But let's suppose that the uthinkable happens, and the site closes its doors to Biopython. So what? It's a distributed SCM; we have all of the code! This isn't CVS or Subversion, where a downed server takes all the revision history with it to the grave. We'll just set up shop somewhere else, point it towards our repositories, and sally on. We can burn that bridge when we get there; in the meantime, don't fret about it.

At this point, I'm sure there's more discussion to have. I just hope it's not too much, given that the transition to Subversion stalled tragically, which I take responsibility for. It would be nice to have this settled by May. I'd rather be fielding "How do I do this in Git/Bazaar?" than discussing "Why should I do this in Git/Bazaar?" My fingers are crossed, my hopes are high, and my stubornness is fiercer than two years ago.

* I'm excluding Mercurial and Bitbucket here because they haven't received consideration on the mailing list. They could be a great solution, but we're least familiar with them, and we have to narrow down the choices somehow. ** Okay, so Launchpad is going to be open sourced, but I don't want to be in charge of running an instance of it if nobody's going to pay me; see 2.