Posts on David Bieber

The Go Note Go Story

Fri, 30 Dec 2022 00:00:00 +0000

This story is meant to be listened to. You can also read along below.

Your browser does not support the audio element.

Hi everyone. I’m David Bieber and I’m going to tell you the story of Go Note Go, which you can see on the screen.

And to do this, I need to bring you back with me to 2014. It’s my senior year of college. I’m in my dorm, which is a beautiful stone building, like a castle. I’m lying awake at the bottom bunk of a bed late at night.

My thoughts are racing. I can’t sleep. There’s this girl. Does she like me? Do I like her? What will I say to her tomorrow?

Every time I finish a thought, I have a new thought, just subtle variations on the thoughts I’ve already had. And they won’t stop. It’s getting even later and I need to fall asleep.

Fortunately, I find a trick that works for me. If I write down what I’m thinking, then there’s just one canonical version of the thought, not two hundred. And it’s on paper, not in my head, and I can sleep.

The only trouble is, I’m in a bunk bed; there’s a roommate six feet above me and I can’t just turn on the lights to write something down. And I wouldn’t want to anyway. It would wake me up. Writing in the dark is no good. Can’t read my handwriting that way. And typing on my phone or computer is out too, because the lights are too bright and they would wake me up.

So, I settle on typing on my laptop in the dark with the monitor turned completely off.

I study computer science, so I write some software to make this even better. And Shh Shell is born. I don’t have to worry about typing and falling asleep on the delete key, or typing and having the window not being in focus. The name Shh Shell is a pun on SSH and sleeping, and it’s amusing to me.

And I pile on the bells and whistles. Shh Shell lets me set alarms, send text messages, hear the time, the works, all without ever opening my eyes. And I love it. I use it for years.

I take it with me when I move out to California to work for Google. I set it up in my shower so I can capture shower thoughts in addition to sleep thoughts. Shh Shell quickly becomes my personal journal, a place I can put my most personal thoughts and then get a good night’s sleep.

Fast forward five years. This is two months ago now. I’m driving in a car that I recently purchased from New Haven, Connecticut, where I live, to a campground in New Jersey to go camping.

I’m listening to an audiobook and I feel an old yearning, the desire to write something down, something important, just to get it out of my head. It’s been a few years since I’ve used Shh Shell. I didn’t use it much after the move east, but I know exactly what I need. I need to write down my thoughts about the audiobook that I’m listening to.

As I get to the campground, the thoughts about the book have evaporated. I set up my tent, and it’s here that Go Note Go was born. A new, improved version of Shell that supports driving and camping, as well, of course, as sleeping and showering.

I design Go Note Go right there in the woods.

It’ll be like Shh Shell, but with a host of improvements. You won’t need a laptop for it to work. It’s just a standalone keyboard. It supports audio with a beautiful red button you can use to record while driving. And it works offline. Perfect for camping. It will transcribe your notes and upload everything as soon as it gets an internet connection.

I am so excited to build this. I order the parts right then and there from the floor of my tent in the middle of the woods.

I don’t even wait for the parts to arrive. As soon as I get home, I write the software.

I configure mine to upload to Roam. But a month later when David, Aria, Sam, and Aaron are building their own, they set theirs to upload to Notion or RemNote or Ideaflow. It works with any note-taking app at all.

David Dohan has connected me with this wonderful group of people after I built the first version. And I give them a live demo, taking my fully assembled Go Note Go out into a tent for the first time, this time to go camping in my own backyard.

I show Go Note Go to Aaron and the whole group. How it handles audio, how as I drift off to sleep I tend to switch from talking to typing the more tired I get, how you can still issue commands like asking it for the time without waking yourself up, just like you could with Shh Shell.

I feel like when I built Shh Shell for the first time. I love it.

And now I have a group of people who love it as well, each building their own.

I finish camping in my backyard here in New Haven and I come inside. Last night, I found myself in a situation like the one from years ago. Getting ready to present today, my thoughts again were swirling. This time, though, I knew immediately what to do. I took the dozens of versions of this story that were racing through my mind, and I used Go Note Go, through a combination of typing and speaking, to put just a single version of the story down on the page. My mind was settled, and I slept.

Sending messages as I drift off to sleep

Sat, 08 Jan 2022 00:00:00 +0000

In this post I describe my new Messager setup, and how it allows me to send messages directly from Roam Research or from a standalone keyboard (no monitor) that I keep at my bedside as I drift off to sleep, and which I take driving and camping.

Currently my setup supports sending messages to Facebook Messenger, Twitter, Slack, Discord, and iMessage. I intend to add gChat and email support next, since there are still a handful of people I want to message that don’t use any of these.

Above all else, the beauty of this project to me is its usability, which is hard to communicate in a write-up like this. Some of the usability comes from the way the components integrate with my existing workflows. Some of the usability comes from design choices (like using a keyboard with no monitor for note-taking and message drafting). I’ll try to point out these usability elements along the way. Bear in mind I designed this primarily for myself (though I have a handful of friends along for the ride as well, trying it out and making their own contributions).

Let’s dive in.

Go Note Go: The Headless Keyboard

For some background, I developed Go Note Go, a headless keyboard designed for note-taking on the go. Go Note Go is a note-taking system for when you’re on the go, with a focus on driving and camping. You can read all about Go Note Go here and learn more on its GitHub page.

Go Note Go’s main purpose is note-taking. Anything you type into Go Note Go or voice-record into it gets transcribed and uploaded to your notes as soon as it gets an internet connection. For me, Go Note Go uploads to Roam Research. It also supports RemNote, IdeaFlow, Mem, and Notion, and adding additional systems isn’t too difficult.

One of the usability perks of Go Note Go is that there is no monitor. This means you can’t get distracted browsing the internet while writing on Go Note Go. You also can’t get distracted wordsmithing your own writing. It makes writing pleasurable, even if it comes at the cost of having typos and stray thoughts in your writing; you can always clean those up later.

Go Note Go does a lot more, so I do encourage you to learn more about it, but it isn’t the focus of this post, which is about sending messages. For the purposes of this post, Go Note Go is a data entry system for Roam. And since I’ve also built a system that allows sending messages directly from Roam, this means I can now send messages directly from Go Note Go. So, I can send messages when driving, camping, and as I drift off to sleep at night.

To send a message in Go Note Go, you simply “@” the person you are writing to. To send a message to Adriana, I would write “@Adriana Message goes here”. To send a message to David Dohan (using first and last name), I could write “@David Dohan: Message goes here”. You can @ someone at any point in the message. You can also @ multiple people, and Messager will send them a group message.

Go Note Go acts as an outliner now too; this is both a big usability improvement for Go Note Go on it’s own, and doubly so for using Go Note Go as a messager. An “outliner” is a system for taking notes as a series of nested bullets. You can push tab to indent notes underneath other notes; I’ve developed a user experience that makes this moderately natural, even in the absence of a monitor or any other visual feedback. For sending messages, the outliner feature is a boon. Any notes you nest under an @’d note will also register as additional messages for the @’d recipients. This makes writing longer multiline messages a pleasure, even without a monitor.

The default behavior of my messaging system is to hold the messages for approval before sending them. So, the messages that I write as I drift off to sleep don’t send as I drift off to sleep. Instead, I see them in the morning and approve them, and only upon being approved are they sent automatically.

This delayed-approve-then-send approach is particularly well suited for Go Note Go, where I likely make typos and want to clean up the messages before they are sent, since I am writing them without a monitor. Later in the post I’ll dive into what the approval process looks like (tl;dr there’s a spreadsheet where I can mark a message as “OK” to send), but first I want to share additional benefits of being able to send messages directly from my note-taking app, which is currently Roam Research.

Drafting Messages in Roam Research

Anything I write on my Go Note Go appears in Roam Research, but I can also take notes directly in Roam. Just like on Go Note Go, the way to draft a message in Roam is to “@” the person you are writing to. Messager has some heuristics it uses to translate your @’s into proper recipients. So, the syntax for @‘ing someone isn’t too strict.

Using Roam offers some possibilities beyond what I can do on Go Note Go, such as including images in messages. Like on Go Note Go, you can use nested bullets to create longer messages, and you can @ as many people as you want for a message. In Roam, however, you can also include images in your messages. Messager will intelligently send those in the native format used by the underlying messaging service, rather than blindly sending the markdown stored in Roam. I find this feature, being able to use images naturally in Roam, and then have them send naturally as messages, quite pleasant.

One future direction I’d like for this project is to add a “send” button in Roam next to any messages I’ve drafted there, as well as to display the status of the messages inline in my notes. That’s not implemented now, but would be a nice-to-have for the future.

Today, all messages are automatically added to my “messager queue” where they wait for approval before being sent. I like having this approve-before-sending approach as the default, since it psychologically frees me up to write things I might not otherwise write if I was instead using a send button that sent immediately. However, sometimes being able to send something immediately is desirable, and that’s not a clean option in the current setup.

Roam to Sheets: The Messager Queue

Any messages in Roam (whether entered directly into Roam, or entered via Go Note Go), are automatically added to a spreadsheet, the “Messager Queue”. These are the spreadsheet’s columns:


Service	Recipient	Text	Date	Time	Approval	Status	Metadata

The Messager Queue sheet is filtered by default to show all pending messages. For each message it shows the Service on which it will be sent and well as the Recipient of the message. The service can be one of FB Messenger, Twitter, iMessage, a Slack server, or Discord. The recipient can either be an individual, a list of individuals, a group, a Slack channel, or a Discord server and channel.

There’s also a sheet that lists aliases, so it’s easy to use a short name or informal name to refer to a longer service or recipient; e.g. I use “@Audio Tools” as a shorthand for sending to a group of 5 people on Messenger all interested in audio tools for networked thought.

The Messager Queue sheet also displays the Text of the message to send, so you can easily clean up the message here for any typos or clarifications before it is sent.

The Date and Time columns allow for scheduling messages to be sent at any point in the future. Leave them blank and the message will be sent as soon as it is approved. Use keyboard shortcuts cmd-; (for date) and cmd-shift-; (for time) to quickly fill out these columns if desired.

Finally, the Approval column is the most important. Mark a messages as OK if you want it to be sent, or as Ignored if you deside to skip it. The Status column is updated automatically by Messager when the message is sent (or if it fails to send, then the error appears here). You can also mark the Status manually with whatever value you want (e.g. if you send the message manually), and the Messager system will ignore that message going forward. If you want the system to retry sending a message, simply clear out the Status column and it will try again.

I include this spreadsheet as a custom service in Ferdi, so it lives alongside all the messaging apps I use. (This in itself might be the subject of a future snippet.)

Why the approval system over immediate messages?

Beyond allowing me to catch a bunch of typos, it’s also a psychologically useful thing for me. I am more willing to write and I feel able to say more things that I might not otherwise, knowing I have the opportunity to cancel or change the message later. In that way, it’s like gmail’s undo feature. In gmail, undo not a true undo, but having those 30 seconds after clicking send to retract the message make a big difference.

Another benefit of sending the messages only after approving them is the ability to sleep at night. If I write a message at night, I know you haven’t responded yet because my messages haven’t even sent yet; they’ll go out in the morning, after I’ve had a chance to clean them up. So, I don’t spend any mental effort wondering if there’s a response from you. It also allows me to set the appropriate tone for the messages, by having them go out at a reasonable time, rather than appearing urgent by being sent in the middle of the night. And it does this without me needing to keep the message draft in my head as I sleep. So, I sleep better.

This is why, while the ability to send messages immediately would also be a good feature to add, I am inclined to leave approve-in-the-morning messages as the system’s default behavior.

Notifications from Bieber Bot

One additional small usability feature is that Bieber Bot will message me in the morning or evening whenever there are messages in the Messager Queue awaiting my approval.

This sounds like a small convenience, and in fact has proven even more useful than it might at first sound. When I wake up in the morning, I sometimes don’t remember that I’ve written messages the preceding night. So, having Bieber Bot send me the link to the Messager Queue and gently remind me to approve the messages has been consistently charming and welcome. Thanks, Bieber Bot.

Automatic Sending from Sheets with Messager

Messager is the underlying system that sends the messages in the Messager Queue system. As noted previously, it supports Facebook Messenger, Twitter, Slack, Discord, and iMessage. It also supports Hacker News, which hasn’t been so useful for this project. And I am thinking I may add gChat and email support next.

I use Messager for more than just sending messages from the Messager Queue spreadsheet; it also backs other projects like my Kangaroo Auto-responder and several parts of Bieber Bot. Its purpose, in the most broad sense, is to support programmatically sending and receiving messages in a uniform manner across all the messaging systems I use.

For Facebook Messager, it uses fbchat to enable programmatically sending and receiving messages as myself. It also uses Facebook’s API to allow sending messages as Bieber Bot.

For Twitter, it uses the API to support public tweets, private tweets (using a separate account; see here for how I use this to reclaim my attention), and DMs.

For iMessage, it uses AppleScript.

Slack and Discord are the most recent additions, and they currently live outside the core Messager system; they are implemented using Browserflow flows. This means that I am effectively sending Slack and Discord messages as myself, rather than using an API to do so. The messages are sent in a browser using clicks and keyboard presses, all in the same human-centric UI that I would use if I were to send the messages manually. I’m so grateful to Browserflow for making this possible, as Slack and Discord have been really key additions to this project.

Reflections on the setup

I’ve been using the setup for only a few days so far, and am continuing to actively develop it. So far, I absolutely love it.

The headless typing experience provided by Go Note Go makes for a great environment for drafting messages. So too does using Roam Research while scrolling through social media. (I’m the sort of person who doesn’t usually like replying publicly to social media posts, but does enjoy engaging with them in 1:1 or small group chat messages.) Drifting off to sleep has proven to be another excellent time to share thoughts with friends. I don’t want to start a conversation with friends as I drift off to sleep, but loads of thoughts come to mind that I do want to share with people, and so adding them to my Messager Queue to send the next day has been quite satisfying.

If this interests you, feel free to get in touch. If you do, I’ll do my best to get back to you, likely as I’m drifting off to sleep.

A Step-by-Step Developer Guide to Building a Personal Roam-to-Twitter Bot

Sun, 07 Mar 2021 00:00:00 +0000

Last month I wrote a little Twitter bot for myself with the goal of reclaiming my attention from Twitter. The idea is to fill my Twitter feed with things that I care about, rather than letting the Twitter algorithm control my attention. By programmatically injecting cards into my Twitter feed I can repurpose the time I spend scrolling on Twitter from time lost to time well spent.

This post explains, step-by-step, how you can build your own private Twitter bot so you too can repurpose the time you spend doom-scrolling.

Overview

In this guide you’ll see how to set up a private personal Twitter bot that only you can follow. We’ll use this to inject content directly into your own feed. In order to choose what to Tweet, we’ll extract content from our Roam Research database. Here are the six steps we’ll follow.

Set up a private Twitter account
Get your developer API keys
Use the twython library to post as your new Twitter account
Write the tweet generator; I pull my tweets from Roam Research
Use the schedule library to periodically tweet
(Bonus) Use supervisord to keep your Twitter bot running

Step 1: Set up a private Twitter account

Go ahead and register a new Twitter account. I called mine “@PrivateBieber6” since my main account is just @Bieber. You can call yours whatever you like!

You’ll have to provide a phone number or email address while registering. If you reuse a phone number from an existing account, it will work, but you’ll lose the ability to use SMS commands and receive SMS notifications on your main account. Only the account most recently linked to this phone number will have these features.

Pro-tip: you can add a plus ("+") string to your email address to turn in into a new email address that still gets delivered to your inbox. E.g. mail to david810+hello@gmail.com still gets delivered to david810@gmail.com (feel free to say hello).

Once you’ve created your new account, two important things to do:

Follow it from your main account!
Set it to protected. To do this, go to your settings ( https://twitter.com/settings/account), choose “Privacy and safety” > “Audience and tagging” > Protect your Tweets.

With this enabled, you’ll be the only one able to see the tweets from this account.

Step 2: Get your developer API keys

First you need to apply for a developer account. This just takes a minute. Go to https://developer.twitter.com/en/apply-for-access to apply for a developer account. Answer the questions honestly. Ping me on Twitter if any of the questions are unclear.

Once you have a developer account, create a new project and app. I called mine “Attention Playground.” Get creative! Choose a fun name.

Navigate to your newly created app, and Edit the “App permissions”. Set the app permissions to “Read and Write”. This will enable your bot to post tweets.

Next navigate to the project you created, and click the key icon next to your App.

Write down your API key and API secret key. Generate an Access token and secret and write those down too. Your access token & secret should say “Created with Read and Write permissions”.

These four tokens will be essential in the next steps.

Step 3: Use twython to tweet as your new account

Now things get a bit technical. Install the twython Python library by running pip install twython from your shell. If you’re not familiar with installing Python packages, you can learn how to do this here¹.

Let’s send a practice tweet. Create a new file called hellotweet.py with the following code:

import twython

client = twython.Twython(
  "YOUR_API_KEY",
  "YOUR_API_SECRET_KEY",
  "YOUR_ACCESS_TOKEN",
  "YOUR_ACCESS_TOKEN_SECRET")

def sendTweet(text):
  return client.update_status(status=text)

sendTweet("Hello world!")

On lines 4-7, place the keys you got from Step 2 in stead of the placeholders I’ve included above.

Now, run your program! python hellotweet.py

If everything’s worked, you’ll have a new tweet from your private account saying “Hello world!”

Don’t move on to the next steps until you get this working.

Step 4: Write the tweet generator; I pull my tweets from Roam Research

The simplest possible tweet generator looks like this:

import random

def choose_message():
  messages = ["Tweet 1: Make spaghetti",
              "Tweet 2: Learn about salmon",
              "Tweet 3: Call Mom"]
  return random.choice(messages)

We’re going to write a more interesting tweet generator that pulls tweets out of a Roam Research database.

To do this, first set up a recurring json backup of your Roam Research database. You can use the project roam2github to set up the backup. There are detailed instructions on its github page for how to do this. You also have the option of using Roam’s built-in automatic backups, which you access from the command palette (cmd-P).

Use git to clone your backup locally. If your github repo is “https://github.com/user/roam-backup", then the command to clone the repo is git clone https://github.com/dbieber/roam-backup.

Create roam.py. You can copy the contents of roam.py from here. This will allow us to pull the latest backup of our Roam database from GitHub programmatically. (1) Set the ROAMRESEARCH_DBPATH variable in roam.py to the location of json backup in your roam-backup directory. For me, this is ‘/Users/dbieber/code/github/dbieber/roam-backup/json/playground.json’. (2) Set the roam_dir variable in the update function to the path to your roam-backup directory. For me, this is ‘/Users/dbieber/code/github/dbieber/roam-backup’.

Using the new roam module we just created, it’s time to write a function that extracts potential tweets from your Roam database. Here’s the function I use:

def choose_message():
  all_children = roam.get_children()
  messages = []
  for child in all_children:
    if ('Fleeting TODOs' in child['string']
        or 'Distraction TODOs' in child['string']):
      children = child.get('children', [])
      for c in children:
        if (c['string']
            and 'DONE' not in c['string']
            and len(c['string']) <= 280):
          messages.append(c['string'])
  return random.choice(messages)

The get_children function gets all the blocks from your Roam graph backup. The for loop looks through all these blocks for ones that say “Fleeting TODOs” or “Distraction TODOs”. It takes the children of those blocks (except those that are too long or say “DONE”) as potential tweets. One of those is selected randomly each time choose_messages is called.

That’s our whole tweet generator. In the next step, we’ll pull the Roam backup from GitHub periodically automatically so it’s always somewhat up to date, and we’ll schedule the bot to tweet every ten minutes.

Step 5: Use the schedule library to periodically tweet

The schedule library is a useful Python library for scheduling tasks to run at a particular time or on a recurring basis.

Here’s a simple demonstration of how to use schedule to print “Hello world!” every 15 minutes:

import schedule

def print_hello_world():
  print('Hello world!')

schedule.every(15).minutes.do(print_hello_world)
while True:
  schedule.run_pending()
  time.sleep(30)

We don’t want to print hello world though; we want to tweet!

We’re going to use schedule to schedule two recurring tasks: (1) every 6 hours we’ll pull the latest Roam backup from GitHub, (2) every 10 minutes we’ll have our Bot pick a random block using the logic from the previous step and tweet it to twitter.

The code for that looks like this:

def tweet():
  message = choose_message()
  sendTweet(message)


def main():
  schedule.every(10).minutes.do(tweet)
  schedule.every(360).minutes.do(roam.update)

  while True:
    schedule.run_pending()
    time.sleep(30)

Putting it all together, our complete code now looks like this:

"""Every 10 minutes selects a topic from my Roam database and tweets it
from a private Twitter account.
"""

import random
import time

import schedule
import roam

import twython


client = twython.Twython(
  "YOUR_API_KEY",
  "YOUR_API_SECRET_KEY",
  "YOUR_ACCESS_TOKEN",
  "YOUR_ACCESS_TOKEN_SECRET")


def sendTweet(text):
  return client.update_status(status=text)


def choose_message():
  all_children = roam.get_children()
  messages = []
  for child in all_children:
    if ('Fleeting TODOs' in child['string']
        or 'Distraction TODOs' in child['string']):
      children = child.get('children', [])
      for c in children:
        if (c['string']
            and 'DONE' not in c['string']
            and len(c['string']) <= 280):
          messages.append(c['string'])
  return random.choice(messages)


def tweet():
  message = choose_message()
  sendTweet(message)


def main():
  schedule.every(10).minutes.do(tweet)
  schedule.every(360).minutes.do(roam.update)

  while True:
    schedule.run_pending()
    time.sleep(30)


if __name__ == '__main__':
  main()

Make sure that both the “tweet” function and “roam.update” run successfully.

Save this as roam-tweeter.py. You can now run it with python roam-tweeter.py.

And that’s it! While this is running, your Twitter bot will post a new tweet to twitter every ten minutes. Using this, you can flood your Twitter timeline with things you care about, whether that’s your TODOs, spaced repetition, or old ideas you want to revisit.

Step 6 (Bonus): Use supervisord to keep your Twitter bot running

This is an optional step, but highly worthwhile! Supervisord is a utility you can use to monitor processes and automatically restart them if they die. If you plan on using your computer for long-running processes like a Twitter bot, this can save you lots of headaches.

Follow the instructions at supervisord.org to get set up. The key steps are:

To install supervisord, use pip install supervisord

Add a config for running your Twitter bot. Mine looks like this:

[program:private-bieber-twitter]
command=/Users/dbieber/.virtualenvs/_3/bin/python roam-tweeter.py
directory=/Users/dbieber/code/playground/private-bieber-twitter

Notice that I’ve included the full Path to python in the command.

Finally, start supervisord with sudo indicating the path to the config. (e.g. for me the command is sudo /Users/dbieber/.virtualenvs/_3/bin/supervisord -c ~/path/to/supervisord.conf)

With supervisord set up, you’re bot is now bullet-proof! (Not bug-proof though.) Even if it encounters an error, it will automatically restart.

Wrap-up

Hopefully this guide makes it clear how you can use a private Twitter bot to reclaim your attention. The main contribution of this guide is its emphasis on using a private Twitter bot to design your own personal Twitter feed. We selected Roam Research backups as the source of tweets for our bot, but of course you can substitute Roam with any tweet generator that you please.

You can use a bot like this for revisiting old ideas, for surfacing your TODOs when you’re distracted, for sending yourself messages, or anything else you come up with.

Since making myself a private Twitter bot one month ago, I’ve been delighted to find that its tweets are often among the highest quality on my timeline. I hope you find the same.

When it says that using a virtual environment is optional, know that it’s definitely worthwhile! ↩︎

Where are all the posts from 2020?

Fri, 08 Jan 2021 00:00:00 +0000

You’ll notice I have no blog posts from 2020, but this is not because I stopped writing. Instead, I have been writing “snippets,” which you can read here.

If you’re going to read just one snippet, I recommend that it be this one. The main idea behind snippets is to allow me to write without the inhibitions I sometimes experience when writing a blog post for publication. By writing in a secluded section of the website with no (now, few) inbound links, and by deliberately keeping the quality bar for publication low to non-existent, I made it easier for myself to write often.

To keep the quality bar low, sometimes I would deliberately post a short half-baked thought, possibly without editing or revision. Despite this aim of keeping the quality bar low, and despite explicitly writing for no one, I often would find myself pleased with what I had written and eager for someone else to read it.

To me, this is the beauty of the setup. The (lack of) quality standards make starting writing easy. I typically start writing a snippet with the mindset that I have a little idea I want to get out of my head and onto paper. As I begin, I think it’s a small idea that will take just a few lines. Perhaps in the back of my mind I even think it will contribute to lowering of the quality-bar for the section of the website. And sometimes it does. But just as often, I find myself ready to elaborate on the idea. Perhaps I explain where the idea came from, or why I find it interesting. And by the time I’ve finished writing, I have a snippet that I’m proud of, and which I would love to discuss.

This is why I sometimes put my email address at the end of a snippet. With just a cursory description of the snippets section (“a place to write for no audience”), seeing my contact info at the end of a post may feel contradictory. Now you see that it is not; the lack of a quality bar and the lack of an audience serve their purpose in allowing me to start writing easily, and to publish readily, but once I have written something and released it to (a tiny unlinked corner of) the world I do not necessarily want to keep it hidden.

So, I am now for the first time linking from my main website directly to the snippets section of my website. It’s not a prominent link, as it’s contained only within this blog post, but it’s a meaningful step for me. Web crawlers now have a means in to index the site. Visitors to the site now have a means in too, not just people to whom I explicitly send the link.

If you do venture in, and read a snippet or two, there are a few things to know. First, you’re welcome here. It’s not voyeurism. These aren’t private posts; they’re simply not written for an audience, but I am happy for them to have one. Second, they’re snippets. Think of them like ideas scrawled in a hurry. I’m not wedded to the ideas I express, and I’m open to reconsidering what I wrote. And thirdly, I value your thoughts (and your time!), and it would delight me to hear them.

You’ll find ideas for side-projects, philosophical musings, technical notes meant mainly for future-me, technical notes meant for other people, project updates, nonsense, personal notes, and more. All of this, here.

Keeping Track of My Life in a Spreadsheet

Sun, 29 Dec 2019 00:00:00 +0000

As many of my friends know, I keep track of much of my life in spreadsheets. There are several pieces to this system, but fortunately it’s the kind of system that you can adopt (and discard) incrementally. So, rather than explaining the whole thing in one go, I will explain one core piece in this post.

In this post, I will detail how I use a spreadsheet as a daily activity log.

My spreadsheet lets me keep a record of all the things I’ve done. It then also rolls up this data in interesting ways that help me with reflecting on and planning my life. Finally, it interacts with a bot system I’ve written for myself to help me keep my life on track.

I will answer all of the following questions in this post:

What’s the organizational system?
What are the main benefits of this system?
Why is this system easy to adopt?
Why is this system easy to keep doing for months on end, without forgetting?
How does this system help me with planning my life?

The System

I have a single spreadsheet with columns Evening, Morning, Keywords, Weekday, Date, People, Notes. In this sheet, I have a single row per day.

The essense of the system is to jot down (1) what I do, and optionally (2) who I engage with each day.

I don’t update the spreadsheet every day, usually just once a week or so I’ll fill in the activity from all of the missing days.

This system serves several purposes, the main one being a journal of sorts so I can see a historical log of my activities. As you’ll see in the Planning and Notifications sections, it also helps me remember to regularly do all the activities I enjoy doing.

I’ll start by providing a brief description of each column in the spreadsheet. Then we’ll get to discuss how I use the spreadsheet, and you’ll see why they’re each important to me. We’ll then also dive into all the benefits I get from this spreadsheet and the infrastructure I’ve built up around it.

Column Descriptions

Column	Type	Description
Evening	Text	A single cell in which to put the main activity I did in the evening (e.g. after work) on that day.
Morning	Optional, Text, infrequently used	The main activity I did in the morning (e.g. before work) on that day.
Keywords	Optional, comma separated	Keywords describing the activities of that day.
Weekday	Filled automatically	The day of the week, e.g. Thursday
Date	Filled automatically	The date
People	Names, comma separated	A list of the people I engaged with on that day
Notes	Optional, Text	A bit like a diary, this is a place to jot down additional notes about the day.

Really all of the columns are optional, but the ones that are particularly optional are marked as such.

Usage

Now I’ll describe the data entry process, and then all the ways I take advantage of this data.

Data Entry

About once a week on average I’ll go to the spreadsheet, and fill in all the activities I did since the last time I updated the sheet.

For each day, I try to capture just the main one to two outside-of-work activities that I engaged in. It’s totally fine if I don’t capture every little thing, and it’s also fine if there are some days that I didn’t do anything of note.

I said above that I do this about once a week, but there’s no strong rhythm to it; sometimes I do it every day for multiple consecutive days, and sometimes I’ll go multiple weeks between updating the sheet.

I’ve been using this system reliably since mid-2018, and it’s served me well. The notifications that we’ll get to shortly, combined with the simplicity of the method (just one cell to fill in per day), have allowed me to keep using this system without being disrupted by travel or high-workload parts of life.

Planning

I have a second sheet that is updated automatically using data from the sheet I’ve been describing so far. This sheet has one row per activity that I enjoy engaging in.

The most important columns in this sheet are:

Column	Type	Description
Category	Text	The category of the activity (e.g. Athletic).
Activity	Text	An activity that I enjoy engaging in. (e.g. Skiing)
Last Date	Filled automatically	The most recent date on which I engaged in the activity.

The Last Date column is filled automatically using the data from the main sheet. The Google Sheets formula that fills in this column is a query a bit like this one:

=IFERROR(QUERY(scheduled,
   "select E
    where A contains '" & C2 & "'
      OR B contains '" & C2 & "'
      OR C contains '" & C2 & "'
    order by E desc
    limit 1
    label E ''"
))

C2 refers to the Activity from the current sheet
A is the Evening column from the Data Entry sheet
B is the Morning column from the Data Entry sheet
C is the Keywords column from the Data Entry sheet
E is the Date from the Data Entry sheet

This lets me quickly see, for each activity that I enjoy, how recently I’ve done that activity. For example, I see that I haven’t gone camping since 2019-07-05, I haven’t played tennis since 2018-07-31, and I haven’t gone swimming since 2019-10-07. Really overdue for some tennis!

My spreadsheet also rolls up this data further, by category. I can quickly see how recently I’ve done any activity in a given category.

The categories I use are: “Athletic”, “Civic Something Or Other”, “Day Trips”, “Educational”, “Entertainment”, “Family”, “Financial”, “Food”, “Music + Arts”, “Outdoorsy”, “Personal Projects”, “Puzzles”, “Social”, “Social Games”, and “Well being”.

Here you can see how recently I’ve done something in each of the categories.

As you can see, I’m doing a pretty good job of doing most of the categories of activities. When I go to make plans next, I’ll most likely schedule activities from the bottom categories. For example, I may do a financial review, schedule a board game night, and plan to attend a City Council meeting.

Notifications

I’ve also given Bieber Bot (my digital personal assistant and robot friend) access to this data, and he helps me make better use of it.

If it’s been too long since I’ve scheduled an activity in any particular category, Bieber Bot will message me on FB Messenger reminding me to schedule something in that category.

Similarly BieberBot will remind me each day of any upcoming activities planned for that day. And if I don’t have any activity planned for a particular day, BieberBot may take the liberty of suggesting an activity.

Finally BieberBot will also remind me if I haven’t been entering activities into the spreadsheet in a while, so I don’t fall too far behind on the logging system.

Setting up these notifcations is a bit outside the scope of this post, but if it interests you how BieberBot works, you can subscribe for future posts here:

You can also learn a bit more about BieberBot here, but there’s so much more to be written.

Archiving Old Events

To keep things clean, my spreadsheet also uses Google Apps Script to archive old events. You can see the code I use for the auto-archiving here.

Getting Started

If this system sounds appealing to you, it’s very simple to get started.

Simply create a new Google sheet with the columns you’d like to track. I use the following:

Evening	Morning	Keywords	Weekday	Date	People	Notes

But for just getting started you might be satisfied with something simpler, say just these three columns:

Activity	Weekday	Date

You can use the formula =TEXT(WEEKDAY(E3), "dddd") to autopopulate the Weekday column, where E3 represents the date (tip: you can use cmd-; to enter today’s date into a cell).

I also create a second sheet called Archive with the same columns as the main sheet, and use the auto-archiving feature mentioned earlier to prevent the main sheet from getting cluttered.

Next create the analysis sheet with the columns:

Category	Activity	Last Date

The first two columns you can fill in with whatever categories and activities you like, and you can add new categories and activities at any time. No need to fill this in too much until you start to see how you’re using the main sheet.

To autofill the Last Date column, the full formula I use is:

=MAX(
  IFERROR(QUERY(scheduled, 
    "select E
    where A contains '" & C2 & "' OR B contains '" & C2 & "' or C contains '" & C2 & "'
    order by E desc
    limit 1
    label E ''"
  )),
  IFERROR(QUERY(archive, 
    "select E
    where A contains '" & C2 & "' OR B contains '" & C2 & "' or C contains '" & C2 & "'
    order by E desc
    limit 1
    label E ''"
  ))
)

This searches both the main sheet (scheduled) and the archive sheet for the most recent occurrence of each activity (C2). For this to work, the rows in the archive and main sheet need to be in reverse chronological order.

Finally, to get the roll-up by category, I use the following Google Sheets formula:

=QUERY(A:E,
       "select B, max(D), count(B) where B<>'' group by B order by max(D) desc
        label max(D) 'Last date',
              count(B) 'Count'
       ", -1)

Though I would like to, I won’t go into how to set up the notifications in this post. For now, that’s just between me and BieberBot.

Conclusion

As a quick recap of my system: (1) I manually enter the activity(s) I do each day; (2) the spreadsheet automatically figures out how recently I did each category of activity; (3) bonus: BieberBot uses this data to help me keep my life on track.

I’ve been using this system for about 18 months now, and I’ve been quite happy with the insight it provides and how helpful it is for planning.

Still, there are many pieces of my system left undiscussed: tracking financial transactions, taking notes on academic papers, staying in touch with friends and colleagues, dealing with social media, and more! I’d love to tell you about the rest, so stay tuned. And don’t hesitate to reach out if you’d like to discuss anything you read. I hope you find this useful, and do let me know if you start using a spreadsheet to track your own life or have a system of your own.

The Weisfeiler-Lehman Isomorphism Test

Fri, 10 May 2019 00:00:00 +0000

Graph Isomorphism

Two graphs are considered isomorphic if there is a mapping between the nodes of the graphs that preserves node adjacencies. That is, a pair of nodes may be connected by an edge in the first graph if and only if the corresponding pair of nodes in the second graph is also connected by an edge in the same way. An example of two isomorphic graphs is shown here.

Figure: Graph 1 and Graph 2 are isomorphic. The correspondance between nodes is illustrated by the node colors and numbers.

In general, determining whether two graphs are isomorphic when the correspondance is not provided is a challenging problem; precisely how hard this problem is remains an open question in computer science. It isn’t known whether there is a polynomial time algorithm for determining whether graphs are isomorphic, and it also isn’t known whether the problem is $\text{NP-complete}.$ The graph isomorphism problem may even be an example of an $\text{NP-intermediate}$ problem, but this would only be possible if $\text{P} \ne \text{NP}.$

The Weisfeiler-Lehman Isomorphism Test

Here is the algorithm for the Weisfeiler-Lehman Isomorphism Test. It produces for each graph a canonical form. If the canonical forms of two graphs are not equivalent, then the graphs are definitively not isomorphic. However, it is possible for two non-isomorphic graphs to share a canonical form, so this test alone cannot provide conclusive evidence that two graphs are isomorphic.

The Algorithm:

For iteration $i$ of the algorithm we will be assigning to each node a tuple $L_{i,n}$ containing the node’s old compressed label and a multiset of the node’s neighbors' compressed labels. A multiset is a set (a collection of elements where order is not important) where elements may appear multiple times.
At each iteration we will additionally be assigning to each node $n$ a new “compressed” label $C_{i,n}$ for that node’s set of labels. Any two nodes with the same $L_{i,n}$ will get the same compressed label.

To begin, we initialize $C_{0,n} = 1$ for all nodes $n$.
At iteration $i$ of the algorithm (beginning with $i=1$), for each node $n$, we set $L_{i,n}$ to be a tuple containing the node’s old label $C_{i-1,n}$ and the multiset of compressed node labels $C_{i-1,m}$ from all nodes $m$ neighboring $n$ from the previous iteration $(i-1)$.
We then complete iteration $i$ by setting $C_{i,n}$ to be a new “compressed” label, such as a hash of $L_{i,n}$. Any two nodes with the same labels $L_{i,n}$ must get the same compressed label $C_{i,n}$.
Partition the nodes in the graph by their compressed label. Repeat 2 + 3 for up to $N$ (the number of nodes) iterations, or until there is no change in the partition of nodes by compressed label from one iteration to the next.

When using this method to determine graph isomorphism, it may be applied in parallel to the two graphs. The algorithm may be terminated early after iteration $i$ if the sizes of partitions of nodes partitioned by compressed labels diverges between the two graphs; if this is the case, the graphs are not isomorphic.

Example of the Weisfeiler-Lehman Isomorphism Test

We demonstrate here the Weisfeiler-Lehman isomorphism test using the example graphs from above. The graphs are shown again here for completeness.

Figure: Graph 1 and Graph 2 are isomorphic. We will apply the Weisfeiler-Lehman isomorphism test to these graphs as a means of illustrating the test.

To initialize the algorithm (Step 1), we set $C_{0,n} = 1$ for all nodes $n$.

For iteration 1, Step 2, we compute $L_{1,n}$. The first part of a node’s $L$ is the node’s old compressed label; the second part of a node’s $L$ is the multiset of the neighboring nodes' compressed labels.

For iteration 1, Step 3, we introduce “compressed” labels $C_{1,n}$ for the nodes:

We now begin iteration 2. In iteration 2, Step 2, we compute $L_{2,n}$:

In iteration 2, Step 3, we compute $C_{2,n}$:

In iteration 3, Step 2, we compute $L_{3,n}$:

In iteration 3, Step 3, we compute $C_{3,n}$:

Since the partition of nodes by compressed label has not changed from $C_{2,n}$ to $C_{3,n}$, we may terminate the algorithm here.

Concretely, the partition of nodes by compressed label may be represented as the number of nodes with each compressed label. That is: “2 7s, 1 8, and 2 9s”. This is the canonical form of our graph. Since both Graph 1 and Graph 2 have this same canonical form, we cannot rule out the possibility that they are isomorphic (they are in fact isomorphic, but the algorithm doesn’t allow us to conclude this definitively.)

Implementation Considerations

One detail omitted is that the algorithm requires canonical forms of the multisets for comparison. Placing the elements of the multiset in sorted order provides one way of doing this.

Additionally, for the representation of the graph that the algorithm emits to be deterministic, we need to establish a convention for creating compressed labels. One possible convention is to use increasing integers starting from 1, and to assign compressed labels to nodes in lexicographical order of their non-compressed labels. Another possible convention is to use the hash of the multiset to create the compressed label.

When comparing the partition of the nodes by compressed label from one iteration to the next to see whether to proceed to another iteration, it is sufficient to compare which nodes are in each partition. If the partitions change, we proceed to the next iteration. If no partition changes, the algorithm can terminate early. When comparing the partitions of the nodes across graphs to see if graphs are isomorphic, we use the canonical form as described above in the algorithm.

Finding the Correspondance Between Isomorphic Graphs

The core idea of the Weisfeiler-Lehman isomorphism test is to find for each node in each graph a signature based on the neighborhood around the node. These signatures can then be used to find the correspondance between nodes in the two graphs, which can be used to check for isomorphism.

In the algorithm descibed above, the “compressed labels” serve as the signatures. Since multiple nodes may have the same compressed label, there are multiple possible correspondances suggested by a Weisfeiler-Lehman labeling. The Weisfeiler-Lehman isomorphism test itself does not provide a way of narrowing down the possible correspondances further.

The Disappearance of Boris Weisfeiler

Boris Weisfeiler – one of the two mathematicians responsible for the Weisfeiler-Lehman isomorphism test – was a professor at Penn State University until he disappeared mysteriously in Chile in 1985. His disappearance in the 80s is unsettling, and I would encourage you to visit boris.weisfeiler.com to learn more about the search for Weisfeiler and the ongoing (2018) judicial proceedings surrounding his disappearance.

Steven List produced an award winning short film called The Colony based on the disappearance of Boris Weisfeiler, available on Vimeo here.

After a brief internet search, I have been unable to find any information about Weisfeiler’s colleague and coauthor A. Lehman, not even a first name. (Update Aug. 2020: A reader pointed me to this recent story about Andrey Leman. Thank you!)

NP-Intermediate Problems

As mentioned in the introduction, it is not known whether the graph isomorphism problem is in $\text{P}$ or $\text{NP-complete}$. In fact, it is not even known whether graph isomorphism falls in either of these categories, or whether this problem is $\text{NP-intermediate}$. The problem of factoring integers is another problem like this, where it is unknown whether the problem is in $\text{P}$, $\text{NP-complete}$, or is $\text{NP-intermediate}$. Bear in mind that $\text{NP-intermediate}$ problems can only exist if $\text{P} \ne \text{NP}$. So if anyone were to show conclusively that factoring or graph isomorphism were $\text{NP-intermediate}$ they would also have shown $\text{P} \ne \text{NP}$ and won the Clay Institute P vs NP Millenium Problem’s million dollar prize.

Applications of Graph Isomorphisms

Now that you’re familiar with the Weisfeiler-Lehman test for graph isomorphisms, what is the graph isomorphism problem good for?

There are applications of the graph isomorphism problem in, for example, computational chemistry and in circuit design.

In chemistry, it is common to represent a molecule as a graph, with the atoms in the molecule being the nodes of the graph and the bonds between molecules being the edges. Here, the graph isomorphism problem can be used to determine if two chemical structures are in fact the same structure. This is important for drug discovery, where scientists are working to create molecular structures that will be useful in order to fight diseases.

A circuit is commonly represented as a graph as well. The components in the circuit form the nodes of the graph, and the connections between components form the edges. The graph isomorphism problem is useful here for determining whether two circuits that are laid out different are in fact identical.

References

Weisfeiler and Lehman first proposed this method in their paper A reduction of a graph to a canonical form and an algebra arising during this reduction in 1968. An English translation of this paper, originally published in Russian, is available here.

More recently the method is also described succinctly in Weisfeiler-Lehman Graph Kernels (Shervashidze 2011).

Now go forth and detect graph isomorphisms!

Writing a Mail Merge in Google Apps Script

Sun, 21 Oct 2018 00:00:00 +0000

Google Apps Script is endlessly useful; in this post you will learn how to write a mail merge with Google Apps Script. It’s less than 30 lines of code! You can use this technique to send personalized emails individually to many recipients. In the second part of this post, you will learn how to schedule these emails to send in the future.

Part 1: Writing a Mail Merge

A mail merge is a program that sends an email to many recipients, automatically filling in the contents of the email according to a template. It’s like a form letter. Each recipient receives a personalized copy of the email.

In this section we use Google Apps Script to create a mail merge that sends personalized “Happy Birthday” messages to a group of our friends. This will involve writing a template email, creating a spreadsheet, and writing a small amount of JavaScript code. With practice, you can perform tasks like this in well under an hour.

First, we draft the email in Gmail. We’ll use angle brackets for the parts of the message we want to replace with data from our spreadsheet.

Subject:
  Happiest Birthday!

Body:
  Dear <name>,

  Happy birthday! I think you, <name>, personally are awesome.
  You may be getting older, but you're still the coolest person I know.
  What are you now, <age>? Wow.

  Party on!
  David

Next, we create a Google Spreadsheet ( sheets.google.com) with three columns like this:

Name  Email  Age  
...   ...    ...

(To add yourself to my spreadsheet and actually receive a happy birthday email on your alleged birthday, just fill out this form.)

Inside your spreadsheet, choose Tools > Script Editor. This creates a new Apps Script associated with your spreadsheet.

Now it’s time to write some JavaScript. We’re going to write a function to perform the mail merge. We’ll call it performMailMerge and it won’t have any parameters, e.g.:

function performMailMerge() {}

It will use the built in SpreadsheetApp library to access the spreadsheet, the SpreadsheetDB library to perform basic data lookup operations on the spreadsheet, and the built in GmailApp library to look up drafts and send emails.

SpreadsheetDB is a small library (a collection of useful functions) that allows us to quickly process rows in a spreadsheet, referencing columns by their header names. Choose one of the two options below for using SpreadsheetDB.

To use the library SpreadsheetDB, choose Resources > Libraries > Add a library and enter 193p7IxBukPVkoxwMH8UDcdkYJiNCDWSylMA_htgliEPxjjqixhzKt_2g. This is the ID of the SpreadsheetDB library. Select the latest version from the version dropdown menu. It should say “SpreadsheetDB” if you’ve done it right. This will allow you to use functions from the SpreadsheetDB library in your project.
To use SpreadsheetDB from source, create a new file in your project and name it spreadsheet.gs (the name is not important). Copy and paste in the source of SpreadsheetDB from the Gist here. If you choose to copy and paste the source into your project, then you must omit the prefix “SpreadsheetDB.” from your calls to the SpreadsheetDB functions in the following code. E.g. SpreadsheetDB.getColumnIndexesFromSheet would become just getColumnIndexesFromSheet.

Now that you’re set up with SpreadsheetDB, I’ll start by showing you the full program, then I’ll explain what each line does.

function performMailMerge() {
  var spreadsheet = SpreadsheetApp.getActiveSpreadsheet();
  var sheet = spreadsheet.getSheetByName("Sheet 1");
  var indexes = SpreadsheetDB.getColumnIndexesFromSheet(sheet);
  var subject = "Happiest Birthday!";
  var draftMessage = getDraftMessageBySubject(subject);
  var templateBody = draftMessage.getPlainBody();
  SpreadsheetDB.forEachRow(sheet, function(row) {
    var name = row[indexes["Name"]];
    var email = row[indexes["Email"]];
    var age = row[indexes["Age"]];
    
    var body = templateBody.replace(/<name>/g, name).replace(/<age>/g, age);
    GmailApp.sendEmail(email, subject, body);
  });
}

function getDraftMessageBySubject(subject) {
  var draftMessages = GmailApp.getDraftMessages();
  for (var i = 0; i < draftMessages.length; i++) {
    var draftMessage = draftMessages[i];
    if (draftMessage.getSubject() == subject) {
      return draftMessage;
    }
  }
}

The documentation for the built in libraries is very good. You can find the documentation for SpreadsheetApp here and for GmailApp here.

Let’s look at the pieces of this function individually:

var spreadsheet = SpreadsheetApp.getActiveSpreadsheet();
var sheet = spreadsheet.getSheetByName("Sheet 1");

This gives us a Sheet object that we can use to access and edit the main sheet of our spreadsheet.

The function call SpreadsheetDB.getColumnIndexesFromSheet(sheet) gives us back a mapping from column name to index. This is useful for accessing elements of a row by name later on. In our case, the result of getColumnIndexesFromSheet is the following mapping: {Name: 0, Email: 1, Age: 2}. This means that Name is column index 0, Email is column index 1, and Age is column index 2. We’re counting columns starting with zero. Perfect!

The call to getDraftMessageBySubject gets a draft from our Gmail drafts that we’ll use later to send the emails to our friends.

We use the SpreadsheetDB.forEachRow function to perform an action for each row in the spreadsheet except the headers. We pass a function with no name (an “anonymous function”) to SpreadsheetDB, and SpreadsheetDB will call that function once per non-header row of our spreadsheet.

    var name = row[indexes["Name"]];
    var email = row[indexes["Email"]];
    var age = row[indexes["Age"]];

The input to our anonymous function, row, is an array with the contents of the row currently being processed. From earlier, indexes[“Name”] is 0, indexes[“Email”] is 1, and indexes[“Age”] is 2. We can use this to get the name, email, and age of one of our friends from the spreadsheet.

var body = templateBody.replace(/<name>/g, name).replace(/<age>/g, age);

This is where the textual replacement actually happens. <name> is replaced with your friend’s name. <age> is replaced with your friend’s age. The ‘g’ indicates that a “global” replacement should be performed, so every time the string <name> appears it will be replaced by the name of your friend from the spreadsheet. Without the ‘g’, only the first occurence of <name> would be replaced.

GmailApp.sendMail(email, subject, body);

Last, this line sends the email!

Since this is JavaScript, you can comment out lines by putting // at the start of the line. Everything occuring after // on the line will have no affect.

I recommend commenting out the GmailApp.sendMail line before running this for the first time to so that you don’t accidently email people incorrectly.

Add the line Logger.log("To: " + email); Logger.log("Subj: " + subject); Logger.log("Body: " + body); in place of the GmailApp.sendMail line until you’re convinced that the correct emails are going to be sent to the correct people. Only once you are sure it’s working as intended, uncomment the GmailApp.sendMail line so it actually sends the emails.

To run the performMailMerge function, choose performMailMerge from the dropdown and click the run (right-arrow) button. To see your logs (the inputs to the Logger.log function calls), choose “View > Logs” or press ctrl-Enter or cmd-Enter.

That’s it. When you run performMailMerge, the emails will be sent with the values from the spreadsheet substituted for the “” and “” placeholders in the draft. Once you’ve got the hang of this, let’s add a few finishing touches before moving on to scheduling emails to be sent in the future.

Finishing Touches

Finishing touch 1: Adding a “Status” column to avoid emailing people twice.

Name  Email  Age  Status
...   ...    ...  ...

Let’s add some finishing touches. Add to your spreadsheet a new column “Status”. When an email sends successfully, we’ll set the status of the row to “Done”. If the status of a row is already “Done”, we’ll skip that row. The updated code is shown here:

function performMailMerge() {
  var spreadsheet = SpreadsheetApp.getActiveSpreadsheet();
  var sheet = spreadsheet.getSheetByName("Sheet 1");
  var indexes = SpreadsheetDB.getColumnIndexesFromSheet(sheet);
  var subject = "Happiest Birthday!";
  var draftMessage = getDraftMessageBySubject(subject);
  var templateBody = draftMessage.getPlainBody();
  SpreadsheetDB.forEachRow(sheet, function(row) {
    var name = row[indexes["Name"]];
    var email = row[indexes["Email"]];
    var age = row[indexes["Age"]];
    var status = row[indexes["Status"]];
    
    if (status == "Done") {
      return;
    }
    
    var body = templateBody.replace(/<name>/g, name).replace(/<age>/g, age);
    GmailApp.sendEmail(email, subject, body);
    
    row[indexes["Status"]] = "Done";
    return row;
  });
}

In order to make this change, we just check the value of row[indexes["Status"]], and use a return statement to skip the row if the status is “Done”. In order to update the row, we modify the row array and return it and SpreadsheetDB takes care of updating the underlying spreadsheet.

With this change in place, we can now add new recipients to our spreadsheet even after having run the performMailMerge function, and if we run performMailMerge again we won’t email anyone who has already received an email.

Finishing touch 2: Adding a menu item that triggers our function.

When we created our Apps Script, it was automatically associated with our spreadsheet. If we add a function named onOpen() to our Apps Script, it will automatically get run when the associated spreadsheet is opened. We can add a menu item to our spreadsheet by writing an onOpen function like this one:

function onOpen() {
  SpreadsheetApp.getUi()
  .createMenu('Mail Merge')
  .addItem('Send Emails', 'performMailMerge')
  .addToUi();
}

Go ahead and refresh your spreadsheet. You should now see a menu called “Mail Merge” with a menu item “Send Emails”. Selecting “Send Emails” will cause your performMailMerge function to be run.

Part 2: Scheduling Emails for the Future

In Part 1 we wrote a mail merge function that emails our friends “personalized” Happy Birthday messages when we run performMailMerge. But why would we want to say happy birthday to all our friends at once? That’s silly, it makes much more sense to send them a happy birthday message on their birthday. In this section, we’ll modify our mail merge to send emails only on the birthday of the recipient.

To start, let’s add a new “Birthday” column to our spreadsheet.

Name  Email  Age   Birthday  Status
...   ...    ...   ...       ...

We’re going to use the moment.js library for working with dates. Create a new file (File > New > Script File) in your Apps Script project. Call it moment.gs, or anything else, and paste in the contents of the moment.js library. You can copy and paste moment.js from here. Note: Be sure to use the non-minified version of moment.js; if you try using the minified version you will likely have trouble pasting the library into your Apps Script project.

Now that we have moment.js, we can get the date of our friends birthday like this: moment(row[indexes["Birthday"]])

We can then write the function isTodaysDate like this. This will check that the month and day of the given date are today’s month and day, irrespective of what year the given date has.

function isTodaysDate(date) {
  var now = moment();
  return now.isSame(date.year(now.year()), "day");
}

Putting it all together, we have this updated version of performMailMerge that will only send the email if the current date matches the birthday of the recipient.

function isTodaysDate(date) {
  var now = moment();
  return now.isSame(date.year(now.year()), "day");
}

function performMailMerge() {
  var spreadsheet = SpreadsheetApp.getActiveSpreadsheet();
  var sheet = spreadsheet.getSheetByName("Sheet 1");
  var indexes = SpreadsheetDB.getColumnIndexesFromSheet(sheet);
  var subject = "Happiest Birthday!";
  var draftMessage = getDraftMessageBySubject(subject);
  var templateBody = draftMessage.getPlainBody();
  SpreadsheetDB.forEachRow(sheet, function(row) {
    var name = row[indexes["Name"]];
    var email = row[indexes["Email"]];
    var age = row[indexes["Age"]];
    var status = row[indexes["Status"]];
    var birthday = moment(row[indexes["Birthday"]]);
    
    if (status == "Done") {
      return;
    }
    if (!isTodaysDate(birthday)) {
      return;
    }
    
    var body = templateBody.replace(/<name>/g, name).replace(/<age>/g, age);
    GmailApp.sendEmail(email, subject, body);
    
    row[indexes["Status"]] = "Done";
    return row;
  });
}

function getDraftMessageBySubject(subject) {
  var draftMessages = GmailApp.getDraftMessages();
  for (var i = 0; i < draftMessages.length; i++) {
    var draftMessage = draftMessages[i];
    if (draftMessage.getSubject() == subject) {
      return draftMessage;
    }
  }
}

Now the only step remaining is to make this performMailMerge function run every day. To do this, click the “Triggers” button in the toolbar (the clock icon). Create a new time-based trigger, and set it up so that it runs Daily (you get to choose when each day it will run). Configure it to run the performMailMerge each day at a time of your choice.

Viola!

You now have a mail merge that’s running daily automatically, sending your friends Happy Birthday messages on their birthdays. Aren’t you a good friend?

While sending birthday emails automatically is somewhat impersonal, the techniques used here are broadly applicable. With Apps Script, you can send yourself reminders, monitor your Calendar events, track your finances, and more. It’s a super useful and super friendly set of APIs, and this automated birthday emailer just scratches the surface of what Apps Script makes possible.

Wrap up

You’ve now learned how to program a mail merge function that uses a spreadsheet as input. You can use a template to send multiple recipients personalized emails. You can schedule these emails to send in the future.

You now have a great power, and with it comes great responsibility. Don’t use this for spam. Be kind and courteous to your fellow citizens of the internet. If you find this useful, do let me know what you use it for (I’m @Bieber on Twitter). I’m using Google Apps Script for a variety of things these days and I am always interested to hear how others are using the same tools.

Introducing Python Fire

Mon, 06 Mar 2017 00:00:00 +0000

Originally posted on the Google Open Source Blog

Today we are pleased to announce the open-sourcing of Python Fire. Python Fire generates command line interfaces (CLIs) from any Python code. Simply call the Fire function in any Python program to automatically turn that program into a CLI. The library is available from pypi via pip install fire, and the source is available on GitHub.

Python Fire will automatically turn your code into a CLI without you needing to do any additional work. You don’t have to define arguments, set up help information, or write a main function that defines how your code is run. Instead, you simply call the Fire function from your main module, and Python Fire takes care of the rest. It uses inspection to turn whatever Python object you give it – whether it’s a class, an object, a dictionary, a function, or even a whole module – into a command line interface, complete with tab completion and documentation, and the CLI will stay up-to-date even as the code changes.

To illustrate this, let’s look at a simple example.

#!/usr/bin/env python
import fire

class Example(object):
  def hello(self, name='world'):
    """Says hello to the specified name."""
    return 'Hello {name}!'.format(name=name)

def main():
  fire.Fire(Example)

if __name__ == '__main__':
  main()

When the Fire function is run, our command will be executed. Just by calling Fire, we can now use the Example class as if it were a command line utility.

$ ./example.py hello
Hello world!
$ ./example.py hello David
Hello David!
$ ./example.py hello --name=Google
Hello Google!

Of course, you can continue to use this module like an ordinary Python library, enabling you to use the exact same code both from Bash and Python. If you’re writing a Python library, then you no longer need to update your main method or client when experimenting with it; instead you can simply run the piece of your library that you’re experimenting with from the command line. Even as the library changes, the command line tool stays up to date.

At Google, engineers use Python Fire to generate command line tools from Python libraries. We have an image manipulation tool built by using Fire with the Python Imaging Library, PIL. In Google Brain, we use an experiment management tool built with Fire, allowing us to manage experiments equally well from Python or from Bash.

Every Fire CLI comes with an interactive mode. Run the CLI with the --interactive flag to launch an IPython REPL with the result of your command, as well as other useful variables already defined and ready to use. Be sure to check out Python Fire’s documentation for more on this and the other useful features Fire provides.

Between Python Fire’s simplicity, generality, and power, we hope you find it a useful library for your own projects.

Ĉu Vi Estas Mia Patrino?

Wed, 01 Jul 2015 00:00:00 +0000

This is my translation of P.D. Eastman’s children’s book “Are You My Mother?” into Esperanto. Enjoy. -David

Ĉu Vi Estas Mia Patrino?

By P.D. Eastman

Patrina birdo sidis sur ŝiaj ovo. La ovo saltis.
“Mi devas ion por mia beba birdo manĝi!” ŝi diris.
Tiel for ŝi iris.

Ene la nesto, la ovo saltis.
Ĝi saltis kaj saltis kaj saltis.
Gis…

…el venis beban birdon!
“Kie estas mia patrino?” li diris.
Li ne vidis ŝin ie ajn.
“Mi iros serĉi ŝin,” li diris.

El la nesto li iris.
Malsupren, malsupren, malsupren!
Plopi! (1)
La beba birdo ne povis flugi.
Sed li povis marŝi.

“Nun mi iros kaj trovi mian patrinon,” li diris.

“Ĉu vi estas mia patrino?” la beba birdo demandis katidon.
La katido nur rigardis kaj rigardis.
Ĝi ne diris nenion.

“Ĉu vi estas mia patrino?” la beba birdo demandis kokinon.
“Ne,” diris la kokino.

“Ĉu vi estas mia patrino?” la beba birdo demandis hundon.
“Mi ne estas via patrino. Mi estas hundo,” diris la hundo.

“Ĉu vi estas mia patrino?” la beba birdo demandis bovinon.
“Kiel mi povus esti via patrino?” diris la bovino. “Mi estas bovino.”

La beba birdo haltis por pensi.
La katido kaj la kokino ne estis lia patrino.
La hundo kaj la bovino ne estis lia patrino.

“Mi havas patrinon,” diris la beba birdo.
“Mi scias ke mi trovos ŝin. Mi. Trovos. Ŝin.”

Ĝuste tiam la beba birdo vidis grandan aferon.
“Vi estas mia patrino!” li diris.

La granda afero diris “SNUFANTE!” (2)
“Ho ne!” diris la beba birdo. Vi estas terura snufanto!

La Snufanto levis la beban birdon supren, supren, supren.

Poste io okazis.
La Snufanto metis la beban birdon reen en la arbo.
La beba birdo estis hejmo.

Ĝuste tiam la patrina birdo revenis.
“Mi scias kiu vi estas,” diris la beba birdo.

“Vi ne estas katido aŭ kokino aŭ hundo.
Vi ne estas bovino aŭ Snufanto!
Vi estas birdo, kaj vi estas mia patrino!”

Footnotes

1: If a bird falls out of a tree in English, it hits the ground and goes “plop!” But if a birdo falls out of a tree in Esperanto, what sound does it make?

2: The word for snort in Esperanto is snufante. It’s not exactly onomatopoeic, but imagining an enormous John Deere tractor going “SNUFANTE” before helping out a baby birdo is actually strangely heartwarming.

Running Firefox in the Cloud

Sat, 08 Nov 2014 00:00:00 +0000

I found running Firefox in the cloud difficult, but with enough digging around I was able to get it working. Now I’m able to drive a headless version of Firefox with Selenium using Python. This lets me do things like have my Scrabble Bot play Words with Friends autonomously on Facebook.

What am I going to show you how to do?

I’m going to explain how you can install Firefox and Selenium on a virtual machine in the cloud. I’m going to show this using a Debian instance on Google Compute Engine, but you could also do this using a virtual machine from Amazon EC2.

Before continuing you should create a virtual machine running Debian on Google Compute Engine (GCE). You can learn how to spin up a VM here. Make sure that you can SSH into your machine.

Installing Firefox

We’re actually going to install Iceweasel. To the best of my knowledge, the only difference between Iceweasel and Firefox is the branding. To do this, we’re going to:

Update the sources list that apt-get uses to find packages. Edit the file /etc/apt/sources.list and add the line deb http://mozilla.debian.net/ wheezy-backports iceweasel-release.
Update apt-get by running the command sudo apt-get update.
Install Iceweasel by running the command sudo apt-get install -t wheezy-backports iceweasel.

I figured this out by searching for “iceweasel sources list”, which brought me to http://mozilla.debian.net/. If you’re looking to install a different version of Iceweasel/Firefox or have a different version of Debian, check there.

Running Firefox

We can now run the browser we installed with the command firefox or iceweasel, but it will shut down because there is no display. To run the browser without a display (that is, to run it “headlessly”), we’re going to:

Install the display server Xvfb (X virtual framebuffer) with the command sudo apt-get install xvfb.
Start the display server with the command sudo Xvfb :10. Here, :10 is the server number we chose for the virtual display we’re creating.
Now run export DISPLAY=:10.

You should now be able to run firefox or iceweasel successfully.

If you’d like to see what’s on your virtual display, Wikipedia provides an example of how to take a screenshot of your virtual display.

Installing Selenium

We can now run a headless version of Firefox. Since there’s no user interface to click around in, the next logical step is to learn how to drive the browser. For this, we’re going to set up Selenium in Python. When dealing with packages in Python, I recommend using pip inside a virtualenv with virtualenvwrapper. There are instructions for setting up these packages below.

Once you have virtualenvwrapper installed and you’re safely working in a virtual environment, just issue the command pip install selenium to install selenium. If you’re not in a virtualenv, you may need to issue the command with sudo.

Try running this Python script to see if it’s all working. It should take you to Google Images, search for cute kittens, and then save a screenshot to the file ‘adorable.png’.

Installing virtualenvwrapper

As promised above, here is a section about virtualenvwrapper.

Managing Python packages effectively is difficult, and I can’t do the topic justice here. I recommend using virtualenvwrapper to help, and you can read more about how to do this here. For completeness, I’ll briefly run through the steps of getting set up with pip, virtualenv, and virtualenvwrapper.

pip is Python’s package manager. Once you have pip set up, you can run pip install package-name to install the package called package-name.

To set up pip, run the following commands:

curl https://bootstrap.pypa.io/get-pip.py > get-pip.py
sudo python get-pip.py

virtualenv is a tool that let’s you have different versions of Python and Python packages for different projects that you work on. Combined with virtualenvwrapper, it will help you avoid headaches of keeping track of where your various Python versions live and where they can find their packages on your machine.

To install virtualenv, run the command sudo pip install virtualenv.

To install virtualenvwrapper, run the command sudo pip install virtualenvwrapper.

Add the following three lines to your .bashrc or another startup script:

export WORKON_HOME=$HOME/.virtualenvs

export PROJECT_HOME=$HOME/Devel

source /usr/local/bin/virtualenvwrapper.sh

You’re now ready to go. Learn more about how to use virtualenvwrapper effectively here.

Last thoughts

I hope you find this helpful. You can now install Firefox on Google Compute Engine, run it using a virtual display, and drive it using Selenium with Python. If something isn’t working, or if something is working really well and has you super excited, I’m happy to talk it through with you. Email’s the best way to contact me. Go forth and automate your life!

Why a Code@Night is not a Hackathon

Fri, 06 Sep 2013 00:00:00 +0000

I’ve written in the past about what a Code@Night is. And they’re wonderful. One of my favorite topics, really. People come to these things and they just code all night. Great experience. Tons of fun. Fruit. PacMan. Yadda, yadda, yadda.

I tell people this all the time, and a pretty common response is “so, it’s like a hackathon?”

So I think for a bit. There’s coding. There’s minimal sleeping. There are tech talks. There are pizzas, snacks, and refreshments. Fruit is hard to come by at a hackathon, but it can be there too. So I concede, “yeah, it’s a bit like a hackathon.”

But I know we’re wrong. Code@Night doesn’t feel like a hackathon at all.

Here, I made you an analogy. It’s not a good one, but you can read it anyway. “A hackathon is to a Code@Night as a track race is to a stroll through a park.”

Or for you symbolically minded folks…

Hackathon : Code@Night :: Track race : Evening stroll

How can they feel so different when the only core difference is that we’ve removed the competitive aspect?

Well, in some ways Code@Night is a lot like a hackathon, but in others it’s not at all. It’s a hackathon minus the competition, plus the assortment of non-hackathon things that happen at Code@Nights. e.g. Pacman, the occasional contest, the tight-knit community, lots of fruit, a tech talk smack in the middle. Also, Code@Night runs for 12 hours in the worst case (if people stay for breakfast, we’ll feed them), but more realistically only goes from 9 pm to 3-4 am. A hackathon usually lasts 24 hours or more.

It sounds like these are small differences from a hackathon, but it’s really a completely different atmosphere. There’s no start and end time for your project. You can work on the same thing Code@Night to Code@Night if you want. And people don’t cluster into teams the way they do at hackathons. You also end up getting large group discussions going, which you rarely get at hackathons because people are rushed to work on their 24 hour product. No one’s worrying about prizes or judges. You don’t have to demo anything to feel accomplished (but we love demos! they’re sure encouraged!) You can build what you want to build, not the thing you think will win over an audience.

Code@Night isn’t about starting. It isn’t about finishing. It’s about doing. It’s about learning. It’s about being a part of a community. By removing the artificial competitive aspects and time constraints that hackathons impose on coding, we’re left with a really rewarding experience. People step back from their coding in ways they don’t at hackathons. They teach about Python, about Hadoop. They sit in boxes ( or put boxes on their heads). They share their projects. They debate bacterial growth characteristics on the surface of crisp orange-colored, peanut butter-flavored flakiness coated in compound chocolate. They talk about technology. And sometimes they write poetry.

So, the next time someone asks if a Code@Night is like a hackathon, I’ll be ready with my “kinda, but not really, actually no” answer. And I’ll be ready to say how they’re different, and how I actually prefer the Code@Night. But hey, I’m biased, I organize the things.

Next Code@Night? It’s Sept 13, 9 pm. Sherrerd, Third Floor. Open to all Princetonites. Hope to see you there!

MathCounts Coach Q+A

Fri, 16 Aug 2013 00:00:00 +0000

Last week I did a Q+A for the Life at KA blog. The last time I did an education-centric online Q+A was more than three years earlier. Let’s take a look.

Originally posted at mathcounts.org in March 2010.

How long have you been a MATHCOUNTS coach?
This is my second year coaching the Van Antwerp MATHCOUNTS team.

What is your favorite part about being a MATHCOUNTS coach?
It’s great to see my students having a good time with math. I love seeing their expressions when an idea becomes clear to them (an Aha! moment, as Martin Gardner would have it) or when they solve a challenging problem. I also like seeing their reactions to the problems I write. They seem to enjoy when I put their names into our practice problems. On some occasions, I’ll have one idea in mind when I write a problem, but I’ll make a typo that makes the question quite absurd. That’s always fun to laugh about.

How do you prepare your students for MATHCOUNTS competitions?
Leading up to the Chapter Competition we met once a week after school. These meetings lasted 90 minutes, with a snack break (often called “Scoff Time!” in excited voices by the students) in the middle. After the team qualified for the State Competition we added a second shorter (45 minute) practice day to the week. On some days we practice with old MATHCOUNTS rounds. Other times we’ve played mathematical games such as variations of Greedy Pig and mathified versions of Jeopardy, Tic-Tac-Toe and Hangman. We’ve also experimented with a variety of practice formats including relays, topic-specific stations, and something akin to HMMT’s Guts Round.

Before the Chapter Competition we also hosted a MATHCOUNTS scrimmage at Union College for six local MATHCOUNTS teams, so that the students got a chance to practice in a competition setting before the official contest date. At the scrimmage we also ran an unofficial Countdown Round for all the students interested in participating, which I had a lot of fun emceeing, and I think the students enjoyed as well.

What interesting techniques have you used in the classroom or after school with your students to get them excited about math?
When our final practice before the Chapter Competition was cancelled due to snow, I put together a set of problems for the team to solve before the competition. Each member of the team received one question by email. What was interesting about these problems was that each one depended on the answer to another. This encouraged the team members to work together in the days before the competition to find a set of solutions that satisfied all of the problems simultaneously.

Another technique is to encourage the students to share their methods of solving MATHCOUNTS problems with the rest of the team. By sharing their ideas, they frequently spot errors in their own work. Teaching is also a great way to become better versed in the material (that’s one of the reasons I do this!), so its a great learning tool for everyone in the room. Often it is easier for someone on the team to explain a difficult concept to another student than for a coach to do this because the students have a better perspective on what parts of the idea are challenging or confusing. It helps a bit that I’m only a few years older than my team!

Why do you think math is so important for middle school students?
To start, it can be fun. The problem solving skills that stem from practicing math are useful in all areas of life. Activities like MATHCOUNTS also help you to think faster and be more creative. And to fully appreciate the world around you or the daily news, it’s really important to be comfortable around numbers.

Speaking as a high school student, I can also tell you that having a solid foundation in mathematics can be a big stress reducer in school, so it’s great to get interested in math early!

Do you have any advice for Mathletes® hoping to advance in the Competition series?
Practice, ask questions, and develop a good sleep habit.

Attending MATHCOUNTS practices is a great place to start, but there are so many ways to go beyond that in becoming a better problem solver. If you’re clever with a search engine, you’ll find there are tons of resources available including problems on the MATHCOUNTS website, these MATHCOUNTS drills, Mr. Diaz’s MATHCOUNTS Bible, and Coach Monks’s MATHCOUNTS Playbook. Get together with members of your team to practice outside of school. Use problems you find online or, better yet, write your own, and don’t forget to ask your parents to provide you with a snack.

If ever there are problems you don’t understand (and this should happen very often), ask about them. At practice it can be difficult to get to all of them because there is so much to do in so little time, but don’t limit your questions to your practices. Talk to or email your coach or teachers or siblings or parents or friends or friends' parents. Anyone might have the knowledge or insight to answer/explore your questions.

What advice would you give to new coaches just starting a MATHCOUNTS program?
Have fun with it, and your students will have fun too (at least in principle, anyhow). Chances are your students will know things and ask things you don’t know or can’t answer, and that’s good. Being a coach is an opportunity to learn as well as to teach.

The most important advice I can leave you with (keep in mind I have more experience as a student than as a coach) to get your MATHCOUNTS team off the ground, though, is to provide snacks. I recommend grapes, as they’re healthy, tasty, easy to distribute, and avoid the majority of allergy problems.

Life at KA: A Khanversation with David Bieber

Wed, 14 Aug 2013 01:00:00 +0000

Originally posted at life.khanacademy.org

We’ll dive straight into the questions for our latest Khan Academy Khanversation, this time featuring rising Princeton senior David Bieber.

David, let’s begin with the question everyone is dying to hear you answer. Are you related to Justin Bieber?

It’s funny you should ask that. People tell me all the time “sorry about your last name” or “I hope you’re not related to that singer” or “can you get me free tickets to the Bieber concert?” And because of that, I’ve started feeling a real connection to the guy. I root for his success. I feel down when people trash his music. I think we could become genuinely good friends if we weren’t both vying for the top spot on Google for the search term “Bieber.”

And no, I don’t listen to his music (much).

So, Bieber, how did you come to work at Khan Academy?

Sounds like life story time.

I had some pretty incredible teachers growing up. Lots of them. And not just the grown-up kind – the peer kind too. Throughout middle and high school I was involved in this fantastic math community, the Albany Area Math Circle. We practiced loads of competition math, and most of what I learned, I learned from other students. Some a few years older than me, some a few years younger.

Here I got involved in coaching MathCounts, and I quickly realized that I love teaching. I ran a math camp for middle school students one summer and I had a blast doing it. Whatever I was going to do with my life, it was going to involve teaching in some capacity.

Fast forward a couple years, and I’m sitting in a dining hall in some castle in Princeton. I’m talking with some friends about startup ideas – I have a lot of friends interested in entrepreneurship – and I have a brilliant idea for a company. (I love getting lost in the excitement of a good idea.) “We’ll take classes, and we’ll put them online. Just Princeton classes to start – it’ll be like MIT’s OpenCourseWare – but we’ll make it scalable. And any student with their professor’s permission will be able to really easily get their classes' lectures online.” Sounds a bit like Coursera. Or YouTube. From there on out, I gave a lot of thought to teaching at scale.

Which brings us to Khan Academy. I’m interested in programming and technology and education, and I’m all for making the world a better place. What better way than the KA way to leverage my interests to facilitate teaching at scale?

Wait… You said you were sitting in a castle?

Yeah, half the buildings on campus are castles. But that’s not the best thing about Princeton. The best thing about Princeton is Code@Night (though I’m biased – I organize them). Every two to three weeks there’s a Code@Night where people gather in this beautiful non-castle building called Sherrerd to code at night. Usually a student or professor or an engineer from a sponsor company will give a tech talk or a demo. People will code all night, they’ll play PacMan and eat pizza. Sometimes they’ll write poetry. It’s really an incredible experience. There are even whiteboards all along the walls making it an ideal environment for puzzles.

Do you have a puzzle you could share with us?

Sure. Here’s one that was going around the KA office recently.

There are one hundred prisoners in a prison, and the warden is a bit too much of a puzzle enthusiast. She finds an empty jail cell, and into it puts one hundred boxes. Under each box she puts the name of one of the prisoners, so every prisoner’s name appears under precisely one box.

Then she tells the prisoners this:

Each one of you will be led individually into the jail cell with one hundred boxes. Under one of these boxes is your name. You will each be allowed to look under 50 boxes, but if even one of you doesn’t find your own name, all one hundred of you will be locked away forever. That said, if all of you find your own name then you will be freed immediately.

The prisoners are terrified, but also excited. They love puzzles too. Help them form a strategy to maximize the chance that they’ll be granted their freedom.

Some caveats: Once the prisoners start going into the cell with the boxes, no communication between the prisoners is allowed. And the prisoners cannot make any modifications to the room with the boxes either – no leaving boxes turned over, no writing on the walls, and no taking boxes out of the room or moving them, even the one with your name under it.

Good luck.

Sounds tough. What’s the answer?

Pshhh, don’t give up yet. It’s a really good puzzle. Who’s asking these questions, anyway?

That’d be me. David Bieber.

So what you’re telling me is that I’m involved in a virtual interview… with myself?

Yeah.

And to think I thought the Turing test was hard… why are you writing all this anyway?

Well, it’s good to take a break from coding every now and again. Plus, I really like having people read the things I write – I found this out after starting my blog a year ago ( take a look!) – it makes the writing process a lot more fun. It also gives you all a better feel for life at Khan Academy, and that transparency is really valuable. We’re real people who make KA run! Seeing all the awesome stuff on this blog is part of why I came to work here. But most of all, the writing really is fun! Especially the Bieber bit – I had a good time writing that. And no, we’re not related.

ssh Passwordlessly

Wed, 14 Aug 2013 00:00:00 +0000

You’re used to ssh-ing into machine B from machine A by typing:

ssh username@B.com
password

Now you wish to ssh from machine A into machine B without having to enter your password. Just follow these steps.

Create a public/private RSA key pair by issuing the unix command ssh-keygen. Do not enter a passphrase.
Put the private key on machine A (the machine you wish to ssh from). Usually the best place to put it is in ~/.ssh.
On machine B, create the file ~/.ssh/authorized_keys. Add the public key as a line to this new file. You can do this with the command cat public_key >> authorized_keys or by copy/pasting.
On machine A, create the file ~/.ssh/config. Add the following to this file. Host anyname
HostName B.com
User username
IdentityFile ~/.ssh/private_key

That’s it. You can now passwordlessly ssh into machine B from machine A just by typing:

ssh anyname

And as a bonus, you can now scp to and from machine B passwordlessly as well.

There’s loads more that you can do with an ssh config file. So have a Google, and see what you can see.

I’m posting this online because it’s something I wish I had known sooner. Hopefully you can benefit from it.

This post isn’t the whole story though! Using an ssh key without a passphrase (a la step 1) is dangerous; if anyone gains access to your private key then they’ll be able to ssh into machine B as you. The solution is to (back in step 1) create a passphrase for your private key, and then ssh-agent will remember your decrypted private key so that you don’t need to enter your password too often.

Teach at Scale

Wed, 07 Aug 2013 01:00:00 +0000

The best way to learn is to teach. It’s also, most of the time, the most fun way to learn. And as an added bonus, the person you’re teaching learns too.

I strongly believe that you, as someone who has found themselves reading strange blogs on the internet, have a lot to offer the world in the way of teaching. So I want you to teach at scale. Yes, I’m looking at you Mom and Dad, former Facebook and Twitter interns, my fellow interns here at Khan Academy. I’m looking at you, my Princeton and Nisky friends, and my former MathCounts students. Yes, even you, HackerNews readers and Google searchers and friends of friends. I want you to give teaching at scale a try.

It’s hard. Don’t mislead yourself into thinking this is an easy task that I’ve laid out for you. It takes courage and confidence. When you teach at scale, you make yourself vulnerable. You have to be ready for people to catch your mistakes and challenge your teachings. People will out-think you, and you should be ready to embrace that.

What, then, should you teach? I offer two suggestions. 1) Teach what you want to learn, and 2) teach what you know best. Sometimes the best teachers are not those with 10 years of teaching experience in a subject, but rather the people who remember what it’s like not to know. Teach something while the Aha! moment that helped you understand it is still fresh in your mind. The second suggestion, teach what you know best, sounds obvious, but the reason behind it has some subtlety. Don’t teach what you know best because you’ll probably avoid mistakes on that topic – do it because it’s something you can speak passionately about. If your teachings get a student genuinely excited about a topic, that is worth so much more than having them quickly master it.

So how should you get started? The simplest way is to blog. You can set up a blog and be started writing in minutes by signing up at tumblr.com. Once you’ve written something educational, share it with your friends and with online communities. Related to technology? Post it on HackerNews.

DIY method two of teaching at scale is to make YouTube videos. I gave this a try making videos teaching Item Response Theory and Bloom Filters. Here’s my setup.

It’s a piece of cardboard with a hole cut out of it for a camera to look through, mounted above some paper. I used my iPhone as the camera.

Here are the resulting Bloom Filter videos, mistakes and stuttering and blurriness and all. I’m pretty embarrassed by them, and I’m hoping future videos I produce will come out better.

https://www.youtube.com/playlist?list=PLTFg4cPjhl-2nrkIR1z_MSTcxNeyPk4PO

I’m going to continue to make videos as I learn awesome things over the years, and I hope you do the same.

How you teach at scale is up to you. Blog. Make videos. Post code. Start a math circle.

Be creative. The most effective teachers-at-scale are not always those who leverage technology to reach a wide audience. They are those who teach passionately, and in doing so infect others with a love for learning and teaching.

Automate Your Life: Sending Emails

Wed, 13 Mar 2013 00:00:00 +0000

In a hurry? Scroll down and skip to the code. That’s the good part anyway.

One of the beauties of programming is the elimination of redundant aspects of life that you find yourself doing over and over again. When you find yourself calling home every weekend, logging hours at your job daily, or getting your hair cut every month, it’s time to learn to program. Write a short script to do these repetitive behaviors for you. Just kidding. Don’t.

These are joke examples – I strongly advise against automating discussions with your mother – but they aren’t so farfetched. Automating processes - shopping, shipping, server maintenance - save countless hours and dollars. Here, we’ll do a thought experiment: imagine a team of people running Amazon.com without automation. As a crude estimate, let’s suppose they sell a mere billion books each year. Let’s imagine they only handle 1% of all Internet traffic. And let’s pretend they only store 1 trillion objects in their S3 storage system. Then with their team of about ninety thousand, each Amazon employee would have to be selling and shipping 30 books a day on top of processing and transmitting over 100 gigabytes of Internet traffic daily, all while handling requests for a hundred thousand S3 objects that they’ve had to memorize. Maybe Jeff Bezos could pull it off. But you’re not Jeff Bezos, so I’d say it’s worth your time to learn to automate.

Even education is being automated these days. Will the next century’s robotic overloads prove to be better professors than the ones we grew up with? To which you may reply: “Oh gee whiz! I sure hope we get better robotic overloads in the future! The ones I grew up with were just miserable professors.” But that’s not what I meant. I’m asking whether Khan Academies and Duolingos will replace our real life Ms. Frizzles and Mrs. Gorfs. A brief excursion with Duolingo makes me suspect it will be a major part of all future language educations. This seems counterintuitive. Languages are inherently about communication. The best way to learn a language should necessarily involve communicating (and the very best way will probably forever remain immersion, but often that’s not an option). Conversation and human interaction are so fundamental to language learning that it seems almost absurd that a computer could be a good language teacher, at least until we have strong AI. But the computer manages to do one thing extraordinarily well that isn’t scalable in a traditional classroom; the computer focuses entirely on me. So, yes, it lacks creativity and the personal touch of a great teacher. It doesn’t tell funny stories or crack foreign language jokes. It doesn’t show cute movies and it can’t relate to me at all. Yet despite all these shortcomings, it’s remarkably good at helping me learn the basics of a foreign language, because it can focus on me, going at my pace and not having to devote its efforts to maintaining the attention of 30 other students.

Automation has clearly been taking over everywhere for years. You have to join it. It’s a real life superpower. So let’s learn to automate something basic. We’ll go with software: a Python script to send emails.

I present: the code. You can run it as is. Save it as a file and run it with Python, and it should just work out of the box. Go ahead, try it.

Awesome. You just sent me an email. What good is this code? Tweak it a bit and you can use it for personal alerts – systematically check when you’re low on milk and email yourself a reminder – have your computer email you daily with upcoming birthdays; you can even have it buy the gifts. You can email yourself automatically when a course opens up on your registrar’s website. We’ll talk about auto-enrolling another time. Or you can set up a script to send different versions of emails to different people, like those spam emails that pretend to be personal by including your first name.

With great power comes great responsibility. Don’t send spam.

With a new Python library in your toolbox, I hope you save yourself thousands of hours. Until next time, try to stay dry.

Discuss on Hacker News.

Lorenz Attractors and Chess

Wed, 13 Mar 2013 00:00:00 +0000

Every pawn is an inverted Lorenz attractor. Not every Lorenz attractor is an inverted pawn.

Blokus Life Lessons

Thu, 31 Jan 2013 00:00:00 +0000

In this post, clicking the bold word Life will replace it with the bold word Blokus.

Life is simple. It gets more complex over time, but nothing insurmountable. In the beginning, your path is pretty much laid out. You have some control over what directions you go, and these are important decisions. They’ll shape your future. They’ll change what opportunities you have down the road. But as important as they are, there are relatively few wrong decisions. Sure, you can screw up at any point, including early on. And sometimes the damage is irreparable. As my dad always says, “avoid the big mistakes.” Aim high, and lay a foundation for yourself to grow. Act with a focus on making your situation the best it can be, always with an eye toward the future. It’s the most important thing you can focus on with so much still ahead of you, with so much still unseen.

Life is all about people. It’s about interactions. It’s important to realize this, that what you decide to make of a situation has a wide impact, bigger than yourself. When you change the world, you change it for everyone. And when you think to yourself, “what should I do now?” you also have to think about how others will respond. How will things be different down the road? How will things be different just a few minutes from now? The person sitting across from you: she has her own mind, her own motives. Your intentions and hers- sometimes they align nicely, fitting together perfectly and making both of you better off. But often they don’t mesh and they get in each other’s way. One must always be wary of this.

It’s interesting that children don’t understand this at first. People don’t develop a theory of mind until the age of three or four. Until then, they don’t grasp that people have different experiences, different thoughts, and different aspirations. Some people with autism spectrum disorders experience a deficit in their theory of mind that lasts beyond childhood, making life supremely difficult. It’s very unfortunate. It takes time before children begin grasp the complexities of life, and at first they cannot puzzle out how others think and how others will behave. Over time, one develops a more analytic mind and the puzzles of behavior become simultaneously more understood and more mysterious.

Take any individual piece of life and you’ll find it mostly transparent. It’s a beautiful, colorful thing, completely understandable, devoid of mystery and yet full of wonder. With thoughtful reasoning, you can pretty much figure out how it fits into the bigger picture. Yet the bigger picture itself, like a stained glass window, is so full of intricacies that even with a remarkable attention to detail it seems there is always more to explore. There is always, it seems, another possibility to consider.

When the pieces of life stack up against each other, the transparency fades. It becomes hard to distinguish one thing from another. Important decisions are more difficult to make, yet remain even more consequential than any before. Sometimes you may look out and see something that really excites you: a thought so fun and so novel it drives you to action; you get the idea that everything fits together so nicely. But there’s always something missing. That one piece of the puzzle that you can’t find, and nothing will fill this void. With everything looking so good, like everything was where it was meant to be, the only way to go from here is down. Your choices are no longer clear, and you can’t tell one from another. What to do? What to do?

There are tough decisions with trade-offs and you simply cannot please everyone. Know what situation you’re in, your strengths and your weaknesses. Lay out your possible decisions clearly in front of you. Organize. Evaluate. You mustn’t rush into big decisions.

When you’re in a tough spot, as you surely will be, do not despair. You may be tempted to look back on your past with regret. Hindsight is 20/20. You couldn’t have known about that poor decision, that wasted opportunity. It’s a sunk expenditure and you must make the best of your current situation. Sometimes you’ll feel like you don’t fit in, and that’s just part of life.

As life progresses, you will have more decisions to make with higher impact. You take on more responsibility with your actions and you have more meaningful interactions with other people. Setting goals, and having them mesh with the goals of others is important. My dad and I agreed to each do 180 days of push-ups and sit-ups this year. Whether or not this sort of goal will lead to success in life is questionable, but having an objective like this is useful. Goals must not be unrealistic, and you should be aware that sometimes it may feel like the world is conspiring against you, and perhaps it is. But often it’s not, and when things don’t go as planned there is no one to blame but yourself. I’ve set out to become a better storyteller this year. Stories, to me, have a lot of value, and they certainly make life more interesting.

It is often said a goal should be objective and measurable. For a goal such as storytelling, though, that seems absurd. Progress here can only be measured by a good tale.

Life, we all know, eventually comes to an end. And as it does so, think about what you want to leave behind. Where do you want your possessions to go? You can’t take them with you. So be charitable, and get rid of as much as you can afford. As much as the world is ready for. Think of those who will follow you, and ask yourself how will they feel about your end. Did you leave them in a good situation? Perhaps you set them up to finish soon, just as you did.

Think of everything. It’s all about strategy. Life is a game.

Sorry, have I been writing life? I meant Blokus.

Blokus is a game.

Here’s the gist. You can click the bold words to go between Blokus and Life.

Get your new Django based web app live with AppFog in minutes

Tue, 25 Dec 2012 00:00:00 +0000

You’ll have your Django web app up and running live on the internet with AppFog in a matter of minutes.

Here’s what you have to do.

The repository is available at https://github.com/dbieber/django-appfog-helloworld.

Here’s what comes with this repo.

It’s a Django project that works out of the box (just download it and python manage.py runserver. If you have Django installed, it’ll work!). It’s already setup to serve static files. It’s all ready to use MySQL or sqlite3. It ships with Bootstrap and jQuery. Django’s admin app is enabled and ready to go. It has templates for 404 and 500 errors so you can quickly see how to get error handling working. And it has one app, named app_name, that demonstrates the core functionality of Django: models, views, templating, and forms, by collecting email address of early users to your site. It’s a rudimentary version of LaunchRock.

My hope is that it will work well both as start code and as reference code for how to get Django working quickly.

This template is great for quick and dirty websites. It’s perfect for a hackathon or for prototyping an idea with friends. And it’s a great template for getting started with Django, Bootstrap, and jQuery. But it’s not production quality in two key respects, security and scalability. Keep that in mind as you start building.

Now, let’s dive deeper into getting your Django app running.

Setting up AppFog

Create a new “Django Python” app. Host it on AWS. And give it a name. I called mine “helloworld-django”. Let’s say you call yours name_of_app.

You’ll want to set up the af command line utility to make deploying your site easier later. On Mac or Unix this is as simple as gem install af. Windows instructions are simple too, and are available in the “Update Source Code” tab of your app on AppFog.

(Optional) Make sure you have a MySQL database service provisioned. To do this, log in to AppFog, choose your app, and choose the “Services” tab. You can either bind an existing MySQL database service to your app or create a new one. If you don’t have a MySQL database provisioned, the app we deploy in a moment will use sqlite3 as a fallback and should work just fine.

Cloning this repository

git clone https://github.com/dbieber/django-appfog-helloworld.git

The web app is already all ready to be pushed to AppFog.

Before you go and push this website live, there are a few things you should do, all marked with TODO in the source code you just downloaded. These are the most important:

Set your SECRET_KEY in settings.py
Set up the ADMINS variable in settings.py
Set default admin account credentials in views.py
Always be aware of whether DEBUG = True or DEBUG = False in settings.py. This changes how errors are handled and you definitely want DEBUG set to false when your site is public.

Now you’re ready to go live!

Pushing to AppFog

Be sure to either run python manage.py collectstatic or place all your static files your static directory before pushing.

Log in to AppFog with the command af login. (Find instructions for setting up the command line tool in the “Update Source Code” tab of your app on AppFog.com.

From inside the django-appfog-helloworld directory (which, by the way, you are free to rename) run af update name_of_app.

That’s it. Your site is live! You’re already collecting email address of beta-testers. Huzzah!

Testing Locally

To test locally, you’ll need to install Django.

To test your app locally, navigate to the django-appfog-helloworld directory containing manage.py:

python manage.py runserver runs your app locally. View it at localhost:8000.
python manage.py syncdb sets up your local database by default in a file called dev.db.
python manage.py collectstatic copies all your static files into your static files directory.
python manage.py help for more.

Django is a Python based web framework with excellent documentation and tutorials. To learn more about Django, visit www.djangoproject.com. Happy hacking!

Comments? Head over to HackerNews.

CRM's: Not Just for Sales Folk

Thu, 29 Nov 2012 00:00:00 +0000

In this blog post I recommend Streak, a customer relationship management (CRM) tool that lives inside Gmail.

Discuss on Hacker News.

What do you use email for? I’m curious, so feel free to let me know before reading further (Google form).

Personally, there are a just a handful of use cases that make up the vast majority of my email usage. Let’s list them.

Organizing talks and events for ACM and Splash!
Applications for summer internships
Read-only-ish mail (Mailing lists announcing talks and events, class related announcements, discussions I minimally partake in, spam, etc.)
Talking with friends

For about three months now, I’ve been thinking that email is woefully insufficient for managing my communication needs, particularly use cases 1 and 2. When I’m organizing a Code@Night I’ll often have 3 or more email threads related to the event. In one I’ll be discussing logistics with a company sponsor. In another I’ll be coordinating with other ACM officers. In still another I’ll announce the event to groups around campus. Often even more form when people start replying about how awesome it is that there will be fruit at the event or asking how they can contribute to making the world a better place.

I thought about how I could improve email so that managing all these communications could be better. Better ways of grouping conversations, by person or by event, would be nice. Having an indicator of what still had to be done and who I still needed to contact would be nice. I realized I could do all of this with Gmail already, combining labels and searches in clever ways. But that would be clunky, and so I was pleasantly surprised when I stumbled upon Streak.

Streak seems to be targeting sales and recruiting folks, and I think that’s the wrong audience for them. Their product was clearly designed with the sales-recruiting person in mind, but I think it’s equally well suited for the hacker, and not just for tracking bugs. To demonstrate this, I’ll write a bit about what the product does and how I’m using it.

I have two use cases for Streak so far, corresponding to email uses 1 and 2 in the list above. I use Streak for organizing events and for managing my applications for summer internships. I don’t know that either of these are use cases intended by the folks over at Streak. For organizing events (primarily tech talks and Code@Nights), I have a “Pipeline” in Streak called ACM Events. Into a Pipeline go boxes, one box per event. And a box contains all the emails, documents, comments, and other information pertaining to the event. It’s simple to add an email thread to a box, so I’m always able to keep everything relevant to a particular event in one easily accessible place in the Gmail web client. As the event goes from being just an idea to a complete plan to being successfully executed, its box moves through the pipeline. In this way I can quickly get at all events in a particular part of the planning process or event lifecycle, and I have a convenient archive of all past events.

For managing my summer internship applications, Streak has also proved useful. I have a box for each company to which I’m applying. And I put all the emails and documents for that company’s application into the appropriate box. On the company’s end, I’m probably being moved through a hiring pipeline in a similar CRM tool, and now the company is moving through my CRM’s pipeline in parallel.

The box and pipeline model took me a bit of tinkering with to really grok. It’s a paradigm shift that takes some getting used to. The email primitive is no longer the conversation, but instead is the thing you are trying to accomplish through the conversation. Say I need to reply to a Facebook recruiter about planning a tech talk here at Princeton. Now rather than using Gmail search to look for a particular email thread I vaguely recall having (perhaps by guessing at words that were in the thread or by scrolling through my email to the date I remember the email having been sent), I simply go to the box I created for the event, and all of the relevant emails are right there. This sounds a lot like labels, but has the distinct advantage that boxes are much more light weight than labels. Once you have more than a dozen labels or so, they become clunky and start cluttering up the Gmail navigation bar.

Don’t just take my word for it. Give Streak a try.

Thoughts on Education

Tue, 30 Oct 2012 00:00:00 +0000

That concludes day one of the TigerTrek. I flew out of Princeton yesterday with a group of 20 to spend the week visiting technology companies, startups and venture capitalists in Silicon Valley. We’re going company to company hearing founder’s stories, discussing entrepreneurship, and as we do this I am forced to introspect. What do I want to do with my life? How can I make a difference?

My passion is education. I was fortunate enough to have some amazing mentors growing up, and to learn in a lot of diverse ways most people never have a chance to experience. I tried out gifted programs, nerd camps, one-on-one classes, online self paced learning, the works. I got involved in math circles, I took summer classes, I really was blessed to have every educational opportunity one could hope for. This, and the diversity of amazing teachers I’ve had has gotten me thinking about how people learn best, and how I learn best. Complementing this interest well, I’ve found I really love teaching- particularly math and computer science to younger students. It’s always a joy helping someone to tackle the Bessie the Cow problem or the game of Nim for the very first time.

Today a lot of young companies bring together my interests of education, computer science and technology. Websites like Coursera, edX, and Udacity virtualize the university experience, and Khan Academy joins them in making quality education accessible everywhere. Startups like Lore and Piazza foster discussions around classes, allowing the learning experience to extend more seamlessly beyond the walls of the classroom.

Today we trekked over to the new office of Khan Academy where our group got to sit down and discuss education with Sal Khan, and then we were off to Piazza’s office where we did Q+A with the question-answer site’s founder, Pooja Sankar.

Sal Khan is a really inspirational person. His collection of tutorials - which now spans a wide range of topics including mathematics and organic chemistry and art history - It all began with Sal tutoring his cousins over the phone back in 2004. Hearing him tell his founding story - how he went from tutoring one cousin, to many, to posting videos on YouTube and being well received all over the world. how he came to tutor the Gates children through his videos, and how after discussions with venture capitalists in the valley Sal made the choice to incorporate Khan Academy as a non-profit - Hearing this story from Sal Khan really drove home how empowering technology is and how tech is shaping the future of education. One smart man with a gift for explaining things clearly now teaches millions of students, an unprecedented feat since most lecture halls barely seat one hundred.

With the resources Khan Academy is producing, the purpose of the classroom is shifting. Lecture time in classes is becoming less and less valuable, and some classes have adopted a flipped classroom model. Students watch the Khan Academy videos at home, and then come to class with an understanding of the material. Students then use the class time to try out problems that would previously have been homework, now in a setting where they can get immediate help from their peers and their teacher. The flipped classroom model has been highly successful so far. One of Khan Academy’s ambitions is to, when someone comes to the site to learn about multiplication, be able to first offer an education on addition or any other prerequisites of the subject. Or after learning about one topic, they’d love to provide an education on the next logical thing to study. They’ve done a fine job of this with their knowledge map of mathematics. Each skill from 1-digit addition to calculus and linear algebra is connected with its prerequisites. Where this problem gets interesting, and where I’m excited to see what Khan Academy does in this space, is with more specialized subject matter. It will be interesting to see how this can be presented when the relationships between material stops being [understanding X is required to know Y] and starts being [X is related to Y] or [X is a good tutorial to follow for people who struggled with Y].

The data that Khan Academy is receiving about its millions of users puts the organization in a fantastic place for improving education. By the sheer number of users, there will always be groups of students studying the same material. In order to achieve this with a traditional education, students were forced to learn at the same rate as their twenty classmates. The class would drag slower students forward, and hold the brightest back. With Khan Academy students can devour mathematics at whatever rate they please, and there will still be folks online learning the same things as them. Self paced learning isn’t new. John Hopkin’s has offered it through their CTY programs for a long time, and MIT’s OpenCourseWare has provided content for free for years, but the content has never been open, free, and as community based as it can be today. Khan Academy could bring these students, all learning the same material at the same time, together, perhaps even offline. Study groups like this are popping up all over the world out of Coursera classes. People meet up to watch the lectures together or help each other through homework problems. That’s a fantastic way to learn.

There’s so much to be gained from this sort of interaction. And there’s so much to be learned from a group of students working on mathematics together, even when they’re all at different skill levels. Traditional classrooms cluster students by grade; they’re all learning the same content at the same time, and they have been their whole lives following the same sequence through mathematics as each other. With students learning at their own pace on Khan Academy, it will be great to see a classroom with a wide spectrum of mathematical ability. The sort of learning that takes place when a fast-paced student, studying algebra ahead of his class, works with a peer struggling to master long division is rare and really valuable. This is the type of interaction I expect will arise naturally out of making K-12 mathematics a self paced endeavor while maintaining the traditional 5 days a week classroom setting.

One other amazing thing that Khan Academy, Coursera, and these other online education websites will be doing is individualizing the teaching experience. I already mentioned how Khan Academy could recommend different tutorials based on your performance or ability or interests. The individualization that can be done can also take place at a finer level. Coursera co-founder Daphne Koller came to Princeton to speak recently and she explained some of the work being done in this area. The data that Coursera is receiving from students is tremendous in quality and volume. With classes of tens of thousands of people, a single numerical answer question will garner hundreds of different answers. There will be clusters in these answers, with thousands of people submitting the same incorrect response. This gives professors insights into the minds of their students they never had before. With this information, we can figure out exactly what sorts of mistakes are common, and how to avoid them. People likely to make a particular type of mistake can be shown content in a way tailored to them. There may be groups of people who tend to share patterns in which mistakes they make, or which types of material they’re good at. The scale at which these courses are being offered and at which the feedback is being received will allow for insights into teaching that were never possible before.

That said, I think that while online education is young and still in its stage of constant iteration, feedback should be collected much more explicitly. The value of people’s behavior on Khan Academy, Coursera, edX, Udacity, etc should not be understated, but these companies could learn so much more about their content and how students and “users” respond to their content it they merely asked. If Coursera put up two buttons next to their videos, one labeled “boring” and one labeled “interesting” (or “simple” and “confusing”, or “easy” and “hard”), or if they placed a feedback box underneath their video lectures, then I think the professors offering the courses would have a much better understanding of their audience. As it is, with both online lectures and live lectures, the professor gets very little feedback. They wonder Are my student’s understanding what I’m saying? Am I going to fast? Too slow? Is that kid in the orange shirt snoring?. This is one area where over the next year we have to leverage technology better. We need to get this feedback cycle right soon. These are the days that content is being produced at an incredible rate. The Khan Academy videos that go up this year, the Coursera classes being recorded every day - this is content that will last forever, and as we produce more of it, it’s so important that we maintain the highest possible standards, and for that feedback is critical.

While quality is important, so too is the opportunity for everyone to become a teacher. More on those thoughts another day. We won’t all produce lectures with the quality of Sal Khan or Kevin Wayne. It is primarily at the top that we must ensure the content is the best it can possibly be. Still, feedback from students is so important to anyone teaching in any capacity.

This notion, allowing anyone to teach, is so fundamental to the way education has to be. People learn from other people. Lore understands this. Piazza understands this. This is the philosophy that makes Albany Area Math Circle such an incredible group. This is why college campuses thrive and produce innovation. During our visit with Piazza, founder Pooja Sankar spoke about (in addition to teaching us some great lessons in entrepreneurship from the company’s founding story) Piazza’s mission to take down barriers to learning. Piazza is really succeeding in this. They’ve empowered people so that everyone in the class can be the teacher in a way that’s never worked in lecture classes before.

Going forward, only good can come of these trends toward moving education online. Khan Academy’s mission seems inevitable. A world class education is becoming available to anyone, anywhere. The best ways to make this happen are still being discovered, and there’s so much to explore in the world of online pedagogy. This is something to be excited about. This is something I want to be a part of.

Do you take online courses?

Wed, 19 Sep 2012 00:00:00 +0000

If you’ve taken any online classes, read on! Online education is growing rapidly, and I want to make the online education experience the best it can be. To that end, I would really appreciate your insight regarding online courses such as those offered at Coursera, Udacity, edX, Khan Academy, and MIT’s OpenCourseWare.

Please help me by filling out this short 4 question form regarding your online education experiences:

The questions asked are: “Which platforms have you used?” “Tell me a bit about yourself.” “How do you take notes on video lectures?” and “Tell me more!”

Pretty open ended, I recognize. And I’d love to hear what you have to say.

What's a Code@Night?

Tue, 18 Sep 2012 00:00:00 +0000

Check out this video showing what Code@Nights are all about!

Since you’re reading my blog, you should know a little bit about myself. I’m a big fan of Scrabble. I love math, puzzles, and playing tennis. I spend a lot of my would-be-free time on Coursera. And I’m ~~vice~~ chair of Princeton’s ACM Group.

ACM stands for Association for Computing Machinery, but that’s not important. I think that name is silly.

One of ACM’s main functions on campus is “promoting interest in computing and it’s applications.” That’s from the ACM Constitution, and it’s a beautiful goal, a goal helped along by Code@Nights, and a goal that I encourage you to participate in. Let’s start with Code@Nights, then I’ll point out how you can make a difference.

In short, a Code@Night is a fun-filled night of coding, powered by enthusiasm, extension cords, pizza and drinks. That’s right! Enthusiasm :D!

Personally, I find all nights of coding are fun-filled, but having a Code@Night guarantees it. Here’s how a typical Code@Night goes. Step 1, the ACM Officers (lovely people by the way) pick a date for the Code@Night and we book a room in the best building on Princeton’s campus, Sherrerd Hall. Sometime we’ll pick a theme (like quines or stenography!) as well.

Sherrerd Hall:

We send out an email (and in this case a blog post) to the ACM Group at Princeton, and everybody marks the date on their calendar, because nobody likes to miss a Code@Night.

Then, hours before the Code@Night begins, the ACM Officers (did I mention how wonderful they are?) scramble to get snacks and beverages ready for the Code@Night, and they inevitably end up with a bit too much Red Bull and Mountain Dew, and never enough fresh fruit.

The fruit is very important to the success of a Code@Night. Sure, hackers will show up because you advertise pizza and sugary things, but after a long night of coding, fresh fruit will bring a smile to any developer’s face.

Around 9pm, the hackery folks begin to arrive. They’ll keep on arriving until well after midnight, so we’re sure to keep the pizza coming and the drinks flowing. Sometimes we’ll plan a tech talk or demos to liven up the night. They come, they code, they conquer. They discuss the latest hacker news. They make the latest hacker news. They share their ideas and expertise with one another, and build pretty cool stuff. And when they’re tired, or realize they have to wake up early the next day for that big Dropbox interview, they leave at 1, 2, 3 in the morning, sometimes later.

Our next Code@Night is this Friday, September 14, and is open to the Princeton Community. Located on the 3rd Floor of Sherrerd.

Most of you probably aren’t Princeton students, but still want to attend a Code@Night. To you, I say “make one!” Find a room and shoot an email out to your computer sciencey and hacker friends. Order some pizza and be enthusiastic. And don’t forget the fruit. And when the event is a roaring success, let me know @Bieber or by email: dbieber@princeton.edu! Try it, you’ll like it.

Princeton students: I hope to see you there.

Friday Sept 14, 2012, 9pm - whenever, 3rd Floor Sherrerd Hall

Collapsible Blog Posts

Tue, 04 Sep 2012 00:00:00 +0000

This is my Hello World blog post of sorts. It's my introduction to the world of blogging, the blog post likely only read by my family and a few techie friends. It's the post in which I make an unrealistic commitment to blog frequently, which I later fail to live up to. And it's the post in which I introduce a new form of blogging, Collapsible Blogging. Here's the tl;dr.

Hello World!

Click any part of this post to see it in condensed form.
Source is here: https://gist.github.com/3616675

Click stuff to collapse. Click again to expand.

Before we get into the details of Collapsible Blogging, let's start with that commitment to blog. Every half-baked blog has it, so here's mine. Once a month. I know my life is sufficiently not boring that something blog-worthy will come along each month. Why, just last Tuesday I ran with Tyson Mao (yeah, the Rubik's Cube guy) and had a near encounter with Top Gear's Richard Hammond (he came by Twitter while I was interning there). This post counts for September. So you'll be hearing from me next at least before Halloween's end. Here's to a happy, healthy, blogging career!

I'll blog once a month. And I like name dropping.

So, how's this whole Collapsible Blogging thing work, why is it awesome, and how can you try it out? Collapsible Blogging works well with my own internet content consumption habits, which I believe I share with many of the other Hacker News obsessed entrepreneurial spirited folks out there on the internet. Possibly you. I choose an interesting Hacker News article, I read a bit, and I quickly lose interest in favor of another well titled article. My goal is really just to consume quickly the highlights of the web. Collapsible Blogging let's me skim faster. But maybe that's a not a good thing. I'm rereading Fahrenheit 451- and there's a passage this reminds me of:

“I sometimes think drivers don’t know what grass is, or flowers, because they never see them slowly,” and

“Have you seen the two-hundred-foot-long billboards in the country beyond town? Did you know that once billboards were only twenty feet long? But cars started rushing by so quickly they had to stretch the advertising out so it would last.”

I don’t want Collapsible Blogging to be those two-hundred-foot-long billboards.

Use Collapsible Blogging to make your content easier to skim.

So with that in mind, I introduce the second use of Collapsible Blogging. Diving deeper into a topic. If you'd like to learn more about diving deeper into a topic, click this passage.

Collapsible Blogging let's people who aren't interested in the topic read quickly, while people like yourselves who want the full blog post can read more in a single click, as you just discovered.

One application of this is resumes. On my resume I list that I have experience with Java, Python, etc. An employer may care just about my lovely Python experience, but my resume doesn't go into details about each technical skill and where I've used them. If I build my resume with Collapsible Blogging, I can make it so that with a single click potential employers can learn more about my experience with just the skills that interests them. They can learn what matters to them without me cluttering my resume with every project I've ever worked on.

Likewise, say I write a blog post about my experiences at Twitter and Facebook. You might be interested in reading about the technical challenges I faced and the programming concepts I learned, while another reader might be more curious about the pool party I attended at Zuck's house. If I write my post using Collapsible Blogging, you, the reader, can choose to read about the parts of my experience that interest you most, or you can choose to just skim the collapsed form of the post.

Some paragraphs don't have collapsed forms. You can only collapse the passages with a left border. There isn't really a shorter way to say that!

If you're interested in using Collapsible Blogging yourself, it's remarkable easy. I include the source as a gist below. Each passage that I want to collapse, I give an id. Then I create an element with the same id but preceded by an underscore. The CSS below hides elements with ids starting with underscores, and the JS below uses JQuery to handle the swapping in and out of elements. If you're blogging with Tumblr, just paste the code into your theme and you're good to go! Anywhere else, it should be as simple as adding the JS and CSS to your page.

Try it. You'll like it.

Inspired to try blogging by a few of my friends: Dan Kang and Harvest Zhang. Thanks guys! Keep on writing!

Discuss on hacker news.