The New York Times offered an applet, but it proved to be finicky and would only let us do the most recent day’s puzzle. A friend’s project offered hope, but only led to “Service Temporarily Unavailable”.

Enter: lmnowave!

lmnowave is a crossword puzzle gadget for Google Wave. To do a crossword puzzle with a friend, you’ll both need Google Wave Accounts.

Once you’ve got that taken care of, click this big link to get going:

lmnowave installer

You should see something like this:

Click the “Install Icon” and create a new wave. You’ll see a crossword puzzle icon in your toolbar:

Click it to add a crossword gadget. It should look like this:

If you’re using Chrome or Safari, you may get a warning about not being able to upload puzzle files. This is fine — just switch to Firefox for a minute or try one of the built-in Onion puzzles.

If you have a .puz file on your computer (perhaps from your times subscription), drag it onto the big lmnowave icon:

The puzzle will load instantly. Now drag a friend into the wave:

and you’re ready to compete or collaborate as you see fit! Each player gets his or her own color, so you can keep track of who’s filled in each square:

lmnowave is an open-source project written entirely in JavaScript. If you’d like to contribute, check it out on github. Run into a bug or have a feature request? Let me know here.

Here were the most common words over the last 12 years, along with the percentage of puzzles in which they occurred:

(you can click column headings to sort.)

So “ERA” appears, on average, in about 23 puzzles per year. How about if we break this down by day of week? Follow me past the fold…

Monday:

The four letter words are more common now. Also look how much higher the percentages are. There’s less variety in the fill of Monday puzzles. “ALOE” and “ARIA” are classic crossword words, not to mention “OREO”.

Saturday:

Lots of three letter words and much lower percentages. “OTIS” is surprising to me, but I don’t do many Saturday puzzles, so who am I to say?

It would be really interesting to combine this with some document frequency numbers for the English language. This would find words which are much more common in crosswords than they are in general, i.e. crosswordese.

I’d include everything necessary to reproduce this here, but the puzzles are not free. See this directory for the program I used to tabulate the statistics and complete word counts, both overall and for each day of the week. The first puzzle in my collection was 2006-10-23 and the last was 2009-01-19.

So we have to come up with a technique that doesn’t involve looking at every single board. And I’ve come up with just such a method! This is the “exciting news” I alluded to in the last post.

Here’s the general technique:

1. Find a very high-scoring board (maybe this way)
2. Consider a large class of boards
3. Come up with an upper bound on the highest score achieved by any board in the class.
4. If it’s lower than the score in step #1, we can eliminate all the boards in the class. If it’s not, subdivide the class and repeat step #2 with each subclass.

Classes of Boards
By “class of boards”, I mean something like this:

The squares that contain a set of letters can take on any of those letters. So this board is part of that class:

and so is this:

All told, there are 5 * 5 * 6 * 4 = 600 boards that are part of this class, each with its own score. Other fun classes of boards include “boards with only vowels” (1,953,125 members) and “boards with only consonants” (794,280,046,581 members).

Follow me past the fold for more…

Upper Bounds
Now on to step #3 of the general technique: calculating an upper bound. This is going to be easier if we introduce some mathematical notation:

b	=	A boggle board
Score(b)	=	sum of the scores of all the words contained on b
B	=	a class of boards, i.e.
Score(B)	=

An upper bound is a function such that , i.e. .

There’s one really easy upper bound: ! This just enumerates all the boards in the class B, scores each and takes the maximum score. It’s very expensive to compute for a large class of boards and hence not very practical. You and I both know that no board in containing only consonants has any points on it. We don’t need to enumerate through all 794 billion such boards to determine this.

With upper bounds, there’s a trade-off between how hard they are to compute and how “tight” they are, i.e. how closely they approximate . is very tight but is hard to compute. At the other end of the spectrum, we know that all the words on a board are in the dictionary. So we could just sum up the scores of all the words in the dictionary and get a number, say 1,000,000. Then is an upper bound. It is very easy to compute, but is not very tight.

The trick is to hit some sort of sweet spot that strikes a good balance between “tightness” and ease of computation. Over the rest of this blog post, I’ll present two upper bounds that do this. Upper bounds have the nice property that if and are two upper bounds, then is also an upper bound. So by finding two bounds, we’ll get a third that’s better than either one alone.

sum/union
The idea of this bound is to find all the words that can possibly occur in a class of boards. Since each word can only be found once, we can add the scores of all these words to get an upper bound.

To get the list of words, we use the same depth-first search strategy as we did to find words on a single board. The wrinkle is that, when we encounter a cell with multiple possible letters, we have to do a separate depth-first search for each.

At first glance, it doesn’t seem like this would be tractable for a board class like this one (alternating vowels and consonants):

In addition to the branching from going different directions on each square, there’s also a huge amount of branching from trying each letter on each square. But we’re saved by the same lesson we learned in boggle post #3: the dictionary is exceptionally effective at pruning thorny search trees. If we prune search trees like ‘bqu’ that don’t begin words, then there doesn’t wind up being that much work to do.

We can find all possible words on the above board in just under 1 second. This is about 10,000 times slower than it takes to score a conventional board, but it’s certainly tractable. The resulting score is 195,944. Given that no board scores higher than 545 points, this is a wild overestimate. But at least it’s a better bound than a million!

This technique does especially well on boards like this one, which contains all consonants:

This takes 0.23 seconds to score and results in a bound of 208 points (it contains words like ‘crypt’ and ‘gypsy’). We’ve already found a single board that has 545 points on it. So we can eliminate this entire class of 794 billion boards. That’s a speed of over 3 trillion boards/second! Of course, this board class is not typical.

It’s also worth pointing out why this upper bound isn’t tight. Consider this class of boards:

You can find both “fir” and “far” on boards in this class, but there aren’t any boards that contain both. So while each “fir and “far” contribute a point to the upper bound, they should only really contribute a single point. The sum/union bound doesn’t take into account the relationships between various letter choices. It’s the best trade-off between computability and “tightness” we’ve seen so far, but it’s not good enough to make the problem tractable.

max/no mark
In the sum/union upper bound, we dealt with multiple possible letters on the same square by trying each and adding the resulting scores (taking care not to count any word twice). But why take the sum of all choices when we know that any given board can only take on one of the possibilities? It would result in a much better bound if we took the max of the scores resulting from each possible choice, rather than the sum. This is the idea behind the “max/no mark” bound.

This is a huge win over sum/union, especially when there are many squares containing many possible letters. It does have one major drawback, though. The sum/union bound took advantage of the fact that each word could only be found once. With the max/no mark bound, the bookkeeping for this becomes completely intractable. The words we find by making a choice on one square may affect the results of a choice somewhere else. We can’t make the choices independently. The optimal set of choices becomes an optimization problem in its own right.

Rather than deal with this, max/no mark just throws up its hands. This is what the “no mark” refers to. In the past, we’ve recorded the words we find by marking the Trie. By not marking the Trie with found words, we accept that we’ll double-count words sometimes. But it still winds up being an excellent upper bound.

Lets try some of our previous examples:

Alternating vowels and consonants:

A class that can be entirely eliminated:

max/no mark isn’t always better than sum/union:

This is something of a worst-case because, while there are relatively few distinct words, there are many different ways to find them.

Putting it all together
Our two bounds do well in different situations. max/no mark works best when there are lots of choices to be made on particular cells and there are relatively few ways to make any particular word. sum/union works best when there are lots of possibilities but relatively few distinct words. Putting them together results in a bound that’s good enough to find the best 3×3 boggle board using the technique described at the beginning of this post.

Given an initial class of boards, we wind up with what I call a “breaking tree”. If the initial class has an upper bound less than 545 points, then we’re done. Otherwise, we pick a cell to split and try each possibility.

Here’s a relatively small breaking tree that results from running this program:

$ ./3x3/ibucket_breaker --best_score 520 --break_class "bdfgjkmpvwxz a sy iou xyz aeiou sy chlnrt chlnrt"
(     0%) (0;1/1) bdfgjkmpvwxz a sy iou xyz aeiou sy chlnrt chlnrt (820, 77760 reps)
                            split cell 4 (xyz) Will evaluate 3 more boards...
(     0%)  (1;1/3) bdfgjkmpvwxz a sy iou x aeiou sy chlnrt chlnrt (475, 25920 reps)
(33.333%)  (1;2/3) bdfgjkmpvwxz a sy iou y aeiou sy chlnrt chlnrt (703, 25920 reps)
                            split cell 5 (aeiou) Will evaluate 5 more boards...
(33.333%)   (2;1/5) bdfgjkmpvwxz a sy iou y a sy chlnrt chlnrt (447, 5184 reps)
(    40%)   (2;2/5) bdfgjkmpvwxz a sy iou y e sy chlnrt chlnrt (524, 5184 reps)
                            split cell (iou) 3 Will evaluate 3 more boards...
(    40%)    (3;1/3) bdfgjkmpvwxz a sy i y e sy chlnrt chlnrt (346, 1728 reps)
(42.222%)    (3;2/3) bdfgjkmpvwxz a sy o y e sy chlnrt chlnrt (431, 1728 reps)
(44.444%)    (3;3/3) bdfgjkmpvwxz a sy u y e sy chlnrt chlnrt (339, 1728 reps)
(46.667%)   (2;3/5) bdfgjkmpvwxz a sy iou y i sy chlnrt chlnrt (378, 5184 reps)
(53.333%)   (2;4/5) bdfgjkmpvwxz a sy iou y o sy chlnrt chlnrt (423, 5184 reps)
(    60%)   (2;5/5) bdfgjkmpvwxz a sy iou y u sy chlnrt chlnrt (318, 5184 reps)
(66.667%)  (1;3/3) bdfgjkmpvwxz a sy iou z aeiou sy chlnrt chlnrt (509, 25920 reps)

The numbers in parentheses are the upper bounds. When they get below 520 (the parameter I set on the command line), a sub-class is fully broken.

Using this technique and the following partition of the 26 letters:

• bdfgjvwxz
• aeiou
• lnrsy
• chkmpt

I was able to go through all 262,144 (=4^9) board classes in about six hours on a single machine. This resulted in the boards I listed in the last post. Six hours is a big improvement over two years!

If that same factor (two years to six hours) held for the 4×4 case, then we’d be down to 380 years of compute time to find the best 4×4 boggle board. Or, equivalently, 138 days on 1000 machines. That’s still a lot. We’re not quite there yet, but we’re getting closer!

Code for the program that went through all possible board classes can be found here. While many people have found high-scoring boards, I haven’t found any previous work on this upper bounding approach. So if you have any ideas/suggestions on how to improve the bound, they’re probably novel and useful!

The ovals are data types and the rectangles are functions. The three basic types are:

• time_t: number of seconds since the start of the UNIX epoch. This is always UTC!
• struct tm: A broken-down date, split into years, months, seconds, etc. In Python, it’s a tuple.
• string: Any string representation of a time, e.g. “Wed Jun 30 21:49:08 1993″.

Generally you either want a time_t (because it’s easy to do arithmetic with) or a string (because it’s pretty to look at). So to get from a time_t to a string, you should use something like strftime("%Y-%m-%d", localtime(time())). To go the other way, you’d use mktime(strptime(str, "%Y-%m-%d")).

This library has been around since at least 1982. It’s been replicated in many other languages (Python, Perl, Ruby). We seem to be stuck with it.

Read on for my rant about why this is all idiotic.

Let me just say that I think this is a horrible system. You almost never want to use struct tm. Most of the time, you want to go between strings and time_t. But lonely ctime is the only function that makes this jump, and it doesn’t let you set the output format or time zone.

The names are not exactly descriptive, either. They all end in “time”, which makes some sense. strptime and strftime are even OK, if a bit cryptic. The p stands for “parse” and the f stands for “format”, ala printf. The parameter order is hard to remember, though. Don’t use gmtime unless you have a good reason. ctime and asctime are non-sensical, but I don’t use them much, either. My greatest loathing is reserved for localtime and mktime. I can never remember which of these does which. Only mnemonic I can think of: mktime makes a time_t from a struct tm.

For another exercise in head-scratching, follow the role of time zones through this chart. time_t knows no time zones — it’s always UTC. To get to struct tm, you need to specify a time zone. This is not made explicit in the struct, however, so you need to do your own bookkeeping. The time zone for conversion isn’t a parameter or anything sensible like that, either. You just get two choices: GM (UTC) time or local time. And if you choose gmtime, you’ll never be able to get back to time_t because that function doesn’t exist. (Some systems supply a mkgmtime or timegm function.)

How would I design it? struct tm would lose its place at the center of everything. There would be sensibly-named functions to go between time_t and string:

• time_t parsetime(format, string[, timezone])
• string formattime(format, time_t[, timezone])

And if you really need them:

• splittime(time_t, struct tm*)
• time_t packtime(struct tm*)

Was that really so hard?

Drag the column headers to rearrange the table. dragtable is incredibly easy to use. To make a table rearrangeable, just add class=draggable to the table tag. And, if you set class="draggable sortable", you can have a table that’s simultaneously sortable and rearrangable! For more details and a download link, check out the dragtable page.

I’m calling this v0.9 since I’m sure there are plenty of bugs and tweaks left to make. I’d love to get some feedback, so take it for a spin and tell me what you think!

Update: I’ve added full-column dragging and bumped the version to 1.0. Head on over to the dragtable, grab a copy, and let me know what you think!

Not so, GW-BASIC. To save space, it stored code in a compact, binary format. This seems like an unnecessary optimization now, but back in 1984 it made a lot of sense. GW-BASIC was an interactive environment, and it stored all your code in memory. Memory was a scarce resource at the time, so every byte counted. Hence the binary format.

I wanted to read my old GW-BASIC programs, so I dug around and found this discussion of the GW-BASIC binary file format. It’s incredibly detailed, which let me whip up a decoder in Python over two solid hours of hacking. Without further ado, here it is:

GW-Basic Program Decoder

For a sample decoding, see below the fold.

Here’s a 20-questions style program I wrote on August 15, 1995:

   10 CLS
   20 DIM G$(15),RP(15),RD(15)
   30 PRINT "In this game I will make up a three digit code. The code will be between 100 and 999. You have 15 guesses. After each guess, I will tell you how many digits are correct and how many are in the right position. If you enter an invalid guess, ";
   35 PRINT "or a duplicate answer, I will let you try again. "
   40 PRINT
   50 PRINT "Press any key when ready. . . "
   60 GOSUB 1000
   70 CLS
   80 GOSUB 2000
   90 IF CODE < 100 OR CODE >999 THEN GOTO 80
  100 CODE$=RIGHT$(STR$(CODE),3)
  110 D$=CODE$
  120 FLAG=0
  130 GOSUB 1500
  140 IF FLAG=1 THEN GOTO 80
  150 FOR I=1 TO 15
  160     CLS
  170     IF I=1 THEN GOTO 230
  180     FOR J=1 TO I-1
  190             PRINT "Guess #";J;": ";G$(J);". results: ";
  200             PRINT "digits: ";RD(J);"positions: ";RP(J)
  210     NEXT J
  220     REM
  230     PRINT
  240     PRINT "enter guess #";I;
  250     INPUT G
  260     IF G<100 OR G>999 THEN 240
  270     G$(I)=STR$(G)
  280     IF LEN(G$(I))=0 THEN GOTO 240
  290     G$(I)=RIGHT$(G$(I),3)
  300     D$=G$(I)
  310     FLAG = 0
  320     GOSUB 1500
  330     IF FLAG=1 THEN GOTO 240
  340     FLAG=0
  350     GOSUB 1400
  360     IF FLAG=1 THEN 240
  370     GOSUB 1100
  380     IF RP(I)<>3 THEN GOTO 410
  390     PRINT "Good job!! You guessed the code in";I;"try's."
  400     END
  410 NEXT I
  420 PRINT "Your guesses are up!"
  430 PRINT "The code was";CODE$
  440 END
 1000 R$=INPUT$(1)
 1010 RETURN
 1100 FOR K=1 TO 3
 1110    FOR L=1 TO 3
 1120            G$=MID$(G$(I),K,1)
 1130            C$=MID$(CODE$,L,1)
 1140            IF G$<>C$ THEN GOTO 1170
 1150            IF K<>L THEN RD(I)=RD(I)+1
 1160            IF K=L THEN RP(I)=RP(I)+1
 1170    NEXT L
 1180 NEXT K
 1190 RETURN
 1400 IF I=1 THEN RETURN
 1410 FOR K=1 TO I-1
 1420    IF G$(I)=G$(K) THEN FLAG=1 : RETURN
 1430 NEXT K
 1460 RETURN
 1500 REM
 1510 FOR K=1 TO 3
 1520    FOR L=K+1 TO 3
 1530            IF MID$(D$,K,1)=MID$(D$,L,1) THEN FLAG=1: RETURN
 1540    NEXT L
 1550 NEXT K
 1560 RETURN
 2000 OPEN "r", 1, "b:amunt.ran", 2: FIELD 1, 2 AS T$
 2010 IF LOF(1)=0 THEN CODE = INT(100+RND*999): T$=MKI$(0): GOTO 2050
 2020 GET 1,1
 2030 FOR I=1 TO CVI(T$): CODE=RND:NEXT
 2040 CODE = INT(100+RND*999)
 2050 LSET T$=MKI$(CVI(T$)+1): PUT 1,1
 2060 CLOSE #1: RETURN

Oh, for the days of unstructured programming and rampant use of the “GOTO” statement. The convention was to number your lines 10, 20, 30, … so that you could go back and add extra lines between your originals. Hence line 35 above. I must have been really ambitious jumping to line 1000!

In my case, this list led to a good use of my Track Parser script, which is in all likelihood the most useful program I’ve ever written. It’s an AppleScript for iTunes (i.e. Mac only, sorry) that lets you apply regular expressions to track names/tags. Here’s how I used it today…

Through some strange turn of events (certainly nothing to do with this), I found myself with a playlist of the top 100 tracks. The music was all there, but none of the songs had their “Artist” field filled in! Here’s where my Track Parser script came in.

I googled around and quickly found this page, which has some commentary on the list, as well as what we’re interested in: a copy of all the songs/artists in simple text form. (For what it’s worth, I agree with his reactions.)

I copied the list and ran two regular expressions to get it down to just the artist (s/ ".*//g and ^\d*: if you must know). The tracks are in reverse order of what we want (100 to 1 instead of 1 to 100). So I ran pbpaste | tac | pbcopy to put the #1 track at the top of the list. Or I would have, if Mac OS X had the tac command. Instead, I ran this monstrosity:

pbpaste | perl -ne 'push @x, $_; END { print for reverse @x }' | pbcopy

to do the same thing. In retrospect, I should have just sorted my playlist in reverse track order.

Next I went into iTunes and selected my songs. I ran “Track Parser (Clipboard)” from the Scripts menu, clicked “New Pattern” and put in “%a” to extract the artist from each line. Track Parser handled the rest. Total time: about five minutes.

If you said “segfault”, give yourself a pat on the back! Bonus points if you know that changing the “20″ to a “16″ will prevent the segfault.

After spending several hours staring at this, I figured out what was going on. Rather than taking a compare function that returns an int (like qsort or Perl’s sort), it wants a “LessThan” function:

If I’ve ever been happy that C++ silently casts ints to bools, I have now done my penance. I’m still somewhat surprised that std::sort segfaults when given a strange comparison function, rather than returning an unsorted list.

This is one of the few games I’ve ever seen with only one control: jump. Your character bounces off the walls and you have to power him up for jumps. My record is 282 floors on Hard. Can you beat it? No fair using the JavaScript version, though. More details below (warning: it takes a hard right turn for the nerdy)…

The main problem with my JS-Tower is that the levels don’t get harder as you climb. To come up with a method for placing platforms, I disassembled the bytecode of the Java version of NS-Tower and discovered this method:

It’s basically a pattern of four repeating platforms, plus some randomness. My JavaScript version would be more fun if I could discover the function used for platform generation by the Mac/PC versions. Here are some of the options, in order of increasing difficulty:

1. Contact Nagi-P software and ask for source. This was painless, but seeing as their site hasn’t been updated in years, I doubt I’ll get a response. I’ve tried tracking Akihiko Kusanagi down, but have had limited success.
2. Disassemble the Windows NS-Tower. This is orders of magnitude more difficult than disassembling Java bytecode, but it may be just the excuse I need to try out IDA Pro. Presumably platform generation uses just a few opcodes, it’s just finding them that will be the trouble.
3. Do a statistical analysis of NS-Tower platforms. I’ve gone surprisingly far down this road, but I’m afraid it only will only lead to madness. The same four-platform sequence is present, but it starts to break down around floor 100. Here’s a sample run.

Any other ideas?

More generally, Java left off a mighty stink back in 2004. Any GUI that I ran on the Mac would look out of place and felt clunky. Performance was poor. But in retrospect, I suspect much of the rank Java smell was really coming from the design patterns gibberish I was being force-fed at the same time. Why use a simple array when you could use an AbstractListFactory that does the same thing with 10x code bloat?

Regular readers only get one guess what program I wrote to get in the swing of things.

The C++ code translated almost line-for-line to Java. Most of the changes were either syntax (“->” to “.”) or swapping String for char*. It took a while to remember that everything is a pointer in Java. The Java statement Trie t = new Trie(); is the equivalent of C++’s Trie* t = new Trie; and not plain old Trie t;, which uses the stack. Another major pain was the lack of unsigned types, which I used as hash codes in some tests.

Here’s the Java code, for those interested. I expected Java code to be an order of magnitude slower than the equivalent C++, so this initial benchmark surprised me:

C++:  Evaluated 21632 boards in 0.694 seconds = 31,162 bds/sec
Java: Evaluated 21632 boards in 1.108 seconds = 19,523 bds/sec

That’s only a 37% performance penalty, far less than the 90% or more I was expecting. Since my code doesn’t create/destroy any objects during the critical section, I’m really testing the effectiveness of the JIT. Since this code is a direct C conversion, it makes sense that it does well.

I don’t understand the internals of Java well enough to know how valid my benchmark is. Profiling C++ code is relatively straightforward, since nothing is going on behind your back. Not the case with Java. For example, does that 1.108 seconds include the time it took the JIT to compile my code? That might explain the whole perf difference. Also, am I fully optimizing? I hear there’s a difference between the client and server JITs. Which one am I using? How can I tell? Lazyweb?

A few more thoughts on the whole experience:

• Eclipse is just great, especially for someone whose knowledge of the JDK is rusty. Having “string.” bring up a list of methods was great. This was less useful for more obscure Java-isms, like System.getProperty("user.dir") to get the current working directory.
• When Eclipse spots an error in your code, it suggests a solution, which you can double-click to perform. This was just perfect for a rusty coder. It added lines like import java.io.File; to my code, which would have taken me a while to figure out on my own. That’s always a nuisance in C++.
• I can’t figure out what command line Eclipse is using to run my code. I have no idea how it’s running my unit tests. Is there any way of finding this out?
• jUnit was very easy to use inside Eclipse. I particularly liked the ability to write unit tests inside the class they were testing. Just tag a method with @Test and it becomes a unit test. Very cool.
• I found the whole StringBuilder business pretty clunky. Writing "a" + "b" is special kludge for new StringBuilder().Append("a").Append("b").toString(). I also missed printf when writing to stdout. Is there no natural way to mix numbers and strings?
• Java GUIs on the Mac still aren’t quite there. When I moused over the icons in Eclipse, there was noticeable flicker.

A tool like Eclipse is a great boon in understanding a large, foreign code base. One thing I’ve discovered in my past year at Google is that, while I’ve become a fairly good coder through experience, I have almost no external experience working with other people’s code. Make no mistake, navigating other people’s code is a skill. And to a surprising extent, it’s a skill disjoint from coding itself. We have a few tools to help with this at Google, but nothing quite so low-latency as what Eclipse pops up when you press type a period. The possibility of tools like Eclipse is a strong argument for languages with a simple syntax. When anyone can parse a language, they can build amazing tools like this one.