This is one of my favorite javascript tricks, because of its effort-to-payoff ratio.

Problem: the default Array.prototype.toString hides any nested structure.

[1, 2, 3, 4, 5].toString(); //-> "1, 2, 3, 4, 5"
[1, 2, [3, 4], 5].toString(); //-> "1, 2, 3, 4, 5"

Solution: override Array.prototype.toString.

Array.prototype.toString = function() {
    return '[' + this.join(', ') + ']';
};

[1, 2, 3, 4, 5].toString(); //-> "[1, 2, 3, 4, 5]"
[1, 2, [3, 4], 5].toString(); //-> "[1, 2, [3, 4], 5]"

I’ve been running Firebug at work for a long time, it’s a really solid tool. A while ago, I started using a Monaco.ttf for Windows. I think it looks much better on my Ubuntu system, but it’s nice to have on Vista.

Firebug was apparently written for the Mac, because it uses Monaco all over the place. That would be very nice, but on Windows, a bold Monaco is a wider Monaco.  This makes the line number gutters uneven, and makes the screen flash when you scroll through text:

I finally got irritated enough to google it, and I found this post by Thomas Scholz (it’s in German, here’s Google’s translation).

The details: Firebug’s CSS files are located in its extension directory, which is probably someplace like this:

• Vista: C:\Users\{USERNAME}\AppData\Roaming\Mozilla\Firefox\Profiles\{alphanumerics}.default\extensions\firebug@software.joehewitt.com\skin\classic
• XP: C:\Documents and Settings\{USERNAME}\Application Data\Mozilla Firefox\Profiles\extensions\firebug@software.joehewitt.com\skin\classic

Search-and-replace the Monaco away, and restart Firefox.

As good as Processing is, it’s still java. I’ve slowly been learning clojure, and trying to use it with Processing, via clj-processing. While the clojure docs and the book are both good, I’m still in that flounder-around stage, trying out tiny experiments to understand how things work.

I wanted a function to make saw-step sequences: (1 2 3 2 1 2 3 2 …). You give it the low bound, the high bound, and the step-size. For example:

• (saw-step 1 4 1) gives (1 2 3 4 3 2 1 2 3 4 3 ...)
• (saw-step 0 15 5) gives (0 5 10 15 10 5 0 5 ...)
• (saw-step 0 5 2) gives (0 2 4 5 3 1 0 2 4 5 ...)

I’ve coded this kind of thing up in ruby, javascript, or java a dozen times, though it’s usually inside a loop, not returning a list. Something like this:

var min = 3;
var max = 7;
var step = 2;

var n = min;
while(shouldContinue()) {
    doSomethingWith(n);
    n += step;
    if (n+step > max || n-step < min) {
        step *= -1;  // turn around
    }
}

My first try in clojure took a similar tack. I never got it right, but it was something like this:

(defn saw-step
  [min max step-size]
    (let [step (ref step-size)]
      (iterate
       (fn [x]
         (if (or ( min (- x @step)))
           (dosync (alter step -)))
         (+ @step x))
         min)))

It’s a disaster — the parentheses don’t even match — but you get the gist. I was trying to cram the same imperative algorithm into clojure: keep some mutable state, add step to it, and when you reach the edges, mutate step to change direction. I kept getting bugs where it would go one step beyond the edge, or it would start working its way back from one edge, only to turn around again, and bounce against the edge forever.

I gave up and went to bed, but a few days later, I had better luck.

(defn saw-step
  [min max step]
     (cycle
      (into (vec (range min max step)) ; vec, so (into) adds to end
	    (for [x
		  (iterate #(- % step) max)
		  :while (> x min)] x))))

The first not-entirely-wrong step I made was to try breaking the list of numbers I wanted into two parts: the going-up numbers, and the coming-down numbers. (0 2 4 5 3 1) is just (0 2 4) + (5 3 1). The (range min max step) part gives you the going-ups, and the (for [x (iterate ...) stuff is the going-downs, as a list comprehension.

(One mistake I made was trying (range max min step) for the going-downs, which yields an empty list; another was using (iterate dec max), which never ends, because it keeps decrementing into the negatives. I found my way out with the list comprehension, but I bet there’s a better way.)

Once you have those two lists, you can use into to add each item from the second list to the first, giving you (0 2 4 5 3 1). That goes into cycle for a lazy, infinite sequence.

The solution’s not too bad: a saw-step is a cycle of the going-ups, followed by the going-downs. The code looks about that way.

(It occurred to me after that I could always use a default step of 1, and pipe the result through a scaling map. That would give me the bounds I wanted, with the extra benefit of evenly-spaced steps. Maybe I’ll remove step later.)

Alan Perlis said, “A language that doesn’t affect the way you think about programming, is not worth knowing.” He also said, “The only difference(!) between Shakespeare and you was the size of his idiom list – not the size of his vocabulary.”  Clojure’s changing my mind, a bit at a time.

What follows is an account from the author’s experience. Some details have been changed for the usual reasons, and a poor memory has fuzzed out the rest.

UPDATE: it turns out the technique described below is known as debouncing. What an awesome name!

My last job was for a company that made case-management software. With this software, you could track all kinds of data about your case: you could categorize it, sub-categorize it, say where and when it happened, note whether it was still open, track who was responsible…all kinds of stuff, great for data fiends.

One of our customers’ favorite features was the reports, and they were pretty spiffin’. But, not being ones to rest on our laurels, we were adding pivot-table reports, so you could count the number of cases, grouped by any criteria you wanted. The input screen let the customer pick their pivot criterion, and which criteria they wanted as columns, for sub-counts. We struggled to name these UI fields — something lucid, something explanatory, something evocative — something to make them understand what kind of report they were in for. After a while, we decided to just show them: when they changed their pivot criterion, we’d run some javascript to render a skeleton report, same as the real report, but minus the data. It would save them time, and save our servers.

The javascript was pretty involved. It had to generate the same HTML as we did for the pivot table, which meant it had to know (or make up) values for each criterion, like all the case categories and sub-categories, and which sub-categories belonged to which categories. And we let the customers pivot on up to THREE, count ‘em, different criteria. And it had to happen each time the user picked different pivot criteria. It took a few tricks, but we got it working. It ran slowly, maybe a few seconds, but it was quick enough, probably, especially once we threw in a little “please wait” spinner. Then we realized we needed to re-render whenever the window resized.

No biggie, I thought, and I quickly added window.onResize(reportPreview);. It worked great, except it re-rendered the report with every pixel-wide movement of the mouse, as the window was dragged to new widths and heights. Calling a function, one that runs for a few seconds, a hundred times in the time it took to widen the browser an inch, meant a locked browser. It meant “time to get more coffee,” and after, “time to fix the bug.”

I knew we could delay calling reportPreview, but we only wanted to delay it when the window was being resized — when the user changed the columns, there was no reason to wait. I was sure window.setTimeout() would do what we needed, but I didn’t want to muck up reportPreview() with it.

I’d been reading The Little Schemer lately, and noticing some striking similarities between javascript and scheme: first-class functions, and higher-order functions that take functions as arguments, and return other functions as values. It was fun reading, with its strange teacher-student dialog style. The material was better than brainteasers, and I knew it would make me a better programmer down the road, but I didn’t think of it as relevant to the day-job, as…applicable.

Then I realized higher-order functions were a way out of this. I could write a function, delay, that would wrap one long-running, slow-poke function in another function: an intermediary that you could call as many times a second as you wanted, but it would only call the slow-poke once things had settled down. delay would let us keep setTimeout out of reportPreview. Something like this:

function delay(millis, slowPoke) {
    var timeoutId = undefined;

    // This is the intermediary.  Call it lots, it won't hurt.
    return function() {
        if (timeoutId) {  // If we're waiting...
            clearTimeout(timeoutId); // re-start the clock.
        }
        timeoutId = window.setTimeout(slowPoke, millis);
    }
}

The first time you call the intermediary, it tells the window to call slowPoke after a bit, but every time you call it after that, it starts the clock over. It’s like when you’re in a five-minute time-out, and you start acting up after only three, so your mom says “Ok, buster, another five minutes.”

var fiveMinutes = 5 * 60 * 1000;
var screamAndShout = delay(fiveMinutes, function() {
    getBackToPlaying();
});

screamAndShout(); // Aw nuts, I'm in time-out.

// I'll be good for as long as I can, but...
screamAndShout(); // dang, five MORE minutes!

Once delay was in place, running reportPreview when the window was resized was no problem.

function reportPreview() {
    // recursion, DOM manipulation, insanity...
}
columnPicker.onChange(reportPreview);
window.onResize(delay(100, reportPreview));

After testing, we found that delaying it for 100 milliseconds made all the difference in the world.

Do you have a war story? Share it, and start the healing: tell a short one in the comments, or link to a longer one on your own damn blog.

I was looking for a good way to illustrate functional composition and higher order functions, and thought that something could be done with Processing, a java-based graphics tool. Among other things, Processing exposes raw pixel data for the images it renders, and you can update the pixels programatically, providing a simple kind of image filtering.

For example, here’s some code that converts a color image to grayscale. Just like with HTML, colors are represented¹ as 3 channels (red, green, blue), and each channel has 255 increments. A gray pixel has equal amounts of red, green, and blue, so if you get the overall brightness of a pixel, and set its color to that much red, green, and blue, it should turn the image gray.

PImage img = loadImage("tattoo.jpg");
size(img.width, img.height);  // set the window size
image(img, 0, 0);  // render the image at x:y = 0:0

loadPixels(); // load the image's pixels into an array
for (int i = 0; i < pixels.length; i++) {
    // get the color for this pixel
    color c = pixels[i];

    // get its brightness
    float bright = brightness(c);

    // Change its color to its grayscale equivalent
    pixels[i] = color(bright, bright, bright);
}
updatePixels();  // render the new pixels to the screen

Here’s the original image:

…and here’s the filtered version:

You can define a bunch of routines like this, each looping through the pixels the same way, but adjusting the colors differently. But if you can separate the pixel-changing part from the pixel-looping part, then you can swap in ANY pixel-changing routine, giving you a flexible image filtering system.

The pixel-changing code is essentially a function that turns one pixel’s color into a new color, and the pixel-looping code uses it to replace each pixel’s original color. The pixel-changing function could be described like this:

color transform(color c) {
    ...
}

Many modern programming languages support first-class functions, which are a natural way to model this. Processing uses a form of Java, which doesn’t have first-class functions, but we can wrap the functions in a class, giving us an object called a functor. I called this one ColorTrans, for “color transformer”.

abstract class ColorTrans {
    public abstract color transform(color c);
}

The ColorTrans class’ only method, transform, is our pixel-changing function. We can re-write the loop from before to use a ColorTrans filter, and while we’re at it, we’ll move it into a method that takes the filename as a parameter, too.

void filterImage(String path, ColorTrans filter) {
    PImage img = loadImage(path);
    size(img.width, img.height);
    image(img, 0, 0);

    loadPixels();
    for (int i = 0; i < pixels.length; i++) {
        // use the filter parameter
        pixels[i] = filter.transform(pixels[i]);
    }
    updatePixels();
}

We can easily recreate the grayscale filter from earlier as a ColorTrans.

ColorTrans grayscale = new ColorTrans() {
    color transform(color c) {
        float bright = brightness(c);
        return color(bright, bright, bright);
    }
};

filterImage("tattoo.jpg", grayscale);

Another really easy filter to write is an inverter. If a pixel has R:100, G:30, B:255, an inverter will return the color R:(255-100), G:(255-30), B:(255-255), or R:155, G:225, B:0.

ColorTrans invert = new ColorTrans() {
    color transform(color c) {
        return color(255 - red(c),
                     255 - green(c),
                     255 - blue(c));
    }
};

filterImage("tattoo.jpg", invert);

The image produced by an inverter is like a film negative.

Now we begin to see the benefits of keeping the-parts-that-change separate from the-parts-that-don’t: we can easily define and swap in new filters. This is one of the big parts of higher-order programming: writing routines that are configured by passing in functions (or functors, in this case) to do part of the work.

Manufacturing a ColorTrans

A useful kind of filter is one that increases (or decreases) the color on one channel: add some red, or remove some green. This kind of filter is easy to create:

ColorTrans aFifthMoreRed = new ColorTrans() {
    color transform(color c) {
    	return color(red(c) * 1.2, green(c), blue(c));
    }
};

This filter will increase the amout of red in the image by 20%. But 20% is a pretty arbitrary number; it’d be better if we could tell the filter how much to increase the red by. Generally, you’d add an “amount” parameter to the transform method, but then filterImage would have to know that this ColorTrans object takes an extra parameter. It’s kind of nice having filterImage treat all ColorTrans objects the same, just passing in the color.

Instead, we can make a method that builds ColorTrans objects: we tell it how much to increase the red by, and it builds a ColorTrans that does it for us.

ColorTrans ampRed(final float amount) {
    return new ColorTrans() {
        color transform(color c) {
            color(red(c) * amount, green(c), blue(c));
        }
    };
}

ColorTrans aQuarterMoreRed = ampRed(1.25);
ColorTrans aThirdLessRed = ampRed(2/3);
ColorTrans noRedAtAll = ampRed(0);

(If you’re curious why amount is final, the short answer is “because the compiler says so,” but there’s a better answer².)

This is pretty nice, because we can use this inside an animation loop, animating the amount of color amplification.

float theta = 0.0;

void setup() {
    filterImage("tattoo.jpg", noChange);
}

void draw() {
    float ampRedAmount = sin(theta) * 1.2;
    filterImage("tattoo.jpg", ampRed(ampRedAmount));
    theta += 0.1;
}

[Here's where I'd link to the applet, nicely hosted somewhere, if I had a hosting service that allowed applets. I'll try to find a place to put all the code, so you can try it yourself.]

Processing calls setup to initialize the animation, and calls draw once per “tick”, to advance the animation. Here, in each frame of the animation, a new ColorTrans is constructed by ampRed, with the amount tied to the sine wave function, oscillating between 0 and 1.2. When viewed, the amount of red in the image swells and falls, and back again³.

This is another big part of higher-order programming: writing functions that build other functions, based on some arguments. Combined with routines that take functions as arguments, it’s a handy way to break down some problems. If done well, the routines that take functions as arguments, and the functions that build those functions, can become a sort of mini-language, a fluent interface, or almost an embedded DSL.

Plugging filters together – filter composition

This is where it gets fun. Suppose you’ve created an ampBlue that works just like ampRed, and now you want to filter an image with both of them. One approach might be something like this:

void draw() {
    filterImage("tattoo.jpg", ampRed(sin(theta) * 1.2));
    filterImage("tattoo.jpg", ampBlue(cos(theta) * 1.2));
}

Using the sine and cosine functions, the image should pulse nicely through reds and blues. The only problem is that it doesn’t really work, because filterImage loads the image fresh from the filesystem each time, so you only get the effect of the ampBlue filter. So how can we apply multiple filters?

We plug them together. We want a filter that does the work of two other filters, and we want it to look like any other filter, so filterImage won’t know the difference. To get this, we can add a method to ColorTrans that returns a new ColorTrans, which calls first the one, and then the other.

class ColorTrans {
    ...
    public ColorTrans then(final ColorTrans applySecond) {
        final ColorTrans applyFirst = this;
        return new ColorTrans() {
	    color transform(color c) {
	        return applySecond(applyFirst(c));
	    }
	};
    }
}

filterImage("tattoo.jpg", grayscale.then(invert));

Combining filters becomes a matter of chaining them together through then. The red-and-blue example becomes:

void draw() {
   filterImage("tattoo.jpg",
        ampRed(sin(theta) * 1.2).then(
            ampBlue(cos(theta) * 1.2)));
}

Processing does it kind of this way, too

If you look at the source for Processing, in PImage.java on line 619, you’ll find code that looks kind of like this:

  public void filter(int kind) {
    loadPixels();

    switch (kind) {
      case BLUR: ...
      case GRAY: ...
      case INVERT: ...
      case POSTERIZE: ...
      case RGB: ...
      case THRESHOLD: ...
      case ERODE: ...
      case DILATE: ...
    }
    updatePixels();  // mark as modified
  }

It basically does just what I’ve been doing, except the operations are hard-coded into the source, rather than being separated behind a class interface. The filters aren’t composable directly, though you can call a number of them in a row:

filter(INVERT);
filter(THRESHOLD);
filter(DILATE);

One benefit of this approach is that it’s easy to see, line-by-line, exactly what’s happening. I’d bet it beats the pants off of the ColorTrans version in benchmarks, too. But filters aren’t composeable, and it’s certainly not extendable. When you’re doing computation-intensive graphics, every bit of speed is important; when you’re illustrating a programming idea, it’s not. Decide for yourself which is more important for your needs.

____________________________________

1. It may seem weird, if you know Java, that the parameter’s type is color — it’s not a java primitive, but it doesn’t follow the normal classname conventions. It’s just the way Processing does it. You can read more about the color constructor in the Processing reference. [back]

2. Taken from the AnonymousInnerClasses page on the Portland Patterns Repository, 2009-04-30:

AnonymousInnerClasses can also be used to create something like closures in the JavaLanguage. However they do not “close over” the lexical environment so they aren’t TrueClosures. (They can capture the value of local variables, but they do not have access to the variable binding so they can’t change the original variable. Which Java makes quite clear by only allowing InnerClasses to refer to local variables that have been declared final (so no-one can change them)).

[back]

3. The noChange filter returns the same color it was given — an identity function. filterImage is called inside setup only so the window size is set correctly, since setting the size inside draw has no effect. And theta is just a Greek letter often used for angles and trigonometry.[back]

I was planning on reading Andy Hunt’s Pragmatic Thinking and Learning: Refactor Your Wetware for a while, but kept putting it off, thinking it wasn’t really central to what I’m interested in, what I’m doing.  I was wrong.

People have different skill levels, from novice to expert.  One sign of expertise is the ability to intuit solutions not available through linear thinking.  Intuition happens on the right side of the brain, so we need to use the right side more effectively.  Rather than “left brain” and “right brain”, he calls them “L-mode” and “R-mode”, for “linear” and “rich”, to emphasize that it’s not literally two halves of your brain working differently, but two different modes of thinking.  Most of the book is about using R-mode more, and using L-mode appropriately, to fact-check your R-mode.

It draws on psychology, neurobiology, cognition studies, managing, teaching, the arts, product design, and math.  The bibliography is ten pages long, and I’ve got a bunch of them highlighted for my next evening at the bookstore.

The most valuable changes I’ve made since finishing the book are:

- Be ready when your R-mode strikes. Since “querying” the R-mode can take a long time, we run it asynchronously, and we never know when the results will come in.  (This explains the “ah-ha!” moments in the shower, on the drive home, and when you’re falling asleep.)  So keep a notebook handy for writing things down.  I’m writing in a notebook almost daily (I’ve joined the cult of moleskine), writing down ideas as they come up.  It’s also a tider home for all the scraps of paper that have lived in my wallet for years, ideas scrawled and smudged into them.  And I’m on my 4th or 5th pocket mod.

- Maintain your ‘exo-cortex,’ your external memory. I wrote a mini-wiki engine with Sinatra, and so far, I’ve used it for everything from clojure, to saving quotes, to robot rights.  I wrote it for practice, but it’s nice to have a searchable, non-linear notebook.

The book mentions Gerald Weinberg’s fieldstone method of writing: to build a stone wall, you gradually gather stones from the field as you clear it, and pile them up.  When you have enough, build your wall.  Both the wiki and the notebook serve this purpose.

- Map out your thinking, and doodle. Drawing pictures engages your R-mode, making it more active, helping you make connections, and understand more clearly.  Besides, it’s fun. This is my 3rd time learning emacs, and it’s finally sticking — I think it’s mostly because I drew up a cheat-sheet, pictorially explaining each command’s effect.  I also map out my learning and career goals this way.

One of my favorite parts is, after you’ve mapped something out, and the drawing looks like a pile of yarn, re-draw it on a clean sheet, fixing up all the awkward placements and messy bits.  The benefit of this second version is in its creation — by considering how to re-organize it, you’re thinking more about the problem, in ways you might not have before.  This works for note-taking, too.

- Take advantage of your brain’s plasticity. You can change how your brain works by believing it works differently: make it smarter, more creative, cheerier.  This is a pretty incredible claim, but he cites two books I’d like to check out.  It sounds impossible, but I’ve seen it work on me, to some extent.  My biggest problem is continuing to believe it works.

Some things I have yet to try:

- Take a walk.  A meditative walk, like around a labyrinth.

- Write morning pages: for three weeks, as soon as you wake, write down three pages of text.  It doesn’t have to be anything specific.  The idea is, right after we wake, our R-mode is more active, so the gems will be more accessible, and they’ll pop out onto the page.  I’m more curious to see what I come up with.

- I have to get better at setting SMART goals: specific, measurable, achievable, relevant, and time-boxed.  I’ve never been much for this kind of discipline — I made a few, but didn’t stick to them, and haven’t gotten into the habit yet.  But like the book says, change is hard:

Change is always harder than it looks — that’s a physical reality, not just an aphorism. An old, ingrained habit makes the equivalent of a neural highway in your brain. These old habits don’t go away. You can make new neural highways alongside, going a different route and making short-cuts, but the old highways remain. They are always there for you to revert to—to fall back on. Practice may not make perfect, but it sure makes permanent.

My next step on the road to lisp is at the International Lisp Conference in Cambridge, MA, this March.

A lot of people are surprised to hear I’m interested in Lisp, but I’ve been studying scheme informally for about 3 years, reading SICP, and finishing The Little Schemer last year. I’ve been interested in clojure for about 18 months, especially after seeing Rich Hickey talk about it last March.  Now I’m studying it in earnest with Stu Halloway’s Programming Clojure.

I don’t use lisps of any sort at work, so it’s not a “practical” language to study like, say, C# is.  But that’s short-term thinking.  Learning lisp is plenty practical, if you’re looking longer-term — even if it’s only three years out.  Learning lisp teaches you about some of the core issues of programming.

I like learning lisp because it’s a window into the workings of programming languages, being so close to the AST.  I want to learn more about meta-programming with macros, hygienic or not.

And for the record, the functional techniques in The Little Schemer have improved my Ruby, JavaScript, and C# in some really solid ways. Functional programming is making its way to the rest of the programming world, and lisp does functional programming.

If you’ll be at the conference, I’d love to hear from you.

As we trudge back to our regular schedules after assorted holiday debaucheries, here’s a heartening thought: Hartford.rb is meeting on Monday, 1/26! Gary Wright will tell us the story of how he learned Git, and some lucky stiff will walk away with a free copy of The Rails Way. We’ll even have pizza — though we’re now on the look-out for a pizza sponsor, so if you know anyone looking to feed us (for only ~$50 a month!), it might even be free.

The usual details:
Monday, 1/26, 6 – 8 PM
GeeZeo
750 Main St, Hartford

See you soon.

I often hear people mis-use the words accuracy and precision, but they mean different things.

To say I’m 117 years, 93 days, 4 hours, 17 minutes, and 48.249 seconds old is very precise, but it’s inaccurate (right now, anyway).  To say I’m in my thirties is accurate, but imprecise.  Precision is about level of detail, accuracy is about truthiness.

They make a wonderful pair, working pretty much orthogonally to each other.

Wikipedia has a nice analogy using a bulls-eye target.

Here’s a little bit of post-Christmas fun, based on the Twelve Days of Christmas.

bugs = %{
     Tell them it's a feature
     Say it's not supported
     Change the documentation
     Blame it on the hardware
     Find a way around it
     Say they need an upgrade
     Reinstall the software
     Ask for a dump
     Run with the debugger
     Try to reproduce it
     Ask them how they did it and
     See if they can do it again.
}.strip.split("\n").map { |bug| bug.strip }.reverse

days = %w[first second third fourth fifth sixth seventh eighth ninth tenth eleventh twelfth]

0.upto(11) do |day_num|
  puts "For the #{days[day_num]} bug of Christmas, my manager said to me"
  day_num.downto(0) do |bug_num|
    puts bugs[bug_num]
  end
  puts
end