Recent Updates RSS Toggle Comment Threads | Keyboard Shortcuts

  • Ben 10:38 on Saturday, March 6, 2010 Permalink | Reply
    Tags: Survey Box Font, Survey Design   

    The Easy Way to add Coded Tick Boxes to your Paper Surveys 

    If you’ve spent any time developing paper questionnaires in MSWord you’ll know how frustrating it can be to add tick boxes alongside your response options.  The boxes included with font sets like wingdings or symbols don’t really cut it.  And if you want coded tick boxes – ones with tiny numbers in them to reduce data entry time and errors once the surveys are received - you are completely out of luck.  Your only choice in that situation is to create the boxes using the ‘text box’ in the draw tool, manually adding in the numbers you need.  Of course, once you start working with drawing objects you then have to fiddle around to get them aligning correctly with the response option text.  Do this over multiple questions in a survey and various iterations of drafts and it is only a matter of time before you start tearing out huge clumps of your own hair.

    Not fun.

    Thankfully, you can now stow away your Rogaine.  I’ve created a couple of fonts that contain discreetly numbered boxes in two different styles. Since the boxes are each separate characters in a font set, they can be treated just like any other text character; you can resize them, align them using tabs, change their colour, put two boxes next to one another, etc.   All it takes to put a box in your questionnaire is for you to hit the related key on your keyboard (hitting ‘1′ will insert a box with the number 1 in it).

    Here’s what they look like:

    You can click this link to download a zip file (314KB) containing the fonts.  The zip file also contains a brief document describing how to add the fonts to your machine, where to find them in MSWord, and what keys to hit to get different numbered boxes.  The fonts are free for you to download and use, but I don’t offer any support or warranties for them.  I’d appreciate you letting me know if you find them useful.

    _______

    ShortURL to this post:  http://wp.me/pnqr9-51

     

  • Ben 16:28 on Saturday, February 20, 2010 Permalink | Reply
    Tags: ,   

    The Incredible Gravitational Pull of Shortcuts 

    It was a beautiful day in Wellington on Friday, so I took a stroll along the waterfront at lunchtime.  As I was returning to town I noticed something interesting between Cable St and Jervois Qy…

    I doubt the goat track is much shorter than the paved alternative, yet there must be some perceived directness about it for a good number of people.  Perhaps it was first formed by those running to catch a green pedestrian light, like the guys in the first shot.  Whatever the case the path now seems to have a life of its own.  Enough pedestrians have taken the dirt route just to get to the waiting point that the entire area next to it has been completely stripped of grass.

    Indeed, I must admit I found the dirt path choice quite compelling even though I wasn’t in a rush and could see the other pedestrian light was already orange.  It was probably the social proof of the well-worn track (“others seem to think it would be quicker, so it probably is”) along with the direct line-of-site to the other crossing that provided the appeal. And the fact that the path was already bare meant there would be no guilt in walking on the grass.

    What’s bizzare is that we’ll choose that route even though it doesn’t really save us any material time.  In fact, even if it did save time, in most cases it would mean we just get to a waiting point quicker!

    This ‘gravitational pull’ of shortcuts is frequently exploited.  Just look at the ads for 30-day weight-loss programs and get-rich quick schemes.  Even when there is little for us to gain we’ll take the short route so as to not miss out on just the potential for a mental, physical, or financial bargain.  It is unfortunate that the designers of this particular pedestrian crossing didn’t take it into account.  It might have made for a little less mud in winter.

    Still, it gave me something to write about!

    Here’s the area in Google Street View if you want to talk a stroll around it yourself…


    View Larger Map

    ________

    ShortURL to this post: http://wp.me/pnqr9-4K

     

    • Scott 9:27 on Monday, March 1, 2010 Permalink

      People will generally do what is right for them at the immediate time. In this case, when crossing in the direction of the two guys in the bottom of your photo. The first crossing is aligned with the exit of the second as you mentioned. As such they continue walking to what they perceive is the correct crossing path for them. It’s not till they get under the cover that they realise that the crossing has been put at a different angle and the timings are not aligned as a single pedestrian crossing. As such, they deviate their path to ensure that the crossing buzzer has been activated. I think you’ll find the deviation is a road safety matter. Not sure it’s the social proof you think it is as it would be quicker had the intersection not been designed to make it not so. Surely it’s a social proof that council walking paths are always the slowest(safest)?

    • Ben 20:20 on Monday, March 1, 2010 Permalink

      Hey Scott.

      I guess we’d have to ask them if they were just looking for a quicker route or whether they actually thought the ped crossing continued over the grass. At least in the beginning (before the grass was destroyed) it should have been clear to people that the ‘official’ route to the next ped crossing was the paved one, and I imagine most people would see that the paved path leads to a ped crossing once they get to the edge of the one they have just finished and look over to the road.

      I’ve also taken to watching people cross from the nearby overbridge every now and then (they don’t let me out much) and people do use the dirt path in both directions. This suggests that at least the folks coming from the cable street side are using what they perceive is a quicker route, since they would take the paved path if they mistakenly thought the ped crossing continued straight ahead (i.e., if they made the same error that you suggest the running guys are making, but in the opposite direction).

      Whatever the case, the original path design has probably led to a less safe environment; the makeshift (‘faster’) grass path is currently attracting people toward the outside edge of the crossing, where they are closer to oncoming/stopped traffic. It’s speculation, but I suspect that if the grass path was properly paved it would direct people to the middle of the crossing more effectively. All the compacting work has even been done already!

  • Ben 21:49 on Tuesday, February 2, 2010 Permalink | Reply  

    One for the Econ Geeks 

    Here’s something I never expected to see; Keynes and Hayek battle it out in a very nicely produced rap vid.

     

  • Ben 11:33 on Saturday, January 23, 2010 Permalink | Reply
    Tags: Market Research   

    The three minute rule

     

  • Ben 20:05 on Friday, January 22, 2010 Permalink | Reply
    Tags: Random Stuff   

    Can You Think of a Personalised Plate for this Ride? 

    Apparently the following picture is from the mid sixties. It would have been something to see Spock staring back at you from his Buick on the freeway.
    Spock (Leonard Nimoy) next to his Buick

     

  • Ben 16:31 on Saturday, January 9, 2010 Permalink | Reply
    Tags: Critical Thought, , Myths   

    How ‘Information’ is made out of Thin Air 

    Have you heard that a weekday edition of The New York Times contains more information than the average person was likely to come across in a lifetime in 17th century England?

    If so, did you also know that this statement probably has no basis in fact?

    I’ve come across the tidbit more than once over the last couple of months, in conversations and online.  It struck me as being in the same general vein as another popular myth: that we only use 10% of our brains, so I did a little digging.  Here’s what I found:

    • This viral YouTube video contains the snippet (at 3:20) and may be the key source of its popularity at the moment.  No sources are cited for any of the information presented.  Not even in fine print.   The video is one of a number of versions of this particular montage of ‘facts’ which, together, have been seen over 10 million times.
    • The snippet has popped up in a number of places around the web, some more reputable than others.  It also appears in numerous books.  Like the YouTube vid, most don’t make any effort to verify the assertion.  The context also changes from repeat to repeat; sometimes it is the Sunday edition and 19th century citizens, sometimes it is exposure to information in a day and ‘our ancestors’.  So, that lesson you learnt playing Chinese whispers as a kid still holds; people are prone to error when retelling a story.  It’s worth being wary of this when retelling something yourself or hearing a startling ’truth’ from someone else.
    • Where sources are cited, they generally lead to a statement made by Richard Saul Wurman in his 1989 book titled Information Anxiety (page 32).  Unfortunately, I don’t have access to the book, so I can’t see whether he presents any evidence to support the claim.  Thankfully, Geoffrey Nunberg at UC Berkeley managed to find a copy and took a look for himself… apparently the book  ”asserts the fact without offering a source or explanation” (p9 of a pdf by Nunberg that in part examines the likely veracity and lack of meaning of the snippet).

    So, after making a reasonable effort aimed at finding evidence to support the claim, I’ve come up empty handed.  Others have had the same experience.  My conclusion is that it is probably nothing more than an statement without foundation that has made its way into popular consciousness by virtue of it being superficially plausible and sufficiently repeated without critical thought.  Fascinating stuff.

    This particular example is innocuous, but sometimes these ‘facts from thin air’ can make their way into places where they might have more impact on policy or business decisions.  For instance, in Damn Lies and Statistics, Joel Best recounts that a published Journal article he once read contained the statement “Every year since 1950, the number of American children gunned down has doubled”.  Go here if you are interested to see why the statement is so absurd, along with the history of how this ‘mutant statistic’ came to be.

    _____

    ShortURL for this post: http://wp.me/pnqr9-4j

     

  • Ben 12:36 on Saturday, December 19, 2009 Permalink | Reply
    Tags: Piracy, Propaganda, Punishment, Rant, Reinforcement   

    Rant: How to Encourage Piracy 

    Imagine you’ve just done something of benefit to someone else.  Lets say you’ve put out the trash for your flat.  Which of the following would you like to hear from one of your appreciative flatmates?

    1. “Hey, thanks for putting the trash out.  It’s great you remember to do this even though we’ve all got busy Monday mornings.  I got a voucher for a free coffee last night – you want it?”
    2. “Phew.  If you hadn’t put the trash out this morning me and the others were going to spit in your coffee and bad-mouth you behind your back.  Remember this next time you consider not putting the trash out. “

    Unless you are a masochist, I’d guess option 1 is both more attractive and most likely to reinforce your behaviour. Under option 2, putting the trash out becomes even more of a chore.  You’d quickly begin to resent doing it and the incentive to continue would reduce to nil if you suddenly found out your flatmates would have no way of knowing whether you had done it or not.

    So what is it with the movie and software anti-piracy groups?  Do they not understand this?  Do they not care?  Or are they simply under the undead control of Sauron-the-lawyer?

    Here is a painful clip from 1992 titled “don’t copy that floppy”

    (In 2009, an even more painful sequel was released targeting college kids.  Apparently you can make anything cool with ‘the kids’ if you rap it.)

    And here is a current clip playing on DVDs in the UK and Australasia.  I think the man featured is supposed to look like Satan.

    Both use fear appeals (watch out, terrorists will eat your babies and we’ll take your mom off to prison) and the assumption that people are too stupid to realise piracy only rarely results in punishment for minor offenders.  What is worse is that these appeals are forced on an audience that is already compliant.  It is essentially the same as rewarding an as-yet-undetermined flatmate for putting the trash out by attaching a note  to the bag that looks something like option 2.

    I’m quite happy to pay for quality content, but it infuriates me every time I have to sit through one of these anti-piracy propaganda pieces.  In fact, on more than one occasion the thought has gone through my mind that it would be easier to go the piracy route – at least that way I wouldn’t have to watch the anti-piracy dross.

    It seems I’m not the only one.

    Anti-piracy organisations should be reinforcing purchasing customers, not punishing them.

    End of rant.

    _____

    ShortURL to this post: http://wp.me/pnqr9-2U

     

  • Ben 10:18 on Monday, December 7, 2009 Permalink | Reply  

    Anything that doesn’t take years of your life and drive you to suicide hardly seems worth doing.http://bit.ly/2vbTI3

    Cormac McCarthy (Author of ‘No Country for Old Men)
     

  • Ben 11:15 on Sunday, December 6, 2009 Permalink | Reply
    Tags: Experimental Design, , Simpson's Paradox,   

    Have You Fallen Prey to Simpson’s Paradox? 

    In a previous post on experimentation at Microsoft I linked to a recent presentation by Ron Kohavi (GM of their experimentation platform).  One point he raised was that you can actually get the wrong answers from split tests because of a phenomenon called Simpson’s Paradox.  You read right; your test might tell you that version A is the best bet when in reality the better performing version is B.

    That should send a shiver down the spine of anyone tasked with improving a website’s ROI.

    Simpson’s paradox can occur in any setting where the proportion of people allocated to split groups (e.g., control and test) varies according to some important attribute in the study.  It is easiest to understand the paradox by example.  Thankfully, the Wall Street Journal presented one a couple of days ago in an article on the Flaw of Averages.  Essentially, it showed that although current aggregate unemployment rates in the US (expressed as % jobless) don’t appear as bad as they were during the 80s recession, they are actually consistently worse when the figures are examined by educational subgroup.  This is because the proportion of people in each educational subgroup has shifted between the 1980s and now, and each subgroup has a different susceptibility to unemployment.

    The WSJ article also presents two other examples (U of C Berkeley admissions gender bias and Kidney stone treatment efficacy).  If you are still scratching your head after reading through the narrative explanations, try having a look at a the data-based explanations of the same examples on this Wikipedia entry.

    Turning to a web-based scenario, in a recent paper outlining pitfalls to avoid in online experimentation, the folks at Microsoft showed how Simpson’s Paradox can occur when a test is ‘ramped up’ over time.  Their example involves a page design test run over two days, with a 1% sample of users assigned to the test group on the first day (Friday) and then a 50% sample assigned to the test group on the second day (Saturday).  Here is the data from the paper:

    (Note: The percentage in the version B ‘total’ cell is different here due to an error in the original)

    On both test days ‘B’ was the winning version.  However, the result is reversed in the aggregated total; Version A is the winner.  This is essentially because both the test split allocations and response levels varied by day.

    Test ‘ramp ups’ are quite common.  It is good practice to do a pilot of the test on a small sample to make sure everything is working OK before unleashing it on a larger sample. So, the potential for Simpson’s Paradox to occur is very real.  If you are analysing split test results, you can make sure your analysis avoids the problem by re-weighting the results from periods with different allocation procedures or by simply discarding the results from the pilot phase.

    _____

    ShortURL for this post: http://wp.me/pnqr9-3X

     

  • Ben 18:17 on Saturday, November 28, 2009 Permalink | Reply
    Tags: , ,   

    Online Experimentation at Microsoft 

    Over the last three years Microsoft embraced experimentation as a mechanism for testing changes to their various online products.  That they are only recently formally adopting a data-driven approach to their design was a little surprising to me, but it is certainly better late than never!

    As part of the process of making the shift away from simply following the Highest Paid Person’s Opinion (HiPPO) to actually testing the ROI of different ideas, the team in charge of experimentation has been disseminating some of their experiences. You can see a recent talk on the topic, presented at a September meeting of Seattle Tech Startups, at the URL below (sorry, the quality isn’t great and I can’t embed because of WordPress.com restrictions).  Alternatively, go to the Microsoft experimentation portal to see other work from this group.

    http://www.ustream.tv/flash/video/2134721

    The talk presents a number of interesting insights, ranging from the results of some tests (winning versions are often different to what you’d think) through to the cultural hurdles arising from an increased reliance on data for decision making (e.g., people with strong opinions get their egos bruised).

    Amazon.com is also mentioned a couple of times.  I think a few of the current Microsoft team originally cut their teeth there, so those of you interested in this topic might also like to see this eMetrics Summit 2004 presentation (pdf).  It showcases the Amazonian approach to deciding on site changes and resolving bitter political disputes over whose pet area should get highly coveted slots on the home page.  Interesting stuff that more and more organisations are going to have to grapple with as their products and services become increasingly digitized.

     

Older Posts →

c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
esc
cancel