@Lathi.net

Simple Expense Tracking with Google Docs

Doug — Mon, 28 Apr 2008 12:57:00 -0500

As my family prepares for our pending Disney trip, I’m thinking about clever ways to take advantage of my wife’s iPhone. One of the things we’ve done is budget the trip based on a few simple categories of expenses. There’s a lot of ways to track expenses to budgets, but I’m particularly happy with how this worked out using Google Documents and the iPhone.

The kicker is that Google Docs are only viewable from the iPhone. My initial hopes of editing a spreadsheet directly from the phone were quashed rudely in a live demo with the wife. Plan B is to use Google Doc’s forms to add individual expenses. This is really a better solution as it’s much easier for my wife to fill out three text fields than to worry about editing a spreadsheet in the right cells.

So the expense sheet of the document is simply the form entries: description, amount, and type. I added help text that listed the valid values I was looking for: food, gas, tickets, junk.

The budget sheet has four rows; one for each budget type. One column has the budgeted amount, and another uses “SUMIF” to sum up all the expense entries if the type matches the budget category. I can then subtract these two to get how much of our budgeted money remains in that category.

I bookmarked the form url on the iPhone for my wife and she can fill it out repeatedly. I also bookmarked the actual spreadsheet. It’s unfortunate that the Google Doc forms don’t let you specify a “thank you” url instead of a thank you message. Anyway, I bookmarked the actual Google Document so we can check the budget remaining easily while out and about. It seems that the iPhone has some trouble changing sheets while viewing Google Documents, so I made the budget sheet the first and default page.

All in all it’s very simple and has a high wife approval factor. Plus, the whole thing took me less than 30 minutes to setup including several “customer approval testing” iterations. Spreadsheets with Google Docs are really fun. Adding in these forms is almost like rapid prototyping custom web apps.

Dynamic Constants and their Pitfalls

Doug — Tue, 05 Feb 2008 15:35:00 -0600

I’ve just fixed a bug in production that took me more than eight hours to find. When I show you the code, you’ll wonder why it took me so long. I have lots of excuses, but it’s a fairly interesting bug to think about. It shows some of the weaknesses in my usual modus operandi. The code that looks something like this:

class Article
  CURRENT="start_publish_on <= #{Date.today} AND stop_publish_on > #{Date.today}"

  def self.find_latest
    find(:all, :conditions => CURRENT)
  end
end

In retrospect the bug is obvious and probably is obvious to you as well. The Article::CURRENT constant is dynamically generated using the date at the time the class is evaluated. With rails in production mode, that could be a long time; certainly more than a day. The conclusion to draw here is to be very, very careful about dynamically generating constant strings. As a rule, I might suggest not doing it.

The most interesting thing about this is you can’t write a test to catch this error. I think that’s the the biggest thing that took me so long to find the error. I tend to be over confident in our test suite. As the new guy to the project, I’m proud of them for how conscientious they are about testing their code. I’m trying to fix this bug by triggering it in a test. Well, it can’t be done. The difference is how the production environment cache classes versus how testing and development does it.

Here’s another tidbit that threw me off the trail for a long time. Our copy editor that is responsible for publishing articles says to fix it she simply back dates the articles by a day. So I spent a lot of time looking for off-by-one errors. I had recently fixed a problem with comparing times to dates and causing off-by-one, so I thought that might be it. As it turns out, this was a red herring. There’s such a tight loop for feature request, implement, deploy that the production environment gets restarted fairly regularly (like nearly every day).

I guess what prompted me to write about this particular bug was what it said about our testing. Clearly automated testing can’t find all the bugs. It also says something about our rapid development. As long as we’re really busy, this bug didn’t bite us. It’s not until our deployment slows down (like a weekend) that it showed up.

My First Day of Freedom

Doug — Mon, 21 Jan 2008 07:49:00 -0600

Friday was my last day at the company that won’t let me blog about them. I am very excited to be 100% independent freelancing! This has been a goal of mine for many years and it’s finally come true.

This morning Mark Windholtz is coming over to pair with me on a contract we share. It’s his good for not making me commute on my first day of independence. I’m looking forward to working with him and several other independent developers.

Hopefully now that I’m not under corporate thumbs I’ll be able to blog more about the types of problems I’m seeing and methods for overcoming. I’ve specifically been asked to write an article on my experiences working remotely on a Scrum project.

In addition to writing more, I’m also hoping to be able to contribute back to the open source community. It’s tricky when your family depends on your billable hours to also give your work away. However, I think it’s good balance to work on things my clients require as well as things I’m interested in. At first, this is likely to be helping Phil Hagelberg shape up Emacs for ruby development.

So, there you go. I’m going to be working with some good guys; doing more writing; and working on more open source. Wish me luck!

Small Victories in Customer Service

Doug — Sat, 01 Dec 2007 11:20:00 -0600

I bought my wife a GPS from Amazon as an early Christmas present before Thanksgiving. I guess I was a little too eager because I ended up paying too much for it. Just a couple weeks later, Amazon advertised to me in one of their “may we suggest” ads the very same GPS for a lot less money. In the end, they refunded me the difference in price; but I can’t help but feel a little miffed by the whole experience.

Let me start by saying trying to sell me the exact same device I’ve already bought from them at a lower price in a targeted ad is pretty stupid. They know everything I’ve bought. In fact, the ad said something like “we know you’ve looked at these GPS’, may we suggest this other one?” Why would they advertise to me something I’ve already bought. And why advertise it to me if you know I paid more for the same device? We in the web development world who control how these ads get generated need to do better than that.

I couldn’t let them rub this price drop in my face, so I called up customer service. The CSR was unsympathetic to my plight. Nothing I said phased her. I was nice and undemanding. I pointed out that I was within my return window and I could just return the device for a refund and then go buy it somewhere else. It all was a no go. There was no getting a refund from her.

So I pulled the standard “let me talk to your manager” trick. She made me wait on hold for what seemed like five to 10 minutes. When the supervisor came on the phone she quickly summarized what she thought my concern was and then just as quickly told me she’d refund me the difference between what I paid and the new price.

One the surface, I’m pleased. I paid way too much for the GPS and I’m glad I didn’t loose out. Money’s tight this time of year and every little bit helps. Also, it didn’t cost me too much effort to get the refund. All told probably 30 minutes. The refund as an hourly wage is pretty good.

On the other hand, I’m frustrated by the way they handled me. First, as I said above, why even advertise the price drop to me? It’s true I was already aware other places (like Wal-Mart) had it cheaper. But the ad was definitely insult to injury. Second, and this is the real kicker, why didn’t the original CSR handle my concern? Why did she resist so hard to giving me a refund when her supervisor just came on the phone and gave it to me without me having to even ask?

I’m trying to think what lesson I can learn about this in dealing with people. I wonder if the original CSR knew her manager was going to give me what I wanted? It seems like there’s times when people have wanted things from me that I knew I’d eventually give them. For some reason though I can recall making them “work for it”. Maybe such a delayed “giving in” has some value, but on the receiving end I’m not so sure. I think this no-then-yes tactic really just comes across as being difficult to work with.

I think the lesson to walk away from here is that when someone asks for something really think about it before you say “no”. Are you saying “no” just because you hope they’ll go away but you know the right thing to do is say “yes”? There are definitely times to say “no” just as there are definitely times to say “yes”. I guess what I’m trying to come up with is don’t play games with your yeses and noes. Shoot straight. Let your “yes” be yes and your “no” be no.

Oh, and the other lesson to walk away from here is don’t give in so easily when talking to customer service.

Jumping to Specific Windows in Emacs

Doug — Wed, 07 Nov 2007 20:58:00 -0600

OK, last emacs tip for a while. It’s ironic, but the last two years I’ve written some cool elisp while at RubyConf. I typically run emacs full screen and split buffers both vertically and horizontally to arrange bits of code. It’s not uncommon for me to end up with three, four, five or even six visible windows in emacs. The problem though is navigating between them.

The out-of-the-box solution is to use other-window (bound to C-x o) and just cycle through them. If you’re fancy you can give a prefix argument to other-window and go backwards. This window cycling is tedious to me though and I’d like something faster. Particularly when I’m mostly bouncing back and forth between two windows.

To solve this problem I wrote jump-to-buffer. Of course, right now I’m really enamored with the fuzzy matching in ido-completing-read. That’s what allows you to type non-sequential characters to match the buffer name. It’s very similar to TextMate’s pattern matching. So, jump-to-buffer uses ido-completing-read. The buffers it gives you as options for completion are only the buffers in visible windows. It also sorts those buffers in the order of the buffer-list; which is in the order of most recently accessed.

To make this work, I’ve written two helper functions: rotate-list, and sort-by-other-list. Without further ado:

(defun dka-sort-by-other-list (to-sort-list other-list)
  (let* ((index 0)
         (other-alist (mapcar (lambda (buffer) 
                                (setq index (+ index 1))
                                (cons buffer index))
                              other-list))
         (swartz (mapcar (lambda (item) 
                           (cons (cdr (assoc item other-alist)) item))
                         to-sort-list))
         (sorted-list (sort swartz (lambda (a b) (< (car a) (car b))))))
    (mapcar 'cdr sorted-list)))

(defun rotate-list (list count)
  "Rotate the LIST by COUNT elements"
  (cond
   ((= count 0) list)
   ((not list) list)
   (t
    (rotate-list (nconc  (cdr list) (list (car list)) '()) (1- count)))))

(defun dka-jump-to-window ()
  "Interactively jump to another visible window based on it's `buffer-name' using `ido-completing-read'"
  (interactive)
  (let* ((visible-buffers (mapcar '(lambda (window) (window-buffer window)) (window-list)))
         (sorted-visible-buffers (dka-sort-by-other-list visible-buffers (buffer-list)))
         (rotated-buffer-list (rotate-list sorted-visible-buffers 1))
         (visible-buffer-names (mapcar (lambda (buffer) (buffer-name buffer)) rotated-buffer-list))
         (buffer-name (ido-completing-read "Enter buffer to jump to: " 
                                           visible-buffer-names
                                           nil t))
         (window-of-buffer
          (delq nil 
                (mapcar '(lambda (window) 
                           (if (equal buffer-name (buffer-name (window-buffer window)))
        window nil)) (window-list)))))
    (select-window (car window-of-buffer)))
)

I’m definitely interested in feedback on this code. The dka-sort-list-by-other-list method was particularly tricky for me to write. I think it’s both right and fast. I haven’t dove into elunit yet to know for sure if it’s right.

Sharing the Mac Clipboard with Emacs

Doug — Wed, 07 Nov 2007 20:09:00 -0600

While I’m sharing emacs hacks, I finally got around to researching this and/or figuring it out. I almost always run emacs inside the terminal in a “no window” mode. It’s pretty natural for me to use M-w to copy something from emacs and then try to Cmd-V it in another Terminal window or likewise Cmd-C something in Terminal and then try paste it into emacs with C-y. Until now, I haven’t known how to share the clipboard with emacs.

(defun copy-from-osx ()
  (shell-command-to-string "pbpaste"))

(defun paste-to-osx (text &optional push)
  (let ((process-connection-type nil)) 
      (let ((proc (start-process "pbcopy" "*Messages*" "pbcopy")))
        (process-send-string proc text)
        (process-send-eof proc))))

(setq interprogram-cut-function 'paste-to-osx)
(setq interprogram-paste-function 'copy-from-osx)

This takes advantage of the pbcopy and pbpaste command line programs to access the clipboard and offers them up as elisp method for emacs’ interprogram-*-functions. Easy, peasy, pumkin weasy.

I need to give some props to Mark Aufflick for his post on Automatic Copy from X11 App to Mac OS Clipboard. My paste-to-osx is pretty much a straight rip from him.

Navigating Your Projects in Emacs

Doug — Wed, 07 Nov 2007 10:04:00 -0600

One of the things that’s really nice about TextMate is the Cmd-T navigation of files in your project. It pops up this little dialog with fancy pattern matching input to select one of the files in your project and then jumps to that file. There’s a similar command once your in a file to jump to symbols in that file. I’ve tried a couple things to achieve that behavior in emacs that I’d like to talk about.

The first thing I tried was a method called find-file-in-project. It was originally implemented by Phil Hagelberg as part of his “Another Ruby on Rails Mode” (arorem) and then also ported over to the current Rinari Is Not a Rails IDE. Basically, it indexes all the files in a “project” and then provides a nice interactive completing read to switch between them. It basically works just like Cmd-T in TextMat. I did quite a bit of work optimizing the completing read so that it would behave nicely. The problem is that emacs is slow to index the files in your project.

What I’m using now is the tried and true find-tag method which is part of etags.el. ETags depends on an external TAGS file as an index of all the symbols in your project. When you invoke find-tag (by default bound to M-.) it prompts you with completing read for the symbol to find. It then jumps directly to the file and location where that symbol is defined. It’s basically combining the find-file-in-project with find-symbol-in-buffer. It’s also very, very fast.

As an added bonus, you can use tags-search to search through your project finding places that tag is used. This is similar to TextMate’s slowly grep all the files and render a buffer with those results in it, but much faster.

The bad news is you have to manually build the TAGS file periodically. Emacs is pretty good about continuing to work when you’ve made modifications to the files, but if you add new methods, new files, or refactor significantly where thing are located then it gets confused. When that happens, simply rebuild the TAGS file.

Jim Weirich gave me a nice little rake task to do the job:

module Tags
  RUBY_FILES = FileList['**/*.rb'].exclude("pkg")
end

namespace "tags" do
  task :emacs => Tags::RUBY_FILES do
    puts "Making Emacs TAGS file"
    sh "xctags -e #{Tags::RUBY_FILES}", :verbose => false
  end
end

task :tags => ["tags:emacs"]

Just put that in lib/tasks/tags.rake and then run rake tags:emacs when you invoke find-tag and it can’t find it. That shouldn’t happen very often. I’ve found it’s also very fast to build the TAGS file. I might consider putting that tags task as part of the normal run tests task, but I’m not sure that’s necessary.

The other catch here is that the rake task above calls out the exuberant ctags rather than the ctags that comes with emacs. The exuberant ctags knows how to parse ruby files whereas the ctags that comes with emacs can’t.

I’ve installed exuberant ctags from MacPorts. It is xctags even though MacPorts doesn’t install it that way. So (for better or worse) I’ve renamed the ctags files installed with ports to xctags. This also gets around the conflict that MacPorts thinks there is between the ctags installed with emacs and the exuberant ctags.

Give this a spin and let me know what you think. I’ve found it to be very accurate and very fast.

Remote Pair Programming

Doug — Tue, 09 Oct 2007 08:47:00 -0500

I mentioned in my post the other day about remote pair programming and someone called me out on it via email. Since I spend probably between 40 – 60% of my work time remote pair programming I thought I’d give some of my experiences.

First, some background. I’ve been moonlighting as a freelance web developer for a lot of years. I work with David Minor at Sus 4. He and I have done remote pair programming on almost all of our projects for the last four years. I also telecommute for my day job into blah blah blah (lawyers made me change this). I travel to the headquarters in VA once a month or so, but work from home the other three or four weeks. Probably half to three-quarters of my programming tasks at blah blah blah (lawyers made me change this) I pair with another developer. There are several different people that I’ve paired with regularly over the almost two years I’ve been there. Finally, I’ve talked a lot about remote pair programming at the Cincinnati Agile Round Table. From their Jim Weirich has picked up on my techniques while doing remote pairing at Edge Case.

What I’m trying to say here is that I have done a lot of remote pair programming over the years and I’ve tried a lot of different techniques. What I’m about to describe here is tried, tested, and proved.

Being on a Mac, I have paid licenses of both TextMate and SubethaEdit. They are both fine editors. I think I prefer TextMat over Subetha simply because I think TM handles projects better. However, I do think Subetha’s networking and multi-user support is really cool. The problem is with Rails projects I always end up with lots and lots of files open. Subetha doesn’t make it easy enough to share the files by default or for the remote guy to join their shared files by default. Plus, with TDD you have to have some way for both partners to see the test output.

While talking about things that don’t work… there’s just not enough bandwidth for VNC. I’ve done all I know to do to optimize it: drop the number of colors, change backgrounds to solid colors, muck with compression algorithms. I just can’t quite squeeze enough bandwidth out so that it feels interactive for both people.

So what does work? Welcome to the past. What I’ve found is that gnu screen running inside a terminal is the most bandwidth efficient, highly interactive, flexible multi-user environment. One person hosts, the other connects via ssh and joins the hosts screen session. For those who’ve never used screen, both users connected see exactly the same thing. Both users keyboards have simultaneous control of the terminal. Combine this with a good voice/video channel with Skype or iChat and it’s nearly as good as actually sitting next to someone.

Inside of screen you can have multiple tabs/windows. What I typically do is designate my first window to run emacs. This is my “IDE”. We do all text editing from inside there. I have another window designated for running tests (although sometimes I run tests from inside emacs). I have another window designated for the mysql prompt. Another is for tailing log files and another for running the server. Screen saves the history (I typically set it to something ridiculously large). Both people can see when you scroll through the buffer history. Both people can see all the data.

I’ve kinda glossed over the use of emacs here. I’ve also done remote pairing with vi (well, vim anyway). The point is that it helps a great deal if you’re using an editor that both people understand. I pair with a guy at work that only begrudgingly uses emacs. It’s not nearly as much fun for him. I’ve also paired with folks who aren’t comfortable with any terminal based editor. That’s no fun either.

Like pair programming when you’re actually together, there are a number of things to consider. Editor environments is one of them. Remote pair programming seems to heighten or exacerbate all the issues with local pair programming. Your communication issues are bigger. You really have to make a point to tell each other what you’re doing. Talk, talk, talk! A definite anti-pattern is one of the people in the pair going off to work on solutions locally on their own computer while the other person can’t see what’s going on. Do all work in the screen session. Make sure that both people can see everything that’s going on. Related to this, it’s hard for one person to “see” when the other is distracted. If one person is checking email while the other is trying to stay on task that’s no good.

The good news is that remote pair programming can work. I find actually pair programming in person to be very fulfilling, fun, and exciting. Remote pair programming isn’t as good as actually being together in the same room. However, it’s a very good substitute when the commute cost are high. If you’re the kind of person that doesn’t really dig pair programming in general, odds are you’ll find remote pair programming to be even less attractive than actual pairing.

I’ll also mention I know one guy (whom I respect a lot) who’s big on pair programming, but just can’t get in the groove of remote pair programming. I suspect his problem is lack of familiarity with the editor, but there may be other philosophical issues as well.

All that said, I don’t find running emacs inside a screen to be some “least common denominator”. In fact, I do all of my development this way—even when I’m not pairing. Lately I’ve been running multiple screen sessions on my box; one for each project I’ve got going. My “state” is always right where I left off. When I switch from independent work to pairing, it’s easy for someone to simply join my screen and we can get started. I’ll stop here and not get too carried away with singing emacs’ praises. That’ll be good info for another post…

Before I close out, it occurs to me that screen can be pretty ugly with zero config. I’ll go ahead an post my hard status config. This dresses things up quite a bit and makes it a lot easier to use. Put these lines in ~/.screenrc:

hardstatus on
hardstatus alwayslastline
hardstatus string "%{rk}%H %{gk}%c %{yk}%M%d %{wk}%?%-Lw%?%{bw}%n*%f%t%?(%u)%?%{wk}%?%+Lw%?"

Resizing Your Terminal

Doug — Fri, 05 Oct 2007 07:24:00 -0500

I thought I’d go ahead and publish this because I found it useful (if ugly). I do a lot of remote pair programming where my partner and I share a multi-user screen session inside a terminal. It’s very important that both partners have the same number of columns and rows visible. Otherwise, whoever is smaller isn’t going to see everything the other person sees and stuff jumps around and it’s generally annoying.

So, the first thing that happens when you start a remote pairing session is you negotiate screen size. I have my Apple Terminal.app set to show the screen size in the title bar as something like 105×54 or 94×71 or whatever. So when my partner says, “I can do 120×62 as my maximum size” I have to click and drag my Terminal window until my size matches. No more!

As I said above, this is pretty ugly but it works. This shell script makes a one line applescript call through osascript to tell the Terminal what size to make the window. I put it in a shell script like this to simplify calling. I started with a straight up applescript, but it was ugly to call it from the command line. So I converted it to what you see here:


#!/bin/sh

/usr/bin/osascript -e 'tell application "Terminal"' -e "tell front window" -e "set the number of rows to $2" -e "set the number of columns to $1" -e "end tell" -e "end tell"

You can invoke it by simply saying:


$ resize_terminal 94 71

and the front most Terminal window (likely the one you’re typing in) will resize to 94 rows tall by 71 columns wide.

New Radiant Website

Doug — Tue, 28 Aug 2007 04:38:00 -0500

blah blah blah (lawyers made me change this) has been going through a major re-branding effort for the last long time. This morning we’re launching a major re-write of our main application. There’s a pretty cool on-line demo that shows off our new sexiness.

Simultaneously we’re rebranding our website and also restructuring most of our internal item catalog. Pretty much everyone in the company has been involved in this effort in some way. After months of work, we’re live!

The new website is running Radiant from index to thank you page. We have a web service for handling all of the EC business logic and multiple user facing web sites that connect to the service for managing their cart. We’ve written a “store behavior” to handle this xml-rpc client interface with Radiant.

This actually isn’t our first production Radiant site. Our first was our Japanese RosettaWorld. That site isn’t getting near the traffic that our US site does though. The US site runs steady-state at about 5 user sessions per second. We had a bit of a problem serving our CSS and JavaScript out of Radiant. It was causing a session hit that forced an update to the DB. Turns out that was a pretty big bottleneck in our load testing. As a fix, we just moved those files out of Radiant for this rollout. I’m sure there’s some in-Radiant solution to tell it not to track sessions on those pages. We’ll figure that out for this next release.

Hopefully, after all the stress of getting this release done we can take a collective deep breath. However, I’m guessing it’s more likely we’ll be scrambling to make all kinds of tweaks and such.