Dave Schweisguth in a Bottle

How many meanings of that can you think of?

Test patterns: red-green-refactor vs. frequent committing

leave a comment »

Agile and lean principles lead to committing frequently, each commit potentially deployable to production. I’ve always been one to save my work often, and I commit often, not just whenever I finish an entire task, but whenever I’ve written enough code to justify running my tests, I do so, and they pass. This always seems like the right thing to do: it’s usually easy to break down big tasks into small, committable, deployable bits, and I can almost always point to the value in each commit, whether an implemented test, a refactoring or just a spelling fix. But I recently found myself in a situation where I realized that committing a (correct) test and its implementation that I’d just written would introduce a bug.

Read the rest of this entry »

Written by dschweisguth

February 18, 2011 at 16:46

Posted in Programming, Rails, Ruby, Testing

latin1, utf8, Rails 2.3.9 and mysql2

with one comment

GWW‘s database.yml (which originated under Rails 1.something) has never specified an encoding. We seem to have gotten away with it because mysql defaults to latin1, our database is latin1 (probably because mysql defaults to it) and the mysql gem was happy with that default. It was clear that non-ASCII characters were being handled correctly, because when using GWW one often sees Flickr usernames that contain them.

Discussions of encodings often include a statement like “start using UTF-8 now, because more and more software will assume it in the future”. They were right, and the future is now. Working towards upgrading to Rails 3, I recently updated GWW to Rails 2.3.10 and replaced the mysql gem with the mysql2 gem (which Rails 2.3.9 and later require). I’m pretty sure this is when GWW began dropping the ball on non-ASCII characters, displaying them all as question marks. I didn’t actually roll back the upgrade to prove it, and I don’t know whether it’s Rails 2.3.9/10 or mysql2. I was more interested in fixing the problem quickly — which, fortunately, it could be.

Read the rest of this entry »

Written by dschweisguth

February 16, 2011 at 11:02

Posted in Programming, Rails, Ruby

Test patterns: One test per path

leave a comment »

I’ve seen many test suites which were much less thought through and much less well factored than the code they tested. But that’s not right: Tests drive the code, so tests are at least as important as code. You should care about the quality of your tests just as much as you do the quality of your code, and refactor your tests just as vigorously as you do your code. Here, then, are some thoughts on one aspect of good test design: the One test per path pattern.

Read the rest of this entry »

Written by dschweisguth

February 15, 2011 at 12:30

Posted in Programming, Ruby, Testing

Filtering spam with GMail and SPF, episode II: “Attack of the Spam”

leave a comment »

A year ago I described how I sent my email through GMail to filter spam, and how I remailed my email instead of forwarding it to satisfy SPF. But there was an onion in the ointment. Because I was remailing my email, GMail learned that mail from my mail server often contained spam, which GMail recognized as spam from other sources. (“Often” here means as often as spamassassin, which I was still running on my mail server, let spam slip through.) GMail frequently guessed that real mail was spam, and although I could correct it sender by sender, I had to watch my GMail spam folder constantly for mail from real human beings whom GMail hadn’t seen before. That was almost as much work as deleting the actual spam myself in the first place. What’s more, although I’m not aware that this ever happened, for all I know GMail’s low opinion of my mail server might make it think that mail that originated on my server was spam.

Fortunately, there was an easy solution: stop passing the buck.

Read the rest of this entry »

Written by dschweisguth

February 8, 2011 at 18:09

self.welcome_back

leave a comment »

After posting once and then going dark more than a year ago, I’m now on sabbatical and am again of a mind to write down things related to what I’m working on. Right now that’s Guess Where Watcher, a Ruby on Rails application used for scoring the Guess Where San Francisco Flickr game. Let’s see where this goes in a month or two.

Written by dschweisguth

February 8, 2011 at 15:53

Posted in Uncategorized

Filtering spam with GMail and SPF

leave a comment »

8 Feb 2011: Caveat lector! I no longer recommend the setup I described in this post. See the followup for the reason why not. The details in this post may still be useful to some, however.

I run my own domain name service (DNS) and mail service (sendmail) on my home Linux server. Until recently I filtered spam with spamassassin, but the spam that it let through grew until it was time to find another way. I could lower spamassassin’s bar for considering something spam, but it would then not be easy to rescue good mail that was wrongly considered spam. GMail allows you to forward mail from elsewhere to it and retrieve mail via POP or IMAP and it has severe but easy-to-tune spam filtering, so that’s what I used.

In my first setup, I aliased my home account to my GMail account in /etc/aliases and used fetchmail to pull mail back from GMail to home. Initially, that appeared to work great: I got all of my mail just as before and no spam at all (in fact, I had to spend some time rescuing bulk mail that wasn’t spam from my spam folder, but GMail knows about that now).

However, I had a residual problem: some of my filtered emails had an Authentication-Results header indicating that, according to the SPF record for the sender’s domain, my home server was not allowed to send mail from the sender’s domain. Forwarding through sendmail results in email that says it’s from one domain (the original sender’s) coming from another (that of the forwarder), which is exactly what SPF is supposed to prevent. All of my mail was getting through, either because it came from domains without SPF, because it came from domains with SPF configured to ‘softfail’ (that is, to be considered as only advisory), or because GMail wasn’t acting on SPF advice. (I have no evidence that the latter was happening; I just never ruled it out.) Although I was getting my mail for now, I might lose mail if any of several things changed.

Read the rest of this entry »

Written by dschweisguth

December 6, 2009 at 12:48