An evening with Jason Fried from 37 Signals

I went to the 37 Signals event last night sponsored by CPB. The speaker was Jason Fried, who is a founder of the company, a designer and all around smart guy.

Jason spoke mainly about small business. He presented a list of things that companies can do to become more productive and better at whatever they do. His list included things such as:

  • 4 day work weeks
  • Virtual companies
  • Saying no to your customers more often than not
  • Do the easiest thing possible
  • Don’t design, just do
  • Don’t go to meetings
  • Turn off email and IM

A number of these things I do each day including turning off email and IM when I’m really cranking, be as virtual as possible, reduce lengthy design and just start working. Overall, Jason has great ideas for small companies. 37 Signals is probably in the 3-5 million dollar range or lower, with about 2-3 million dollars of overhead or more. They are a life-style company and will not grow beyond a certain threshold because of these ideals.

Having worked for a large company, worked on vastly more complex systems than Jason, created many open source frameworks and now starting my own company, I found Jason’s ideas exciting and total crap at the same time. I regularly find myself battling with creating solutions that will scale, versus creating the simplest solution. This naturally occurs to anyone who has ever worked with applications that do hundreds of millions of transactions per day (or per hour).

I pushed Jason a bit to see where he would break ranks with some of his ideals. From his responses, he absolutely would not abandon these ideals at any cost. However, I think this is a marketing tool created by 37 signals to promote their products as well as Ruby on Rails. The reason I know this to be true, is that Jason is intelligent and capable or understanding moderately complex problems with more ease than say a college student. In addition, he spoke briefly about how a single feature request such as adding time-stamps to ToDo list items can have a larger impact than any customer might think. These features requests require a large amount of thought and planning because they impact the entire system. Similarly, changes to the Rails internals require a large amount of thought and testing to ensure they don’t break existing applications.

This dichotomy is something that is a fact of life. It is something engineers, product managers and executives must deal with, no matter how small or large your company is. 37 Signals has built a reputation that they can safely ignore these problems and be wildly successful. However, it is 95% smoke and mirrors. The reality is that 37 Signals is composed completely of phenomenal employees who can work in complex domains without losing their minds. The can distill complexity into manageable pieces and then combine those pieces in the end to produce a complex yet functional system.

Do we really need code reloading?

With Grails, Rails, Python, etc offering developers the chance to change code, hit refresh and see the changes I often wonder if reload truly a feature that is a must have these days? It seems like it is one of the main selling point by most of the folks pushing those frameworks.

I think this is not a requirement, and doesn’t really increase productivity. Here’s why:

  • This promotes less tested code as you can code an entire app without testing at all by simply hitting refresh
  • The testing cycle is lengthy by nature, so you do any testing during development, it is going to take some time
  • Rather than using reload, if you use the development cycle of write, test, run or test, write, run, than starting up the application incurs minimal overhead in addition to running the tests
  • So, what are we really trying to accomplish while developing? Better code, less code, better tests, faster development. These things seem to imply great conventions and excellent extensibility and not reloading.

Handling strange Rails URLs

Came across an interesting situation where my Rails application was going to be receiving URLs with characters like (dash) and . (period or dot). Something like this:

http://example.com/my_controller/action-with-odd.characters

Ruby doesn’t allow dashes or periods in identifiers, so this was going to cause a problem with method naming. I couldn’t figure out how to handle this from the web and so I dove into the source and found that when the action method doesn’t exist Rails calls a catch all method called perform_action. Adding my logic to this method worked well. Looks something like this:

def perform_action
  ...
end

[tags]rails urls,ruby identifiers[/tags]

Handling Rails 404 and 500 errors

I spent a couple of hours trying to figure out how to handle 404 and 500 errors in Rails. This is not simple and actually really annoying. Hopefully future versions clean this up because right now it sucks pretty badly. Anyways, I found a page on the wiki and some other blogs, but the issue was that they wouldn’t handle all the cases. So, here’s the solution:

1. Edit your app/controllers/application.rb file and add these three methods:

  def rescue_404
    rescue_action_in_public CustomNotFoundError.new
  end

  def rescue_action_in_public(exception)
    case exception
      when CustomNotFoundError, ::ActionController::UnknownAction then
        #render_with_layout "shared/error404", 404, "standard"
        render :template => "shared/error404", :layout => "standard", :status => "404"
      else
        @message = exception
        render :template => "shared/error", :layout => "standard", :status => "500"
    end
  end

  def local_request?
    return false
  end

The first method will be explained in the next step. The second method is the method that Rails calls to handle most errors. This method will not capture a certain class of errors where neither the controller nor the action requested exist. The third method tells rails to stop sucking. Normally Rails handles requests made to localhost or 127.0.0.1 differently than all others. This might be for debugging purposes, but it sucks when testing error handling.

2. Edit config/routes.rb and add this line TO THE END OF THE FILE:

  map.connect '*path', :controller => 'application', :action => 'rescue_404' unless ::ActionController::Base.consider_all_requests_local

This tells Rails that if it can’t find any other route to handle the request (i.e. the *path) it should call the rescue_404 action on the application controller (the first method above).

3. Edit config/environments/development.rb and add this line:

ActionController::Base.consider_all_requests_local = false

This additionally tells Rails to stop sucking and stop handling requests to localhost and 127.0.0.1 differently.

Anyways, happy coding.

FCGID timeout causing Rails 500 errors

The default timeout for FCGID is pretty low I think (maybe 5-10 seconds). This was causing me SO much grief that I was just about to lose it. I found these posts out there that helped a lot:

http://wiki.rubyonrails.org/rails/pages/Debian+mod_fastcgi+Notes

http://weblog.rubyonrails.com/2005/01/03/watch-for-huge-requests-on-default-fcgi

Of course Ubuntu uses different config locations and such for FCGID, but the configuration names are the same.

I set my timeouts to 2 minutes just to be safe. Besides for an admin application, I could wait for a long time without thinking things were amiss. Anyways, help that helps prevent some folks from losing their minds.

RoR plugin woes

I’m been doing a lot of work with RoR plugins trying to make things generic. The RoR plugin mechanism is just plain horrible. Plugins do not work like most ruby classes because they are loaded in a slightly convoluted way and most errors end up getting swallowed by RoR. Of course this can be fixed, but it is really annoying and makes programming even more stone-age than it should be. You end up with errors like this:

/usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:123: in `const_missing': uninitialized constant AjaxValidation (NameError)
        from /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:133:in `const_missing'
        from script/../config/../vendor/plugins/ajax_validation/init.rb:3: in `load_plugin'
...

The issue here is NOT that AjaxValidation isn’t defined. Instead, it is that in the definition of AjaxValidation an error has occurred and this caused the load of AjaxValidation to completely fail AND there is absolutely no logging of the error that caused AjaxValidation to fail. So, you resort to writing to stdout each line of code to figure out where the problem is. That sucks.

Another issue I have with plugins is that I cannot define standard action methods in a plugin. There should be a mechanism for doing this. The issue seems to be that how RoR is handling action method invocation is not standard reflection because this would mean that all methods on the object could be actions. Instead it only allows the methods defined in a controller to be called. This makes sense, but makes life hard on plugin developers.

Ruby Locale 0.1

I wanted a ruby version of the Java Locale object and couldn’t find one, so i created a project for it. Here’s the address:

http://rubyforge.org/projects/locale/

It is pretty minimal, but it does contain all the current ISO country and language codes as well as a simple class for accessing them. It is also in a rails compliant plugin layout so it can just be dropped into the rails vendor/plugin directory.

The unit tests are failing, but it is just a numbering issue, which I’ll fix this afternoon or next week.

Enjoy.

Configuration vs. convention

My buddy Dave Thomas has an interesting blog entry concerning convention vs. configuration. I’ve given it a considerable amount of thought today because it is something that hits you square in the face when switching between frameworks and languages (i.e. WebWork to Rails and back each day).

Both have upsides and downsides and nothing seems to really bring one out as a front runner. Dave’s point about ramp up time is completely accurate. But those configuration nightmares also have that same issue right? No one is really going to memorize what a WebWork validator XML file looks like. Who wants to do that? But they also suffer from double work. The work of coding and the work of configuring, which makes it difficult to compare with convention based programming where configuration is removed.

The major problem I see with convention based programming is always constraints. Of course Rails and Ruby allow you a lot of freedom to modify classes and intercept method calls and stuff like that, which alleviate some of the constraints, but there are constraints. For example, in Rails it is difficult to externalize an action chain and reuse actions for multiple events. Something like this:

Action #1
one -> two -> three

Action #2
four -> three -> five

If each of the steps are atomic units of work and simply state their outcome as success or failure, this makes sense. You can reuse three because it doesn’t know who it chains to. The chaining is externalized into some XML configuration somewhere. You can pull this off in Rails using YAML and some hacking around inside your controllers, but it can get ugly quick. In something like WebWork, this can be handled entirely in the XML configuration without the code changing at all.

Dave’s second point about maintenance seems to me to be an issue with language rather than framework. Creating crazy method-missing handlers and classes that are defined in 15 files are features of languages and this is where I see the cost of maintenance is incurred. Rails itself doesn’t really let you tweak too much. It seems to me that Ruby let’s you tweak just about anything. Of course, not being an expert in either, I won’t speak definitively about that.

Dave’s last point about Perl I’ve kinda covered already. This happens with configuration and convention. However, with IntelliJ and configuration, you get some pretty good help and code completion that can really make the re-learning much faster. You still have to recall how to code on the configuration side as well as the convention side and nothing can really make that less teadeous and error prone except documentation and good exception messages.

I’m not certain that I fully grok meta-frameworks per se. But I will say this, I have developed systems that are both configuration and convention and they always seem to make me happy. To this day I’m still floored at how few frameworks do this and even more floored and how few people have used mine even if they realize that it supports both styles. The JavaLobby link on Dave’s blog entry covers this a little bit but I think folks still lack the understanding of configurable/overridable conventions.

Take for example the Savant 2.0 alpha I released. This system I love for its use of convention and configuration. If you want to get up and running fast, you can run Savant to create a project layout for you (or just follow the convention for layouts and build one yourself) and that’s it. You can now build, clean, jar, deploy, whatever. The standard layout (the convention) is like this:

src/java/main = The main source files that will be included in the JAR
src/java/test = Unit tests for the source files (not included in the JAR)
target/classes/main = The location where the main source files are compiled to
target/classes/test = The location where the test source files are compiled to
target/jars = The location where the JARs are placed after they are built

Its simple, straight-forward and works with absolutely NO configuration. This is where it gets good though (and personally I feel really Savant excels past Maven and Ant). Let’s say you can’t use the convention or need to tweak it. Let’s say you have this project layout:

src/java/library-1
src/java/library-2
src/java/shared

Now the Maven folks will say, “that’s three projects with separate JARs” and perhaps it is, but you can’t change it right now or you can’t change it ever and really would like a nice build system to help you out. Well, Savant to the rescue:

java-project.xml
--------------------
<java-project>
  <directories id="library-1">
    <source location="src/java/library-1"/>
    <output location="target/classes-1"/>
  </directories>

  <directories id="library-2">
    <source location="src/java/library-2"/>
    <output location="target/classes-2"/>
  </directories>

  <directories id="shared">
    <source location="src/java/shared"/>
    <output location="target/classes-shared"/>
  </directories>

</java-project>

This configuration overrides the default project structure and allows you to define as many source directories, build directories, test directories and JAR files as necessary. It even lets you define which JDK to use to compile each one if you want.

The moral of my story is that I think frameworks should always work based on convention right out of the box with as little configuration as humanly possible. Then if you want to tweak, slap down some configuration and go to town! Perhaps this is the meta-framework that Dave is talking about or perhaps not. Either way, this to me seems like a good solution that reduces the overhead as much as possible while still allowing the flexibility to tackle tough situations and problems.

New Inversoft Product

I’ve finished launching my latest Inversoft product – The Inversoft Bad Word Database. The website is written entirely in Ruby on Rails and uses PayPal for purchases. All and all the coding was extremely straight forward and simple. Here’s the address:

http://badwords.inversoft.com

The product idea essentially sprouted from my work on https://www.naymz.com where I need to add some filtering logic to the site so that folks could not enter any restricted words (cuss, slang, swear, superlatives, derogatory, etc.) when constructing their ad for the search engines. Google, Yahoo and the others reject advertisements whose content is not valid or contains restricted words. As I built out the functionality for Naymz, I wanted to test it with some data. Unfortunately there was no data to be found.

I did quite a bit of searching and found that there were services out there who offered a web service you would sent the content to and they would validate it. This just didn’t make sense to me since I wanted to have as few external dependencies as possible and most of these sites could not guarentee reliability, up-time or accuracy of the content filtering. I also found a few other sites that had lists of words, but they were not in a database friendly form. These websites also suffered from the fact that some words I would not consider “restricted” at all and yet again the data would need major scrubbing. Another site I found offered hundreds of thousands of words entered completely by its users. This was probably the worst considering that I had no way of making a determination of which words I wanted to load and which I didn’t because as we all know users will add anything when there aren’t any checks and balances.

What I really wanted was a list of words, with definitions, ratings and categories. So, in my spare time I started building this out. I surfed the web and found all the words I could. I started adding definitions to them as well as a ranking system that would allow applications to adjust the level of leniency they wanted to allow. I also found that many times a list of alternate spellings was need because folks are smart and can beat filters by typing something like sh7t.

After quite a bit of data scrubbing, tweaking and coding, I finished the website and did an initial data load. The initial load contained approximately 500 words and another 500+ misspellings (or something close). The website is live and I’ll be adding words to the database each day and also be building out an multi-language version that contains bad words from as many languages as possible. Check it out and feedback is much appreciated.

Rails with FCGID and puts

Just figured out something very humerous. I’ve just launched a new website that is written in Rails and I got it working with FCGID and Apache2. Works really well. During development I made use of print, puts, p and the other stdout printing functions to output to the console windows that WebBrick runs in. This works great considering that I haven’t really gotten too far into Ruby debugging and unit tests just can’t cover everything.

Needless to say, I should have been using the logger that is available to all ActiveController classes, but the amount of logging in the development.log file gets really excessive. Anyways, there’s is a slight issue with using the stdout methods when you convert over to use Apache and FastCGI or FCGID instead of WebBrick. The issue is that stdout seems to be redirected to the apache mod, which directly feeds back into Apache and then back to the browser. So, what could happen is that your print statements look like HTTP headers to Apache. This of course explodes violently in the Apache error.log file. Quite humerous. Perhaps a better solution is to redirect stdout to the logger or capture stdout in production mode and log errors to the logs or something.

Anyways, just something to watch out for.