An evening with Jason Fried from 37 Signals

I went to the 37 Signals event last night sponsored by CPB. The speaker was Jason Fried, who is a founder of the company, a designer and all around smart guy.

Jason spoke mainly about small business. He presented a list of things that companies can do to become more productive and better at whatever they do. His list included things such as:

  • 4 day work weeks
  • Virtual companies
  • Saying no to your customers more often than not
  • Do the easiest thing possible
  • Don’t design, just do
  • Don’t go to meetings
  • Turn off email and IM

A number of these things I do each day including turning off email and IM when I’m really cranking, be as virtual as possible, reduce lengthy design and just start working. Overall, Jason has great ideas for small companies. 37 Signals is probably in the 3-5 million dollar range or lower, with about 2-3 million dollars of overhead or more. They are a life-style company and will not grow beyond a certain threshold because of these ideals.

Having worked for a large company, worked on vastly more complex systems than Jason, created many open source frameworks and now starting my own company, I found Jason’s ideas exciting and total crap at the same time. I regularly find myself battling with creating solutions that will scale, versus creating the simplest solution. This naturally occurs to anyone who has ever worked with applications that do hundreds of millions of transactions per day (or per hour).

I pushed Jason a bit to see where he would break ranks with some of his ideals. From his responses, he absolutely would not abandon these ideals at any cost. However, I think this is a marketing tool created by 37 signals to promote their products as well as Ruby on Rails. The reason I know this to be true, is that Jason is intelligent and capable or understanding moderately complex problems with more ease than say a college student. In addition, he spoke briefly about how a single feature request such as adding time-stamps to ToDo list items can have a larger impact than any customer might think. These features requests require a large amount of thought and planning because they impact the entire system. Similarly, changes to the Rails internals require a large amount of thought and testing to ensure they don’t break existing applications.

This dichotomy is something that is a fact of life. It is something engineers, product managers and executives must deal with, no matter how small or large your company is. 37 Signals has built a reputation that they can safely ignore these problems and be wildly successful. However, it is 95% smoke and mirrors. The reality is that 37 Signals is composed completely of phenomenal employees who can work in complex domains without losing their minds. The can distill complexity into manageable pieces and then combine those pieces in the end to produce a complex yet functional system.

Talkinator and profanity

Just read an interesting new I, Cringely about a piece of software named Talkinator. The technology seems just like a new IRC for the web with some APIs and Web 2.0 style, but that wasn’t what caught my eye. What caught my attention was that Cringely states that Taklinator doesn’t allow profanity. Being that Inversoft’s main product is a profanity filter, I had to try it out. It failed pretty badly.

Of course it catches your straight forward profanity like ass and fuck, but it does very little for anything more complex.

Here are some examples of stuff it doesn’t handle:

$hit
asslips
assface
phuck
fuk you
|)ick
sh it

That’s just a few, but the list goes on. I definitely think that people don’t understand the complexity that comes with attempting to filter profanity. It is much more difficult than it seems. I definitely think Talkinator should check out Inversoft’s Profanity Filter.

Restoring a SubVersion repository from an old backup

I have recently gone through the exercise of restoring a medium sized SubVersion repository from an older backup and wanted to share my experience with everyone. First, the problem:

The problem

After you restore the older backup, if any work was performed between the last backup and when the repository crashed, the repository will be “older” than what developers have on their boxes. Here is an example:

  • Let’s say you have a backup from Monday evening
  • The repository crashed sometime Tuesday
  • There was work done during the day Monday

Now, let’s say there is a project A that looks like this:

  • The last change to the project was at revision 100 just before the repository crashed
  • The last change to the project from the backup was 80
  • Frank’s computer contains a checkout of the project at revision 85
  • Mary’s computer contains a checkout of the project at revision 100

This means that both Frank and Mary’s computers contain newer code than the repository, but not the latest code. Mary’s computer contains newer code than the repository, but might not be a complete snapshot of what version 100 was before the repository was corrupt. The reason why Mary’s computer might not be 100% correct is that Mary might have committed files to the repository but not performed a “svn update” prior to committing.

SubVersion is like CVS in that each file contains a version number. So, you might have a local checkout that contains version 100 for one file and 90 of another file. Therefore, you might be missing an updated version of the file from revision 93 when it was checked in.

Okay, now onto the fix:

The fix

Each developer’s computer must be analyzed before anything new is put into the repository. You must have a complete picture of the entire company, otherwise you might miss some changes. These changes can be merged in by hand from each developers machine, but this could be error prone and lengthy process. It is usually better to script out as much as possible.

Step 1

In order to determine the local “revision” of a project on a developers computer, you will need to look at each file in the checkout. You can run an ‘svn stat’ on each file to determine the version number of that file. Write a script to output a file like this for each local checked out project on all developers machines:

src/java/foo.java: 90

The first part is the file and the second part is the revision of that file.

Next, once you have the complete list of revisions for all projects on all developer’s computers in the entire company, you can compare each file with the current revision in the restored repository to determine if the developer has a later version of a file than the repository. This should ignore all files that the developer has modified locally, but not committed.

This comparison will look like this:

Restored repository
src/java/foo.java: 80

Mary's checkout
src/java/foo.java: 90

Mary has a later version of foo.java!

You should script all of these comparisons out. If a developer doesn’t have later revisions than the repository or any locally modified files, they can safely take these steps:

  1. Make a backup of the local checkout
  2. Delete the local checkout
  3. Re-checkout the project
  4. Don’t do anything until the restore is complete

Step 2

The next step is to make a list of the revision that each file in the project was lasted changed on in the restored repository. This report will look like this:

build.xml: 10
src/java/main/foo.java: 16293
src/conf/main/logging.properties: 12093

Next, for each project, collect all of the revision reports from the previous step into a single location. These reports look like this:

frank-project-a-report.txt
build.xml: 10
src/java/main/foo.java: 16301
src/conf/main/logging.properties: 16405
mary-project-a-report.txt
build.xml: 10
src/java/main/foo.java: 16410
src/conf/main/logging.properties: 16321

Combine each of the developer reports into a global report that gives you the revision number for each file on every developers computer. Next, use this global developer report to determine whose computer contains the latest version for each file in the project. Based on the examples above, you can see that build.xml didn’t change recently enough to matter since the version in the repository is the same as the version on Frank’s and Mary’s computers. However, Frank made the most recent commit in revision 16405 to logging.properties and Mary made the most recent commit in revision 16410 to foo.java. Both of these files are more current that the repository and therefore need to be re-committed.

Step 3

Finally, setup a staging area that contains a copy of every checked out project from every developers computer who has a later revision than the repository (from step 1). This will be pretty large, but necessary. Based on the results from step 2, copy the latest version of each file over to a clean checkout of the project from the restored repository. Once you have all of the changes copied over for a single project, commit all of those files for that project back to TRUNK.

You should now have a fully restored repository based on the files from various developers computers.

Open source operating system quest

While waiting to see if my X300 sells on Craig’s list and before I eBay it, I figured I would go back to my roots and install a bunch of operating systems on the machine. It is very new hardware and I wanted to see what else is out there. For those who don’t know or haven’t ever read my white-paper, I wrote a fairly lengthy paper around 2000 that outlined how to build an open source operating system that was geared 100% towards desktop and laptop users. It covered mainly changes to Linux, but could be easily abstracted out to any operating system. It had things like:

  • Drop LHS (linux file system standard) in favor of a more friendly naming convention like /config, /system (or operating-system), /users, /applications, /libraries, etc.
  • Drop X-Windows in favor of OpenGL foundation with no networking and everything vector based where possible
  • Standard system APIs for everything, including graphics. No more Gnome vs. KDE vs. whatever
  • Better file permissions
  • Better login
  • Remove TTYs
  • Remove termcap and all that jazz
  • Assume latest modern hardware everywhere, even shell/terminals
  • Fix run-levels and services
  • Standard hardware abstraction
  • Better packaging (no more littering files everywhere)

It had a bunch of other stuff, but you get the drift. Some of this stuff has actually happened in the last decade or so. However, a lot of it hasn’t quite gotten there and Linux has suffered from more and more server syndrome that it probably will always be rough around the edges for desktops and laptops.

Anyways, back to the main point… I downloaded and installed a bunch of different operating systems and here’s what I found.

Linux (Ubuntu, Fedora, OpenSuse, etc)

Same old story. ACPI is rough, the CPU is a hog, the battery life is short, 3D desktop doesn’t quite work, missing drivers, bloated, etc, etc. Definitely a thumbs down on my scale, even though I am currently running Ubuntu on all my machines and love Linux to death, it still sucks for desktop and laptop compared to other OS’s.

OpenSolaris

Sorry to say that this is just like Linux with a different kernel and some minor tweaks here and there. It still runs X-Windows, Gnome and all the other Linux upper layers. Plus, it has worse driver support and very little modern desktop and laptop necessary support such as ACPI, CPU scaling, low voltage, etc. Definitely a thumbs down for now. We’ll see what Sun does. If they are smart, which they don’t appear to be yet, they would drop all the upper layers and build something better and new. They would also drop all the file system standard crap and just start fresh. They have the man power and the money to do it, just not the vision or the drive it seems.

Syllable

This is a very promising OS. They have a lot of the key components, but they have also been using Linux and Unix too long to deviate drastically enough to make it truly usable for the average Laptop and Desktop user. However, it still isn’t 1.0 and they might update some of these things. I never actually got it running, but from what I’ve seen on their website, they have the right idea. If I had some advice for all the developers and users of this OS it would be: hang a sign that says, “can my grandmother use this OS?” on your wall next to your computer and if that answer is ever “no”, fix things until it is “yes”. If I had a million bucks or so laying around, I’d definitely put my money on this project.

Hackitosh

Yeah, just wanted to see if I could get it running and the answer is no. Apple definitely has a great OS. Although some things are still lacking, it is probably the best out there right now. However, it runs best on Apple hardware and I’m not about to fight that. I’m planning on a 100% switch to Mac OSX here soon.

There are a few others that I didn’t get to like Haiku, but they seemed quite new as well and probably wouldn’t have worked all that well. Another thing that was lacking from most of the newest OS variants out there was 64bit support.

I can't sell a laptop on Craig's list????

So, some of you might have seen that my latest laptop purchase has been a complete disaster. I’ve decided to sell it and switch entirely to Apple hardware and software because at this point I just need an OS and machine that I can count on and is some form of Unix. The only ones out there that almost fit this bill are Apples. With Linux, there is always something.

I figured I would put it up on Craig’s list, save the fees and make it really affordable. Thus far my experience with Craig’s list has been absolutely horrific. Not a single person has responded to my posting that is actually in my area and NOT a scammer. I’ve received a number of emails asking me to ship the laptop to Nigeria after payment via PayPal, but those always sketch me out, even with PayPal for the money transfer. I’ve heard of folks using PayPal and then the credit card companies coming after them later for a refund.

It seems that people are still in the habit of using eBay for everything computer related. I find this sad because Craig’s list is much better on both sides because it is free.

As a last ditch effort, before I eBay this machine, I figured I would post my Craig’s list posting to my blog and see if anyone that reads my blog might be interested in the laptop. Here’s my Craig’s list posting for anyone that might be interested:

http://denver.craigslist.org/sys/688891861.html

X300 first impressions

Got my Lenovo Thinkpad X300 last week and I’ve been using it for development for only a few days. Here are my first impressions:

Pros

  • Very lightweight
  • Fast – the SSD drive seems to make some things pretty fast and the system is fast enough for the work I do
  • Nice screen
  • THE BEST KEYBOARD ON A LAPTOP (as all Thinkpads have)
  • The Trackpoint (never leave home row again)

Cons

  • The fan is the WORST PIECE OF CRAP EVER!!!!!

After much research and pain it appears that Thinkpads in general have a horrible issue with fan noise. Apparently, the Thinkpad BOIS pretty much refuses to turn the fan off and the fan goes directly from 1000 RPMs when cool to 6000 RPMs when hot. There is no intermediate fan speed. Well, needless to say that 6000 RPMs is extremely annoying.

I would definitely not recommend this machine to anyone looking for a solid work laptop because the fan noise is considerable enough that after an hour or two of working with that drone you are certain to have a monster headache and feel like throwing up. I’ll probably sell it and get an Apple instead. At least then I know I’ll have very little problems with the hardware.

Google Code could be EXTREMELY dangerous

I host a number of projects including JCatapult over at Google code. We use the wiki over there for our documentation because it is simple and centralized. The wiki is stored inside the SubVersion repository and when you update the wiki it performs a commit to the repository. Pretty straight-forward.

One of the project members, James Humphrey, was editing our wiki last night, finished editing a page and hit Save. Rather than just updating the wiki page in SubVersion, Google’s custom built SubVersion server decided it wanted to completely revert our entire project back to revision 1. Yeah, I’m totally serious!

Well, the old revisions appear to be in the repository, but in order to clean this clandestine (hehe) mess up I’ll have go in by hand and revert our entire repository. This consists of roughly 10 sub-projects and 5 tags for each project plus branches, etc, etc. Really nasty.

So, here is my warning to all those out there that might be using Google Code, be careful. I’m working with Google right now on trying to figure out what happened and how to fix it. I’ll update this post once we figure it out.

Directv media sharing

Okay, this took me a long time to get working and I wanted to set out my steps. The most important part is noted below and it is the step that nearly everyone misses and if you don’t do it, nothing will work.

  1. Ensure that you network layout is correct by plugging in a computer to the cat5 cable you will put into the Directv DVR
  2. Attach network cable to Directv DVR
  3. Reboot Directv DVR (red button next to card)
  4. Enable and test network using Setup->Network->Connect Now and also Setup->Network->Test
  5. Turn on the media server PC and install Windows Media Player 11
  6. Open My Computer and share your My Music folder by clicking “Share this Folder” on the left side
  7. Turn off Windows firewall
  8. IMPORTANT! Open My Network Places and click “Show icons for networked UPnP devices” on the left hand side
  9. Reboot Directv DVR again
  10. Open Windows Media Player 11 and click the menu under Library called Media Sharing
  11. Check the share checkbox and the new devices checkbox
  12. You should also see “Unknown Device” in the list and click on that
  13. Hit okay and you should now see the Music and Videos option on the Directv DVR menu

The new year

It’s 2008 and I’m hoping to start blogging more regularly again. To start, I figured I’d tell my readers my plans for 2008. Here’s the quick run down:

1. Continue building Inversoft’s Profanity products and increase sales

I did a lot of work last year getting the profanity database and filter working well. These products are slowly selling and Inversoft is getting more customers each week. I’m hoping to add some more advanced logic this year to reduce false positives even further.

2. Rebrand Inversoft and finish the website redesign

We have been working with the excellent designers from We Are Super Ordinary to rebrand Inversoft with a new logo and a new website presence. This should help us leverage our website as a good marketing tool for the business

3. Launch JCatapult and promote the heck out of it

JCatapult is starting to get to a 1.0 stage. I’m hoping to finish up the last few things in the next couple of weeks, including migrating SmartURLs into the Struts source repositories and make it one of the default plugins shipped with Struts. JCatapult will be based primarily on Struts2 and the SmartURLs port (called the Struts2 Convention Plugin). Once that migration is complete, JCatapult should be ready for a solid beta release.

4. Build a web application as a JCatapult example

I’ve had a few ideas and domains registered for nearly 7 years now and I’m going to try and build out one of these ideas using JCatapult. Hopefully I can find 3-4 developers that are interested in helping build this application in their spare time. Anyone who is interested, send me an email.

That’s the run down for 2008. I’ll try and post a few times each month on my progress, in addition to the standard technical ramblings I put up, which usually get more traffic than the rest of my posts. Time to get coding! Happy New Year!

GMail IMAP doesn't work for mailing lists (conversations)

Well, I think I’ll have to ditch GMail’s hosted email completely because of the way that GMail handles conversations (i.e. threads and mailing lists). After switching Inversoft and Pontarelli email to Google, I realized that GMail IMAP has a huge problem handling email sent to mailing lists. Apparently, if you send an email to a mailing list, the GMail MX server doesn’t place that email into your inbox when the mailing list sends it back to you. This is usually done by mailing lists programs like mailman and elmzm to allow you to track an email thread correctly. If you don’t have your own messages in a threaded view, you won’t be able to determine who is replying to you or what emails you have replied to.

The reason they do this is because rather than managing the GMail conversation view based on the emails in a folder, it instead manages it based on the emails in the folder and the emails you have sent. This is pretty obviously not following the KISS methodology and is probably best dropped completely in favor of a more traditional approach to threaded views. Thunderbird 2.0 provides a pretty decent threaded view of any folder and I’ve never had any issues. This is completely handled using only the emails in that folder and Thunderbird doesn’t rely on you sending emails from Thunderbird in order to handle threaded views. This is pretty smart since in many cases you can send email from an account via different SMTP servers and different programs or web applications.

Here’s the note I sent to GMail support about this issue:

This issue is known, but I wanted to let you know that it is really painful and would be a good thing to fix. The issue is that email I send to a mailing list via the GMail SMTP server or my companies SMTP server do not show up in my GMail inbox ever. The mailing list daemon is sending the email to GMail’s MX server, however, since the email was sent by the account that is receiving the email, it is never put into my inbox. Here’s an example:

1. Open Thunderbird
2. Write an email to dev@struts.apache.org
3. Send the message via SMTP server at newton.inversoft.com
4. Apache’s elmzm daemon sends the email to all the list members including brian@pontarelli.com
5. GMail’s MX server receives the email from Apache’s elmzm daemon addressed to brian@pontarelli.com from dev@struts.apache.org
6. GMail’s MX server drops the message and doesn’t deliver it because it is addressed to the account it is receiving the message for.

This is a huge issue with IMAP support and makes using GMail extremely difficult because I cannot manage threads correctly from Thunderbird since I cannot see my replies or the root message if I started the thread.

Please fix this problem because it makes the entire hosted GMail solution unusable for anyone that uses mailing lists regularly (90% of my mail comes from mailing lists).

If you need more information about the issue or help testing, please feel free to contact me. For now I will have to switch over to another email hosting provider until this is fixed.

I really hope they fix this soon because with the Google hosted business solutions this is going to cause MAJOR issues for businesses that use mailing lists for internal communication.