I have to admit, the magic is gone. For nearly a year, I was addicted to a very cool web service called Twitter. It was, and still is, one of the coolest ideas around: let you update your friends, family, and others of whatever you're doing at the moment by text message, IM, or the web. Unfortunately, as cool of an idea as it is, the service is also a testament to what not to do when designing a software service.
The service has experienced so much downtime in the last year that it's gotten to be more of a shock when I go to the site and it is fully available than when it isn't. While it's not usually the whole service that's down (that has happened) there is usually some kind of failure somewhere in the system that prevents users from enjoying the full Twitter experience. Last month it was the Replies tab that was disabled, this month it's seeing your timeline past the last 25 tweets. The Twitter team, through its blog, is always aware of the issues happening in the service but the repeated nature and length of the downtime seem to indicate that they either don't have the resources to fix the problems (they just got a ton of VC money, so that's not it) or they have fairly inept software developers.
I used to make excuses for Twitter. I used to tell people that developing software for massively scalable systems was really hard and that we should give Twitter a break. But, as a software developer myself, I know how hard developing that technology is and it's not nearly so hard that a small team of good developers armed with the right tools couldn't fix it in a few months much less nearly a year. MySpace did it, Facebook did it, and even Google, who's public consumption ranks orders of magnitudes higher than that of Twitter, has done it. I'm left to believe that, the reason the problems aren't fixed are because the developers are simply not qualified to fix them. It's a huge step from being a good developer to being a good developer of a massive system such as Twitter where several variables are out of your direct control. Such systems present an enormously larger problem and the issues aren't those that can be solved by 'average' or even 'good' web developers.
I also think Twitter has largely built its house on a foundation of sand by choosing Ruby-on-Rails (RoR) as its foundational development language. While Ruby is a beautiful language with a lot of positive advantages, its relative newness means a lot of general envelope pushing, a lack of experienced developers, and the lack of a lot of the niceties other, more mature (or more developed) languages have built in. Personally, I think the smart guys who run Twitter made a very stupid mistake by building it on RoR.
So, what do I believe could fix Twitter? I don't really think it would take a whole lot. But it would take a commitment by Twitter to scrap almost everything they've done and redesign from the ground up.
First, they need to abandon RoR and go for something more stable like C# or even Java. Personally, I prefer C# as it feels more modern than Java and offers a little more stability and ease of use. But either way, C# or Java, Twitter has to move away from RoR. Everywhere.
Next, they need to redesign their message queue. This is probably the most important part of Twitter (as it routes messages around the system) and it's broken. It doesn't work reliably and there's a lot of inefficiency They need to rethink the design of the queue with millions of users in mind and make it better and more robust. The current queue simply isn't cutting it.
Twitter also needs to open its mind to other technology. While I'm not a Microsoft fanboy, I've seen some impressively massive setups of SQL Server and Windows Server 2003. One thing Microsoft does very well is scalability. Twitter might want to consider, even with the additional cost, buying some non-open source Microsoft love.
Lastly, and this is probably going to be the most painful of all, I think Twitter needs to replace its developers. I'm not saying they're bad developers. But they have proven to be very 'wall gardened' developers, not thinking far outside the RoR box for the solutions they need. When your service is in crisis and facing mass user defection, you don't debate language ideology or evil vs good companies. You find technologies that will save your service and you use it. Twitter developers have utterly failed on this point and I think it's time to bring in some new blood.
Do I think Twitter can be saved? Yes. But I think it's going to take some massive overhauls and more work than many developers have ever done in their lives.
Is Twitter up to the challenge?
I'm just not too sure.