Ada Lovelace Day

About The Authors

Suw Charman-Anderson

Suw Charman-Anderson

Suw Charman-Anderson is a social software consultant and writer who specialises in the use of blogs and wikis behind the firewall. With a background in journalism, publishing and web design, Suw is now one of the UK’s best known bloggers, frequently speaking at conferences and seminars.

Her personal blog is Chocolate and Vodka, and yes, she’s married to Kevin.

Email Suw

Kevin Anderson

Kevin Anderson

Kevin Anderson is a freelance journalist and digital strategist with more than a decade of experience with the BBC and the Guardian. He has been a digital journalist since 1996 with experience in radio, television, print and the web. As a journalist, he uses blogs, social networks, Web 2.0 tools and mobile technology to break news, to engage with audiences and tell the story behind the headlines in multiple media and on multiple platforms.

From 2009-2010, he was the digital research editor at The Guardian where he focused on evaluating and adapting digital innovations to support The Guardian’s world-class journalism. He joined The Guardian in September 2006 as their first blogs editor after 8 years with the BBC working across the web, television and radio. He joined the BBC in 1998 to become their first online journalist outside of the UK, working as the Washington correspondent for BBCNews.com.

And, yes, he’s married to Suw.

E-mail Kevin.

Member of the Media 2.0 Workgroup
Dark Blogs Case Study

Case Study 01 - A European Pharmaceutical Group

Find out how a large pharma company uses dark blogs (behind the firewall) to gather and disseminate competitive intelligence material.


free page hit counter



hit counter script


All content © Kevin Anderson and/or Suw Charman

Interview series:
at the FASTforward blog. Amongst them: John Hagel, David Weinberger, JP Rangaswami, Don Tapscott, and many more!

Corante Blog

Friday, February 23rd, 2007

Six Apart spins like a Whirling Dervish

Posted by Suw Charman-Anderson

I’ve refrained from blogging about Six Apart lately, because I have nothing positive to say about them or their products right now, but I’m afraid I can’t let their latest marketing email pass without calling bullshit.

I have spent the best past of the last four or five months listening to various friends struggling on a daily basis to keep Movable Type up and running. In fact, if you’re a regular reader then you’ll have experienced for yourself some of the problems that Corante have had with MT: the slowness, the failed page loads, the inability to post comments and, at one point, Strange’s total absence. I know of at least four large commercial installations of MT that have struggled - and, at times, failed - because Movable Type simply did not scale. (Although the new Rebuild Queue has helped.) I have personal friends who have had significant problems with MT, even though their sites are relatively small. And I have consoled more than one developer as MT saps their will to live, with significant bugs in 6A’s code being found and, eventually, fixed.

(Note: I am not going to name names, other than Corante’s - you knew about that anyway. Businesses in particular seem to be very wary of admitting when they are having software problems, but I am talking about household names both in the UK and the US who are having problems, and not small ones.)

With all this in mind, I find it totally disingenuous of 6A when they write:

We talk a lot about helping bloggers succeed with Movable Type, and that requires us to also focus on an important rule: Failure Is Not An Option.

You see, one thing Movable Type users often have in common is that, whether they’re writing a personal parenting blog for friends and family, or they’re publishing their opinion on case law for a law blog, they just can’t accept downtime on their blog. Fortunately, Movable Type was designed from day one to be super-reliable, standing up to the heaviest traffic load, even if you get linked to be a huge website.

This is nothing more than marketing department spin. MT is not super-reliable. If it was, then I wouldn’t keep hearing of yet another blogger who has abandoned MT, or another company that’s fighting to keep its MT installation going.

Six Apart is talking about MT as if it’s only used by individual bloggers, and that the only problem is when you get linked to by a big site. But whilst there are plenty of individual bloggers who are having problems, there are also business who have paid good money to MT for a commercial licence and are now finding MT to be a liability. And it’s not necessarily a big link that’s causing the problem, but fundamental flaws in the way that MT deals with spam and comments, and other bugs in the code that frankly should have been picked up years ago.

The spam problem, as I understand it, is that MT doesn’t differentiate between a spam hit and a proper comment until it has hit the database. It does the same amount of work in both cases, and the only difference is where that comment eventually turns up: on your blog or in the junk folder. So if your blog is hammered by spammers, the database does the same amount of work as it would do if it were hammered by real commenters. Of course, a spambot can hit your database with more comments more quickly than a human being can, and that alone can bring a blog down.

I heard of one case where, every time a comment was made, it caused 250mb of data to be transferred between servers. Scale that up to 100s or 1000s of spam comments, and suddenly you have the kind of load that can melt a server.

So no. MT is not super-reliable, and it cannot stand up to the heaviest traffic loads.

Six Apart go on:

How does it work? Well, unlike most blogging tools, Movable Type supports two different ways of publishing your pages — it can look in your database and choose which posts to display each time someone visits your site, or it can just generate a regular HTML web page that gets displayed without having to touch your database. That’s what we’re talking about when we say Movable Type supports “static” or “dynamic” publishing — static publishing doesn’t talk to your database every time someone visits your blog, and it’s the default in Movable Type. We let you choose between both so you can set the right balance of performance and scalability. (Static publishing takes longer for you as an author, but less time for your readers — so if you’ve ever waiting for your site to “rebuild”, you can take some consolation in the fact that your readers will have less of a lag when they visit you.)

Aaah yes, the rebuild. They talk as if this is a good thing. The trouble with rebuild is it’s really not very efficient, and frequent comments cause superfluous tasks to be queued for the rebuild, so you end up wasting a lot of server capacity. God knows the number of times I’ve sat there, waiting for a blog to rebuild… and waiting, and waiting, and waiting.

If you have the very latest version of MT, you have Rebuild Queue, but if you don’t then it doesn’t matter whether your site is static or dynamic, the problem is total comment load, including both spam and valid comments.

Most other blogging tools don’t do rebuilds the way MT does, and I can’t think of another tool that I use that suffers as much from bugs and downtime as MT. Doing things differently doesn’t mean you’re doing them right and everyone else is doing them wrong.

Digg-proof

Now, if you have a huge farm of servers and lots of technical staff, you can make dynamic publishing work at very high traffic volumes, too. In fact, our LiveJournal team here at Six Apart invented a lot of the open source technology that makes that work — the people behind sites like Facebook and Digg and Wikipedia and our own Vox use it, too. But if you’re running on a regular web server at a standard hosting company, they’re going to get kind of annoyed if your blog is hitting the database thousands of times just because you wrote a popular post.

Most commercial installations don’t have big server farms, nor do they have lots of technical staff. Yet even if you do chuck a few extra blades and a couple of developers at the problem, it’s still difficult to make MT work in either mode, static or dynamic, if you’re being hammered by spammers. Again, writing popular posts isn’t the problem. Serving pages isn’t the problem. Comments are the problem.

Now, it’s very easy to blame the spammers, but the sad fact is that spammers aren’t going to go away, and tools have to be built to withstand their onslaughts. MT isn’t. It didn’t matter how many servers you threw at MT 3.2x, comment spam could still kill them.

Oh, and just to nitpick… all that lovely open source stuff from LiveJournal? Well, let’s remember that minor point of fact that 6A bought LJ for its open source goodies. No sneakily trying to claim credit for LJ, please.

You might’ve seen this effect already — ever check out a link that’s been promoted on a big site like Digg or Slashdot and been faced with a “database connection error” when you visit the blog that got Dugg? Well, Movable Type is designed to prevent you from ever having to face that problem.

I feel like a broken record. Spam, guys, spam. Not the Slashdot Effect. (For the record, I’ve noticed that the Slashdot Effect is nowhere near as strong as it used to be anyway.)

For more tips on how to make sure your blog is performing as reliably as possible, our community’s put together some resources:

* MT Wiki
* Performance tuning Movable Type
* Enabling FastCGI
* Movable Type System Architectures

MT was always a tool that you needed to have a reasonable amount of expertise to install. Then they made it a bit easier, so you didn’t need to have quite the developer chops that you used to. Now you need to be a developer again to make the damn thing work. Make up your minds, 6A. Either MT is a developer tool or a consumer tool - you can’t keep wavering between the two.

And of course, we haven’t yet achieved this goal of making blogs failure-proof. Some of the steps for making a Movable Type blog bulletproof are too obscure or confusing. So we want to collect your feedback on the questions and concerns you have about the reliability of your Movable Type site — if you’ve ever missed out on some page views or potential readers because your blog wasn’t reachable, let us know or briefly summarize your story on this Movable Type wiki page.

OK, so 6A haven’t achieved their goal of making blogs failure-proof, why spend five paragraphs claiming they had?

If they want to understand where the problems are, they should start offering some support instead of expecting the people they’ve let down put the time and effort into writing it all up for them on their wiki. I know of people who have paid good money for MT who have had to fight to get 6A’s attention for support - 6A have complained when people ‘don’t use the ticketing system’ when the ticketing system was in fact broken. Hell, I even know of companies that have had to fight to pay them for a licence to use their software as per their terms and conditions. What sort of way is that to run a business?

Give proper support to the people whose MT blogs are failing, and you’ll soon gather all the stories you need to figure out what’s screwed up with MT. Instead of asking us to put the effort in, why don’t you, for a change?

We’ll start blogging about the reliability stories we’ve heard, both where MT has held up under pressure as well as where MT didn’t do what you’d expect, and how to fix it. Until then, you can help by pointing us at examples of Blog Failure, whether it’s on Movable Type or not, and we can all work together to help solve the problem.

Frankly 6A’s marketing department should be given, at the very least, a strong talking to for this email and especially the first and last paragraphs. Why should we do your work? It’s not the consumer’s job to figure out what’s wrong with your software - that’s your job, and if you provided decent support you’d have most of the answers by now anyway.

MT 3.34, released on 17 Jan 2007, has helped a few of my friends and contacts, but they are still having to do significant work to get all the plug-ins installed and working efficiently. Spam is still a problem. FastCGI gives a perceived speed increase, but frankly is a bit like faking it.

And whilst Rebuild Queue helps, it comes too late for many individual users and large MT installations. In commercial settings, MT’s damaged reputation has rubbed off not just on other third-party blog-related tools, but also on those evangelists who championed blogs in the first place, obscuring blogs’ benefits with serious performance issues that blot everything else out. It also makes it much harder to sell other Web 2.0 applications because of the fear that they too won’t scale.

The truth is that 6A have dropped the ball. They abandoned MT and their users, and their lack of support and updates has caused significant problems for even those people who are paying to use the software. Instead of keeping on top of MT and ensuring that it can cope with a rapidly changing environment and increasingly sophisticated spammers, they’ve spent the last two years focused on Vox.

Personally, I find it hard to have faith in Six Apart’s commitment to developing, improving and supporting Movable Type, which is why I now advise clients to avoid it at all costs.

Technorati Tags: ,

Email a copy of 'Six Apart spins like a Whirling Dervish' to a friend

EMAIL THIS ENTRY TO A FRIEND



Separate multiple entries with a comma. Maximum 5 entries.



Separate multiple entries with a comma. Maximum 5 entries.





E-Mail Image Verification

Loading ... Loading ...

52 Responses to “Six Apart spins like a Whirling Dervish”

  1. Lloyd Davis Says:

    Wow! Put the fisking gloves down ma’am and step away from the body :)
    Great stuff - I smiled wrily at the irony of getting this e-mail the day after I had to make an emergency switch to wordpress.com because MT couldn’t cope with the spam issue you allude to.

    Not so sure about ’sophisticated’ spammers though - the idiots who hit me continue to ping trackback scripts whose name I’ve changed and ignore the resulting 404s (which of course added to the traffic problem).

  2. Anil Says:

    There’s a lot to respond to here, and I can’t go point-to-point (candidly, I think it’s a rather shoddy blogosphere tradition to fisk messages, because you’re necessarily taking things out of context) but I do want to respond to many of the issues you raise.

    First, I wrote that email. Remember me? We’ve met. I’m not a psycho, and I’m not a marketing droid, I just write what I really think. If I’m spinning, it’s no more or less so than you’re doing here. I do firmly understand and believe that you’ve had a bad experience — I would hope you see from the email that this is not only not what we want for you, but what we’re actively working to fix. As the message itself said, we’re not there yet.

    As far as our commitment to Movable Type itself goes, there’s no question that we’ve had a lot on our plates over the past few years, and MT hasn’t gotten the love and attention the community deserves. But instead of looking at the past, you might want to look at today: We’ve got the biggest dedicated development, support, and product team working on Movable Type that has ever been focusd on any installed blogging tool ever. And they’re not drifting off into creating bulletin board software, or writing wiki apps, they’re just focused on kicking ass with the most powerful blogging platform around. There have been four major releases of Movable Type in the past year, between the Yahoo hosting version, MT 3.3, Movable Type Enterprise and Movable Type Enterprise 1.5.

    With a bigger team, we’ll be able to do a lot more, and bring a lot of the features and benefits of MT Enterprise into the non-enterprise space. A company like Corante, that depends on its blogs, should absolutely be able to get first-class support from us. If that’s not the case, or if the company isn’t taking advantage of it, then please let me know. We’re happy to help troubleshoot the problem and implement a fix. But you say:

    “So no. MT is not super-reliable, and it cannot stand up to the heaviest traffic loads.”

    This is false. I can recite countless examples, from the Washington Post to the Huffington Post, but what we have here is a case of selection bias. So let me be clear: You’re having problem — I’m sorry for that, and we’ll help you fix it. But to extrapolate from your experience to generalize about the platform is absurd and unfair and frankly, I think it’s intellectually dishonest.

    Now clearly, there is some part of your configuration at Corante that isn’t set up to handle the load of comment spam that you’re seeing. And it’s absolutely our responsibilty (and our failing, if you want to point figures) that the best practices in that regard aren’t well-documented. But even following the minimum recommendations in the spam fighting guidelines linked in that email (http://www.lifewiki.net/sixapart/MTSpamFighting) would handle the vast majority of these problems.

    And then, “Well, let’s remember that minor point of fact that 6A bought LJ for its open source goodies. No sneakily trying to claim credit for LJ, please.”

    Sneaky? For mentioning that we’re learning from our coworkers? Ooh! You caught me! Movable Type itself actually predates Six Apart’s incorporation — more sneakiness! Six Apart is claiming credit for Movable Type!. C’mon, you have no qualms about giving credit to other companies for being open source, when they release less code to fewer users supporting fewer platforms that impact much smaller audiences, but we can’t mention the work that our team does? If Facebook and Digg are allowed to learn from LiveJournal and benefit from its platform, why wouldn’t I mention that we intend to do the same thing on our own platforms?

    “Why should we do your work?”

    That’s one way to look at a request for responses. Another way is to understand that many Movable Type users feel they’re part of a community, and that we recognize the community can do better work collectively than we could ever do on our own. We do put a lot of work into these resources, but we also give people the chance to give their feedback and insights as well.

    Hell — you’ve put a lot of work into this post, haven’t you? That counts for something.

    “If it was, then I wouldn’t keep hearing of yet another blogger who has abandoned MT, or another company that’s fighting to keep its MT installation going.”

    Part of the reason you hear about this is because there are other businesses using their resources to promote this message. I think it’s an absurd idea that it’s easier to redo all your templates and styles, pay more for a hosting account that can handle heavy database load, and accept database connection errors in lieu of spam load errors. I can understand why they make argument, but if it comes down to it, I’d prefer a system that fails when under malicious attack from spammers than one that fails when interested readers come by. Call me crazy.

    And you don’t have to take my word for for where we’re going with MT. The process is open. The next version of MT is being created with the guidance and feedback of the community. Daily builds of development versions are available. And we’ll actually listen to your feedback respectfully and take it to heart.

    Nobody is pretending that we don’t have a lot of work to do with Movable Type. But to criticize the platform for not being updated more frequently when we release new versions with more features more often than our competitiors is disingenuous. It’s completely fair to say that in the past we had a hard time focusing on MT while we built up our hosted services, but now that they are all launched and thriving, we can reinvest in MT — do you feel comfortable recommmending other platforms when they’re just starting to learn the lessons from dedicating all their resources to a hosted service?

    Here’s the bottom line — I understand why you may be reluctant to recommend Movable Type based on your own past experiences. I do think you should give us the chance to show you both what’s possible today (not the atypical experience you’ve having) as well as the roadmap to where we’re going in the future. There is simply no better choice than Movable Type for building a career around in blogging. I know it, because I see consultants and experts every day recommending MT. I see people putting out successful million-dollar proposals for their work, with Movable Type as the engine for their work.

    I’m not spinning, I’m telling our story. Call it “fuckwittery” if you must, but I’ve no reason to think that enhances your credibility, and I don’t particularly think it informs anyone about Six Apart’s intentions or actions. You say, “Doing things differently doesn’t mean you’re doing them right and everyone else is doing them wrong.” It follows naturally that, if your experience is different from that of many others, it might be worthwihle to see what you can learn from them. I’d like to help you do so, if you’ll do the same for us.

  3. Joe Clark Says:

    Anil, you can’t whitewash the problem by saying, in effect, “You may claim that a structural flaw in MT causes huge performance deficits, and you may even know of multiple sites with that problem, but trust me, I work here and MT is working *fine*.” MT demonstrably is *not* working fine. The perfect installations don’t matter, just as, say, how well MT works for nondisabled people doesn’t matter for people with disabilities. But that’s another sensitive issue that Six Apart would prefer not to address. There seem to be a lot of those.

  4. LSF Says:

    Nothing much to add other than the fact I also have clients - three in total - that are having issues with Movable Type. It’s not the spam - that can be dealt with using a number of plugins - but the rebuilding of the pages. This requires - IMHO - too much resource and needs to be rewritten from the ground up. RebuildQueue is nothing more than band-aid.

  5. Jesse Newland Says:

    The ironic thing here is that memcached (what “make[s] dynamic publishing work at very high traffic volumes”) could greatly help some of the rebuilding issues people are experiencing with MT in large installations, but it’s not currently possible to use memcached with MT due to some poor design decisions.

  6. Jesse Gardner Says:

    Anil said: “The process is open. The next version of MT is being created with the guidance and feedback of the community.”

    I can personally vouch for this. We’ve been invited to periodic phone calls with Six Apart staff where we’ve been able to make requests, grind axes and even just complain a little. There’s nothing cooler than downloading a nightly build to see your own suggestions implemented!

  7. Jesse Newland Says:

    Jesse G: These changes are great, but the license of wheeljack (http://code.sixapart.com/svn/movabletype/branches/wheeljack/LICENSE-BETA) explicitly forbids production use. How good are fixes if you can’t implement them?

  8. Fahim Says:

    I’m sorry but this sounds a lot like a rather poorly thought out rant. The biggest thrust of the argument by Ms. Suw seems to be that her blog (and those of others) can’t handle the spam load that’s thrown at it and so Movable Type is at fault for that.

    Sure, spam is an issue which is not going to go away but at the same time, you can’t simply expect a blogging platform to do all the work for you. I’ve used WordPress (heck, I switched from MT to WordPress) and that doesn’t (or at least didn’t at that point in time) have any spam combating measures out of the box. I had to write a plugin of my own, experiment and find the solution that worked best. I still found WordPress to be not quite what I wanted and that it caused too much server load because of the constant mySQL queries.

    Again, some of that is not WordPress’s fault - some of the mySQL queries were due to incorrectly written plugins and other issues. But there certainly were load issues. In fact, I’ve worked for web hosts who claim that most of their server load is from WordPress installations and so will look for WP installations when the load goes up. Does that make WP a bad platform? Of course not! It’s again a matter of knowing how to find the issue, tweak it and optimize things till you can get things working correctly.

    I switched back to Movable Type from WP because I wanted static page loading and to get rid of the mySQL load. I was happy to see that MT had some spam protection tools out of the box. But the tools didn’t do enough to help me with the spam issues. So I looked farther, searched a bit more and found more tools. Now, I hardly have any comment spam on my blog at all.

    The moral of the story? Just sitting back and complaining that something doesn’t work and it’s the fault of the code isn’t enough - you might actually have to do some digging to find the solutions :) Or maybe it’s that the viallian isn’t always who you think they are :p

  9. LSF Says:

    Fahim, Let’s not turn this into a Movable Type vs Wordpress discussion. That’s just a distraction from Suw’s main point - which is that Six Apart neglected Movable Type during the building of Vox and they continue to offer little support to customers experiencing real problems.

  10. Fahim Says:

    Actually LSF, if you read my comments closely, you will notice that this was not about MT vs. WP but the fact that Suw’s main point appears to be that MT is not doing anything much about spam. I didn’t see much about Vox and MT and a whole heck of a lot about spam and how it affected her blog and those of others she knows. My response was aimed at that. Of course, everybody reads what they want into a comment - including me :p

  11. Timothy Appnel Says:

    Jesse N: The database driver code in MT pre-dates mem_cache and the Live Journal acquisition. Not predicting the future correctly is not a bad design decision. I think its often forgotten that MT is over five years old and was first written during a time when PHP wasn’t common and standard hosting environments didn’t come with a database either. It seems hard to imagine, but its true.

    Anil elluded to the open development process, but didn’t mention any specifics. MT 4 is due out end of second quarter (give or take a month I suppose) and is being refactored to use the database driver system developed for Vox. That system (Data::ObjectDriver avilable from CPAN or Six Apart’s open source code repository) not only supports memcache, but database partitioning and a whole host of other database scaling goodness.

  12. Timothy Appnel; Says:

    LSF: “…but the rebuilding of the pages. This requires - IMHO - too much resource and needs to be rewritten from the ground up. RebuildQueue is nothing more than band-aid.”

    This happens a lot and in my experience (and I’ve helped lots of high-traffic weblogs using MT through my consulting work) most are self-inflicted and easily remedied. MT/Six Apart is not entirely blameless here. They have to make it easier to avoid and/or diagnosis these types of situations. Still, to blame problems that are mostly of ones own doing solely on the software vendor seems a bit unfair.

    The biggest performance drain I see in large and well established MT systems are “killer sidebars.” This is where a MT blog template (usually an archive) inserts something like recent posts or recent comments statically in the sidebar. Besides the fact you need to rebuild the whole site to keep it current, this causes MT’s template engine make repetitive database queries to produce the *exact* same content on every page it generates. This type of thing wasn’t apparent in the early days when blogs were mostly a hundred entries or less, but over the past 5 years many blogs are reaching the thousands, even tens of thousands, of entries and this bad idea is really starting to show why. See this post from nearly 2 years ago: http://appnel.com/2005/05/mt-performance-tips. (Start at the fifth paragraph, “Over on the ProNet mailing list…”) The difference can be quite dramatic. Six Apart changed their default templates to avoid these situations, but many well established blogs missed that and still have these killer sidebars in place.

    Another issue that has come up because of growth/age of blogs is category archives. MT creates an archive page for each category. Thing is, unlike a date-based archive, categories keep growing and growing. One client that I helped with their performance issue had a category with over 4500 entries in it. I think the lights dimmed at the server farm when it had to rebuild that one archive page which was often. The good news is that Andy Yaco-Mink of The Huffington Post released a plugin called CatCalendar (not what it sounds like) which lets you generate archives by category AND month.

    http://yacomink.com/CatCalendar/
    http://www.majordojo.com/movable_type/how_to_increase_advertising_revenue_with_movable_type.php

    For lack of a pages facility I’ve also seen blogs that have dozens of so-called index templates which are really static content like About Us or Contact Us pages. The problem here is that, being an index page and not switching off the autorebuild switch that is on by default, MT rebuilds these pages when a post is made thinking they need to be updated.

    There are some plugin tags that when implemented drag rebuilds down. MTPaginate is often identified as a performance killer amongst slow moving and maxed out systems. That plugin is known to suck up tons of CPU cycles and memory when implemented. I haven’t studied the code to be sure why. I’ve always found a way around it. With CatCalendars there is almost no reason for it.

    One last thing I’ve seen impact performance (or the perception of it) is the excessive pinging of services like weblogs.com. During a new post the publishing process has to make a request and wait for a response from each of those servers. Some of them are rather slow (weblogs.com was notoriously slow to the point where it would timeout) and add a few seconds or more to the process of publishing a post. Personally I shut all of those off and use FeedBurner’s PingShot service to handle it for me.

    My point is that like any tool, there is a right way and a wrong way to use it. One fellow consultant once equated some of these rants as the equivalent of buying a Ferrari and stabbing the tires with a switchblade then calling it a piece of crap because it does drive fast.

    LSF: If you are having problems and need some advice fixing problems there is a host of nice guys and girls that don’t work for Six Apart to ask.

    http://www.sixapart.com/mailman/listinfo/pronet
    http://tech.groups.yahoo.com/group/mt-dev/

  13. Timothy Appnel; Says:

    LSF: “…but the rebuilding of the pages. This requires - IMHO - too much resource and needs to be rewritten from the ground up. RebuildQueue is nothing more than band-aid.”

    This happens a lot and in my experience (and I’ve helped lots of high-traffic weblogs using MT through my consulting work) most are self-inflicted and easily remedied. MT/Six Apart is not entirely blameless here. They have to make it easier to avoid and/or diagnosis these types of situations. Still, to blame problems that are mostly of ones own doing solely on the software vendor seems a bit unfair.

    The biggest performance drain I see in large and well established MT systems are “killer sidebars.” This is where a MT blog template (usually an archive) inserts something like recent posts or recent comments statically in the sidebar. Besides the fact you need to rebuild the whole site to keep it current, this causes MT’s template engine make repetitive database queries to produce the *exact* same content on every page it generates. This type of thing wasn’t apparent in the early days when blogs were mostly a hundred entries or less, but over the past 5 years many blogs are reaching the thousands, even tens of thousands, of entries and this bad idea is really starting to show why. See this post from nearly 2 years ago: http://appnel.com/2005/05/mt-performance-tips. (Start at the fifth paragraph, “Over on the ProNet mailing list…”) The difference can be quite dramatic. Six Apart changed their default templates to avoid these situations, but many well established blogs missed that and still have these killer sidebars in place.

    Another issue that has come up because of growth/age of blogs is category archives. MT creates an archive page for each category. Thing is, unlike a date-based archive, categories keep growing and growing. One client that I helped with their performance issue had a category with over 4500 entries in it. I think the lights dimmed at the server farm when it had to rebuild that one archive page which was often. The good news is that Andy Yaco-Mink of The Huffington Post released a plugin called CatCalendar (not what it sounds like) which lets you generate archives by category AND month.

    http://yacomink.com/CatCalendar/
    http://www.majordojo.com/movable_type/how_to_increase_advertising_revenue_with_movable_type.php

    For lack of a pages facility I’ve also seen blogs that have dozens of so-called index templates which are really static content like About Us or Contact Us pages. The problem here is that, being an index page and not switching off the autorebuild switch that is on by default, MT rebuilds these pages when a post is made thinking they need to be updated.

    There are some plugin tags that when implemented drag rebuilds down. MTPaginate is often identified as a performance killer amongst slow moving and maxed out systems. That plugin is known to suck up tons of CPU cycles and memory when implemented. I haven’t studied the code to be sure why. I’ve always found a way around it. With CatCalendars there is almost no reason for it.

    One last thing I’ve seen impact performance (or the perception of it) is the excessive pinging of services like weblogs.com. During a new post the publishing process has to make a request and wait for a response from each of those servers. Some of them are rather slow (weblogs.com was notoriously slow to the point where it would timeout) and add a few seconds or more to the process of publishing a post. Personally I shut all of those off and use FeedBurner’s PingShot service to handle it for me.

    My point is that like any tool, there is a right way and a wrong way to use it. One fellow consultant once equated some of these rants as the equivalent of buying a Ferrari and stabbing the tires with a switchblade then calling it a piece of crap because it does drive fast.

    LSF: If you are having problems and need some advice fixing problems there is a host of nice guys and girls that don’t work for Six Apart to ask.

    http://www.sixapart.com/mailman/listinfo/pronet
    http://tech.groups.yahoo.com/group/mt-dev/

  14. dennis Says:

    “I heard of one case where, every time a comment was made, it caused 250mb of data to be transferred between servers.”

    I don’t disagree that MT is facing some performance and usability issues, but…

    As someone who deals with enterprise networks on a daily basis — and while we’re calling bullshit — the statement above requires some sort of elaboration.

    250Mb of traffic due to a single comment? If there were a serious software or network configuration flaw, perhaps. I could even buy a poorly written script/plugin causing such havok. However, the odds seem rather long that MT itself would be the lone culprit.

  15. tyler Says:

    I love it when someone writes something that seems a little aggressive, clearly passionate and perhaps insulting and naturally one of the proprietors has to get all defensive by the third comment.

    I know you don’t know me, Anil, but you’re probably better off just sharing a laugh around the office rather than trying to prove anything.

    You’re not going to change any opinions, instead you’re just going to make them all right. It’s not like you’re going to change anyone’s mind here, and this isn’t one of those “any publicity is good publicity”. No, you could’ve left this one alone and a lot less people would have probably seen it.

    Besides dude, if she wanted your help, she would have called you and asked for it.

    Just let it go man, let it go. It’s not worth it.

  16. Anil Says:

    “Just let it go man, let it go. It’s not worth it.”

    Well, this is a valid point to debate — do you engage with your community or not? I know Suw a little bit, and respect her opinion, and I see this as a conversation between peers about a topic we’re both passionate about.

    If you see it as a marketing person going out to defend a product that’s facing legitimate criticisms, then I’m sure some MBA handbook says “ignore it and try to sweep it under the rug”. I care too much about the work our team does, and about our reputation for being passionate and involved in blogging, to *not* be part of the conversation.

    And call me a hopeless optimist, but I do think Suw’s gotten some information from me that she didn’t have before, as I have from her. That’s the great thing about working for a blogging company instead of for a tire manufacturer or something — I get to really interact with people who care about what we’re doing. But yes, it might be *easier* to not respond. :)
    And I hope I wasn’t too defensive — I really do welcome people talking about what we’re doing right or doing wrong.

  17. Matt Says:

    “We’ve got the biggest dedicated development, support, and product team working on Movable Type that has ever been focused on any installed blogging tool ever. And they’re not drifting off into creating bulletin board software, or writing wiki apps, they’re just focused on kicking ass with the most powerful blogging platform around.”

    As someone that makes blogging software with probably not the biggest team ever, and also drifts off to make bulletin board software and wiki apps, I can safely say that it hasn’t had an effect on product success in the market. One of those drifting distractions was also an anti-spam system, which seems to have worked out well and is now one of the most popular plugins for MT.

    Anil, I have a lot of respect for you. MT actually has a lot of great things in its code, and I’m sure you can make its case without making oblique digs at open source projects and competitors whose entire company is probably smaller than your aforementioned product team.

  18. Jeremy Zawodny Says:

    Wow, I’m impressed. How were you able to configure MT so that it’s so unusable?

    You comments about static publishing seem completely, 180 degrees, off base to me. I’ve been using static publishing in MT since before MT had any other option and have never had reliability problems. And this is on a 4 year old server without and performance enhancing mod_perl or FastCGI stuff.

    When you’re using static publishing, MT isn’t even involved until someone posts a comment. What kind of pageview to comment ratio do you have, anyway?

  19. Rick Says:

    Just on a sidenote: I’ve looked at most mainstream blogging software including MT and WP for very high traffic sites. One consistent conclusion for all apps was that they don’t scale properly. All of these apps were never build for the kind of success blogs have today.

    It will probably take a major and complete rewrite to create blogging software that is as scalable and reliable as the blogging community currently needs it.

    I’m not dissing the software: the makers of the big scalable “enterprise grade” content management systems are still playing catchup with the creative, innovative and user friendly nature of most blogging software. The blogging features in most major CMS’s are a joke.

    But MT is facing a major challenge: keeping your own services up and running and scaling is one thing: building commodity software that scales by definition is a whole different ballgame.

  20. Jesse Newland Says:

    Tim: I still think it’s a shame to ship an “enterprise” web application without any way to implement a caching layer, especially when the authors of such caching technology are employed in-house.

    The “poor design decision” I was speaking of, which I think I’ve heard you agree with on pronet, was to implement the additional features in MT 3.3x and MTE before refactoring the codebase to use Data::ObjectDriver. It’s a herculean feat, but more important in my mind than an even Asset Manager, or some of the other things I’m seeing in my local wheeljack install.

    I appreciate your continued encouragement of Six Apart to implement Data::ObjectDriver on pronet and mt-dev - believe me, I would greatly benefit from it. I have a couple servers with 8GB of RAM just waiting…..

  21. Timothy Appnel Says:

    Jesse N: In a perfect world you would have taken your time shipping something so your engineering team could refactor all of the MT code to support those features, but things are more complicated then that. Unlike say an open source project (not knocking them Matt), a commercial business like Six Apart is balancing technical AND business issues. This is actually a gross simplification in that they are balancing probably 50 different things.

    Working in enterprise IT in some form for the 15+ years of my career they tackled the show stoppers first — LDAP integration and Oracle support. the one thing they missed IMHO in that release was Microsoft SQL Server support. The next version out a few months later addressed that in addition to external authentication other then LDAP and managing many users and blogs.

    Believe it or not, I’ve yet see any of the enterprise clients reach a scale (yet cause they will) where memcache or partioning is necessary. They are just starting to blog. I think sites like the Gothamist, Huffington Post and Seed Media who are running the standard version have a greater need for advanced database features then the Enterprise clients currently. That’s won’t last of course, but that is the way it is currently.

    “…was to implement the additional features in MT 3.3x and MTE before refactoring the codebase to use Data::ObjectDriver.”

    I think you may have misunderstood me then or I you. I’ve been arguing on Pronet and in private email (until MT4 plans started ocming out) that in order to take care of some of the core issues and shortcomings of MT that (like Data::ObjectDriver and memcache support) a complete rewrite in the form of MT4 needed to happen and the sooner the better. MT is an evolution of code that was written over 5 years ago. There has never been a redo or wholesale refactoring of the code. I think MT4 will pretty much be that.

  22. Craig Danuloff Says:

    As a long time MT user, I’m massively disappointed and disgusted with what they’ve done over the past 3-4 years, which is close to nothing. From a feature and ease-of-use point of view release after release adds almost nothing. Their documentation is virtually useless. The spam issues are terrible. They had such a huge lead and such a great reputation and goodwill, its been just sad to see it all squandered. I plan on moving to Wordpress very soon.

  23. Robert McLaws Says:

    You guys should move off MovableType and onto CommunityServer. It’s a much more solidly-designed engine than MovableType, and leverages things like object caching to keep the server from hitting the database every time, while keeping the benefits of a dynamic website.

  24. Anil Says:

    Matt, you’re right. I was frustrated (mostly due to air traffic control issues at O’Hare) and that was lame of me. Sorry.

  25. Phil McClure Says:

    “I’d prefer a system that fails when under malicious attack from spammers than one that fails when interested readers come by.”

    The results are the same using this argument. High traffic articles are going to draw spammers like a fresh paint job attracts insects.

    More interested readers yields more spammers which results in a crashed site that more interested readers have to pull from a google or coral cache.

  26. Vax Says:

    I helped run a hosting provider where a number of the clients ran MT. The load problems were so terrible we eventually had to blacklist the application, as well as disabling posts to mt-comments at the perimeter. At the time, I think MT’s solution was to “change the name of mt-comments to something else”, which is completely insane to think that average humans can process that. (First, SSH in…HA!)

    Movable Type is a *huge* liability to anyone that brings it in their systems, it is not worth the trouble or the potential costs. Wordpress, even though its developer community is beyond arrogant and often unhelpful to the point of pain, is open and scalable enough that it gets the job done.

    MT can say what it wants, but from someone who was in the trenches, I’d have to say the writer catches the essence of my frustration with MT quite well.

  27. Su Says:

    was to “change the name of mt-comments to something else”, which is completely insane to think that average humans can process that. (First, SSH in…HA!)

    Your company requires SSH to rename a file?

  28. LSF Says:

    Su: “Your company requires SSH to rename a file?”

    Well, that’s probably how most users connect to their machine nowadays. Telnet is generally frowned upon for security reasons.

  29. Fahim Says:

    LSF: “Well, that’s probably how most users connect to their machine nowadays. Telnet is generally frowned upon for security reasons.”

    Umm … I believe Su’s point was that most webhosts these days have a web control panel for their customers which also includes a web-based file manager. All you need to do to rename a file is to login to your control panel, use the file manager, browse to the file and rename it. Just as you would on your own desktop :p

  30. David Kaspar Says:

    Ms Charman, I disagree with your conclusions about MT.

    For me the static pages vs dynamic pages have a been a life saver in few instances of traffic surge.

    There was an annoying performance issue due to spam comments but a solution was just a Google search away.

    Renaming scripts from default, removing post button from first page and blocking (in)famous IP blocks and user agents in .htaccess has always been my first line of defense.

    After that, the webserver can serve static HTML pages without breaking a sweat.

    The occasional site rebuild causes a CPU spike but I try not to rebuild aggressively.

    Static, pre-built web pages FTW!

  31. Bernie Goldbach Says:

    I use Six Apart’s Typepad implementation of MT and believe it is the most efficient anti-spam blogging platform on the planet. It is heads-above the Wordpress anti-spam capability that I have to manage with my WP blog at podcasting.ie. I’ve also run several versions of MT and have endured hours of plugging the spammers from taking over the discussions.

    I would seriously investigate a hosted MT solution like Typepad because it’s saves me more than the $150 annual hosting charges I pay to use it on a blog containing 3900 posts, 4500 comments and 12000 pages that serve no fewer than 2000 page views every day of the week.

  32. Suw Says:

    Wow, I didn’t expect to kick of such a discussion. I want to respond thoughtfully, but I can’t right now because today I have to prepare a video for De Montfort University ready for tomorrow. I’ll come back and address a few points as soon as I can give it the time it deserves.

  33. Su Says:

    Actually, my point was that if someone just needs to rename a file, especially one of these users who wouldn’t understand SSH, they’d probably just use their ftp client, and grab the config file(which they’d have to modify) at the same time.

  34. Kevin Says:

    Just to clarify, until 3.34 and Rebuild Queue, the load issues that MT generated had nothing to do with static versus dynamic serving. And as Suw points out, 3.34 was released on January 17 of this year. This isn’t the distant past.

    The load issue for us was down to flaws in the object driver that generated redundant calls to the database and unacceptable loads if our system was dealing with even a few hundred comments per hour, whether the comments were spam OR valid. From reaching out to the MT community itself to find answers, I can say our experience was common, not atypical. And to blame customers for poor configurations seems a ‘blame the victim’ strategy.

    I do appreciate that Six Apart has rededicated itself to MT after focusing on other products for the last two years. In conversations with Six Apart, you have been contrite and helpful. But to send out a marketing e-mail like this felt like a slap in the face to those of us who struggled while Six Apart’s focus was elsewhere.

  35. LSF Says:

    Kevin, I agree. Perhaps Six Apart will offer the next major upgrade of Movable Type free of charge to the customers that have bothered to put up with the lack of development for so long.

  36. billg Says:

    I think it’s fair to consider every blogging platform that’s out there as first generation. None of them were built from the ground up to cope with today’s environment, which includes people attempting to use them as CMS tools managing large, popular, multi-author sites, and coping with torrents of spam.

    My own experience is limited to running personal blogs on MT and WordPress. That said, here some thoughts:

    1. Saying blogging software is the right place to stop spam is a bit like arguing that all those homes in New Orleans’ Ninth Ward should have been immune to the flood that washed over them when the levees broke. The blogging tool is the wrong place to fend off spam. That job ought to be done at the server and network layer. That means that if you are running a blog on a shared host, e.g., Dreamhost or Pair, then the primary burden of keeping spam away from you falls to that hosting company, not you. If you run your own servers, guess what? You get to be your own spam fighter. If spam is taking down your MT install, why blame MT? Blame the software that allowed the spam to get that far in the first place. Remember, a spam flood will take down WP or any other similar product, too, if you let it get that far.

    2. Changing the name of the comment script — a common recommendation for both MT and WordPress — is a bit of a waste of time. Whatever you call the script, it’s easy to find by simply scanning the source. The underlying problem is the use of (X)HTML forms to implement commenting.

    3. People who suggest using a hosted service and letting them deal with spam are on to something. However, I don’t want to put up with the restrictions of a typical hosted service. I’d love to see someone start a “comments-only” hosting service (yes, I know about Haloscan) that allows me to offload my spam problem to them. I’d pay for that. Let’s be clear: Once written, blog pages are usually static. They don’t really need to be stuffed into a database. But, comments really are dynamic, and it makes sense to store them in a database. If we could concentrate on the blog writing part and pay someone else to deal with comments and databases, wouldn’t we?

    4. 6A’s advocacy of FastCGI makes sense in that it’s a way to run an app like MT in a persistent Perl process. I don’t think that’s a magic bullet for personal bloggers because I don’t think FastCGI is as widely available on shared hosts as some might think.

    5. 6A has, over the years, made rebuilding more granular, i.e. less unnecessary rebuilding. It needs to be even more granular. Most suggestions for optimizing an MT site revolve around fiddling with the templates toward this goal. (Clear instructions for this are sadly lacking on the 6A sites.) MT itself needs to be clever enough to avoid rebuilding any part of a file that has not been changed. That means if I make a one-character edit in an 10,000 character file, MT’s goal should be to rebuild only that single character, leaving the other 9,999 characters, in effect, in place. (No, I don’t have a clue how to make that work.)

    6. The major credit for WP’s perceived ability to fight spam better than MT belongs to the Akismet service, not to the WP code. Akismet has been available for MT and other platforms for some time and works just fine. But, again, Akismet only comes into play when spam makes it to an individual form in an individual blog. We need to move the burden of spam fighting from the blog to the server and the network hosting the blog.

    7. Perhaps we are expecting too much from the commenting capability of blogs. If a site is so popular that its comments section becomes an active discussion board with hundreds of daily comments, then perhaps it is time for that site to consider running discussion board software.

  37. Rafe Says:

    Suw says: Wow, I didn’t expect to kick of such a discussion.

    This sort of exposes you as at best disingenuous or at worst stupid, doesn’t it? You write a highly inflammatory post about one of the oldest and most popular blogging platforms on a high visibility blog, and you think what, people will just ignore it?

    Lame.

  38. Jonic Says:

    Anil: Hi, it’s Jonic again ;)
    “A company like Corante, that depends on its blogs, should absolutely be able to get first-class support from us.”

    Hang on a second… Shouldn’t every MT user be able to get first-class support from 6A?

    Suw: Great article, I’ve only just caught my breath from bollocking 6A myself, but that was over Vox. I won’t link to the post for fear of looking like a spammer, but it’s a very recent entry on my blog :)

  39. Sketchee Says:

    Suw, trying searching the popular digg.com for stories about Movable Type. You may find the results–or lack thereof–interesting. (Also search for say wordpress…)

  40. Peter Says:

    “250Mb of traffic due to a single comment? If there were a serious software or network configuration flaw, perhaps. I could even buy a poorly written script/plugin causing such havok. However, the odds seem rather long that MT itself would be the lone culprit.”

    I am involved in debugging that particular installation but did not design it. That 250MB/comment figure came from looking at the reported bandwidth consumption across the WAN between MT and the database server and the comment rate. I assumed that I’d confused bits with bytes or seconds with comments when I worked that out, but no, it is correct.

    The core problem would appear to be the ObjectDriver and the way MT and plugins use it. It tends to result in unnecessary (and sometimes repeated) iterations over mt_blog and mt_author which might be fine for a single blogger writing on their own blog with a handful of their mates commenting, but it doesn’t scale so well on tens of blogs and hundreds of authors.

    Badly-written plugins don’t help either of course - CustomFields is particularly buggy and doesn’t integrate well with the rest of MT - but since the required functionality isn’t in MT one has to turn to third parties for it. Of course, some of the problems with plugins are caused by ObjectDriver not being expressive enough to avoid doing dumb queries.

    Yes, our templates could be rewritten and the site architecture redesigned to work around these scalability issues. This would reduce the amount of queries that eventually go over the wire. But if we’re going to have to basically redo the site from scratch, we might as well just do it with a different engine that doesn’t have the same limitations.

    P.S. what cretin “designed” that database schema? Would it have been too hard to mark primary keys as SERIAL in PostgreSQL so you don’t have to explicitly use NEXTVAL(’…’) in INSERT queries done outside of MT? Had they heard of referential integrity? UNIQUE constraints? Third Normal Form? No, thought not.

  41. Suw Says:

    OK… lots of comments, and I’m unlikely to be able to address every point, but I’ll respond here to the key issues as I see them.

    Anil: Yes, I remember you. I didn’t know you wrote that email because you didn’t sign it, but that’s irrelevant. This isn’t an ad hominem attack of you as a person, but a reaction to what I consider to have been a particularly ill-judged email.

    I stand by my point that I think you put an unsupportable positive spin on things by saying that MT has always been super-reliable from day one. I don’t think that trying to gloss over the problems that MT has had does 6A any favours - a bit of honesty and an apology for having dropped the ball would have gone a lot further to engender goodwill within the community of MT users who have struggled with your software.

    A marketing department is more than just you, but I think that you and 6A marketing department need to reconsider the way that you communicate with your customers and the tone you use when you respond to people on their blogs. I’ve read a few of the comments you’ve left on the blogs of people that complain, and I think there’s a gulf of difference between what I suspect you think you wrote, and what I (and other people I’ve spoken to) have read.

    I hope you really do a lot more to bring MT up to date now that you have done over the last couple of years, but again, the fact is that Corante hasn’t had first-class support, and nor have a number of other companies that I know or know of.

    Your selection bias accusation is a cul-de-sac. Many businesses having problems with MT that you will never hear about, because businesses just don’t like admitting in public that they are having troubles, instead they just either migrate to another blogging platform, or abandon it altogether. If my examples are inadmissible, then so are yours, and where does that leave us?

    I don’t want a system that fails under malicious attack, because malicious attacks are common, and if it fails under attack then interested readers don’t get to read. I would prefer a system that could cope with spam.

    Other people have said that spam should not be the responsibility of the blogging software, but that’s a bit like saying that Ford, or other car manufacturers, shouldn’t need to include suspension on their cars because potholes are the responsibility of the government or local council. Spam is a fact of life. It’s ugly, we all hate it, but it’s there. Reliable network level spam blocking would be makes life easier for us all, but we don’t have it yet, so tools that work in a spammy environment have to deal with spam. The alternative is the sort of unhappy user experience that I’ve been talking about.

    I don’t think other businesses are deliberately promoting an anti-MT message. Everything I think about MT is informed by either my own experience, or the experiences of people I know - I’m not a conduit for other blog vendors to preach anti-MT propaganda. This is my opinion, informed by my experiences and my conversations with other people. I don’t think that underhand jibes at Matt Mullenweg and Wordpress are fair, and I’m glad that you acknowledge that point in a later comment.

    You’re over-egging the pudding to say that MT has updated more frequently with more features than your competitors. I don’t see MT making any great innovations - I just see bug fixes that should have been done a long time ago.

    You ask if I feel comfortable recommending platforms when they’re just starting to learn lessons, and the answer is no. I recommend platforms that have shown themselves to be reliable, to have a history of ongoing development, and either good support or a supportive community. I recommend based on the client’s actual needs, budget, and my experience of specific tools.

    It’s clear that 6A has now turned its attention back to MT, but that’s after two years of inattention and lack of development. It’s hard to recommend a solution that languished for so long, by your own admission. You have re-establish your track record with respect to MT, in terms both of releasing a better version of MT and proving your ability to support corporate clients.

    Finally, I’m not here to inform anyone about 6A’s intentions - I can’t possibly do that. I’m here to tell people about my experience, and the experiences of those around me. You can, of course, choose to ignore me, but it seems from the volume of comments both on the blog and the ones I’ve had personally via other channels that I have hit a nerve. You might want to think about that for a bit, and re-examine your responses to people like me who are unhappy with MT and 6A.

    Jesse Gardner: It’s good to hear from someone from the community who’s involved in the next version of MT. But business shy away from nightly-builds, especially if there are licensing issues, and I can quite understand why. In fact, many won’t roll out alpha or beta software on their production servers at all, and I’m sure that’s a pretty common policy.

    It’d be great if 6A fixed some of the more serious problems with MT and released reliable and tested upgrades more regularly.

    Fahim: it would be nice to wish away spam, but sadly, it doesn’t work. Spam exists. And it’s just not possible for every MT user to write plug-ins - not everyone has that level of expertise.

    I do expect a blogging platform to handle spam and to do the work for me, the same way as I expect Gmail to filter out the vast majority of spam email. Spam does not have any real impact on my experience of using Gmail because their filters are so good and they handle the spam gracefully. That’s the sort of user experience MT needs to provide.

    In actual fact, my main point was not about spam, but the fact that 6A abandoned MT for two years whilst they focused on Vox. The MT user experience went down the pan, and they did nothing. To then claim that MT has been super-reliable since day one, and to talk up MT the way that Anil did, that really rankled.

    Timothy: If MT is over five years old, wouldn’t an update before it got too old and cranky have been a good idea? I’m glad to hear that something’s being done to improve that old code, but for many it will be too late.

    You’re dead right that when you have 100s or even 1000s (I’m not kidding) of authors, and thousands of posts, with tens of thousands of comments, and millions of page views, a system built for small-scale blogging can exhibit unpredicted problems.

    But it’s not just large-scale deployments that are struggling, and the five paragraphs of happy-clappy marketing speak that pretended all is well is MT-land were a red rag to a bull. It’s less like buying a Ferrari, stabbing the tyres and then complaining that it won’t run well, and more like buying a 2CV that breaks down all the time, only to have the salesman trying to pretend that its a Ferrari.

    Anil: I’d never advocate sweeping criticism under the carpet, or not responding. But I do think your response was defensive, and it left me feeling even more cross than I was when I got your original email. One of the reasons I didn’t respond immediately was because I wanted to sit back, chill out, and let all the kneejerk reactions just evaporate. If I had any one single piece of advice for you out of all this, it would be to reassess how you respond to people’s blogs and to be a bit more willing to use words like ’sorry’ and mean them.

    Jeremy: I’m glad you’ve had a good experience of MT. Am I right in thinking that Yahoo! got a custom version of MT from 6A, the Yahoo Small Business build, that included goodies not generally available? (Has the Yahoo! version now been released to the public?) If you’re talking about your Yahoo! experience, that might be partly why you have had a better experience.

    But either way, I think you’re misunderstanding what I’ve written. Serving static pages isn’t a problem, commenting is. And when you’re taking in hundreds of comments an hour - whether spam or valid - the load that creates on the database causes significant problems. Obviously, the more pages you have to comment on, and the more more spam you get the worse the meltdown, but I know of individual bloggers whose MT installations have just disintegrated under the weight of spam.

    So, just to wrap up a few key points from the comments:

    1) Serving pages isn’t the problem; comment load and poorly written code, including ObjectDriver, are the problems.
    2) There’s been a serious lack of customer support even for paying customers.
    3) Updates haven’t addressed these issues.
    4) Marketing emails which ignore or gloss over these problems with overly positive ‘messaging’ get right up people’s nose, adding insult to injury.

  42. Jeremy Smith Says:

    I posted some numbers regarding our MT installation here — http://blog.case.edu/jms18/2007/02/26/movable_type_scaling.

  43. Nik Cubrilovic Says:

    Anil: it’s funny because those guys might be “distracted” but they are producing damn fine code. Doing more than a single thing at once and solving problems you encounter yourself is what hackers do

  44. billg Says:

    I don’t have a spam problem with GMail either, because Google throws an awful lot of resources at the problem. No single piece of software, no matter how clever, is going to be able to provide the same level of protection. That’s what I meant by asserting that spam control needs to be seen as a problem for the network: Because individual pieces of software just can’t cope, and it’s foolish to pretend otherwise. Again, take away Akismet — a network tool — and think about the spam problem WP would have.The woods would be alive with the sound of MySQL crashes.

    I suspect 6A’s fundamental puzzle with MT — how to combine static pages with the need to change those already published files without constant file regeneration — is unsolvable. So, yes, they allowed MT to languish while they were busy elsewhere. But, I don’t think they could have “fixed” MT in any case.

  45. Ezra Says:

    As a MT-to-WordPress switcher, I’d have to guess that it took me less time time to move to WordPress than it took you to complain about MT. Shoot, at the end, it was taking me more time to hit “publish” on a post in MT than it took me to switch.

  46. Timothy Appnel Says:

    I don’t work for Six Apart and have had my own share of disagreements with them, but I think you went a bit overboard which is my problem with your post. What bothered me the most about you rreport and reply is that your criticizing technical issues you don’t see to understand or appreciate. As an end user I’m not suggesting you do, but if you are going to be a critic, especially as harsh a you are, you should.

    > I don’t want a system that fails under malicious attack, because malicious attacks are common, and if it fails under attack then interested readers don’t get to read. I would prefer a system that could cope with spam.

    Understood, but just know any PHP tool wouldn’t cope with spam well either if they were installed and run as a CGI.

    A few years back the Apache Web Server team decided to start bundling PHP with the core distribution. (I believe after MT was initially built, but I could be wrong here — the main point is PHP was NOT common 5+ years ago and CGI was.) That meant that not only was PHP spreading, but with a couple of quick configuration directives it was a built-in part of the server that was always running. The built-in nature of the PHP engine gave it peristence — its always running. No other programming language has the “defaults” advantage except for say ASP if you consider the Microsoft stack.

    So any weblog tool written in Python, Ruby or any other cross platform scripting language no matter how well coded could be immune to a spam flood unless the adminstrator takes the added extra steps to make it scale and protect it.

    In order to address this sort of thing, it has becoming common practice to use FastCGI. Other web servers (like Ruby fans favorite, Lighttpd) require PHP to be run with FastCGI in order to make web apps scale. I don’t think criticizing it’s use in MT as if it is some gross neglegence of Six Apart’s is completely unreasonable. I can understand criticisms that they weren’t fast and thorugh enough in their roll out of that support. That is true and they’ve heard that countless times from me that I don’t need to dwell on it. It’s here now and it works.

    > I do expect a blogging platform to handle spam and to do the work for me, the same way as I expect Gmail to filter out the vast majority of spam email. Spam does not have any real impact on my experience of using Gmail because their filters are so good and they handle the spam gracefully. That’s the sort of user experience MT needs to provide.

    I’d just like to point out that you are comparing a hosted service (GMail) with software that you host and install “yourself.” This is truly an apples and oranges comparison that indicates you don’t appreciate the issues in which you are speaking. Comparing TypePad and Gmail would be accurate and from what I understand TypePad’s comment spam handling is quite good. Both system put more resources and work into their spam protection then we could probably appreciate.

  47. billg Says:

    Ezra: In my experience, posting in WP is often slower than in MT. I’m sure this varies depending on the configuration of different MySQL setups. Still, I’ve seen WP time out during a post, but never MT.

    The truth is that both MT and WP (or any other PHP-based tool) will succumb to excess load at some point. The causes of that failure will be different, but they will fail just the same.

  48. Leslie Says:

    Anil: We’ve got the biggest dedicated development, support, and product team working on Movable Type that has ever been focusd on any installed blogging tool ever. And they’re not drifting off into creating bulletin board software, or writing wiki apps, they’re just focused on kicking ass with the most powerful blogging platform around.”

    lol. I just find this statement funny. Thanks for the laugh. Back to drifting.

  49. Timothy Appnel Says:

    http://www.crunchnotes.com/?p=357

    Every tool has it’s day.

  50. John Handelaar Says:

    Like Suw, I’m sick to the back teeth of 6A’s attitude to MT’s failings.

    One more time for the Panglossian Anil: It’s The Spam, Stupid. We’re drowning, and you’re doing absolutely nothing about the problem.

    Add to this a European representative whose response to my question about when if ever 6A was going to get around to working on *something* was to giggle around with his buddies and shout out loud across the room “Thanks for using the product” in the most sarcastic tone imaginable…

    …before getting drunk and barging one of us across a room…

    …and then (much later) hooking himself to a right-wing reactionary politician and forcing a conference-ful of people to listen to him drone at them…

    We’re basically done with this crap. The outfit in question dropped MT like a hadful of dogshit some time ago, and I’ve successfully prevented at least a dozen multiblog license sales for 6A since. Treat your customers like crap and just *see* what we’ll do back to you.

  51. Brian Despain Says:

    While I am a little late to this party, I find it truly amusing that you focus on comment spam and altogether ignore the failings that Wordpress has a platform (constantly hitting the database), along with the obvious security implications of shipping a rooted product.

    I will agree with you in one regard. MT was treated like a back burner development product but I was pleasantly surprised that 3.34 was such an improvement. I also expect that MT will improve faster than Wordpress this year. Furthermore your post simply isn’t fair to MT. They have built channels to gather feedback. You berate them for building the channels. You say this,”so 6A haven’t achieved their goal of making blogs failure-proof, why spend five paragraphs claiming they had?”

    It’s all relative right? You can work to improve a product and still have room for improvement.

    As far as your support claims, they are purely anecdotal and completely at odds with my own anecdotal experience. I have the support system at 6A responsive and effective. They have solved the problems with my installation MT quickly.

    As far as the Spammer problem, are you suggesting that Wordpress has no such problems.

    Here’s my take on the issue.
    http://thalasar.com/archives/movable_type_vs.html

  52. Veronica Says:

    I’m waaay late to the party but I just have to make a point. My (currently on vacation) blog is on Squarespace and I have never had spam of any kind. I’ve used both Typepad and Wordpress, briefly, and Squarespace is far superior, in my opinion.

    My blog has never been “down” that I’m aware of and I have complete control over who comments, who visits, etc.

    Squarespace has a free trial. My advice is to try it just to see how amazing a blog service can be. Even if you don’t switch because you’re “invested” in a service someplace else, you’ll at least know the level of quality that you should expect.

    My guess is that if you tried it, you’d switch.