Why you should take Kicktraq trends with a grain of salt

Projections are now available. You should give them a try! More Info.

I’m seeing a couple of recurring questions in various discussions or project comments. As soon as a new, popular project shows up on Kicktraq that looks (at that moment) like it’s trending to do millions of dollars, one of 2 people show up:

"My Kickstarter will earn one meeeelion dollars"

Person A:

OMIGOD <insert project name> IS GOING TO DO TEN MILLION DOLLARS!!!

Person B:

That’s a shitty estimate. No way. / That isn’t accurately estimating what the real end-value is!!! / Oh god all the yellow on their site!!!

Person A:

I hate to burst your bubble (over and over and over), but sadly your project isn’t likely to keep up with the initial pace. Unless you’re in the same league as Wasteland 2 or DoubleFine, it just isn’t going to happen.

Lucky for you – DoubleFine caught the attention of other studios, so I fully expect more larger titles to show up on Kickstarter in the future. We’ve already seen awesome projects like Shadowrun Returns and Leisure Suit Larry popup this past week. I’ll wait patiently for a Wing Commander remake to popup.

Person B:

You are absolutely right, well at least about the estimation bit (I like yellow, ok – don’t judge). The trend line is not an estimation or a forecast, it’s just a trend that takes the current value and casts it over the remainder of the project’s funding period. Because of the way projects initially fund, especially popular ones, the initial few days are highly skewed when casting over the remainder of the project. This is why the accuracy of the overall trend is so off-kilter the first few days, but gets better as the campaign progresses – because it’s constantly being adjusted based on the current funding level. This is also why once it gets out of the initial pledge ramp, the projection will get more in-line with realistic projections. I’ve discussed this phenomenon extensively on BoardGameGeek in this post, and I’ll probably write a similar post here in the near future.

This is actually the reason I’ve changed all the labels from “projection” to “trend” recently – because that describes more accurately what it’s doing. In its current form, it’s absolutely a trend cast over the life of the project – not a realistic projection, not an estimation, not a forecast, and not even a guess at what it will end at.

So, why did it ever say projection? The simple answer is: it used to work! Unfortunately it’s totally borked now, and here’s why.

When I first started Kicktraq, I was using a weighted model that would scale the values of the daily contributions to try and guess a more accurate projection. When it was my little secret pet project for my own personal tracking of boardgames, it worked great. However, because I originally built the tool around data that I’d gathered for boardgames (because that’s what I was tracking originally, almost exclusively) I later discovered that it was waaaaaay off when I started adding other non-boardgame projects. So, a couple of weeks ago I removed the weighting and thought nothing of it – until this past week when I suddenly saw a traffic spike from some folks who must have seen some of my initial sample charts on BoardGameGeek started posting links to Kicktraq in the comments of the Yogventures! and the Wasteland 2 campaigns.

Needless to say, I’ve been scrambling to just get things working and behaving properly as I’d never anticipated the site getting much use until the Firefox add-on got approved. I was so wrong. Once people jumped on the site, they started figuring out that they could add projects and I’ve gone from tracking a handful of projects to over 200, almost over night!

Long story, short: don’t worry, I’m still adjusting my weighted model and hopefully once I have a larger sample of projects to build my model I’ll have more realistic projection functionality implemented soon! In the mean time, thanks for stopping by, and for all of your feedback and support!

Update:

If you’re new around here, you probably haven’t heard I’m working on new projection reports. They’re coming soon! Here’s a sample chart from Colossal Cave:

Click for larger image

I’m also working on a trend progression so you can see how the trend has moved over time and how it converges with the actual pledges. Here’s one for Ogre: