I’m seeing a couple of recurring questions in various discussions or project comments. As soon as a new, popular project shows up on Kicktraq that looks (at that moment) like it’s trending to do millions of dollars, one of 2 people show up:
Person A:
OMIGOD <insert project name> IS GOING TO DO TEN MILLION DOLLARS!!!
Person B:
That’s a shitty estimate. No way. / That isn’t accurately estimating what the real end-value is!!! / Oh god all the yellow on their site!!!
Person A:
I hate to burst your bubble (over and over and over), but sadly your project isn’t likely to keep up with the initial pace. Unless you’re in the same league as Wasteland 2 or DoubleFine, it just isn’t going to happen.
Lucky for you – DoubleFine caught the attention of other studios, so I fully expect more larger titles to show up on Kickstarter in the future. We’ve already seen awesome projects like Shadowrun Returns and Leisure Suit Larry popup this past week. I’ll wait patiently for a Wing Commander remake to popup.
Person B:
You are absolutely right, well at least about the estimation bit (I like yellow, ok – don’t judge). The trend line is not an estimation or a forecast, it’s just a trend that takes the current value and casts it over the remainder of the project’s funding period. Because of the way projects initially fund, especially popular ones, the initial few days are highly skewed when casting over the remainder of the project. This is why the accuracy of the overall trend is so off-kilter the first few days, but gets better as the campaign progresses – because it’s constantly being adjusted based on the current funding level. This is also why once it gets out of the initial pledge ramp, the projection will get more in-line with realistic projections. I’ve discussed this phenomenon extensively on BoardGameGeek in this post, and I’ll probably write a similar post here in the near future.
This is actually the reason I’ve changed all the labels from “projection” to “trend” recently – because that describes more accurately what it’s doing. In its current form, it’s absolutely a trend cast over the life of the project – not a realistic projection, not an estimation, not a forecast, and not even a guess at what it will end at.
So, why did it ever say projection? The simple answer is: it used to work! Unfortunately it’s totally borked now, and here’s why.
When I first started Kicktraq, I was using a weighted model that would scale the values of the daily contributions to try and guess a more accurate projection. When it was my little secret pet project for my own personal tracking of boardgames, it worked great. However, because I originally built the tool around data that I’d gathered for boardgames (because that’s what I was tracking originally, almost exclusively) I later discovered that it was waaaaaay off when I started adding other non-boardgame projects. So, a couple of weeks ago I removed the weighting and thought nothing of it – until this past week when I suddenly saw a traffic spike from some folks who must have seen some of my initial sample charts on BoardGameGeek started posting links to Kicktraq in the comments of the Yogventures! and the Wasteland 2 campaigns.
Needless to say, I’ve been scrambling to just get things working and behaving properly as I’d never anticipated the site getting much use until the Firefox add-on got approved. I was so wrong. Once people jumped on the site, they started figuring out that they could add projects and I’ve gone from tracking a handful of projects to over 200, almost over night!
Long story, short: don’t worry, I’m still adjusting my weighted model and hopefully once I have a larger sample of projects to build my model I’ll have more realistic projection functionality implemented soon! In the mean time, thanks for stopping by, and for all of your feedback and support!
Update:
If you’re new around here, you probably haven’t heard I’m working on new projection reports. They’re coming soon! Here’s a sample chart from Colossal Cave:
Click for larger image
I’m also working on a trend progression so you can see how the trend has moved over time and how it converges with the actual pledges. Here’s one for Ogre:
Click for larger image
A lot more neat features coming soon!
By Tomer April 10, 2012 - 9:29 pm
A suggestion for a simplified, yet smarter model:
why don’t you take the past trend into account?
Let’s assume a project earned 100$ 2 days ago, 60$ yesterday, and 27$ today. Sure, maybe tomorrow it would get 1000$ if some mark zuckerberg falls in love with it, but we would tend to guess it’d be more likely to be around the 18$.
Of course, there are other factors – the initial boost of excitement, the final boost of panic, and probably other stuff that might be considered… but without major complications one could really improve this over-simplified, linear forecast!
However – thanks a lot for the site and the idea!
Tomer
By Tomer April 10, 2012 - 9:37 pm
p.s. I haven’t read carefully enough what you wrote. You said you have in the past, but it failed… I do still think it’s more likely to give better results than the current technique.
By Adam April 10, 2012 - 9:40 pm
Tomer: You’re quite welcome!
I agree with you, there is a lot of room for improvement and I’m currently working on a few different ways to approach a more reasonable projection than a linear trend. Like I mentioned above, I have a few models I’ve used in the past but so far I’ve not found a universal one that doesn’t either punish small projects, or over-inflate larger ones, or skew projects that don’t follow the normal high-low-high curve.
Don’t fret, it’s a work in progress. Technically I’ve only been “live” for a few days so thanks for being so patient and taking the time to share your suggestions. I really appreciate your feedback!
- Adam
By Tomer April 10, 2012 - 9:57 pm
Thanks a lot for the (quick!) reply.
I’m really curious to hear about your ideas and models. How boring would the world be if everything were linear…
It’s very unlikely that you’d be able to find a universal model. As you’ve mentioned, smaller projects are almost definitely going to behave otherwise than larger ones.
Especially not when humans are involved (although god knows we all obey the law of large numbers, so we must consist of *some* logic!). However, if you feel like sharing your ideas or asking for opinions, feel free to write me. I have some friends who are immensely talented when it comes to such models and do exactly this for a living – so I might even pass it on.
I have actually done my bachelor in maths and physics and doing my masters in physics, but I cannot say I’m amazing at statistics
I’m currently following a project I’m personally interested in (Jane Jensen’s Studio) using simple excel charts and statistical calculations, but it’s only been 3 days so I hardly have gotten any enlightenment (other than being able to disprove linearity!)
Anyway good luck with it
By Adam April 10, 2012 - 10:01 pm
You may be quite right. I’m happy to have enough data to start working with so we shall see once I get some time to really dig in.
I’ll hit you up via email.
Thanks again!
By Tomer April 10, 2012 - 10:02 pm
you do that — thank you!
By darius4242 April 18, 2012 - 8:18 pm
Kicktraq is so much fun. I’m only missing one piece of valuable information, namely Average Pledge Per Backer. With this we could easily see how generous average pledger each project has.
By Adam April 18, 2012 - 10:00 pm
Consider it done!
By darius4242 April 18, 2012 - 10:51 pm
Great! Jane Jensen’s Pinkerton Road seems to have very generous backers indeed; $70 at the moment. Wasteland 2 got $48 in average from each and every one; amazing. But I’m getting carried away… Thanks for the extra info.
By DW Nelson April 18, 2012 - 9:44 pm
Best trending would seem to come from historical stats, no?
I.e., if current rocket ‘Pebble’ is about 20% of the way through their project time window, and judging from this chart:
http://www.kickstarter.com/blog/trends-in-pricing-and-duration
which is based on KS historical trends, they’ve raised about a third of their funding, so they should end up at about $13.5M
By John redman April 19, 2012 - 1:46 am
Pretty neat site you have here! I learned about it from a pledge that came from here! You may want to show optimistic and pessimentic ‘trending toward’ values. It would be neat to see a cone from the last data point to high and low trend values… Just a thought!
By Adam April 19, 2012 - 2:40 am
Thank you for the compliment! I’m happy to hear people are promoting projects after seeing them here.
Hrm. I’d not considered that as a possible view for the trend. Let me play with it and see if I can make something like that work.
By Kim April 19, 2012 - 5:01 pm
Not to spam, but if you want a Wing Commander remake, there’s something on Kickstarter right now that’s pretty darned close. Starlight Inception is basically a revival of the space sim genre, and it’s goal is to have world-building gameplay where the player’s choices in missions matter, so it’s like Wing Commander in a bunch of ways.
http://www.kickstarter.com/projects/732317316/starlight-inceptiontm?ref=live
By Adam April 20, 2012 - 12:37 am
Whoa… thanks for the heads up! That looks awesome! Trend isn’t looking so hot, but I’m in!
http://www.kicktraq.com/projects/732317316/starlight-inceptiontm
By Brian April 20, 2012 - 2:57 pm
What a great tool! Too bad your trending isn’t delivering perfect info yet — I kind of liked the idea of my own Kick Starter project (a doc about the history of Star Wars toys) earning so much money! Ah well…
I’m very interested in seeing how the funding trend for my project evens out as you get more data, and whether or not it matches my own best (but only slightly informed) guess. And then how they both compare to the end result. Science in real time!
Best of luck with the site’s continued development.
By Adam April 20, 2012 - 3:17 pm
Thanks Brian, and good luck to you on your project!
I can assure you I’m just as excited as you in regards to improving the projection model. I really enjoy the “figuring it out” aspect of making things better.
By John April 23, 2012 - 3:33 am
The “Probability Cone” approach sounds interesting. Another consideration is using a model that also considers past performance within the same KS category, or funding goal or duration? I’m sure each as a “filter” would provide some fairly fascinating comparisons. Maybe multiple views?
Not like I’m adding much work, or anything…
By Adam April 23, 2012 - 3:37 am
Ha!
The multiple views was another option based on different factors. I’m playing with different pieces right now and still debating on setting a permanent projection once, or a moving projection daily.