14-Nov-2013, 7:55 PM
Ok, so what follows is some sort of an explanation on the ranking system for this competition.
All six participating voters submitted a list of all 13 of the levels in their preferred order. If someone submitted an incomplete list, even that could have been used. For the most part I disregarded any votes a designer would have given his own levels, but had I needed that information for some ranking method, I would have considered them to have voted their own levels as the highest for balance.
I looked at each pair of two levels in the pool and pitted them against each other by counting how many of the voters who didn't make either level ranked one higher than the other. A majority here awarded a level with one point. So if three voters out of five eligible ranked Impedence
higher than Bricks
, one point went to Impedence
for being generally more liked than Bricks
. It didn't matter if all five preferred the level or only three out of five, and it didn't matter what the designer of either level thought. After all twelve pairings for a single level pitted once against each other submitted level, its score added up. This is the first number you see in with the results announcement. Impressively Shemozzle
won all of its 12 pairings!
But what about second place? There were two levels with the same number of points and even further ties lower down the list. For just two tied levels it might be possible to rank them according to which level won the pairing between the two of them, but even that might be a tie on occasion. So we take into account a form of standard deviation. If one level wins all of its matches really tightly, like 3-2, it could be considered less successful than one that wins all of its matches with a larger margin, like 4-1 or 5-0. What if points were awarded only for matches where a single vote could not decide the outcome? So this is basically the second number in the results announcement. It practically records how the scores would have changed for a level if one person who ranked that level decently high, had instead voted that level as the lowest. This results in That Ticking Noise
beating out Protect Me as I Protect You
for having clearer match-up wins, even though the match-up between those two levels was actually 3-2 in favor of the latter.
I think the last number in the results tells of how many ties a level was involved in, as this is better than out-right losing a match. After this the tiebreaker was to look at the match up between those particular levels. So In or Out
and Impedence
both had the same amount of match wins, and the same amount of match wins if someone had voted differently, and the same amount of ties in matches, but their own match favored the former 3-2 so it was ultimately ranked higher.
To finish off, I want to note that I compared the results of this ranking to simpler methods, and the top levels stayed the same in just about all of them, with some differences appearing lower in the rankings. I liked the method used because it seemed fair enough for all the levels no matter who voted on them, and it didn't require everyone to rank all the levels. The method did benefit from having a good number of entries to compare against each other, and it would have been even better if more people had ranked the levels.
The downside of doing any type of voting like this was that participants in the competition were forced to wait for the results. It also requires active voting, which truthfully can take some time. Hopefully this time the effort and waiting was worth it!
-Miika
All six participating voters submitted a list of all 13 of the levels in their preferred order. If someone submitted an incomplete list, even that could have been used. For the most part I disregarded any votes a designer would have given his own levels, but had I needed that information for some ranking method, I would have considered them to have voted their own levels as the highest for balance.
I looked at each pair of two levels in the pool and pitted them against each other by counting how many of the voters who didn't make either level ranked one higher than the other. A majority here awarded a level with one point. So if three voters out of five eligible ranked Impedence
higher than Bricks
, one point went to Impedence
for being generally more liked than Bricks
. It didn't matter if all five preferred the level or only three out of five, and it didn't matter what the designer of either level thought. After all twelve pairings for a single level pitted once against each other submitted level, its score added up. This is the first number you see in with the results announcement. Impressively Shemozzle
won all of its 12 pairings!
But what about second place? There were two levels with the same number of points and even further ties lower down the list. For just two tied levels it might be possible to rank them according to which level won the pairing between the two of them, but even that might be a tie on occasion. So we take into account a form of standard deviation. If one level wins all of its matches really tightly, like 3-2, it could be considered less successful than one that wins all of its matches with a larger margin, like 4-1 or 5-0. What if points were awarded only for matches where a single vote could not decide the outcome? So this is basically the second number in the results announcement. It practically records how the scores would have changed for a level if one person who ranked that level decently high, had instead voted that level as the lowest. This results in That Ticking Noise
beating out Protect Me as I Protect You
for having clearer match-up wins, even though the match-up between those two levels was actually 3-2 in favor of the latter.
I think the last number in the results tells of how many ties a level was involved in, as this is better than out-right losing a match. After this the tiebreaker was to look at the match up between those particular levels. So In or Out
and Impedence
both had the same amount of match wins, and the same amount of match wins if someone had voted differently, and the same amount of ties in matches, but their own match favored the former 3-2 so it was ultimately ranked higher.
To finish off, I want to note that I compared the results of this ranking to simpler methods, and the top levels stayed the same in just about all of them, with some differences appearing lower in the rankings. I liked the method used because it seemed fair enough for all the levels no matter who voted on them, and it didn't require everyone to rank all the levels. The method did benefit from having a good number of entries to compare against each other, and it would have been even better if more people had ranked the levels.
The downside of doing any type of voting like this was that participants in the competition were forced to wait for the results. It also requires active voting, which truthfully can take some time. Hopefully this time the effort and waiting was worth it!
-Miika