19-Aug-2014, 10:15 PM
Quote:The CCLP1 voting packs had 1,647 levels total-were these picked by the staff or the community? How big of a drawing pool for those levels were there? Depending on the actual process, 1500 may not be close to enough for a "first cut". I will say that the CCLP1 voting packs have been quite manageable, and breaking up the sets into 50 level anonymized chunks seems like the ideal solution-if not very close to it. It just comes down to what method of generating them is wanted, and that's where I think there is no perfect way to do it.
If I counted correctly, there were 4,484 levels across the sets in the submission pool for CCLP1. What we did before voting was to narrow down the list by eliminating levels that fit into the following categories:
[1] Levels that were automatically ineligible by default: in other words, levels that were already in another CCLP, levels that contained some sort of inappropriate content in their design (yes, believe it or not, we had to deal with this), special-purpose levels that would not be appropriate for general voting like cyphers and credits lists, dummy levels, blank levels, levels that were superseded by newer versions in a newer set, levels created after the submission deadline, and levels that were specifically requested to be removed from voting by their designers.
[2] Levels that were incompatible with one of the two rulesets and could not be fixed without basically constructing an entirely new level.
[3] Levels that were determined by the staff to be too trivial, too difficult, of poor design quality, or any other characteristics that didn't make them a good fit for CCLP1.
As part of a check / balance system, we made sure that levels that fit into categories [2] and [3] had a chance to be nominated back into the voting process by non-staff community members who didn't want to see them eliminated. The grand total, after all was said and done, was the remaining 1,647.
I didn't get into the reasons for suggesting a cap of 1,500 in the previous post because, honestly, that could be an entire blog post. But I'll try to summarize my thoughts: whenever CCLPx is released, the disappointed designers who didn't get levels they wanted to see in the set are often told, "Don't worry; there's always CCLPx+1!" That is true - I mean, just look at the almost decade-long gap between Lazy Hourglass's original appearance in CCLP2 voting and its eventual induction into CCLP3. But after working on building a couple of CCLPs, I can say with some certainty that this is more or less the exception to the rule. CCLPs are generally a reflection of the levels and level design preferences of the time, which are always in flux as people come and go, and new official sets affect the tastes of players and the tendencies of designers. By the time the next set is constructed, you're dealing with new levels built after the paradigm shift that will more than likely garner quite a bit more praise than the material of yesteryear. Perhaps it won't be the case with CCLP4 because there are a lot of CCLP1-inappropriate levels that would be given more positive attention now, but this would be even more reason to take a good look at what already we have and, by extension, what we need for variety's sake. This is what's been done for another community-produced puzzle game I've been involved with: the designer put out a call for level submissions, then he gradually adjusted that prompt to account for the kinds of levels that weren't nearly as well-represented.
Don't get me wrong - I'm the last one to suggest that the CC community turn into a CCLP-producing factory rolling out a new set every several months. But I think we need to pay attention to just how fast levels are being made. More than that, I'd hate to see more people walk away disappointed after not getting something they made in an official set and then submitting the same material next time just to see it get overshadowed by newer levels and the "contemporary" design tactics of the day after so much time has passed. (Thankfully, the voting pack system can help with this a bit.) I'm not a fan of abandoning the formal submission / permission-granting process either, since increasing the size of the pool with each CCLP will just get more cumbersome for the staff and the community at large. The CCLP3 process is a perfect example of just how much more difficult it can be to manage the project when the submission process is open for so long and contains so many levels. Nor am I a fan of ditching the quality-based method of level selection for a designer "fairness"-based method, since doing so can easily compromise the quality of the overall product. So by process of elimination, that leaves limiting the submission pool as the only viable option to ensure that [a] the possibility of making it into CCLPx+1 is a bit more of a likely possibility, and the testing / voting / assembly processes do not take a ridiculous amount of time.