For anyone who is on the CCLP4 staff

[rockdet]

It's all suggestive at that point, I say. For example, I intentionally didn't submit levels that I thought were too hard for what I had in mind for CCLP1 (stuff like "Overlap" or "Dominion"), and it probably bit me in the behind, because some are actually super fitting in late CCLP1, but hey, I respect others and don't only think "Hey, I want 89 levels in the set! Let's submit all my stuff!"

As for voting itself, one's idea is not gonna change, just like CCLP3 and CCLP1 voting phases. We all have our general preferences in terms of a single level, and that stood out in the voting. The difference was that we actually did split them levels into categories for CCLP1, which allowed for more variety. And as long as the staff in place keeps doing that, we're safe.

[jblewis]

I'm going to post this in this thread since whatever eventual CCLP4 staff is created will most likely find this information handy.

I recently posted a blog post listing some reasons why I believe CCLP4 needs to be produced sooner rather than later. One of the reasons that I didn't list there is that keeping an eye on the submission pool can help ensure that the voting process isn't a cumbersome task for the community. I think the voting pack approach for CCLP1 did a fair job breaking up the pool into manageable chunks, but 1,647 levels is arguably pushing it. It's a lot to play and rate with care. We're also no longer in the CCLP3 age, where there was more of a gap between the number of submitted levels and the number of levels that were actually Lynx-compatible. Now, Lynx gameplay has become much more mainstream, and most designers have made Lynx compatibility a priority when building levels. For CCLP1, at least 4,484 levels were submitted. This means that 36.73% of the submissions actually made it into voting, with the remaining 63.27% eliminated on the basis of:

1. Incompatibility: possessing a significant difference in difficulty between the MS and Lynx ruleset, containing a known bust that makes the majority of the level a moot point, being unsolvable in one or both of the two rulesets, or being unplayable in the Lynx ruleset.
2. Ineligibility: appearing in a previous CCLP, being a credits / cypher / blank / dummy / other text level that would be out of place in an official set, being based on a CC1 level, or being disqualified at the designer's request.
3. Not for CCLP1: eliminated by virtue of low fun rating from staff or on the basis of being too difficult for a set targeted at beginners.
   

Of course, how that last category works for CCLP4 would need to be determined by its staff one day: should there be a difficulty ceiling for the set? Should there be a pre-voting elimination of levels that aren't enjoyable to play, are poorly designed by fairly objective standards, or are just plain trivial? No matter the case, there seems to be a higher consistency in quality among the levels in the pool this time around, and I fully expect that the percentage of levels that will appear in voting compared to the total number submitted will be higher for that reason - not to mention that the set's difficulty range will most likely not be nearly as restricted (at least from what most people are expecting). So with that in mind, here's a rundown of how many levels are in the pool or are expected to be in the pool one day before a pre-voting elimination takes place on the basis of the above three categories.

First, the sets that are listed on the CC Wiki's CCLP4 page:

• BigOto Returns by Trevor Hedges, containing 100 levels.
  
• Vitor 10 levels by Victor Tsukamoto, containing 10 levels.
  
• MyBest37-Lynx by Ian Wilson, containing 37 levels.
  
• BHLS2 by Ben Hornlitz, containing 70 levels.
  
• geodave4 by Dave Varberg, containing 44 levels.
  
• 60 Minutes by lookatthis, containing 60 levels.
  
• Pit of 100 Tiles by ajmiam, containing 100 levels.
  
• TS_CCLP4 by Tyler Sontag, containing 88 levels.
  
• JBLP1 by John Lewis, containing 149 levels.
  
• Extra_Extra by syzygy, containing 31 levels.
  
• C1059-1 by Chipster1059, containing 50 levels.
  

This totals 739 levels. Next, let's take a look at the sets that haven't been added to this list at the time of this post but have been submitted for CCLP4, either through a notice specifically stating so or a blanket statement providing permission for use in any future CCLPs. Of course, these could be superseded by future sets, such as those featuring a pared-down version of a set or an amalgamation of multiple sets submitted specifically for CCLP4.

• Alpha by Victor Tsukamoto, containing 47 levels.
  
• Arcade by Victor Tsukamoto, containing 16 levels.
  
• ArchieP1 by Archie Pusaka, containing 149 levels.
  
• BHLS1 by Ben Hornlitz, containing 149 levels.
  
• Chip56 by Henry Potts, containing 56 levels.
  
• Chip's Happy Day by Victor Tsukamoto, containing 28 levels.
  
• DanielB-Lynx by Daniel Bouwmeester, containing 101 levels.
  
• EricS2 by Eric Schmidt, containing 24 levels.
  
• History-Places by Ian Wilson, containing 60 levels.
  
• JacquesS2 by Jacques Smith, containing 148 levels.
  
• JCCLP2 by Josh Lee, containing 149 levels.
  
• JRB_CCLP1 by Jonathan Birchall, containing 38 levels.
  
• KTNUSA2 by Kevin Stallman, containing 45 levels.
  

This group totals 1,010 levels. Finally, let's take a look at the sets that haven't been submitted yet but most likely will be, particularly since many of these designers can be contacted or are active community members:

• Ida4 by Ida Roberthson, which contains 30 levels.
  
• JoshuaBoneLP by Joshua Bone, which contains 47 levels.
  
• JoshL5 by Josh Lee, which will contain 149 levels.
  
• MikeL3 by Mike Lask, containing 85 levels.
  
• The Other 100 Tiles by Andrew Menzies, which will contain 100 levels.
  
• Ultimate Chip 3 by Jeffrey Bardon, containing 149 levels.
  
• Ultimate Chip 4 by Jeffrey Bardon, which will contain 149 levels.
  

This group has at least 709 levels, and there are plenty of other sets that I didn't bother listing from designers who have made "best-of" submission sets in the past and will probably do something similar for this one. So all in all, there are at least 2,458 levels that will most likely appear in the pool so far, with many more to go.

To the future CCLP4 staff: you may want to consider putting a cap on this pretty soon, at least for the voters' sake, if not that of the designers who hope to be included in the final product.

[Ihavenoname248]

• Ultimate Chip 3 by Jeffrey Bardon, containing 149 levels.
  
• Ultimate Chip 4 by Jeffrey Bardon, which will contain 149 levels.
  

I can't speak for anyone else, but I was planning on picking through these sets (and UC2, but I don't expect much representation from there) and compiling anything I thought had a chance into one set specifically for submissions. Granted, this just means that instead of 298 it'll probably be around 250...but every level counts, right?

[M11k4]

Hmm.

That's a lot of levels already. Perhaps the staff needs to think about some other ways to limit the number of levels seriously considered for the set. Just throwing ideas around here:

-Limiting the number of levels submitted per designer, like 149 levels. A designer might not have the same view of their own levels as others, but they could still be able to pick which levels they would be most happy with ending up in a community set, or enlist the help of others to do so.

-Limiting the number of sets submitted by a person. I guess this isn't very different from the previous one and might create extra 'best of sets' but it might weed out something. Yeah, this isn't a useful idea.

-Limiting the people who can submit sets. It sounds unfair, but there could be a simple criteria that could be easily met. In practice this could aim to allow the more active and current members of the community would be the ones whose levels would make it into the voting pool.

-Limiting the types of levels allowed to be submitted. Quality is a hard thing to judge so we'd have to choose some other more specific criteria. It would be good for the set to have a clear goal in mind, like CCLP1 did.

-Limiting the method of how levels are allowed into the submission pool. One way to do this would be to require nominations of individual levels, rather than full sets. There could even be a limit for the number of nominations per person (though this might have some stalling effects due to wanting to see other people's nominations first).

Currently, it sounds like I could make a set with 500 levels of varying quality, ask for it to be included for consideration, leave the community, and come back years later to see how the set turned out and which of my levels made it into the set. This doesn't sound efficient or even fair to the people who would put a lot of work into compiling the set.

Happy Holidays!

[jblewis]

Perhaps the staff needs to think about some other ways to limit the number of levels seriously considered for the set. Just throwing ideas around here:


...

-Limiting the method of how levels are allowed into the submission pool. One way to do this would be to require nominations of individual levels, rather than full sets. There could even be a limit for the number of nominations per person (though this might have some stalling effects due to wanting to see other people's nominations first).

Currently, it sounds like I could make a set with 500 levels of varying quality, ask for it to be included for consideration, leave the community, and come back years later to see how the set turned out and which of my levels made it into the set. This doesn't sound efficient or even fair to the people who would put a lot of work into compiling the set.Happy Holidays

You're absolutely right, Miika. I think your last suggestion is probably the best. It would help pare down the voting pool into something that's reasonable for <i>both </i>the voters and the staff. The staff could set some kind of rule on this, such as two nominations from people who aren't the designer of the level. Plus, this saves the staff a lot of testing work. Rather than comb through every single level in every single submitted set for busts and compatibility issues, the pre-voting testing can be limited to the levels that would actually appear in voting. It would also help quite a bit with respect to levels that do have these issues, as they'd be included on the condition that they'd be fixed by virtue of community nominations, not staff decisions that encompass other eliminated levels. (The blacklist / whitelist / marked-for-blacklisting list system in CCLP1 was a bit confusing for many, I think.) Of course, the one drawback is that there are some sets with quality material that may not be nearly as popular and would not receive near the amount of nominations, though this could easily be remedied by the staff keeping tabs on what's hot and what's not and nudging the community to check out what's neglected.

[ajmiam]

Hmm, I'm not sure if requiring every level to be nominated by 2 others is the best idea, just because of the sheer number of nominations it would require--to get a 149-level set would take 298 nominations! Also, some sets are almost certainly played more than others, and I haven't had time to play most of the recent sets out there. What if the level designers were able to choose a modest number (maybe 50 or even just 25--or lower if you think even that's too much) of their own levels to submit for voting, and more could be nominated by the community using the 2-other-nominations rule? (Perhaps the designers could make recommendations for what the other players should look at, but it'll be the other players' decisions ultimately.) This would seem to cut down the burden of having to play a ton of other people's levels in a short time to give them a fair chance, when they'll have to be played anyway to vote.on them. Also, it would be the "best of both worlds" between promoting designers' levels that they'd like to see in the CCLP, and levels that are loved by the community despite the designers not expecting them to be so well-received.

Note: I meant "designers" as "level designers", not the CCLP staff.

[jblewis]

I think this would be even more reasonable. It addresses the everyone-would-have-to-be-familiar-with-most-everything issue. Plus, having a number like 50 levels for designers to choose from their own submissions would strike a balance: it would force the designers who crank out a lot of content to cull their submissions down to the best (hopefully with some variety in there) while allowing less active designers fair representation. In the past, I haven't been a huge fan of the "everyone-is-entitled-to-a-certain-number-of-levels-in-the-set" approach because there are obviously some designers who are simply more consistent quality-wise and varied in their submissions. But something like this could work for voting under the condition that the community can nominate even more levels from a designer past the 50 (or whatever number we can agree on) as a check against designer bias.

[KeyboardWielder]

agreed

[Ihavenoname248]

I think this would be even more reasonable. It addresses the everyone-would-have-to-be-familiar-with-most-everything issue. Plus, having a number like 50 levels for designers to choose from their own submissions would strike a balance: it would force the designers who crank out a lot of content to cull their submissions down to the best (hopefully with some variety in there) while allowing less active designers fair representation.

Though this is true, it's still not a perfect solution. Though yes, I'm aware of the problem because it directly affects me, I'm not the only designer who would be in this position (Josh and Zane have similar amounts of levels available in the pool, though admittedly both already went through the CCLP1 process).

The potential "abuse" in the system of nominations is that designers could purposefully leave out levels they know would be nominated from their submission pool. This in turn favors the more well known (and by extension, older) sets. I'm not sure if there's a way to prevent this, unless we started with a nomination phase, then had designers submit 50 (though to me this number feels a bit low, I suppose it would depend on how the nominations go) and then open back up for a final round of nominations if there were any deserving levels that were missed.

Another potential change could be 50 or some % of the total number of MS&Lynx compatible levels by the designer, up to a maximum. This is probably more trouble than it's worth to calculate, though.

[jblewis]

Although it's certainly playing the system, I'm not sure I would call that tactic "abuse" any more than a designer lobbying for some of their levels to be nominated. Also, I'd contend that the older sets would probably not be favored nearly as much even if they are well known, as many have been subjected to previous CCLP voting processes that have, in many cases, pushed their best levels into official sets. The whole point of a method like this would be to make sure that the levels in voting are actually levels that the community would want to see in voting. So, if we're dealing with a designer who cranks out quality material on a fairly consistent basis, then great. There's a good chance that the material is fairly well-known, especially if the designer is that active, and the community can push or be persuaded to push additional levels into voting if need be. But there's also the other side of the coin as well: at the very least, if we're dealing with a designer who submits a thousand levels for CCLP4, and the vast majority of them are more than likely not going to make it into the set by virtue of being mediocre design- or gameplay-wise, at least they can get their 50 or so best in voting without the staff or voters having to worry about rating the remaining 950. This is also the reason why I wouldn't be in favor of a percentage-based system as well.