UPSB v3

General Discussion / World Cup '10 - Brainstorming/Information Thread

Zombo
Date: Sat, Jun 6 2009 06:04:13

I had the chance to work with Crash 2 years ago on the format of the inaugural World Cup. Although the next World Cup is still far, we still need to work on it early and make sure it's ready in time.

The purpose of this thread is to share ideas on how the next WC may proceed. Nothing discussed in this thread should be taken as real or final. I would also like to get as many feedback, comments, suggestions as possible from as many communities as possible. Spread this thread around your local community and tell them to send me and Crash your ideas. This is an international event, therefore its format should be enjoyable to everyone.

-------------------------------------------------------------------------------

IMO, these are the main issues that need to be addressed for the WC

- Communication: the main problem has always been communication. Because of the different communities, language has always been a restriction on the rules we could implement in the tournament. Not only does it limit what we can do in the tournament, it also makes it unfair for teams that don't understand the rules very well. There is also the problem of communication with the judges. Sometimes the judges themselves don't really understand how to judge due to language which results in poor results.

-> Solution: Two things must be done,
-- Full translation of the rules in all languages of participating communities as early as possible. This means we need to finalize the structure of the competition well in advance and ensure that every community has a translated copy of it. This is especially important for asian communities, since their english skills are often poor. Once we have those translations, they should be put on the main site and spread to all communities.

-- Managers who understand English. If you remember, each team has a manager which serves as a spokeperson for that team with the officials (i.e. Crash). In the last WC, we had trouble finding good managers and I remember the Korean team was temporarily managed by Crash himself. It is important for each community to start finding a competent manager who can communicate easily in English. However, that is not the only job of the manager. S/he must also take care of the roster and make important decisions during the competition. Therefore, it would be unfair to demand that the manager speaks English if there is a better candidate at the tactical aspect of the position. Therefore, the manager role could be split into two persons, where one takes care of the actual management, and another as the spokeperson. Ideally, one person can fulfill both roles. Also note that the manager himself could be a spinner in a team. Therefore, one spinner could be doing the actual managing and an extra person is included just to communicate with the officials.

- Judging: we've always had problems with judging. One aspect of it is related to the communication problem above. The other is having a good system. One of the problems is the use of criterion. Because we try to simplify the scheme so that it is understandable by everybody, we use very simple criterion and make their weight all equal. Sometimes a judge has a well-defined opinion of a battle but it doesn't translate very well in the score. Furthermore, the idea of summing up points means judges who tend to give big grade differentials have more power than judges who grade more tightly. One judge who gives a -4 relative score in a battle offsets 4 other judges who give +1 score. The real problem here is that there is no set standard as to what a score represent. There's nothing that says what a score 9/10 is compared to 6/10. Maybe for some judge 7/10 is bad, for another it means good.

-> Solution: There are many ways to fix this, here are a few ideas
-- Use voting: by voting, I mean that each judge's opinion count as 1 only. We remove summing of score. This eliminates any "big difference" bias. The only issue with this is that it doesn't contain any nuance. Maybe 2 judges voted for a spinner but they didn't really feel strongly about their vote, but another judge noticed a very bad mistake and makes him vote against the same spinner with a strong opinion. In this situation, maybe the spinner deserves to lose.

-- Eliminate criterions, use justifications: One idea would be that you cannot really quantify pen spinning. Therefore, judging is best described in words, not numbers. In this sense, judges should just write for each battle who deserves to win and why. The only problem with this kind of system is that it may give too much freedom to the judge to vote however he wants (possibility of corruption) and just write a random justification and say "that's my opinion" and there's nothing you can do about it. Also, the justifications should be written in the judge's own language and we can translate later (or english if possible).

-- Use a relative system: It is much easier to grade 2 videos relative to each other. We can use a +/- system to grade videos. We must make sure all judges understand the standards of the system and truly understand what a +1 means compared to +3 (for example).

-- Spinners give their breakdown: Maybe if the videos came with a breakdown, it would be easier for judges to analyze the combo in great details without spending an immense amount of time. The only problem is that breakdowns require a lot of time to write and might need translation.

- Number of communities: Last time, we had to merge some communities to reduce the number of teams. Ideally we need a system which can accept any number of teams, so that as many communities as possible can participate.

-> Solution: Most solutions require a way to eliminate a large number of teams during a preliminary round.
-- Make a qualification event: We could create a qualification event which takes place before the main event. This could happen as early as Fall '09. One idea I had is to make a big "collab" battle. The collabs would then be graded in some fashion. The top collabs would then be qualified for the main event. The participating spinners could be anyone from each community, not just people who will take part of the final roster, since the final roster is likely to be small (6-8). This could give motivation for communities as a whole to work hard and make the collab to contribute to the qualification of their community. The qualification could apply to all communities or just a subset, with the others being already seeded. There are advantages and disadvantages to both approaches. Only after the qualifications should the communities submit their final roster for the main event.

-- No qualification required? It's possible qualification is not even required. We could support 16 teams and I don't think there are much more than 16 communities out there. If there are however, maybe merging needs to occur or those extra teams need to settle on the final spot or something. However, 16 teams is a fairly large number and it would impose some big constraints on the structure of the tournament. I much prefer only 8 teams in the main event because it allows for more interesting structure.

- Time and number of rounds: We must make sure the event does not take too long. One round will take 1 month, since it takes more organization to gather the videos, but also to judge. The problem of the number of rounds is less apparent in a team event than in the WT. This is because not all spinners have to participate in every round. Teams can rely on their depth and rest spinners for certain rounds. It will be an interesting decision for a manager to evaluate whether having one of his top spinners (his "starter") at only 60-80% capacity (because he's been used for several rounds) is better than using a backup spinner at 100% capacity for one round only. Therefore there should be no problem if the WC requires 5 or 6 rounds. Time is the bigger factor.

- Seeding: In the last WC, we used a somewhat arbitrary method of seeding teams by using WT results. The problem is that the events are sparse and the world of PS can change dramatically within 1-2 years. Therefore, I don't think it would be the best idea to use, for instance, the WC08 results to seed the WC10.

-> Solution: seeding could be integrated in the qualification process outlined earlier such that not only the top collabs get qualified, but the ranking of those collabs matter for seeding purposes.

-------------------------------------------------------------------------------

Here are some things I thought were done well last time and needs to be kept and/or improved upon:

- Team concept: I thought the concept of teams to be well done and needs to be improved upon. We need a better way to seperate the responsabilities of each spinner within a team. This can be achieved through the concept of themes. By having themes which are fixed in advance, teams will have to build their roster based on those themes. This mean spinners should be chosen to cover a wide range of different themes. It also means each spinner will have a different role to accomplish and makes their abilities more meaningful. Not to mention that themes make judging more easy since battles are more likely to match combos of similar style.

During a team vs team battle, the theme for each match in that battle is chosen or determined. THEN, the manager of each team sends their lineup, placing their spinners in the battles with specific themes. This means managers select their spinners which do well in the themes that are present.

-> There are many ways themes could be integrated:
-- Totally fixed themes: suppose each team vs team battle comprises of 3 matches (2 singles, 1 double). Then we know in advance the themes of these matches before the tournament begins and every battle always use the same 3 themes for every battle. The advantage of this is that there is no luck and every team knows exactly what the themes are going to be. It's also easy to understand and simple. The problem is that it doesn't allow for much variety. Each team only need to make sure those 3 themes are covered, which means they are likely to roll with the same roster for every battle.

-- Limited randomness: we could have a longer list of themes, and themes are regrouped in categories. For example, have a category for "technical" themes (speed, smoothness, difficulty, all-around), one for "original" themes (2 handed, 2 pens, stalls, use of body), one for "tricks" themes (infinity, continuous tricks, aerials, busts, backarounds, etc.). Then the three battles have three different categories and we select at random the theme from each category. This allows for some balancing over totally random themes in the sense that you won't pull out 3 themes that are very similar (3 "power tricks" themes like backaround, bust, aerials) and at the same time provides some variety. The problem is making sure everybody understand what each theme means (solved if communication is not a problem) and also that it's still possible for luck. For example, if a team totally suck at backaround but is godly at busts then you hope to draw the bust theme, although you could argue it's your team's fault for lacking depth and making sure you cover all themes.

-- Teams select themes: A list of possible themes is fixed before the WC begins. The two managers of a battle could then decide what the themes are going to be using some sort of mechanism. This could be interesting in the sense that the managers will try to select themes that not only maximizes their own abilities, but minimizes the opponent's, which requires greater analysis of opponents. Example, each manager selects the theme for one battle, and the doubles match always has no theme (just a all-around battle). After the themes are set, the lineups are then decided. As a manager, I would try to select a theme I'm good at and they're not. The problem is that this process might take too much time to resolve and might too complex to use.

- Multiple elimination: I liked the use of pools because it gave more opportunities for teams to participate longer. Since teams have more spinners than needed in a single round, it also gave the chance to every spinner to participate at least once (if their manager is willing, of course).

-- If we go with 8 teams, I would make 2 pools of 4 teams. Each team would then battle every other team in their own pool. This means 3 battles total. Each win is 3 points, tie 1, loss 0. The top 2 teams of each pools would then advance to the semi-finals, and then the finals. In total there would be 5 rounds, thus 5 months, which is reasonable.

-- With 16 teams, we could go with 4 pools and then either top 2 or top 1 of each pool advance to go to quarter/semifinals, for a total of 6 or 5 months. The only problem with 16 teams is that there will be 3 rounds with 64 participating spinners, which is a lot to grade for judges...

-- Double-elimination knockout: In double-elimination, you must be eliminated twice to be out. In this format, the number of rounds is 4 for 8 teams, 5 for 16. I still like pools better though, because each team has the chance to battle more teams.

-------------------------------------------------------------------------------

Feel free to spread this information around, translating if necessary. Please collect all feedback and send it back to me or Crash.

Also if you have ideas for themes, post them or give them to me.

Thanks for reading!
danduts
Date: Sat, Jun 6 2009 06:34:17

is Indonesia participating??? if not, how can we participate???
PenMaster
Date: Sat, Jun 6 2009 08:14:14

hella lots of words there zombo..

the collab battle idea is good, i think it should..
AyySoLo
Date: Sat, Jun 6 2009 14:05:01

i translated to Chinese and posted on PSH and HKPSA

i can see you are trying to eliminate all luck factors, but imao, luck factor is a big factor in any competition, it makes the event more interesting and dramatic, rather than monotone. So i think a reasonable amount of luck factor is good. Remember how everyone was excited when we were picking matches for WT?
chrisPS
Date: Sat, Jun 6 2009 14:22:53

I didn't bother to read the rest (as I am getting sleepy), but I agree that there should be at least a spokesperson who understands English.

but there's gonna be somewhat a difficulty here at this part:

"Also, the justifications should be written in the judge's own language and we can translate later (or english if possible)."

note to all Nothing discussed in this thread should be taken as real or final.
Glamouraz
Date: Sat, Jun 6 2009 14:51:05

Something like the asia cup would be good.

I think collab battle for finals.

For prelims, maybe a 1-1 or 2-2 battle with reserves.
Zombo
Date: Sat, Jun 6 2009 16:54:19

QUOTE (AyySoLo @ Jun 6 2009, 10:05 AM) <{POST_SNAPBACK}>
i translated to Chinese and posted on PSH and HKPSA

i can see you are trying to eliminate all luck factors, but imao, luck factor is a big factor in any competition, it makes the event more interesting and dramatic, rather than monotone. So i think a reasonable amount of luck factor is good. Remember how everyone was excited when we were picking matches for WT?

there is a difference between luck and hidden information.

luck means no control for the players, hidden information means the lineups are selected secretely then revealed at the same time.

therefore matches still have to be revealed.

QUOTE
is Indonesia participating??? if not, how can we participate???

later there will be registration thread.
CrAsH
Date: Sat, Jun 6 2009 21:01:01

Hi,

Firstly i think the idea of deciding on teams by the use of collabs is a really good idea and would save a lot of time rather than doing a qualification event. It gives every community a fair chance at their shot of participating.

About the judging, zombo and I discussed a while ago about different solutions to this and one that we came up with was for each community to choose 1 representative of their board who they think is good enough to judge to be one of the judges for the event. The advantage of this is that everybody knows who the judges are this time as there has been a bit of critiscm about it over the past few years.

Zombo is right by saying we should start thinking about organizing the the WC2010 as soon as possible to get everybody prepared for it.

If anybody has any critiscm to say about the past tournaments or any new ideas we could introduce to the next tournaments, dont be afraid to tell us so we can improve the future events.

Thanks,

CrAsH
hoiboy
Date: Sat, Jun 6 2009 21:18:06

QUOTE (Zombo @ Jun 5 2009, 11:04 PM) <{POST_SNAPBACK}>
- Communication: the main problem has always been communication. Because of the different communities, language has always been a restriction on the rules we could implement in the tournament. Not only does it limit what we can do in the tournament, it also makes it unfair for teams that don't understand the rules very well. There is also the problem of communication with the judges. Sometimes the judges themselves don't really understand how to judge due to language which results in poor results.

-> Solution: Two things must be done,
-- Full translation of the rules in all languages of participating communities as early as possible. This means we need to finalize the structure of the competition well in advance and ensure that every community has a translated copy of it. This is especially important for asian communities, since their english skills are often poor. Once we have those translations, they should be put on the main site and spread to all communities.

-- Managers who understand English. If you remember, each team has a manager which serves as a spokeperson for that team with the officials (i.e. Crash). In the last WC, we had trouble finding good managers and I remember the Korean team was temporarily managed by Crash himself. It is important for each community to start finding a competent manager who can communicate easily in English. However, that is not the only job of the manager. S/he must also take care of the roster and make important decisions during the competition. Therefore, it would be unfair to demand that the manager speaks English if there is a better candidate at the tactical aspect of the position. Therefore, the manager role could be split into two persons, where one takes care of the actual management, and another as the spokeperson. Ideally, one person can fulfill both roles. Also note that the manager himself could be a spinner in a team. Therefore, one spinner could be doing the actual managing and an extra person is included just to communicate with the officials.

Pertains to my idea of Ambassadors, go betweens between [central government/UPSB] and their board. Guys fluent in English and (the other language, if applicable).

Also, we need a list of possible boards.

QUOTE
- Number of communities: Last time, we had to merge some communities to reduce the number of teams. Ideally we need a system which can accept any number of teams, so that as many communities as possible can participate.

-> Solution: Most solutions require a way to eliminate a large number of teams during a preliminary round.
-- Make a qualification event: We could create a qualification event which takes place before the main event. This could happen as early as Fall '09. One idea I had is to make a big "collab" battle. The collabs would then be graded in some fashion. The top collabs would then be qualified for the main event. The participating spinners could be anyone from each community, not just people who will take part of the final roster, since the final roster is likely to be small (6-8). This could give motivation for communities as a whole to work hard and make the collab to contribute to the qualification of their community. The qualification could apply to all communities or just a subset, with the others being already seeded. There are advantages and disadvantages to both approaches. Only after the qualifications should the communities submit their final roster for the main event.

-- No qualification required? It's possible qualification is not even required. We could support 16 teams and I don't think there are much more than 16 communities out there. If there are however, maybe merging needs to occur or those extra teams need to settle on the final spot or something. However, 16 teams is a fairly large number and it would impose some big constraints on the structure of the tournament. I much prefer only 8 teams in the main event because it allows for more interesting structure.

This goes to the idea of a "Board Ladder" which could be set up eventually. For '10 though, it sounds like a great solution. Random seeding again? The PS community overall has to be more organized.

QUOTE
- Time and number of rounds: We must make sure the event does not take too long. One round will take 1 month, since it takes more organization to gather the videos, but also to judge. The problem of the number of rounds is less apparent in a team event than in the WT. This is because not all spinners have to participate in every round. Teams can rely on their depth and rest spinners for certain rounds. It will be an interesting decision for a manager to evaluate whether having one of his top spinners (his "starter") at only 60-80% capacity (because he's been used for several rounds) is better than using a backup spinner at 100% capacity for one round only. Therefore there should be no problem if the WC requires 5 or 6 rounds. Time is the bigger factor.

Orgy the top x amount of teams from the pools in a collab battle? Maybe something like what happened in the AC R1.

QUOTE
- Seeding: In the last WC, we used a somewhat arbitrary method of seeding teams by using WT results. The problem is that the events are sparse and the world of PS can change dramatically within 1-2 years. Therefore, I don't think it would be the best idea to use, for instance, the WC08 results to seed the WC10.

-> Solution: seeding could be integrated in the qualification process outlined earlier such that not only the top collabs get qualified, but the ranking of those collabs matter for seeding purposes.

Good temp. solution again, but it won't hold up very well long term etc.

QUOTE
-- Double-elimination knockout: In double-elimination, you must be eliminated twice to be out. In this format, the number of rounds is 4 for 8 teams, 5 for 16. I still like pools better though, because each team has the chance to battle more teams.

I prefer Double Elim because it gives the teams a chance to come back and capitalize on their mistakes.
AyySoLo
Date: Sat, Jun 6 2009 23:38:17

QUOTE (Zombo @ Jun 6 2009, 12:54 PM) <{POST_SNAPBACK}>
there is a difference between luck and hidden information.

luck means no control for the players, hidden information means the lineups are selected secretely then revealed at the same time.

therefore matches still have to be revealed.

I'm not talking about revealing the matches, obiously they need to be revealed.
I'm talking about the theme selection, maybe some randomness in there can make the event more interesting.
Fripãƒ¡ECãƒ„
Date: Sun, Jun 7 2009 04:00:48

Kk, just make sure the judges judge in a way where highest and lowest scores are emitted.
Zombo
Date: Sun, Jun 7 2009 12:51:58

if you translate this somewhere, tell me so that I know which communities are aware of this yet
Resonance
Date: Sun, Jun 7 2009 13:07:13

I actually like the Themes idea. It gives the WC a more special approach, rather than the normal battles (which are nice, but a change once in a while is always nice)
I also though about different awards than simply "best spinner" (or community in this case). it came to my mind after the argument with Frat during the WT... How about trying to award spinners for creativity? maybe a viewers favorite? I know it'll work better in WTs, but I think it could be used in the WC as well, if we want. Just throwing the idea into the air.
Pudels Kern
Date: Sun, Jun 7 2009 15:09:17

German translation was posted on GPC by the Dane.
defect
Date: Sun, Jun 7 2009 16:55:27

Hm, the whole world speaks english. I dont think its necessary to translate the rules or this long text. Its all pretty easy to understand. But asian communities might need translations, i understand that.
Vikroal
Date: Sun, Jun 7 2009 18:08:56

QUOTE (Zombo @ Jun 7 2009, 02:51 PM) <{POST_SNAPBACK}>
if you translate this somewhere, tell me so that I know which communities are aware of this yet

Spanish translation has just been posted on SPSC by Vikroal
We'll make you know our ideas if there are some
Thewave
Date: Sun, Jun 7 2009 22:49:00

I tried to do a similar concept in the UPSBTT but since it didn't turn out as active as I wanted it- I just settled with what I had.
I still think a tag every round with partners rotating could be a nice addition to the WC, something else than just the ordinary single videos each round and makes room for so much more creativity.
Also- if there are so many teams and judging could be a problem since time is of the essence- you could form 2 teams of judges, 1 for each bracket\couple brackets.
This could make room for 16 teams as each judge will still have to judge just 8 teams. You could go even further and enhance this idea by selecting judges for every brackets and have 4 brackets.
This of course requires a nice amount of judges (and knowing how hard acquiring judges can be- this could be a problem).
Also- I liked the 4 teams per brackets or even 5 and from there first (and maybe also second) go on to finals\semi finals.
Zombo
Date: Mon, Jun 8 2009 00:12:46

the WC had a double battle and 2 single battles

yes exactly, judges are supplied by communities, which means half grade the other half, just like it was done in WC08
x1213
Date: Tue, Jun 9 2009 09:08:20

QUOTE (Zombo @ Jun 7 2009, 08:51 PM) <{POST_SNAPBACK}>
if you translate this somewhere, tell me so that I know which communities are aware of this yet

I have posted it in TWPS. (With link to Ayysolo's translation)
Zombo
Date: Wed, Jun 10 2009 04:35:02

I think I found someone to take care of the organization for JEB, unfortunately that person does not speak English.

I need someone to act as a translator between English and Japanese. PM me if interested.
Joshy
Date: Wed, Jun 10 2009 04:44:53

translating it to korean

but i'll probably finish tomorrow cause is is 10 pm right now
Zombo
Date: Sun, Jun 14 2009 04:17:33

The WC10 has been generating a lot of discussion at the FPSB and a couple of relevant points were raised, I want to share them here to know what's the opinion of others around the world on those issues.

- Judging: probably the most discussed topic. Several suggestions were made:

= Some liked the idea of removing criterias, some didn't. Other suggested a middle ground where criterias would remain, but instead of giving grades, you would give your reasoning. Example: A was smoother than B because ..., B had more difficulty than A because... Most also liked the idea of relative judging (giving +3/-3 on criterias).

= Some introduced the notion of public voting. I don't think unrestricted public voting would work because it becomes a popularity contest and the team with the largest number of members will always win. My suggestion was to make the qualifications (by collab) voted by public in each community. After the collabs are released, each community creates a poll for the public. They can specify some requirements in order to vote. People will vote on the top 8 collabs excluding your own team collab. Then each community compile their own results into a list. We then compile the final list from the lists of all communities. This means each community opinion has an equal value, regardless of number of members for that community.

= Some suggested a greater number of judges, for example 50. I think this is unfeasible and too difficult to manage. My suggestion is 2 judges per board and ideally the 2 judges from each board should talk to each other and discuss how they're going to vote. This is how Mats and I evaluated videos together for WT07. I think this way will give better quality if judges work in pairs. Since each pair of judges come from the same community, I think communication is not a problem (they can speak same language and live in same place).

- Theme: some people like it, some don't. Theme is an idea I would like to defend:

= Some people said that if you force a theme it limits creativity. I don't think this is a problem because managers choose their spinners after the themes are determined. Obviously, you don't put someone who is bad at the theme. Also we will choose themes are wide enough to be used or that focus on an aspect of pen spinning. For example: speed. Those aspects are general enough that teams can build their lineup while considering the themes.

= Some people didn't understand how judging would work. For example in a speed battle, it obviously doesn't mean that the person who spins the fastest is the winner. It means that the person who uses speed the best way wins. If we use criterias for judging, then this is going to mean that there is an additional criteria called "Theme" which corresponds to the question "Which spinner has best applied the theme in his combo?"

= Some other people thought that the theme battles should only be used in the pool round (first round after qualifications). To give more flexibility to the teams, we could always the double match with no theme. In fact, we could say that the theme of the double match is always the transition between the two spinners.

= Others understood very well the strategic depth themes would bring to a team competition and supports the idea.

- Mods: some people said that grading should be affected by modding. For example, you give a coefficient to mods. Buster CYL would be 1.0, KT 1.02, Comssa Metallic 1.04, MX 1.07, Unmodded Comssa, 1.09, etc. You multiply the score by coefficient of the mod which means smaller and lighter mods would get bonus marks. The problem with this is that it's easy to cheat and many people use personal mods so there would be no way to evaluate how heavy those mods are. Also some people are just against the idea that bigger mods are easier to spin. In my opinion, sometimes the choice of mod is important and can give a good visual effect (like spinning an MX with fast tricks) but not always.

I will write a draft of the rules and pass it around the communities. Everybody should then tell me what to change.

===========

I'm still looking for someone fluent in Japanese to talk to the JEB manager
Thewave
Date: Sun, Jun 14 2009 09:14:25

The idea of grading mods is stupid...
It just ruins the equality and affects the spinner by the mod.
Also- there can be personal mods and how can you know if it's better than X pen and worse than Y pen.
There is also the fact that heavier pens are harder to control when doing direction changes, so in this particular situation spinning a heavier pen should get a higher score.

About the judging- I do prefer your system, too bad I didn't notice it in time for the UPSBTT so we could test it out
I think public voting for the collabs is a good idea, where you have to vote for the top 8(?) communities excluding your own.

About the themes idea- this is a bit trickier, since this is almost a totally new aspect and the judging for it can be difficult.
It's also highly influential by the judges view even more than the normal judging since each criteria enhanced to a theme is affected by personal views more than judging a combo plain.

There is also something that I highly agree on- the communication between judges.
This idea could be used to minimize the interference of personal views as you compare your views with another individual which affects your judging and makes you more flexible and more open to things that you might have missed while judging the combos.
AyySoLo
Date: Sun, Jun 14 2009 18:27:14

about public voting, i like that idea, here's my suggustion on how to do it

i think we can create a forum or some sort of website that allows only members to vote. The organizer(Crash/Zombo) should set this forum/site to not allow anyone else register. Then the organizer himself registers maybe 32 accounts, then asks the participating 8 communities (assume 8 teams, each community 4 judges) to send a list of 4 trusty judges and their e-mail. The organizer sends these 32 accounts to these 32 judges. Half of the accounts should only be allowed to vote for battles in one poll and the other half should be allowed to vote for the other poll, this is during the poll rounds. Once the semi/final starts, judges from the communities that are still in should be banned from voting, and all other judges should vote. The organizer himself should also have 1 vote in case a tie score comes up.

This voting should be open to public so everyone can see how the votes are going.

another advantage of this voting system is that if a few judges don't vote in time, we don't have to wait for them. And voting is much easier for the judges than grading, therefore might take less time.
Zombo
Date: Mon, Jun 15 2009 14:12:56

you're not supposed to vote after seeing others vote... that gives more power to influecen the results to the people who vote later
AyySoLo
Date: Mon, Jun 15 2009 14:31:42

ok, then set it to private during the vote, and publish afterwards
CNstar
Date: Mon, Jun 15 2009 14:41:13

world cup/tournament is the most important event in PSing every year, but every year we hear so much complains. I don't understand wut is the organizer thinking about, every year people complain, every year they still keep the scores secret, keep the judges secret. Some times i even feel China is the most justice country here. I thought about it for a long time, couldn't come up with any reason why Crash wants to keep these stuff under the table. Now we are starting to talk about WC10, but all we talk about is the mode of the competition. I feel the problem is that we have no set rules to control the event, in another word, the event is lagging so much. It doesn't look like a world class event at all: from judges to organizer, there are no rules set out to restrict them. This causes the lagging, we have to wait for weeks for the results, etc.

about the WT, i don't understand why does this thing goes on for so long, really, is a half year competiton fun?? PS is not developed to such a level where we can have "seasons". And plus, why do we have to do this 1v1 thing? Penspinning is not a battle type of sport, but rather a grading thing. Some people suggusted the voting thing, but really, there is no sport or any type of competition existing that uses voting system in world level game. Some people say grading is to subject, i say NO, they CAN be justice. The way to do this is through training of the judges: give them all kinds of videos to grade, so we can set up a specific set of rules for judging, try to minimize the subjective factor.

I see the trend for world tournament, a filtering round>rounds and round >final, do a seeding thing so the great spinners won't meet too early. But this just drags the event for too long. In my opinion, any grading sports such as pen spinning should use ranking. I know people might say this is too hard for the judges, but that is just bullshit. I have been a judge myself, say 5 people per community we have 20 communities(not even), that is 100 videos max. Grading a film takes less than 3 minutes, 300 minutes is 5 hours, spread out into maybe 3 days, the result should be out within 5-6 days. Yeah you may say not all judges are responsible like that, but look, this year's WT, last few rounds they had less than 10 videos, but it takes them more than 1 week to judge, that is fucking bullshit, if you can't be responsible, why the hell are you being a judge?

my ideal WT mode, is 2-3 rounds, 3weeks for 1st round to collect video, half a week for judging, score for every spinner and a rank. Do this for 3 rounds max and we have the top spinners, compile the event into 2 monther, otherwise the exictment is gone, and the spinnes don't have anything new to show, the competition becomes boring.
Zombo
Date: Mon, Jun 15 2009 15:22:13

this is discussion for World Cup, so I don't know why you're talking about the format for a World Tournament. the style of both competitions are different. but we already said for WT11 there will be a filtering round with ranking like you said. and we also plan it to be much shorter than WT09. but right we are concerned with WC10 so give us ideas for world cup please.

the problem with judges IMO is that often the best judges are also the best spinners, and the best spinners are already in the competition! this means the judges are eithe people who don't want to participate (and those ppl are usually busy so that means they wont even judge) or people who are not as good as the ppl in the tournament, so their evaluation may not be as good.
Rarka
Date: Tue, Jun 16 2009 08:45:36

You can appoint about 15 judges and for each round you can end voting when 10 judges have voted. Or when one judge is busy during the week of voting, he can say it to organizers be repleaced by other jugde since we have more judges than we need to vote. Assuming that all judges are the same skilled in judging, it doesn't matter which judge will vote this round.
SpinFo
Date: Tue, Jun 16 2009 10:17:16

I beleive that a system where each community says for example " we have 4 spinners and 2 of them are know and top" so we build a tournament/cup where the top spinnners from each community will be met only in the final rounds is wrong.

Because this is going to create an influence and many people including the judges will vote for the victory of a not so known but good spinner against a known and good spinner only if the first spinner do something incredible in the combo and the second spinner do something not so good and not so bad.

Even if the first spinner win against the other spinner the community of the known and good spinner will propably argue for the decision.

We have now problems like that. For example, a lot of people didn't like that meteor lost in the first round with such a good combo and many people said there was no reliability. Imagine how many people will argue if there was a system with good spinners and not so good spinners and the not so good spinner won even if he deserved to win against the good spinner.

P.S. i agree with CNstar about "the things under the table"
Zombo
Date: Tue, Jun 16 2009 14:02:45

that's a problem with judging, not seeding. the solution is to fix judging, not to avoid seeding. creating favourable matchups so that the favourite spinners fight in the end is perfectly normal and used in all manner of competition.

BTW this is more applicable for individual tournaments like WT than WC.
SpinFo
Date: Tue, Jun 16 2009 18:15:30

I think this is perfectly normal to competitions where you can tell the difference of someone winning or losing. For example, in football you can tell the winner if someone has more goals from his opponent. The referee has some influence in the result but not so much. In penspinning the judge has 100% influence in the result and there is no clear way to say the winner. Even if theme battles exist its still hard to tell the winner if the videos are equal in the difficulty.
AyySoLo
Date: Tue, Jun 16 2009 19:07:02

i agree with the seeding influnce judging arguement, but like Zombo said, we just have to work on the judges. All of the talk about seeding, theme, teams, rounds are just not going to work out if the judges are not good enough.

Obiously the judges(some of them) from WT09 are not the best choice, if they can't even submit their results on time, how can we trust them for a reasonable score? The system where each community provides 1/2 judges also has problems, Asain cup already proved it. My suggustion is a professional judging system, each community should provide a few trusty people for judging, and these people will be trained(like CNstar said) and tested, this should be done throughout the year. For example, give them videos once a month and ask them grade, then give them feedbacks about their standard compare to other judges. Slowly adjust them so they can all judge at a same level. Nowadays there are so many events going on other than WT/WC, stuff like AC, the global team tournament, etc, these professinal judges can be used. We should have more than 1 set so when events comes up, those who has time will be judging.

When a judge wants to quit judging, each community will take turns to provide new judges. If a community's judge never does the "homework"(grading those practice videos), they will be kicked out of this group, then that community will have no judges in important events.

I know this might sound too ideal, but there are actually lots of people with "justice" minds out there, and willing to take part in these events.
Zombo
Date: Tue, Jun 16 2009 22:58:38

QUOTE (SpinFo @ Jun 16 2009, 02:15 PM) <{POST_SNAPBACK}>
I think this is perfectly normal to competitions where you can tell the difference of someone winning or losing. For example, in football you can tell the winner if someone has more goals from his opponent. The referee has some influence in the result but not so much. In penspinning the judge has 100% influence in the result and there is no clear way to say the winner. Even if theme battles exist its still hard to tell the winner if the videos are equal in the difficulty.

well anyway this is not applicable for WC, because each individual match is decided by the 2 teams, not by seeding. if two videos are equal in value, the correct verdict is tie, simple as that.

QUOTE
i agree with the seeding influnce judging arguement, but like Zombo said, we just have to work on the judges. All of the talk about seeding, theme, teams, rounds are just not going to work out if the judges are not good enough.

Obiously the judges(some of them) from WT09 are not the best choice, if they can't even submit their results on time, how can we trust them for a reasonable score? The system where each community provides 1/2 judges also has problems, Asain cup already proved it. My suggustion is a professional judging system, each community should provide a few trusty people for judging, and these people will be trained(like CNstar said) and tested, this should be done throughout the year. For example, give them videos once a month and ask them grade, then give them feedbacks about their standard compare to other judges. Slowly adjust them so they can all judge at a same level. Nowadays there are so many events going on other than WT/WC, stuff like AC, the global team tournament, etc, these professinal judges can be used. We should have more than 1 set so when events comes up, those who has time will be judging.

When a judge wants to quit judging, each community will take turns to provide new judges. If a community's judge never does the "homework"(grading those practice videos), they will be kicked out of this group, then that community will have no judges in important events.

I know this might sound too ideal, but there are actually lots of people with "justice" minds out there, and willing to take part in these events.

this is good but to me the role of judge feels more like a job than fun. it is one of the most thankless job in PS. Not only do you volunteer substantial amount of your time to do the job and there is no reward for it. If it's anonymous, then obviously nobody know you did the job so nobody gonna thank you. If it's not anonymous, then most likely if you did good job people will just say "results as expected", if you did a bad job, they're going to flame you very hard. It is one of the most charitable things to do in PS aside giving a donation. So I think it's really difficult to get a lot of judges that are motivated. Most are just people that were going to watch the videos anyway so they take a little more time to fill the scoresheet. But it's not real motivation.

To make a pro league is difficult because you have to train a lot of people, because it takes time to become good. So that means either those people quit cuz they quit PS altogether, don't have time anymore, or has become good enough to participate in events. Why would someone stay a judge if he's good enough to play? Some pen spinniners can't even respect deadlines for collab, what makes you think they have dedication to follow a training program to become... a judge??

How are you going to decide what's the right way to judge? If you ask all the communities, everybody will give you different answer.

IMO this will only work if judging is a real paid job, then the motivation is to make money.
AyySoLo
Date: Wed, Jun 17 2009 03:15:10

i understand wut you are saying, but look, if the organizer go look for judges(i don't know how he does, but we assume he does...), he asks around for people interested in judging. These people are from a position of "being asked", they might not even want to judge but still agreed to because they don't want to turn CrasH down, then these people (or some of them) won't be so motivated, this is why they feel this is more of a "job".

But, if we some how make people sign up for judges and filter them, those judges will be much more motivated cuz they applied for the position. True that some spinners are irresponsible, but not ALL. Some don't make the deadline, but MOST do, that's why we need training and testing, filter out those irresponsible ones. And plus, there ARE people out there that finds judging fun, from what i know, there ARE people that are willing to contribute. It's not such a boring job if you are making the result for the most important event of Pen spinning. As you said, if there are people willing to give donations, there got to be people that are willing to judge. From the forums i go to, PSH, HKPSA, maybe TWPS, i can even name people that for sure will be willing to do such jobs and have the ability to. And i'm sure there will be at more from other communities.

Plus, we don't really need that many judges, i think WT09 had 5 judges? Rather than having 5 not-so-interested judges from the "being appointed" position, we can easily get 10+ much more motivated people from all communities and select a few that best suits the position.

and yeah, you are right, there is no right way of judging, but there must be some sort of Standard, things like AC R2 PDS Taru grading must be avoid. I can see that every community has different views on pen spinning nowadays, but long as your score is SOMEWHAT REASONALBLE, and it can be explained, then i think most people will understand. They might ARGUE, but they won't FLAME, if they FLAME a relatively REASONALBLE score, then it's their problem, no manner or just stupid. Again, pen spinning is not a scoring competition, there is not absolute right or wrong, everything is relatively speaking, long as your grading is relatively reasonable, then it's ok. And besides, if the CrasH can post the specific judging results, it can save a lot of trouble. Cuz most of the judging critirias like smoothness, difficulty or those themes like speed, etc are relatively easy to judge, easy to give a fair score, these things are not that subject so any community should have basically same view. And stuff like style, effectiveness are hard to argue, so everyone will see why the results are what they are.
Zombo
Date: Wed, Jun 17 2009 05:00:33

IMO, I still think the biggest problem is not the judges themselves, but the judging system. A good judging system means anybody with good PS experience and good knowledge of PS can grade and express his opinion correctly. like for example even Taru, I don't think its his fault, its just nobody explained to him how it works. nobody told him what a 0 mean, what a 20 mean, what theme mean. the rules just give you 1 word criteria, in a language u don't know well, you can interpret any way you want. imo u don't need training, just better explainations that judges can read before the tournament start. it's just a problem of system and communication. and if you have 2 judges per community and they have to discuss with each other when judging its even better imo because they have to convince each other that their judging is good and I think it makes it less boring if you judge with someone else. feels less like work. so I like the idea that judges from own community discuss with each other the results before giving them to crash.

IMO problems from judging is not because judge has a bad opinion, but because the grading was done wrong. like i said, you're never told what 7/10 means compared to 5/10 or 9/10. most of the grading systems I see don't even tell you what each number means (5 = average, 6 = above average, 7 = good, 8 = very good, 9 = excellent, 10 = outstanding???) my system gives very detailed guidelines... the problem is that it takes very long to follow