Algorithm of 400 Clan Ranking Algorithm

[jigger1986]

This is some interesting info and I thank you for it. According to FD's initial calculations there we alot of surprises in the league.

[Qwert]

farang can you give results start from 2011.
If i understand correct-every clan start with 1000 pt.
Can you tell me if THOTA play against T4c- how many pts will get thota if win,and how many pts will get T4C if win?

[jpcloet]

JP why are results from 3 years ago important? Clans are fluid and the skill of the clan 3 or four years ago don't matter. A true ranking should show who you are now. It should have a beginning and an end to the ranking period and take into account the results during that period. This system has merit if used appropriately but the results of the first ever clan war don't matter now, maybe in a historical ranking but does the 1927 Yankees winning the world series have any impact on the 2011 Yankees winning it? Nope. So define a clan year and base the rankings on challenges begun during that year.

You have to look at the history of clans, some are not as active as the others. I simply used all the data to start with. Various other ELO's I used removed older data as we went along. 2 Year RPI was the last 24 months when it was in use.

[FarangDemon]

The rankings presented below are based on all clan wars I got from jpcloet and all Clan League (1-3) that had finished at time of posting.

http://killersapp.com/Algorithmof400.htm

Note: I'm not advocating that the particular model with these specific parameters is the best model to use to compare clan performance. There are a lot of parameters that could be tweaked to result in higher accuracy. I used rough estimates for end dates of CL1 and 2. So consider this post as an illustrative example of how to use my script to create and evaluate your own models. More details below.

I've updated the script to:

• Graph the top clans' performance over specified time window
  
  

  Click image to enlarge.

  

  
  The graph above used Data Window of 2 years and Graph Window of 3 years. Data Window of 2 years means that whenever performance rating is computed, clan wars from 2 years before computation date are weighted 0% and clan wars from computation date are weighted 100% (values in between are scaled linearly between 0 and 100%).
  
• List all clan wars, in order of decreasing weight, which contributed to each clan's final rating (Display Basis of Each Clan's Rating)
  
  
  If the final rating was lower or higher than expected, you can look at the reason why
  

  Click image to enlarge.

  

  
  
  Note: Yes, TOFU only got a rating of 476 from beating The 4th Crusade with 80% win rate (applied with relatively high weight of 13% unfortunately for TOFU, as it was a big challenge / relatively recent in time). At the time T4C had a rating of only 76 because they lost to TIME when they were 433 and Knights of the Future when they were 524. Kind of a funny situation... But there are two ways to possibly alleviate this injustice (both described in more detail below):
  
  
  • Tweak parameters to modify point gain/loss based on win margin,
    
  • Use more accurate dates for end of CL1 and CL2 challenges.
• Compute accuracy of the model, in terms of how often the higher ranked clan actually won
  
  
  Now you can know how accurate the model is in real terms, measured by how often the higher ranked clan wins for various rating difference intervals
  
  Min Weight to include in Accuracy = 100
  
  
  This means, that given a match between two clans, where one clan is 109 points above the other, it is 82% likely to end in a non-upset. Btw, this 109 point value was auto-calculated as 10% of the range between min and max ratings (after excluding clans below weight threshold).
  
  You can take a look at the ratings below to see if it makes sense to your perceived chances of success against a clan.
  
  

  Click image to enlarge.

  

  
  
• Change parameters to allow rating for a clan war to increase/decrease based on win margin. The model I have provided is just an example. You guys can:
  
  
  • Add/adjust Data
    
    I didn't look up when every clan war from CL1 and CL2 actually ended, so I just used one date for each regular season. It would be great if someone could update this to reflect the actual dates because this model is time sensitive and the clan wars took place over many months.
  • Play around with the parameters
    
    Try to use parameters that result in high model accuracy

[FarangDemon]

Also, if you add criteria with 3 wars completed - then the results should be much better.

I forgot to implement this! So what I've implemented now, if a clan has under 3 wars, any query on their score will result in the average of 1000 and the otherwise regularly computed score. This is what gameknot did, though they may have used a different cutoff value.

So if a clan has only completed 1 war against a clan with 1000 points, they are considered a 1200 instead of a 1400 at that time. This prevents clans from skyrocketing positive or negative by playing just 1 or 2 games, which adversely affected TOFU in the example in my previous post.

This is what it looks like with this fix and using 600 points max delta (if one clan wins/loses 100% of games in a war), 200 points min delta (if a clan barely wins/loses).

(All screenshots from the same model)

Click image to enlarge.

Click image to enlarge.

Click image to enlarge.

As you can see above, this model is a lot more forgiving to higher-ranked clans that take on lower-ranked clans (as all clans that participated in CL3 did). Using the standard implementation of algorithm of 400, a higher ranked clan is automatically penalized even for winning against a clan less than 400 points below them. With this model, they can make up for the point differential by having a high win margin.

Example, in this model TOFU is penalized 200 points for the 80% win against T4C. In my opinion, this is fairer than the 400 point penalty resulting from previous posts' min=400, max=400 standard model (with top-of-post-mentioned fix for clans with less than 3 wars).

Click image to enlarge.

For the accuracy calculation, I've made the interval length a user-defined parameter, so you can play around with different values that you think provide more meaningful results.

This table shows that according to this model, any clan that is about 22-66 points ahead of another has a 67% chance to beat or tie the lower-ranked clan. For an 67-110 point lead, the chance is 83%.

[Dako]

Wow, great job. I will look at it again to reread it cause there are a lot of info here.

[danryan]

I have to say, this is a very credible model from what I'm seeing so far. For one, it clearly shows what effect your input data has. Secondly, it removes all incentive for "farming" a clan. Lastly, it penalizes a clan (but not too severely) for inactivity, due to the decay factor. I'd like to hear jpcloet's take on it since he has put a lot of work into a lot of different models over the years to try and come up with a mathematical model.

[jpcloet]

I have a few question on the final rankings to help clarify.

1) TSM is due to lack of challenges correct?
2) MM seems to be the biggest outlier in my mind, considering the overall quality of opponents. Is S2 playing a part? This is the only real anomaly for me.
3) Is the O&H loss by LOW keeping LOW from the top 3?
4) What does the top 10 look like with a KORT win over THOTA in CCup1?

[josko.ri]

I very like this system of calculation, results are very reliable I think. decay idea is great. only one complaint, you put 32-28 result in TOFU vs KORT challenge in database, which couldnt be valid. for achieving that result TOFU used forbidden bonus according to the official rules of the challenge, so it shouldnt be validated as regular result.

[Gold Knight]

Wow, great job. I will look at it again to reread it cause there are a lot of info here.

Agreed, someone finally put some colorful pictures up so us peasants can understand... It does seem like a pretty close representation of what the clan landscape is IMO. Where the hell have you been in the CLA the past year while we've been bickering over 40 different ranking systems?

[Leehar]

I very like this system of calculation, results are very reliable I think. decay idea is great. only one complaint, you put 32-28 result in TOFU vs KORT challenge in database, which couldnt be valid. for achieving that result TOFU used forbidden bonus according to the official rules of the challenge, so it shouldnt be validated as regular result.

It's already been validated under clan results so I doubt this is really the place for that argument

[Leehar]

I have a few question on the final rankings to help clarify.

1) TSM is due to lack of challenges correct?
2) MM seems to be the biggest outlier in my mind, considering the overall quality of opponents. Is S2 playing a part? This is the only real anomaly for me.
3) Is the O&H loss by LOW keeping LOW from the top 3?
4) What does the top 10 look like with a KORT win over THOTA in CCup1?

I was wondering more along the lines of if cl3 results were taken into account or something for tsm to be so far down?
I'd think the O&H loss would be too far back to have that much waiting, and it's still a surprise that IA is in the top 3 after losing to Low in their most recent full-scale challenge?

[FarangDemon]

Admittedly, I'm not the data guy and I don't want to be the data guy. If someone wants to maintain a data file(s) I can link to it from my site.

All you have to do to change/add data on your own:

1) Click the link to the data file from my site (http://www.killersapp.com/algorithmof400.htm)
2) Open it in Excel or notepad
3) Change existing war data / add new entries
4) Paste it into the input box of my site

[FarangDemon]

I have a few question on the final rankings to help clarify.

1) TSM is due to lack of challenges correct?

You can see the basis of the rating for every clan by checking "Display Basis".



Tieing Grim Reapers and losing to BSS and Gen 1 gives TSM an 800 weighted at 17%, so that's what is bringing them down the most. I'm guessing that was CL3, so it's recent. The weight that is applied to each challenge is the number of games in the challenge times the decay factor (a number between 0%-100% based on how long ago the challenge took place). They don't have as much total weight as other clans do, so these CL3 losses/ties affect their score more than a clan with more wars and hence more weight.

I did not include their recent win in CL3 Quarterfinals as the final score was undecided when I put the data file together. This should give them a boost.

2) MM seems to be the biggest outlier in my mind, considering the overall quality of opponents. Is S2 playing a part? This is the only real anomaly for me.

Looks like 39% of their rating weight (losses to BSS, AOD, EMPIRE, Devils Brigade) averages below 800. Three out of four of these losses were brutal, winning 28-33% of games. In this model (min delta = 200, max delta = 600), clans that lose by a larger margin are penalized more.

3) Is the O&H loss by LOW keeping LOW from the top 3?

That O&H loss is weighted at just 5%, but at 599, it is still quite damaging.
Losses to THOTA, KoRT and TOFU weighted at combined 27% are also lowering them a little from where they otherwise would be based on other, lower weighted results (perhaps from past).

4) What does the top 10 look like with a KORT win over THOTA in CCup1?

What margin should I use? It would probably end up #1 KoRT, #2 TOFU.

I'd think the O&H loss would be too far back to have that much waiting, and it's still a surprise that IA is in the top 3 after losing to Low in their most recent full-scale challenge?

The IA loss to LOW accounts for 12% of IA's rating. It drags us down the most, but we are buoyed by our strong performance in CL3, which is very recent and accounts for a ton of games (90 games in regular season).

[jpcloet]

Tieing Grim Reapers and losing to BSS and Gen 1 gives TSM an 800 weighted at 17%, so that's what is bringing them down the most. I'm guessing that was CL3, so it's recent.

Yes that was CL3, that mean's this is now a fully inclusive measure. Some will argue the league should not be included. One of the things I did before was to have the final league score against a generic team so that it got some weight as the divisions. More so for CL2.

Eg.

TSM 55 CL3A 45

[FarangDemon]

Yes that was CL3, that mean's this is now a fully inclusive measure. Some will argue the league should not be included.

I don't want to take your job, I'm just providing you a tool you or others can use to generate rankings based on your data. It's up to you to decide what data to use for your official ranking.

1) Regular Clan Wars (no CL)
2) All Data (Regular Clan Wars and CL)

Just put it into the right format and paste it into my script. http://killersapp.com/AllData.txt

In my example of how the script works, I included all data for two reasons:

Most Data

The more data used, the more accurate the results of this algorithm.

Most Fair

Choosing to discount certain results would, in my opinion, be a can of worms, as each clan would be motivated to lobby to exclude the wars where they performed the worst.

Note: Individual clan league war results are automatically weighted less than regular wars, because they consist of fewer games.

So you could maintain two clan war data sets and provide graphs for both, in order to please all of the people. But I'd recommend using all data for the above reasons.

One of the things I did before was to have the final league score against a generic team so that it got some weight as the divisions. More so for CL2.

Eg.

TSM 55 CL3A 45

Ok so that would count as TSM beating a team with rating 1000? You can definitely implement this and see what it looks like. Some drawbacks I see with this:

1) Dilutes the data
2) Assumes competitiveness of each division is equal
3) Obscures the data - Clans on the bottom end up losing to a clan of rating 1000 at a high-to-moderate loss margin instead of to high ranked clans at high margins and medium ranked clans at moderate margins.

Due to item 3) I have no idea what the end effect will be, but it is confusing. I have a vague idea of what your intent is, but I'm not sure it will be accomplished due to this obscuring effect.

So I personally think it's a lot more straightforward and cleaner if we just keep data in original format, Clan 1 vs Clan 2 for each CL War.

[josko.ri]

Tieing Grim Reapers and losing to BSS and Gen 1 gives TSM an 800 weighted at 17%, so that's what is bringing them down the most. I'm guessing that was CL3, so it's recent.

Yes that was CL3, that mean's this is now a fully inclusive measure. Some will argue the league should not be included. One of the things I did before was to have the final league score against a generic team so that it got some weight as the divisions. More so for CL2.

Eg.

TSM 55 CL3A 45

I think that small number of games in CLA3 is actually validated by smaller weight factor in FG's calculations, and I agree with that totally. Weight factor may be dependent on number of games played and time passed from the war, not only by time passed.

I dont like your idea JP about TSM55 CL3A45 because it in any way does not include strength of opponents in Division. the fact that both teams from Divisions C&D and zero teams from Division A&B advanced to semis proves that some Divisions were stronger, and in ranking like this one it should be included strength of opponents in CL3 Divisions, which I think is implemented in FG's graphs.

[Dako]

I disagree that Div C&D were stronger than Div A&B. You can only tell that one clan was stronger than the other at the moment with given settings, but you cannot propagate it on the whole division strength.

[josko.ri]

I disagree that Div C&D were stronger than Div A&B. You can only tell that one clan was stronger than the other at the moment with given settings, but you cannot propagate it on the whole division strength.

so you then think that Divisions C&D were not the strongest? if so, again my point has sense, that it is not fair to validate score of 55 compared with average score of 45 in Division because not all Divisions were with the same strength (it is questioned and subjective opinion which one were stronger and which one easier, but I think everyone agrees that Divisions were not equal, some were stronger and some were easier). because of that I very like to validate Division results according to opponents faced in Division.

and I think that FD very good implemented factor of only 18 games challenge because Weight factor from Empire-Kort war is 10%, and weight factor from TSM-Kort Division challenge is 6% no matter that Empire war was from May and TSM challenge was from December. so obviously, number of games per challenge is included in weight factor next to time passed, and I totally support it. it is true that small challenges can sometimes be decided due to some bad luck/drops but then it is validated with small weight factor, so it makes good point to me.

[Dako]

I didn't say you were wrong in your opinion on "each clan against score of the division" method. I haven't said your point didn't have sense ether. Please read my post more careful next time as it will save breath for both of us :).

[Leehar]

I didn't say you were wrong in your opinion on "each clan against score of the division" method. I haven't said your point didn't have sense ether. Please read my post more careful next time as it will save breath for both of us .

Then I'll say it.
I vehemently disagree Josko, specially with the Div A/B weaker than C/D. It's almost like saying in the Cricket World Cup Group A was overall better than Group B because they had 3 semifinalists. After all, over there the 2nd in group B made the Final, while the 1st lost in the quarters. It all just depends on who performed better on the day with the conditions, and I think that analogy holds some truth over here as well.

Again, I'm not sure why I still have an issue with the rankings, but something still feels a bit off

[josko.ri]

@ Leehar

do you think that all Divisions were equally stronger/weaker or some Divisions were stronger and other ones were weaker?

[danryan]

Division C was clearly the strongest because as awesome as FOED are we didn't even make the playoffs. Quod erat demonstratum.

[Dako]

@ Leehar

do you think that all Divisions were equally stronger/weaker or some Divisions were stronger and other ones were weaker?

What does equally stronger mean? I reread your sentence like 3 times and couldn't grasp what you were trying to say.

[josko.ri]

@ Leehar

do you think that all Divisions were equally stronger/weaker or some Divisions were stronger and other ones were weaker?

What does equally stronger mean? I reread your sentence like 3 times and couldn't grasp what you were trying to say.

equally strong (or) weak. means that noone Division were stronger than any onther Division. true or false?