Algorithm of 400 Clan Ranking Algorithm

[FarangDemon]

Welcome! For the example clan ranking based on all clan wars and Clan League data prior to Mar 29, go to this post:

viewtopic.php?f=438&t=140929&start=15#p3088157

The remainder of this post is my original post, describing the method and my initial application of it to the results of Clan League 3 only (nothing else). You have to start small, right?

Performance Rating is a hypothetical rating that would result from the games of a single event only. Some chess organizations use the "algorithm of 400" to calculate performance rating. According to this algorithm, performance rating for an event is calculated by taking (1) the rating of each player beaten and adding 400, (2) the rating of each player lost to and subtracting 400, (3) the rating of each player drawn, and (4) summing these figures and dividing by the number of games played. This can be expressed by the following formula:

Performance rating = [(Total of opponents' ratings + 400 * (Wins - Losses)) / Games].

I've taken this formula and modified it to get a clan ranking based on CL3 performance.

I treat each set of 18 games (19 games in finals) from CL3 as a single game between two clans where each clan either wins, loses or ties.
The points a clan gets for each set is a function of their opponents rating at that time.
The rating of a clan is the weighted average of all such accumulated points.
(weighting is by games in the set, so reg season sets are weighted at 18, finals weighted at 19)

I started each clan at 1000 points.



I assumed that KORT beat TSM in the regular season 10-8 (the data in the thread are incomplete at 9-8 favoring KORT).
If TSM tied KORT please let me know and I'll update the ratings.

[jpcloet]

I assumed that KORT beat TSM in the regular season 10-8 (the data in the thread are incomplete at 9-8 favoring KORT).
If TSM tied KORT please let me know and I'll update the ratings.

10-8 was the final. That game lagged for a while so the table has not been updated.

[Dako]

Looking at the regular season it seems pretty accurate I must say. Good job.

[FarangDemon]

Looking at the regular season it seems pretty accurate I must say. Good job.

Thanks!

I created a script that takes clan challenge data as input and provides a ranked list of all clans by performance rating.

http://killersapp.com/Algorithmof400.htm

I thought it may be interesting to add data from more clan wars, both before, during and after CL3, and see what performance looks like then. So now anybody can come up with a ranking for any event or set of events.

You can set the Rating Calculation Date to be any date, it will tell you what the clan's score was at the given time (not counting future challenge outcomes).

You can also set the Data Window, which is how many years back challenge outcomes will be used to calculate your performance rating. I have it set to 1 year as default. So clan challenges from 2 years ago won't be used to calculate your performance rating. They are still relevant because they established how many points your opponents gained/lost when they played against you over the previous year.

Format for Clan Challenge Input

http://killersapp.com/Data.xls

One war per line, tab separated columns: (ClanName1,ClanName2,WinCode,Games,Date)

• ClanNames - use whatever you want but be consistent, matching is case-sensitive
• WinCode = 1 means Clan1 beat Clan2, WinCode = 0 for ties
• Games is number of games in challenge
• Date must be dd/mm/yy, 0 in the tens place for any single digits

[skillfull]

Nice work Farang

[White Moose]

Very nice work indeed.

[josko.ri]

very nice indeed. very accurate results shown

[jpcloet]

Here is what I get when I put all the wars into that system....

Rank Name Rating
1 z-Dark Defenders 1800
2 The Horsemen of the Apocalypse 1583
3 The Spanking Monkeys 1474
4 z-The Rogue State 1400
4 z-Dragon Lazorz Pew Pew 1400
4 The Pack 1400
7 The Odd Fellows Union 1382
8 The Immortal Assassins 1345
9 Nemesis 1320
10 Knights of The Round Table 1278
11 The Fantastic Four-Skins 1275
12 Legends of War 1271
13 EMPIRE 1267
14 Knights of the Round Table 1229
15 The Devil's Brigade 1199
16 A Fistful of Sixes 1195
17 De Veroveraars der Lage Landen 1188
18 ++The Legion++ 1167
18 The Brethren of the Fat Mermaid 1167
20 Angels of Death 1159
21 Agents of Chaos 1149
22 z-The Untouchables 1132
23 The Fraternal Order of Exceptional Drinkers 1129
24 Memento Mori 1112
25 The Imperial Dragoons 1095
26 Generation One: The Clan 1079
27 One Step Ahead 1036
28 z-Knights of the Empire 1000
28 z-Snipers 1000
28 z-Disciples for Christ 1000
31 z-Exictium 976
32 The Dark Knights of Chaos 974
33 z-Warlords of the Wort 970
33 z-Order of Odin 970
35 The Last Warriors 955
35 z-Left4Dead 955
37 Risk Attackers 954
38 Black Sheep Squadron 943
39 The Bullet-Proof Bandits 931
40 z-Eternal Empire 927
41 z-Knights of the Future 924
42 z-Sky Force 910
43 z-The Regulators 903
44 z-Outlaws & Highwaymen 875
45 Mythology 868
46 z-Imperial Britain 857
47 Legends of the Zone 829
48 z-Project Enigma 815
49 Grim Reapers 810
50 Soldiers of War 798
51 z-Divine Domination 785
52 z-Bounty Hunters 763
53 Death By Comity 760
54 z-Freemium Forces 690
55 z-Flame Whores 659
56 z-Grey Company 636
57 z-Churchill War Cabinet 600
57 z-Jesus Freaks 600
57 z-Bushwhakers 600
57 z-The Army of Darkness 600
57 z-Hell's Messengers 600
57 z-Project Firestorm 600
57 Always Kicking Ass 600
57 Kill On Arrival 600
65 z-Water's Fury 587
66 Dynasty 568
67 z-TIME 555
68 The 4th Crusade 496
69 z-Lunatics 458
70 z-Faith Becomes Reality 444
71 z-Marvel/DC Heroes 372
72 z-Gorgonzola Pirates 317
73 Legends of The Zone 207
74 Karma Clan 173
Total Records: 180

[Dako]

Interesting results. New clans with 1-2 wars get an overpowered rating.

[jpcloet]

Notes for those that are not aware.

1 z-Dark Defenders 1800 (1-0)
4 z-The Rogue State 1400 (1-0)
4 z-Dragon Lazorz Pew Pew 1400 (2-0 and farmed both)

Essentially this is ELO which requires lots of good data and I see a number of clans that have not faced any decent clans yet. This is one of the usual faults in ELO and RPI systems.

[jpcloet]

Interesting results. New clans with 1-2 wars get an overpowered rating.

If you are a new clan and go 3-0 against new clans (then 1000 v 1000 etc.) will give you a higher than perceived ranking.

[Qwert]

farang these not going to work- because THOTA its not no1. Its better to give up. Like blitz say- hes and Chuuck ranking its only acurate ranking, and all other its pointless mathematic who are not acurate.

[danryan]

A lot of very odd looking results in that output...

[FarangDemon]

Wow, I hadn't hoped for such a quick response!

Thanks, jpcloet.
Please provide your data file so I can take a look at improving this metric.

I should add some sort of filter to only include clans that have played a minimum number of wars in the ranking.
It is only meaningful after a certain amount of data, as I believe most metrics would be.

Regarding the non-existent clans, there must be some bug, though, any clan that hasn't played a game within the past year, if you use default setting, should have a score of 1000. I'll take a look and report back.

[danryan]

I should add some sort of filter to only include clans that have played a minimum number of wars in the ranking.
It is only meaningful after a certain amount of data.

I'd say 3 wars or a total of 100 games played as a minimum amount of data, then it would make sense. The trouble is, some decay needs to be in there as well as clans' strength change over time. This is a hard nut to crack.

[jpcloet]

A lot of very odd looking results in that output...

Yes but at some point you say clan x is overrated and clan y is under rated and they should battle, but that is not happening. In theory it should sort itself out. The same principle applies to the leap. Speaking of that, I have put an updated model in the CLA for them to review/discuss.

[Leehar]

A lot of very odd looking results in that output...

Not really, it's mostly understandable how it came out.
Maybe if you add some restrictions like minimum wars played or only privies/medals using ones it could weed out some of the more wonky ones. (Or it could just be that I'm looking forward to seeing Nem at #5 )

I'm actually trying to just discount the 'z's completely since they're no longer existing clans I presume

[jpcloet]

I'm actually trying to just discount the 'z's completely since they're no longer existing clans I presume

yes, the z-clans are dead clans. The z was put in front so that we could display better in the clan results thread.

[Dako]

Also, if you add criteria with 3 wars completed - then the results should be much better.

[jpcloet]

Also, if you add criteria with 3 wars completed - then the results should be much better.

In one of the ELO versions I had, I used a display factor of (.5 for wars of 1 or 2, .75 if 3 wars and 1.00 if 4 or more wars) You could do something like that. I called it a Reliability Rating.

[FarangDemon]

I should add some sort of filter to only include clans that have played a minimum number of wars in the ranking.
It is only meaningful after a certain amount of data.

I'd say 3 wars or a total of 100 games played as a minimum amount of data, then it would make sense. The trouble is, some decay needs to be in there as well as clans' strength change over time. This is a hard nut to crack.

Yes, I think we'd want a 100, or maybe 150 or 200 game minimum for a clan to appear in the ranking.

As for decay, I agree with you. That's why I created a Data Window parameter. Using the default parameter (1 year), I intended for only the past 1 year of games to be included to determine the performance rating. Either jpcloet increased this value or there is a bug I need to fix.

jpcloet, which value did you use for Data Window?
Also, please provide me the data files you used so I can investigate.

[jpcloet]

jpcloet, which value did you use for Data Window?

I used all of time, so I set it to 5 years.

[FarangDemon]

Also, if you add criteria with 3 wars completed - then the results should be much better.

In one of the ELO versions I had, I used a display factor of (.5 for wars of 1 or 2, .75 if 3 wars and 1.00 if 4 or more wars) You could do something like that. I called it a Reliability Rating.

This is definitely a good idea. I appreciate you testing out this metric and your feedback, as I know you have a lot of experience doing this.

I came across the algorithm of 400 by playing chess online on Gameknot. The way they implemented it, if you played under 10 games, your score was computed the same as everybody else, but it was noted as being "provisional". Perhaps I could add option to include/exclude provisional clans from the ranking list.

I used all of time, so I set it to 5 years.

It's not necessary to set it to 5 years if you want to take 5 year-old data into consideration. It's a bit confusing. It will help if I explain a little of what exactly the program does.

The way the program works, it starts at the first challenge, no matter what the Data Window value, and adds challenge by challenge, recomputing each clan's rating as it goes.

In order for this to work, every time I calculate the rating for a clan, I must pass a time parameter. The time parameter specifies what point in time I am calculating the rating. This is the date used to establish the data window that determines which challenges are counted.

Each clan war provides one data point, which is either +/- 400 of the opponent clan's rating at the time of the challenge. For ties, the opponent's rating is used unaltered. Data points are dated to the time of conclusion of the challenge.

For example, when I calculate THOTA's rating for Mar 25, 2007, it will only be looking at challenges that happened prior to that date. If Data Window is set to 1 year, the rating will be computed by games-weighted averaging the data points (i.e. results from challenges) between Mar 25, 2006 and Mar 25, 2007. However, given that one of these THOTA challenges (against Z-Nihilists) concluded on June 6, 2006, the value of that data point would be based on Z-Nihilists rating at that time (June 6, 2006), meaning it would have been computed by averaging data points of the Z-Nihilists' challenges between June 6, 2005 and June 6, 2006. If the Z-Nihilists played against Z-Beelzebubs on July 14, 2005, it would then need to calculate rating of Z-Beelzebubs on July 14, 2005. This required looking at all of Z-Beelzebubs' challenges between July 14, 2004 and July 14, 2005. And so on and so forth until the beginning of time.

So since it starts at the beginning, you can be assured that all 5 years of data will be used as the algorithm computes and recomputes ratings over the past 5 years no matter what the Data Window setting.

[FarangDemon]

I've updated the script.

http://www.killersapp.com/algorithmof400.htm

Decay Factor

There is now a decay factor. If the Data Window is set to 2 years (default), then a war today is weighted 100%, a war 6 months ago is 75%, 12 months ago is 50% and 18 months ago is 25%.

decay factor = Elapsed time from start of data window to clan war record / total length of data window

The weight changes continuously, day by day, so that over time, a clan's score will will not perceptibly spike up or down except immediately after conclusion of a new clan war.

Min Weight to Display in Ranking List

Restricts display of clans in the ranking list to those with minimum weight total. Total weight = Sum of products (decay * games) for each clan war record within data window. This ensures reliability of ranking.

[shocked439]

JP why are results from 3 years ago important? Clans are fluid and the skill of the clan 3 or four years ago don't matter. A true ranking should show who you are now. It should have a beginning and an end to the ranking period and take into account the results during that period. This system has merit if used appropriately but the results of the first ever clan war don't matter now, maybe in a historical ranking but does the 1927 Yankees winning the world series have any impact on the 2011 Yankees winning it? Nope. So define a clan year and base the rankings on challenges begun during that year.