Experimental Ranking System

[Peter]

We have a new experimental ranking system. You can see the rankings for all players in the new system at our ranking page.

Player's now have an assigned skill value. This value starts at 100 for new players and can go up and down depending on the outcome of each game session. Winning matches will increase your skill level while losing can decrease it. The system determines how your skill level changes after each match using an adopted TrueSkill algorithm.

You can read more about this ranking algorithm if you follow the link, but in short it works as follows. Each player has an average skill value and an uncertainty value. The uncertainty tells us how much the system trusts that the average skill is actually correct. Initially the system will guess your skill is 100, but it says that with a big uncertainty. After playing a few matches your uncertainty will shrink as the system can compare you with other players and your mean will move towards your proper skill. If 4 new players compete in a match we get a ranking from 1st to 4th. We can then say with a little more confidence than before that the 1st player is probably above average, 2nd is only slightly above, 3rd may be a little below and 4th is further below average. Their skill levels will be adjusted accordingly and their uncertainty is decreased. The system is still more uncertain about the top and bottom players because maybe they could be miles above or below average, you can't really tell from only this match.

How much the skill level changes depends on how surprising the outcome is. A player with a low skill level winning over a player with high skill level will get a large skill increase. She is probably better than we previously thought. On the contrary, if the skilled player wins it's not surprising and the skill levels will not change much.

Another example is a low ranked player getting 3rd place in a 4 player game of otherwise skilled players. This can still increase the skill of the 3rd player because she did beat one high skill opponent. In the same game the 1st player will get a normal increase for beating opponents at mostly  the same skill.

Compared to TrueSkill we use a lower uncertainty to begin with. This means that the system is quite confident that new players are average and they will need to prove in several games that they get 1st or 2nd position. In practice this means that the rankings change at a slow pace. However, a new player can get to the top 20 in only a few games if she's playing against skilled opponents. It's also possible to reach the top by playing and winning several games against average and above average players.

We have chosen to weight games by the number of human players. A 4 player game will give 100% of the skill points, a 3 player game 60% and 2 player only 10%. AI opponents does not count anything for ranking so losing or winning to them does not affect your skill.

If a player abandons a game and the game is still finished legally it will affect the abandoning player's ranking.

All decimal places count in the skill level even if only the integral part is visible. When players get to the higher skill levels the increases and decreases will decline and the skill can never go above 1000.

We're working to show how each game affected your skill level. It will be linked from the player names in the ranking page and display plus or minus (+/-) the number of skill points per game.

The outcome of new games are not added to the rankings yet.

Note that this ranking system is in it's experimental stage and will likely be modified. We'd like your feedback so please discuss it in this forum thread:

Do you think the new ranking is fair or not?
Do you have any questions?
Would you like to see another way of ranking?

Regards,
The Dev Team

[piete]

Interesting update. (Congrats Kipley, you finally made it! ) Not being able to study the system more deeply at the moment, my personal ranking seems a bit high considering that I have played very few games lately.

Is there any weighing favouring more recent games? If not, I think even good players need to show constantly their skills, so that historical success is not a proof of current skills.

What is very much welcome addition is that not only victories count, which really gives chance to players that can't invest as much of their time as others (when I retire and my kids are grown up, I'll have a chance to play more...)

[MuleyMan]

I have not studied ranking algorithms and don't intend to.
So my ideas may clearly not work, but here they are.

There are players highly ranked and have not played in 6 months (ex. BaronHelix).  Use last 90 days on ranking or have games over 90 days old have less weight in the ranking.  I started with Muleyman, got typical abandons and losses while lerning the game, then started a nic (Wheetfarmer) that is higher ranked than MM even tho I have only played a few games recently with Wheet.  I went back to Muleyman since total games played mattered and wanted to concentrate stats mainly on one character.
Also, Abandons don't seem to matter much in rank (ex. BaronHelix)
I think tho that the main problem is using all stats and not limiting to a 2-3 month timeframe.

[data2008]

Thanks for the comments!

We plan to do the "diminish rating time-factor" next.
That means the Ranking displays the ranking among the currently active and consistently playing mule folks.
Like in tennis, if you do not show up in the future, your ranking will slowly drop over time.

So the new ranking is an actual measurement between competing players,
while we still keep the alltime-hiscore list which shows overall achievement regardless when it happended
(more like the alltime olympic medal statistic).

Since we store all information of each game, we can even reconstruct, who was in the top1000 at what time, so you see players from the past having much higher positions at one point in time, while they currently do not play anymore and therefore have lost their topspot ranking position. We could therefore display for each player a detailed page how his ranking and skill evolved over time and what players he played against together with the ranking outcome. A very basic page exist, if you click on a players name in the ranking list, but it will be much more improved.

[kipley]

(Congrats Kipley, you finally made it! )

Thanks, piete!  Though my being at the top of the new rankings may indicate a problem with the system.  I know for a fact there's a handful of players out there that can consistently outplay me.  This is not false modesty on my part, it's just the way it is.

And yet under this new system I outrank these players.  I'm not certain why.  Perhaps it's because these same players also tend to be very particular about whom they play against... as they are also players that go out of their way to only play in games with experienced opponents (and often highly skilled opponents, as well).  I'm not nearly as picky... when I want to play Mule, I tend to want to play now, and so I play against whatever first three players show up, whether they be experienced or complete newbies.  The only folks I refuse to play against are those that have an excessive amount of abandons (or certain "infamous" players mentioned ad naseum in these forums).

So perhaps the new system overly rewards playing against weaker opponents?  Because that's the only reason I can think of why I should outrank some of the worthier competitors out there.

[Mt-Wampus]

Who cares about rankings! just play for the fun of the game.

[Death_Mule17]

So perhaps the new system overly rewards playing against weaker opponents?

  I think your correct kip, as i noticed my rank shot past 120/ from current #20...I usually play as the lowest ranked in my nightly games with rodz an friends.

I guess im not as good at mule as i thought, oh well.

ps. @mt.wampus , I CARE ABOUT THE RANKINGS!!, I played for the fun of it back in the 80s... and never dreamed of being ranked on a global scale.
you know , people can still have fun while climbing the ladder against the best of the best and i think its a great addition to mule!

[Mt-Wampus]

Rankings are based on number of wins. Guys who have jobs and lives have no shot at a high ranking while the guy who spends his entire life on Planetmule is "Worlclass". Just dont agree with that. There is no realistic way to rank people that is fair to all. Hence the fact that i could care less about rankings.

[Chuckie Chuck]

I saw my ranking drop slightly, but I also feel it is more accurately reflective of my skill vs others.  There might be a feature to ad.  Be able to view both "Overall average ranking for duration of membership", and a second sort that ranks based on a 3 month history.  Perhaps include two ranking columns in the game engine that will reflect both sorts, so people can know both the players long term success and current average.

[rodz]

i have found my ranking drop from 1 to 5 with new system.
fair enough i suppose if this is the new system.
i will have trouble improving my ranking due to the time differences between me (nz) and other countries.
to play i need to play when i can and this tends to be out of sync with other top ranked players.
i am not to concerned because i have achieved what i set out to do initially which was to #1 enjoy playing mule,#2 play against real people. #3 play to the best of my ability

all i can say is thanks for making mule a game we can all enjoy and play against people from all over the world

[piete]

to play i need to play when i can and this tends to be out of sync with other top ranked players.

You're out of sync anyway because of the lag you cause!

i will have trouble improving my ranking due to the time differences between me (nz) and other countries.

Not exactly true, every victory still improves your skill score, although maybe in smaller steps. Keeping up your pace of games and winning majority of them still will get you to 1st place eventually.

[dynadan]

The new system seems like a good idea.    I can't really tell if the system is fairly implemented or not yet.  Need to see some more cause and effect.

I do agree with Piete and Muley Man about weighting the more recent games heavier.  And/Or make the rating go down 1 point every day of no playing.  Whatever the system, ideally you don't want people to get to the top and then be able to camp there without playing.

Anyway great addition yet again. Thanks for all the work.

[Bumbes]

Well, I like the new ranking system, especially with the coming "diminish rating time-factor".
Pre version 1.3.3 host advantages will disappear in the stats slowly, at least for that ranking.

The opponents statistic list is very interesting to watch also, keep up the good work.

[Rhodan]

I propose when the new ranking system goes live to reset everyone to zero.

[C64 nostalgia]

I propose when the new ranking system goes live to reset everyone to zero.

+1