Games in Progress: 3 | Players logged in: 4 | Players Registered: 37413 | Games Played Total: 68649
   Home   Help Search Login Register  
Pages: 1 2 3 [4]
  Print  
Author Topic: Experimental Ranking System  (Read 43804 times)
Mt-Wampus
Mule Senior
****
Posts: 125


View Profile
« Reply #45 on: November 01, 2010, 14:18 »

I considered punishing hosts, but it would just lead to fewer hosts and fewer games.  The developers have done a fantastic job removing the host advantages.  There are no longer any actual play advantages for the host.  So I don't think i can endorse a system that punishes hosts.]


     Punishing hosts ? Guys come on Planetmule and are begging for somebody to host a game so they can play and your talking about PUNISHING hosts Huh? Talk about ungratefull. Slap in the face to all legit hosts of the past who have been kind enough to fire up a game.
Logged
C64 nostalgia
Prototype Tester
Mule Senior
****
Posts: 159



View Profile
« Reply #46 on: November 06, 2010, 08:58 »

I noticed the new "Ranks" link on Planet MULE's main navigation bar. Along with it, the prototype ranking system appears to be continuously updating... I believe the developers have proceeded with their plans using TrueSkill as the basis for Planet MULE's ranking system.


TrueSkill's design allows itself to be used with a wide variety of Xbox Live games. This ability stems from almost singly counting wins and losses. However, MULE is more than just wins and losses. Thus, I suggest adding some MULE specific data to enrich the ranking results.

My idea starts by letting TrueSkill behave as it normally does, but before players' skill values are adjusted after a game, a step is added. This step is a skill point modifier (SPM) consisting of the unique things that make a M.U.L.E. win or loss special. The SPM is applied to the individual player's skill point change before it adjusts their skill value. Let me start listing and explaining the factors making up the SPM.

1.6x to the winner's SPM. This gives strong weight to a first place finish. Winning a game of MULE means taking first. First ought to be counted as being more important. The cliché of second is first loser is very apt for MULE.

0.2x (max) is added to the winner's SPM giving weight to the margin of a win. How much of 0.2 is calculated by taking the difference between first and second's score, dividing that by the second place's score, and finally multiplying that by 0.2.

0.2x (max) is added (or subtracted) to all player's SPM's based on colony scores. Using the colony achievements as a guideline, 0.2x is awarded for the highest achievement, a score of 120,000+; no award for the middle achievement, 60,000 to 79,999; and a penalty of 0.2x for lowest achievement, <20,000. The in between achievements use 0.05 and 0.1, respectively.

0.015x penalty goes to the host. This component is to compensate for having the best pings. While changes to the game have made for much less of the host advantage, it still exists. Notice the very small value of this component.


Implementing my suggestion would give meaning to wins and colonies (and help compensate for the privilege of hosting). The implementation would also help make the ranking system reflect specific qualities showing truly better MULE players.

edit: If I've been unclear or something doesn't make sense above, please feel free to point it out. I will try my best to explain my suggestion better...
« Last Edit: November 06, 2010, 19:55 by C64 nostalgia » Logged
C64 nostalgia
Prototype Tester
Mule Senior
****
Posts: 159



View Profile
« Reply #47 on: November 06, 2010, 20:50 »

I wanted my Skill Point Modifier idea to be self-contained in its own post. But, I wanted another post to add my other suggestions and thoughts. I will first lay out my full desire. Then, I will provide a compromise. (I apologize if some of the following is repetitive from other posts. My only excuse is the following is that important, and I'm trying to consolidate my positions in this and the previous post.)

Reset the ranking data This is a longstanding request of many players. Old game data is derived from many long outdated versions of Planet MULE. Versions that included countless bugs and significant changes in game mechanics. The old data will only negatively affect the integrity of game data from the current version of Planet MULE. Not only that, but resetting the ranking data will better show the best players because the rankings will reflect game strategies evolved from hundreds of games. (Resetting the rankings every major bug fix and/or gameplay change would continue to honor the best players of tomorrow's Planet MULE, as well.)

At the very least, reset the rankings starting from the last update. This game data is relevant and reflective of current strategies within the gameplay of the current game. Keep the old data displayed somewhere, but do not let it taint the new rankings.

Make games that start with AI's unranked I'm so very happy serious players as of late rarely play games that start with AI's. Earlier in Planet MULE's history, it was fairly common. This "botless" phenomenon is truly a marvelous evolution. With good reason, players avoid the AI. The AI is painfully laughable (My apologies to the guy who tweaked the AI's. My statement has nothing to do with him personally.). The old 1983 AI's play better. The ranking system should not reward wins using a predictable and bad player. AI's are worse than new players because they are extremely easy to game. They always play similarly. Furthermore once you learn their patterns, they become more like colluders and much less like real players.

At the very least make the skill value for an AI, the same as the lowest ranked player, currently 57 -- relatively accurate. Games that start with 2 AI's should definitely not be ranked. Again AI's are horrible players, and more importantly, doing this would also eliminate the ruse of players who habitually play 2 bots and a new player.
Logged
dynadan
Mule Regular
***
Posts: 93


View Profile
« Reply #48 on: November 07, 2010, 11:18 »


My idea starts by letting TrueSkill behave as it normally does, but before players' skill values are adjusted after a game, a step is added. This step is a skill point modifier (SPM) consisting of the unique things that make a M.U.L.E. win or loss special. The SPM is applied to the individual player's skill point change before it adjusts their skill value. Let me start listing and explaining the factors making up the SPM.

1.6x to the winner's SPM. This gives strong weight to a first place finish. Winning a game of MULE means taking first. First ought to be counted as being more important. The cliché of second is first loser is very apt for MULE.

0.2x (max) is added to the winner's SPM giving weight to the margin of a win. How much of 0.2 is calculated by taking the difference between first and second's score, dividing that by the second place's score, and finally multiplying that by 0.2.

0.2x (max) is added (or subtracted) to all player's SPM's based on colony scores. Using the colony achievements as a guideline, 0.2x is awarded for the highest achievement, a score of 120,000+; no award for the middle achievement, 60,000 to 79,999; and a penalty of 0.2x for lowest achievement, <20,000. The in between achievements use 0.05 and 0.1, respectively.

0.015x penalty goes to the host. This component is to compensate for having the best pings. While changes to the game have made for much less of the host advantage, it still exists. Notice the very small value of this component.


Implementing my suggestion would give meaning to wins and colonies (and help compensate for the privilege of hosting). The implementation would also help make the ranking system reflect specific qualities showing truly better MULE players.

edit: If I've been unclear or something doesn't make sense above, please feel free to point it out. I will try my best to explain my suggestion better...

I was wondering if it was possible for you to show us an example of how you would work the numbers?  I would very much like to see the difference for wins and losses from a 5 game sample using the equation you had in mind next to the same sample with just the TrusSkill system.  I think the margin of victory is a good idea, but i am not quite understanding your formula for that one.

Also I think you should just take out the host negative factor, a lot of votes will go against that.  And we really don't want to have less hosts on planet mule.  The land auction still isn't perfect, but other than that there is no more host advantage anymore.



Reset the ranking data This is a longstanding request of many players. Old game data is derived from many long outdated versions of Planet MULE. Versions that included countless bugs and significant changes in game mechanics. The old data will only negatively affect the integrity of game data from the current version of Planet MULE. Not only that, but resetting the ranking data will better show the best players because the rankings will reflect game strategies evolved from hundreds of games. (Resetting the rankings every major bug fix and/or gameplay change would continue to honor the best players of tomorrow's Planet MULE, as well.)

At the very least, reset the rankings starting from the last update. This game data is relevant and reflective of current strategies within the gameplay of the current game. Keep the old data displayed somewhere, but do not let it taint the new rankings.

Make games that start with AI's unranked I'm so very happy serious players as of late rarely play games that start with AI's. Earlier in Planet MULE's history, it was fairly common. This "botless" phenomenon is truly a marvelous evolution. With good reason, players avoid the AI. The AI is painfully laughable (My apologies to the guy who tweaked the AI's. My statement has nothing to do with him personally.). The old 1983 AI's play better. The ranking system should not reward wins using a predictable and bad player. AI's are worse than new players because they are extremely easy to game. They always play similarly. Furthermore once you learn their patterns, they become more like colluders and much less like real players.

At the very least make the skill value for an AI, the same as the lowest ranked player, currently 57 -- relatively accurate. Games that start with 2 AI's should definitely not be ranked. Again AI's are horrible players, and more importantly, doing this would also eliminate the ruse of players who habitually play 2 bots and a new player.


Since they are apparently keeping the high scores page, you have convinced me that resetting the stats may be a good idea.

I wouldn't make AI games completely unranked, but maybe a simple way to do it would be 1/2 value for all rating points for 1 AI games and 1/4 points for 2 AI.  On the current system this seems about right to me, but I am not sure how it will mesh with what you have in mind.  I am not exactly sure how TrueSkill is handling the AI's right now, it almost seems like they just leave them out.  if thats the case maybe leaving them out and just sticking a modifier on would be the way to go.

I am not against anything you have in mind, I would just like to see some more numbers so we can discuss the weighting.
Logged
C64 nostalgia
Prototype Tester
Mule Senior
****
Posts: 159



View Profile
« Reply #49 on: November 08, 2010, 03:48 »

My SPM suggestion and my Reset/AI Unranked suggestions both concern the ranking system. However, they pertain to separate aspects. So, I will reply in separate posts in an effort to not confuse the two.



Reset the ranking data This is a longstanding request of many players. Old game data is derived from many long outdated versions of Planet MULE. Versions that included countless bugs and significant changes in game mechanics. The old data will only negatively affect the integrity of game data from the current version of Planet MULE. Not only that, but resetting the ranking data will better show the best players because the rankings will reflect game strategies evolved from hundreds of games. (Resetting the rankings every major bug fix and/or gameplay change would continue to honor the best players of tomorrow's Planet MULE, as well.)

At the very least, reset the rankings starting from the last update. This game data is relevant and reflective of current strategies within the gameplay of the current game. Keep the old data displayed somewhere, but do not let it taint the new rankings.

Make games that start with AI's unranked I'm so very happy serious players as of late rarely play games that start with AI's. Earlier in Planet MULE's history, it was fairly common. This "botless" phenomenon is truly a marvelous evolution. With good reason, players avoid the AI. The AI is painfully laughable (My apologies to the guy who tweaked the AI's. My statement has nothing to do with him personally.). The old 1983 AI's play better. The ranking system should not reward wins using a predictable and bad player. AI's are worse than new players because they are extremely easy to game. They always play similarly. Furthermore once you learn their patterns, they become more like colluders and much less like real players.

At the very least make the skill value for an AI, the same as the lowest ranked player, currently 57 -- relatively accurate. Games that start with 2 AI's should definitely not be ranked. Again AI's are horrible players, and more importantly, doing this would also eliminate the ruse of players who habitually play 2 bots and a new player.


Since they are apparently keeping the high scores page, you have convinced me that resetting the stats may be a good idea.

I wouldn't make AI games completely unranked, but maybe a simple way to do it would be 1/2 value for all rating points for 1 AI games and 1/4 points for 2 AI.  On the current system this seems about right to me, but I am not sure how it will mesh with what you have in mind.  I am not exactly sure how TrueSkill is handling the AI's right now, it almost seems like they just leave them out.  if thats the case maybe leaving them out and just sticking a modifier on would be the way to go.

I am not against anything you have in mind, I would just like to see some more numbers so we can discuss the weighting.

Cool, note dynadan and I agree resetting the stats is a good idea.


First, I want to make sure everyone understands when I refer to AI games, I mean games that begin with AI players. I am not talking about games where an AI fills in for a player (timed out, bailed, router crash... et cetera).

I looked at akire1's (TrueSkill) ranks page. Most recently, when there are 2 AI's and 2 humans, the skill point changes tend to be 0.2 or 0.3 (If you go further back to older games, the change tend to be be higher but generally under 0.5.). For the higher placed human the change is positive and negative for the lower placed player. This seems to happen regardless of where the AI's place. The AI's appear to be placeholders more than anything else. I'm sure the developers could explain how they are currently treating AI's better than I can.

Nonetheless, AI's are horrible and predictable players. I'm not sure why you think games that start with AI players should be ranked. Maybe you could explain why you feel games that start with AI's are important to demonstrate a player's skill.

The reason I stress 2 AI and 2 human player games be unranked is because of the abuse repeatedly demonstrated by certain players. Taking a new player and using the AI's to dominate them should not be rewarded. Plus, with 2 AI's the dynamic of a game is very simple and consistently predictable -- hardly an arena to show high level skills.

That leaves games that start with one AI. Since AI's are horrible and predictable players. They should have horrible skill values. Players should feel pain when they lose to an AI. Another way, I suppose, is to treat the AI like a real ranked player. Then, it will receive a skill value according to its talent. To simply ignore an AI makes them more meaningless than they already are.


Logged
Rhodan
Prototype Tester
Mule Senior
****
Posts: 135



View Profile
« Reply #50 on: November 08, 2010, 04:34 »



Reset the ranking data This is a longstanding request of many players. Old game data is derived from many long outdated versions of Planet MULE. Versions that included countless bugs and significant changes in game mechanics. The old data will only negatively affect the integrity of game data from the current version of Planet MULE. Not only that, but resetting the ranking data will better show the best players because the rankings will reflect game strategies evolved from hundreds of games. (Resetting the rankings every major bug fix and/or gameplay change would continue to honor the best players of tomorrow's Planet MULE, as well.)

At the very least, reset the rankings starting from the last update. This game data is relevant and reflective of current strategies within the gameplay of the current game. Keep the old data displayed somewhere, but do not let it taint the new rankings.



Agreed.
Logged
C64 nostalgia
Prototype Tester
Mule Senior
****
Posts: 159



View Profile
« Reply #51 on: November 14, 2010, 00:31 »

I don't mind a new ranking system per se, however awarding people points for 2nd and 3rd place has a MAJOR flaw:

I would make certain plays during games to win or bust.  Sell a plot of land late, buy stite @108, etc.  Sometimes I take major gambles to WIN, not to play it safe and hope to come in second.  I have finished last many times when I could have just played it out and finished 2nd.  

I won a game last week making almost no ore or stite, but instead took all 4 river plots and won with food and energy; against very good players ranked in the top 100.  If a statagy like that bombs on me and I finish 4th and lose ranking, where is my motivation to try to experiment with something new?

The basic premise is wrong, the 4th place player is NOT nessessarily worse than the 2nd place guy and maybe even played better.  

If there is a new rating system, still award points for winning and nothing else.  Don't reward a player for just being the first loser.

One way to do it is by rewarding the winning player extra points for how much he wins by.
Winning by 5000 is a lot more impressive than winning by 500.  Although frankly I am happy with the ranking system the way it is.
My post above is my intellectual argument.  Here is my emotional one:

PLEASE, PLEASE, PLEASE, do not go to a Truskill type of ranking system.  I have played other games that use it and I can't stand it.  

It especially will not work in MULE because the sample of games isn't nearly large enough.

MULE is about winning people!

WINNING!  I don't want to feel good about coming in 2nd.

I did a very quick reread of this thread -- skipping dynadan's and my posts. While dynadan and I have posted at length in this thread, I wanted to bring back a couple of posts I really liked by GambitTime.

In agreement, M.U.L.E. is about winning! Wild risks for a play at first founder can make M.U.L.E. exciting. Furthermore, giving an incentive for a margin of victory gives the player in first a reason to take risks, too. Then, to balance that margin and give everyone a chance at a boost, a colony award is offered. This is my basis for almost all of my SPM.


A quick rerun through my skill point modifier (SPM) idea. The SPM is multiplied to the skill point change determined by the ranking system after a game, but before the skill point change is added to the player's skill value. (These exact numbers are somewhat arbitrary, but they serve as a general guide.) :

0.6x to the winner's SPM

0.2x (max) is added to the winner's SPM giving weight to the margin of a win. How much of 0.2 is calculated by taking the difference between first and second's score, dividing that by the second place's score, and finally multiplying that by 0.2.

0.2x (max) is added (or subtracted) to all player's SPM's based on colony scores. Using the colony achievements as a guideline, 0.2x is awarded for the highest achievement, a score of 120,000+; no award for the middle achievement, 60,000 to 79,999; and a penalty of 0.2x for lowest achievement, <20,000. The in between achievements use 0.05 and 0.1, respectively.

The 0.X factors were not explained well. The SPM always starts at 1x. The 0.X parts are added or subtracted. So, a win with the worst colony penalty becomes 1+ 0.6 -0.2= 1.4x. Or, a 2nd, 3rd, or 4th with the best colony award becomes 1+0.2= 1.2x for positive skill point changes and -1+0.2= 0.8x for negative skill point changes.

Another way to think about the SPM is by moving the decimal 2 places, thus everything becomes a percentage. 1x becomes 100% of the skill value change. A win, 1.6x becomes 160% of the skill value change. For positive skill point changes, one wants the highest percentage for the biggest upward movement. For negative skill point changes, one wants the lowest percentage to minimize negative movement.


The last bit about a hosting penalty... While I applaud the developer's efforts to mitigate "host advantages," a host still gets the best pings. Their side of their games will be the most responsive. Additionally, if the lag is big enough (as little as 250-300), their low pings can overcome the host advantage mitigators. For example, a higher-placed host can steal a plot from a lower-placed player. The penalty is small, but serves a noble purpose -- it tries to make things a bit more fair.

0.025x penalty goes to the host.


The problem with being able to give results from a 5 game sample is the developers are the only people who know Planet MULE's actual equations and variables in calculating the leaderboard. I don't know how a SPM will affect sigma values and player skill values (mu) over many games. Will the SPM make a meaningful difference? Will it muck up the uncertainty portion (the sigma stuff) and create strange and unreliable results? I just don't have the needed information (and potentially the skill and/or tools) to do the statistical samples, but I am very curious and hopeful.


[edit: proofing, more clarification, and added the percentage analogy]
« Last Edit: November 14, 2010, 03:39 by C64 nostalgia » Logged
C64 nostalgia
Prototype Tester
Mule Senior
****
Posts: 159



View Profile
« Reply #52 on: November 14, 2010, 01:05 »

Punishing hosts ? Guys come on Planetmule and are begging for somebody to host a game so they can play and your talking about PUNISHING hosts Huh? Talk about ungratefull. Slap in the face to all legit hosts of the past who have been kind enough to fire up a game.

The primary reason why people beg for hosts is because not enough people know how to port forward... Learning how can be time-consuming, frustrating, and for some, virtually impossible. Please do not confuse a lack of hosts with adding more fairness to the rankings.

The reason I advocate a cost for hosting is because I've seen (and played with) the benefits a player gets by hosting. I want games to be fair. It's that simple. Last, more often than not, the times I've seen someone choose not to host (assuming a reliable game can be set-up) is to give an advantage to a weaker player or a player with high pings.

[Part of the above was cut from my previous post. Players with Macs check out: A simple way to Port Forward on a Mac -- Port Map. Maybe it can help you with your hosting woes.]


[edit: proofing and clarifying]
« Last Edit: November 14, 2010, 04:13 by C64 nostalgia » Logged
data2008
Administrator
Mule Expert
*****
Posts: 288



View Profile
« Reply #53 on: November 14, 2010, 09:53 »

@C64-Nostalgia:
very good and thoughtout ideas!
We will think of implementing something like "SPM"s in the near future.
Logged
Peter
Turborilla
Administrator
Mule Expert
*****
Posts: 379


Planet M.U.L.E. Team


View Profile WWW
« Reply #54 on: November 15, 2010, 14:20 »

Thanks for all the feedback.

Quote
I'm sure the developers could explain how they are currently treating AI's better than I can.

To clear this up. AI opponents are removed when the ranking is updated. A game with 2 humans and 2 bots will be treated as a 2 player game. It doesn't matter where the AI's are placed.

Games with 2 humans are weighted by 10%
Games with 3 humans are weighted by 60%
Games with 4 humans are weighted by 100%

Weighted means that the change in the players mean skill value and uncertainty is reduced to the specified percentage.

For example if a player has a skill mean of 25 and uncertainty of 8 which should be updated with +2 mean and -1 uncertainty, and it was a 2 player game, then the change is reduced to +0.2 and -0.1.

Like C64's SPM idea, it's of course possible to weight the change in skill by any factors. C64's modifiers will mainly give you less of a penalty if you lose games and make 2nd place less valuable.

« Last Edit: December 22, 2010, 07:19 by Peter » Logged

Pages: 1 2 3 [4]
  Print  
 
Jump to: