Rating System
How bot rankings are calculated
Overview
RiichiLab uses the OpenSkill rating system (Weng-Lin, Plackett-Luce model) to rank bots. Unlike traditional Elo, OpenSkill natively supports multiplayer games and tracks uncertainty (sigma) alongside estimated skill (mu).
How It Works
Each bot has two internal parameters:
- mu — Estimated skill level (starts at 25.0)
- sigma — Uncertainty / confidence (starts at 8.333)
The conservative estimate of skill is called ordinal:
Display Rating
The ordinal value is converted to a familiar 1500-based scale:
All bots start at 1500. Winning increases mu and decreases sigma, making the rating rise.
Why OpenSkill?
| Feature | Elo | OpenSkill |
|---|---|---|
| 4-player native | No (pair decomp.) | Yes |
| Uncertainty tracking | No | Yes (sigma) |
| Inflation resistance | Low | High |
| Patent-free | Yes | Yes |
Skill Dynamics (tau)
OpenSkill assumes fixed skill levels. Without adjustment, sigma converges after many games and the rating stops responding to improvements. To keep ratings responsive, RiichiLab applies a dynamics factor (tau) that adds a small amount of uncertainty before every rating update:
With tau = 0.3, sigma never drops below ~1.5, keeping a floor on how much each game can shift the rating. If you improve your bot, the rating will catch up within roughly 100 games without needing to re-register a new bot.
Sigma Decay (Inactivity)
If a bot does not play for more than 30 days, its sigma increases by 0.26 per month (capped at 8.333, the initial value). The decay is applied weekly, every Monday at 00:00 UTC, in prorated steps of about +0.06 sigma (roughly −4 display rating points per week). This lowers the display rating, reflecting that the bot's true strength is uncertain after a long absence.
When the bot resumes playing, each game has a larger impact on its rating, allowing it to quickly converge back to its true skill level.
Leaderboard
The leaderboard shows bots that are active (status = active) and have at least one game.
FAQ
Why is my bot's rating lower than expected?
New bots start with high uncertainty (sigma = 8.333). The display rating uses a conservative estimate (mu - 3*sigma), so it starts low and rises as the bot plays more games and sigma decreases.
How many games does it take to stabilize?
Sigma decreases rapidly in the first ~30 games and stabilizes around ~50 games. After that, rating changes per game become smaller and more stable.
I improved my bot. How long until the rating catches up?
Thanks to the dynamics factor (tau), your rating will track skill improvements over roughly 100 games. There is no need to create a new bot — just deploy the improved version and keep playing.
Does beating weak bots inflate my rating?
No. OpenSkill considers opponent uncertainty. Beating a bot with high sigma (uncertain skill) provides less rating gain than beating a well-established bot.