Saturday, March 08, 2008
The meaningfulness of Tests and ODI's
Today I want to statistically show what is obvious and logical — that there's a lot more luck in ODI's than in Tests. But something not quite so obvious comes up later.
I got the idea for this sort of analysis from this post by Tangotiger, who usually studies baseball statistics. (In that post, he gives a method for finding the standard deviation of the talent distribution in a league. I think that he actually estimates a lower bound for this quantity.)
I took all Tests and all ODI's between the top eight nations since 2003. (I did this in Statsguru. I didn't realise it before, but in the advanced filter, if you click on the 'view' near the right edge of the table, you get checkboxes rather than a dropdown menu. Do this for Team and Opposition, check the major nations, and in eighteen clicks you've excluded Bangladesh, Zimbabwe, and all the weird teams.) I then threw away any draws, ties, or no-results, and calculated the fraction of wins for each team. (So, e.g., a team with five wins, five losses, and five draws will have a fraction of wins of 0,500 — not 0,333.)
Tests:
ODI's:
The gap in winning percentage between top and bottom is much larger in Tests than in ODI's, which is what we would expect — Tests better differentiate between the quality of the teams. The standard deviation of the win percentage for Tests is 0,219; that for ODI's is 0,119.
So far, nothing you wouldn't have guessed. But it's interesting to compare this to what you'd expect from chance. That is, if every match that ends in a result were decided by a coin toss, what standard deviation would you expect? The SD for the number of wins out of n games would be sqrt(n/4), from the binomial distribution. The variance, being the square of the SD, would be n/4. The fraction of wins is the number of wins divided by n. Now, Var(aX) = a²Var(X), so the variance in the fraction of wins would be 1/(4n). So the SD would be sqrt(1/(4n)).
If you take n as the average number (for each team) of result matches in each set (31,75 for Tests; 106,25 for ODI's), you get the SD's expected from chance as 0,088 for Tests and 0,049 for ODI's.
What you'd like in a distribution of winning percentages is that it's clearly wider than what you'd expect from chance (so that you can conclude that the differences between teams are due to differences in the quality of their play, not just the luck of the day). Since the SD you'd expect from chance for ODI's is smaller than that for Tests (because more ODI's are played), you actually don't need the real SD for ODI's to be as large as that for Tests, in order to sort the teams out.
A simple way to quantify this (there may be a better way) is to take the observed SD divided by the SD expected from chance. For Tests, this is 0,219/0,088 = 2,47. For ODI's, it's 0,119/0,049 = 2,46.
Almost exactly the same! That's probably a little bit lucky — those numbers would probably be a bit further apart if I'd picked a different period — but it shows that, in terms of sorting out which the ranking of the teams, the balance between Tests and ODI's is about right. There were, over this period, about 2,4 ODI's played between these teams for every Test.
That doesn't mean I like all these ODI's! Each one of them is, in itself, much more meaningless than a Test match (at least outside World Cups, and except for draws, which are happily a minority of Tests these days). And even though they take up fewer playing days in total than Tests, each match is independent of the others. A bad day for a team doesn't matter — the teams start from scratch again in two days' time. In a Test match, of course, a bad day directly affects the remainder of the match.
I got the idea for this sort of analysis from this post by Tangotiger, who usually studies baseball statistics. (In that post, he gives a method for finding the standard deviation of the talent distribution in a league. I think that he actually estimates a lower bound for this quantity.)
I took all Tests and all ODI's between the top eight nations since 2003. (I did this in Statsguru. I didn't realise it before, but in the advanced filter, if you click on the 'view' near the right edge of the table, you get checkboxes rather than a dropdown menu. Do this for Team and Opposition, check the major nations, and in eighteen clicks you've excluded Bangladesh, Zimbabwe, and all the weird teams.) I then threw away any draws, ties, or no-results, and calculated the fraction of wins for each team. (So, e.g., a team with five wins, five losses, and five draws will have a fraction of wins of 0,500 — not 0,333.)
Tests:
team w l w%
Australia 35 7 0,833
England 25 16 0,610
South Africa 21 18 0,538
India 14 11 0,560
Pakistan 13 16 0,448
Sri Lanka 10 14 0,417
New Zealand 5 14 0,263
West Indies 4 31 0,114
ODI's:
Australia 93 34 0,732
India 61 69 0,469
South Africa 53 41 0,564
New Zealand 50 52 0,490
Pakistan 50 53 0,485
Sri Lanka 50 56 0,472
England 38 61 0,384
West Indies 30 59 0,337
The gap in winning percentage between top and bottom is much larger in Tests than in ODI's, which is what we would expect — Tests better differentiate between the quality of the teams. The standard deviation of the win percentage for Tests is 0,219; that for ODI's is 0,119.
So far, nothing you wouldn't have guessed. But it's interesting to compare this to what you'd expect from chance. That is, if every match that ends in a result were decided by a coin toss, what standard deviation would you expect? The SD for the number of wins out of n games would be sqrt(n/4), from the binomial distribution. The variance, being the square of the SD, would be n/4. The fraction of wins is the number of wins divided by n. Now, Var(aX) = a²Var(X), so the variance in the fraction of wins would be 1/(4n). So the SD would be sqrt(1/(4n)).
If you take n as the average number (for each team) of result matches in each set (31,75 for Tests; 106,25 for ODI's), you get the SD's expected from chance as 0,088 for Tests and 0,049 for ODI's.
What you'd like in a distribution of winning percentages is that it's clearly wider than what you'd expect from chance (so that you can conclude that the differences between teams are due to differences in the quality of their play, not just the luck of the day). Since the SD you'd expect from chance for ODI's is smaller than that for Tests (because more ODI's are played), you actually don't need the real SD for ODI's to be as large as that for Tests, in order to sort the teams out.
A simple way to quantify this (there may be a better way) is to take the observed SD divided by the SD expected from chance. For Tests, this is 0,219/0,088 = 2,47. For ODI's, it's 0,119/0,049 = 2,46.
Almost exactly the same! That's probably a little bit lucky — those numbers would probably be a bit further apart if I'd picked a different period — but it shows that, in terms of sorting out which the ranking of the teams, the balance between Tests and ODI's is about right. There were, over this period, about 2,4 ODI's played between these teams for every Test.
That doesn't mean I like all these ODI's! Each one of them is, in itself, much more meaningless than a Test match (at least outside World Cups, and except for draws, which are happily a minority of Tests these days). And even though they take up fewer playing days in total than Tests, each match is independent of the others. A bad day for a team doesn't matter — the teams start from scratch again in two days' time. In a Test match, of course, a bad day directly affects the remainder of the match.
# posted by David Barry : 08:41
Comments:
<< Home
First let's do this theoretically using my standard device of assuming that partnerships follow an exponential distribution (this should give us at least an order of magnitude estimate). The average partnership for each wicket (in Tests) is as follows:
1: 37,53
2: 39,18
3: 42,17
4: 40,99
5: 36,21
6: 33,02
7: 26,39
8: 21,54
9: 17,09
10: 13,39
Use these averages as the parameters in the exponential distribution that each wicketth partnership follows, and do a lot more algebra than I thought would have been necessary (2^9 - 1 terms come out of the resulting integral), and you get about 1 in 106.
The empirical answer is about 1 in 54 - there have been 87 innings in which the tenth wicket stand has been the largest, out of 4691 innings that included a tenth wicket partnership (these numbers are out of date by a couple of weeks - haven't updated the SA-Bdesh and Eng-NZ Tests).
I'm fairly happy with the agreement between theory and practice here - partnerships don't really follow the exponential distribution, and when you consider ten partnerships, any error gets magnified.
Post a Comment
1: 37,53
2: 39,18
3: 42,17
4: 40,99
5: 36,21
6: 33,02
7: 26,39
8: 21,54
9: 17,09
10: 13,39
Use these averages as the parameters in the exponential distribution that each wicketth partnership follows, and do a lot more algebra than I thought would have been necessary (2^9 - 1 terms come out of the resulting integral), and you get about 1 in 106.
The empirical answer is about 1 in 54 - there have been 87 innings in which the tenth wicket stand has been the largest, out of 4691 innings that included a tenth wicket partnership (these numbers are out of date by a couple of weeks - haven't updated the SA-Bdesh and Eng-NZ Tests).
I'm fairly happy with the agreement between theory and practice here - partnerships don't really follow the exponential distribution, and when you consider ten partnerships, any error gets magnified.
Subscribe to Post Comments [Atom]
<< Home
Subscribe to Posts [Atom]
