Pappus' plane - cricket stats

Note on the probability of century

2012-01-08T06:52:00.002+01:00

This is mostly for myself so that I have something to refer to in future. In the previous post I skipped some algebra and gave a formula for the probability of a century, given a hazard function which is 3/(avg+3) on nought and 1/(avg+3) for all scores greater than nought.

The algebra I originally posted was slightly wrong (I have corrected it in the second edit), but could have been simplified. The probability of a score of at least 100 is [avg/(avg+3)] * [(avg+2)/(avg+3)]⁹⁹. It is a little bit annoying having such a high exponent. Re-write (avg+2)/(avg+3) as [1-1/(avg+3)]. Then from a definition of the exponential function, raising that to the power of 99 is approximately equal to exp[-99/(avg+3)].

So, the probability of a century is [avg/(avg+3)] * exp[-99/(avg+3)].

Empirically, the simple exp(-100/avg) is a better predictor of the fraction of a batsman's innings that are centuries. If we exclude not-outs below 100, then it is much of a muchness between those two and exp(-99/avg).

Duck to century ratios

2011-12-26T06:38:00.004+01:00

Ric Finlay on Twitter:

SMarsh Oz's 1319th Test duck. 747 Oz centuries. Oz 1.77 ducks per century Ind 2.01 ducks per century. Bangladesh 9 ducks/100!

As a bit of pointless fun, let's pretend that all representatives of each country are equally good batsmen (a terrible assumption, obviously, but perhaps the tail-enders' ducks and top-order centuries will roughly cancel out in the right ratios) and apply the hazard function mentioned at the end of this post.

This hazard function is the simplest extension of the constant-hazard model, the latter saying, effectively, that a batsman always has his eye in. In the "new" model, the batsman is three times more likely to get out on nought than to get out when on some other score, but gets his eye in as soon as he's off the mark. When studying the proportion of ducks, this is an important change to make, because the constant-hazard model will under-estimate the number of ducks.

The algebra is boring and I will skip it. The probability of a duck is 3/(avg + 3), and the probability of a score less than 100 is 3/(avg + 3) + {1 - [(avg+2)/(avg+3)]⁹⁸} * avg/(avg + 3). Subtract the latter expression from 1 to get the probability of scoring a century.

Throw it into an Excel formula, and you find that a duck:century ratio of 9 corresponds to an average of about 21.1, a ratio of 2.01 corresponds to an average of about 30.1, and a ratio of 1.77 corresponds to an average of about 31.1.

The actual averages are 20.6, 31.5, and 32.3 respectively.

(Edit: Fixed the exponent in the probability for a score less than 100.)

(Edit 2, 8/1/2011: The earlier edit fixed the typo in my transcription from my handwritten work to the blog, but the handwritten work was wrong. The exponent should be 99, not 98. The corresponding averages to the three duck:century ratios should be about 21.3, 30.4, and 31.4 respectively.)

The World Cup group stage

2011-02-08T07:09:00.002+01:00

Russ has done some ODI ratings and looked ahead to the group stage of the World Cup. His main point is that it is more likely than a side from outside the "top eight" will make the quarters. (I need to use scare quotes, because Bangladesh are actually ahead of the West Indies in the ICC ODI rankings now, though not in Russ's rankings.)

But there's more to the group stage than seeing if Bangladesh will beat the Windies (I'm writing off Ireland – they were easily beaten by Bangladesh in the one series they've played in Asia). In the quarter-finals it'll be A1 v B4, A2 v B3, etc., so finishing top of the group will probably mean an easy quarter-final. I'm not sure if this'll be enough to keep us interested, but it is a real effect.

Using Russ's ratings (so don't blame me for India being easily below Australia and South Africa) and assuming that the "top eight" sides make the quarters in rating order, I ran 100000 simulations of the knockout stage. These are the probabilities of overall victory than I get:

Aus 0.28     SA  0.28
SL  0.13     Ind 0.18
Pak 0.04     Eng 0.08
NZ  0.01     WI  0.01

Now suppose that Australia, through bad luck, loses its games to SL, Pak, and NZ, and so finishes fourth in group A, thus playing South Africa in the quarters:

SL  0.22     SA  0.20
Pak 0.05     Ind 0.21
NZ  0.01     Eng 0.11
Aus 0.19     WI  0.01

Sri Lanka went from having less than half Australia's chance at overall success, to having a better chance than Australia. Obviously South Africa takes quite a hit in this scenario as well.

What if we only swap India and South Africa?

Aus 0.28     Ind 0.21
SL  0.13     SA  0.25
Pak 0.04     Eng 0.08
NZ  0.01     WI  0.01

A little change.

These ideas should motivate the teams to play hard in the group stage – there won't be any really dead rubbers against the major sides (except maybe at the very end). Of course the winning team will still have to win three straight games to take the Cup, so my suspicion is that from a fan's perspective, this won't add much excitement to the group matches.

Conversions from 50 to 100

2011-01-21T13:21:00.002+01:00

In comments at Kartikeya's blog there was a little aside about Samit Patel's conversion rate – he has only 10 first-class centuries, despite reaching fifty 34 times. Kartikeya said that such a low conversion rate was typical of players who bat at 6 or 7, and gave the example of VVS Laxman.

(From the few scorecards I've checked, Patel seems to often bat at 4 for Notts.)

The breakdown of Laxman's record is indeed stark: batting at 6, he averages 51 and has made 5 centuries having reached fifty 25 times; batting at 3, he averages 47 and has made 4 centuries having reached fifty 10 times.

The obvious question is, is this typical? This seems like as good an excuse as any to use the aside mentioned at the bottom of this post. In that basic model, batsmen effectively bat like they average 2 runs more per innings once they get off the mark. So, their conversion rate from 50 to 100 should be, on average, exp(-50/(avg+2)).

Here is a scatterplot of actual conversion rate against expected conversion rate, for all batsmen who've scored 2000 runs batting at positions 1-4:

The red line is y=x. There are 79 batsmen above the line and 75 below, so the model seems pretty decent.

Now here is the same scatterplot for batsmen at positions 6-7:

Only 6 of the 27 batsmen are above expectation, presumably because they're left stranded or have to start hitting out with 9 wickets down. Laxman is the point (0.406, 0.179). He's on the bottom edge of the scatter, so his very low conversion rate somewhat unusual, even for lower-middle order batsmen. The regression line forced through the origin is y = 0.82x.

Returning to the "purer" sample of top-order batsmen, we can ask whether conversion from 50 to 100 is a skill. Using the same method as in the post I linked to earlier, we can treat "scoring a century, having reached fifty" as a binomial random variable, which happens with probability p = exp(-50/(avg+2)). If a batsman has reached fifty N times, then we can calculate z = (actual number of hundreds - Np) / sqrt[Np(1-p)]. If "scoring a century, having reached fifty" is a skill separate from the batting average, then we'd expect the standard deviation of z's to be greater than 1.

As it happens, the standard deviation is 0.93. Perhaps that'd increase a little bit if you treated not-out innings between 50 and 99 properly. But it looks to me like most of the variation in conversion rates between top-order players is down to differences in their general batting ability (as measured by their average) and random luck.

Update on no-ball rates

2010-12-29T13:58:00.002+01:00

It's been a few years since I wrote this, noting the sharp decline in the no-ball rate in ODI's since the start of Twenty20. The trends don't appear to be a blip, as we now have three more years of data to look at, and the evidence is pretty solid:

The drop-off is also quite visible, though not as dramatic, in Tests:

In less than a week, my almost month-long holiday from work will be over, so the recent burst of activity on this blog will probably end. I'll have a couple more blog posts in the next few days, and then I'll probably go back to just being a commenter around the place.

18th century statistics

2010-12-15T15:27:00.004+01:00

The ACS seems to have settled on a list of 'great' matches from the 18th century, starting from 1772, considered equivalent to first-class for statistical purposes. CricketArchive now lists these matches as first-class, starting with Hampshire v England.

If any of the few remaining readers that I have were here three years ago, they might remember a series of posts on 19th-century first-class cricket in England. The biggest challenge was estimating bowling averages for the very early scorecards, which don't credit catches to the bowler, and don't record the number of runs conceded by the bowler. The method is described in this post, with results (with funny +/- values!) here.

Over the last few days I've dug up my old code and run the estimations on the "new" first-class matches, and added these numbers to the 19th-century estimations where applicable.

Things to remember:

- The error in the bowling average "should" be about 8%, in the sense that about 68% of the true averages should be within 8% of the estimated figure. But in the testing runs I did a few years ago, very high wicket-takers seemed unusually likely to get anomalous estimates of their average (out to around 15%).

- The exercise is even more speculative for these very early matches. The training data is from 1854-1863, and I'm here using it to find stats in games played more than 80 years before that.

- Nevertheless, they can't really be too far wrong.

- The estimates of wickets, unlike bowling averages, can be atrocious. They are all almost certainly under-estimates, by seemingly random amounts.

- I only consider matches where the dismissals are all known, and it was eleven-a-side. (So the number of matches played is slightly lower than what you'll see on CricketArchive.)

Here are the leading wicket-takers (at least 150 estimated wickets) of players who began their career in the 18th century, ordered by estimated bowling average (their batting stats are also there):

                                                              batting          est. bowling
name                start   end     mats    inns    n.o.    runs    avg    wkts   runs      avg
D Harris            1789    1798      72     124      42     467     5.7   481.7  4985.1    10.3
John Wells          1789    1815     138     256      19    2927    12.4   551.3  6134.9    11.1
T Boxall            1790    1803      81     147      28     822     6.9   465.7  5251.4    11.3
T Walker            1789    1810     166     315      19    5757    19.4   486.0  5488.7    11.3
T Lord              1790    1815      55     100      15     831     9.8   217.0  2495.4    11.5
Lord F Beauclerk    1791    1825     124     229      19    5259    25.0   577.5  7047.1    12.2
R Purchase          1773    1803     109     199      18    1820    10.1   373.8  4572.2    12.2
E Stevens           1773    1789      71     126      37     694     7.8   496.0  6112.4    12.3
W Beldham           1789    1821     178     328      18    6709    21.6   376.0  4704.5    12.5
J Hammond           1790    1816     114     206      13    3741    19.4   244.3  3098.8    12.7
R Clifford          1789    1792      68     131       7    1484    12.0   343.5  4396.1    12.8
T Brett             1773    1778      24      43      11     231     7.2   154.4  2180.9    14.1
W Fennex            1790    1816      82     155      14    1881    13.3   220.6  3239.0    14.7
W Bullen            1789    1797     108     207      43    1697    10.3   289.5  4271.4    14.8
R Nyren             1774    1786      41      76      11     824    12.7   162.2  2459.1    15.2

My knowledge of 18th-century cricket is pretty close to zero, so I had to look most of these players up. The Who's Who of Cricketers says of David Harris: The greatest bowler of the Hambledon Club, it is impossible to gauge his real success owing to lack of bowling analyses. Well now we have a bit of a gauge.

When I did the estimates for the 19th century, there were players (such as Alfred Mynn) who played in many matches with their bowling analyses recorded, and in many without. These players serve as a useful check on the method – if their estimated average was similar to their actual average in matches with known figures, then that adds to the confidence we can put in the estimates.

For the 18th century, that's not possible. We instead have to add another link in the chain – take players who played in both centuries, whose 19th-century estimates we're pretty confident about because of the Mynn-like players from later on. For whatever that's worth, we can look at Wells, Walker, Beauclerk, Beldham, and Hammond, and make the comparison.

player    18th C   19th C
Wells     10.9     11.4
Walker    11.2     12.0
Beauclerk 11.3     12.6
Beldham   12.5     12.4
Hammond   12.9     12.1

I don't think I'd want to draw too many conclusions from such a small sample, but it at least passes a sanity check.

In case anyone's interested in batting stats, here they are (qual. 2000 runs):

                                                              batting           est. bowling
name                start   end     mats    inns    n.o.    runs    avg    wkts   runs      avg
Lord F Beauclerk    1791    1825     124     229      19    5259    25.0   577.5  7047.1    12.2
R Robinson          1792    1819     102     196      15    3992    22.1    46.6  1142.7    24.5
W Beldham           1789    1821     178     328      18    6709    21.6   376.0  4704.5    12.5
T Walker            1789    1810     166     315      19    5757    19.4   486.0  5488.7    11.3
J Hammond           1790    1816     114     206      13    3741    19.4   244.3  3098.8    12.7
J Aylward           1773    1797      99     194       6    3611    19.2     5.0   103.8    20.8
H Walker            1789    1802      93     171       4    2518    15.1     0.0     0.0     0.0
J Small sen         1773    1798     100     189       8    2724    15.0     7.1   158.1    22.4
J Ring              1789    1796      83     164      10    2088    13.6     2.5    48.3    19.3
J Small jun         1789    1810     134     252      13    3216    13.5     0.0     0.0     0.0
A Freemantle        1789    1810     125     235      28    2674    12.9     0.0     0.0     0.0
John Wells          1789    1815     138     256      19    2927    12.4   551.3  6134.9    11.1
Earl of Winchilsea  1789    1804     124     235      10    2048     9.1     6.5   112.5    17.2

Are some batsmen nervous starters?

2010-12-03T10:51:00.002+01:00

Probably. But the ability to get off the mark seems to be determined by how good a batsman is overall. There is of course variation between batsmen in the percentage of ducks they make, but no more than would be expected by random chance.

The starting point is to work out what the relationship is between a batsman's average and the percentage of innings that are ducks. (Ideally I would exclude scores of nought not-out from this analysis, but I did everything with Statsguru because it's easier. This won't make much of a difference.)

I took all batsmen with at least 20 Test innings against top-eight sides and put them into 'buckets' – the first bucket had batsmen who averaged less than 10, the second averaged 10-19.99, the third 20-20.99, etc., up to 50-59.99.

Then for each bucket, I sum up the number of ducks and divide by the number of innings to get the percentage of ducks. I also find the overall average of all the batsmen in the bucket.

Now, as discussed in this old post, the probability H(x) of getting out on a particular score x is related to an 'effective average' µ(x) by µ(x) = 1/H(x) - 1.

Since we will be plotting against the overall average, it makes sense to use the effective average on nought rather than the percentage of ducks. The result is a lovely linear plot:

Note that the problem of nought not-out innings is particularly acute for the first data point, which is full of people who batted at number 11. These innings make it look like the batsmen were better at getting off the mark than they really were, thus improving their apparent effective average. The regression line has been forced through the origin, both because logically it should do so, and so that the problem of the nought not-outs is reduced.

By a wonderful quirk, the effective average on zero is (on average) one third of the overall average. This makes the algebra relatively easy (details left as an exercise): a batsman's expected fraction of ducks is 3/(3 + avg).

What I then did was, for each individual batsman, calculate the number of binomial standard deviations his actual number of ducks was from his expected number of ducks.

As an example, consider Shane Warne. Average 17.65, so expected duck fraction 3/(3 + 17.65) = 0.145. He played 194 innings, which gives an expected number of ducks of 28.18. Warne actually made 34 ducks. A standard deviation for a binomial random variable is sqrt[N*p*(1-p)] = sqrt(194*0.145*0.855) = 4.9. Warne's number of ducks is therefore (34 - 28.18) / 4.9 = 1.2 standard deviations above expected.

If getting off the mark is a particular skill that some players are better at than others, independent of their overall batting abilities, then the standard deviation of the standard deviations should be greater than 1. If the only two factors going into the number of ducks are the overall batting average and random luck, then the sd of sd's should be 1.

The sd of sd's for all the batsmen who average more than 10 is 0.98, pretty close to 1.

(The breakdown by bucket goes like this. 0-9.99: 1.16 (but remember the problem of nought not-outs). 10-19.99: 0.82. 20-29.99: 1.01. 30-39.99: 0.98. 40-49.99: 1.04. 50-59.99: 1.07.)

By contrast, if you assume that there is no distribution of skill whatsoever in getting off the mark, and just assume that everyone (from Chris Martin to Sachin Tendulkar) gets off zero with equal probability (0.0917 in this sample), then the sd of sd's is 1.34, much greater than 1.

So my conclusion is that if someone seems to make an unusually large number of ducks, then he's almost certainly just unlucky.

Mathematical aside: Usually when I need to model the distribution of a batsman's scores, I use the geometric or exponential distribution. One level more advanced than this would be to have the hazard function take on a particular value at zero, and then a constant for scores greater than or equal to 1.

Using the above result, such a hazard function is this:

H(0) = 3/(avg + 3), H(n) = 1/(avg + 3) for n > 0.

My database

2010-07-31T04:42:00.002+02:00

Tests, first-class matches. Have at it.

Post any questions you have in comments, or by email. I have no idea how easy it is for someone else to make sense of what I've done.

The co-efficient of variation

2010-07-18T11:44:00.002+02:00

Gabriel Rogers debuted at It Figures with a post on batsmen's consistency. The main tool he used was the co-efficient of variation – the standard deviation divided by the mean. In general I think this is OK, but there is a problem with including players with short careers in the analysis.

The problem is that shorter careers might tend to have lower CV's. (I haven't checked this empirically.) To show this I'll play with exponential random variables. The distribution of a batsman's scores is reasonably close to an exponential distribution, so the results below should apply to real batsmen.

I generated 10000 "careers" of 2 innings, 10000 careers of 3 innings, 10000 careers of 4 innings, and so on. For each career length, I calculated the average CV. This is a graph of the results:

(I've used the "N-1" version of the standard deviation here.)

The theoretical CV for an exponential distribution is 1 (the standard deviation equals the mean; for real cricketers the typical CV is about 1.05, because the distribution is skewed by lots of ducks and low scores, and occasional very big scores), and you can see that for moderately large careers, this is true – the average CV for a 50-innings career is about 0.98. But for short careers the CV's are noticeably less than 1. For a two-innings career, I think the expectation of the CV is 1/sqrt(2).

My guess is that, if this effect carries over to real cricketers, then the trend shown in Figure 1 of the linked blog post is actually stronger than it looks – batsmen with shorter careers tend to be worse and have lower averages, so there'll be disproportionately many dots in the lower-left part of the scatterplot.

Of course I could check this myself, but I am pretty lazy with stats these days, as evidenced by the very long break in posting here!

New cricket stats analysis blog

2009-03-22T00:51:00.002+01:00

While I'm being lazy and not doing any real updates (I am getting closer to writing all the readme files etc. for my database), you can get your stats fix from this blog, which has been around for a few months and is generally very good. It is likely that some of the new ideas will need to be improved over time, but they are very valuable first step, especially on declaration stategies (on this, Elliot Tonkes has done some academic work on this, but I don't know if it's been published) and wins above replacement for ODI's.

Do check it out.

Batted ball speed

2009-03-15T11:33:00.002+01:00

Hey look, I have this thing called a blog, maybe I should update it.

I should update my profile – I'm no longer a student (sort of, my Masters thesis is under assessment), having got out of my PhD and started a 9-5 job, which is in geostatistics, if anyone is wondering.

I've been very slack with cricket stats lately, with work and chess taking up more of my time. Anyway, this is a long and interesting discussion about a baseball hitting a baseball bat. The very counter-intuitive result is that the batted ball speed doesn't depend on the grip, as long as the ball hits the "meat" of the bat. The same is true for cricket (this page gives a few physicsy aspects of cricket). All that matters for batted ball speed is the speed of the bat at impact.

On a different topic, I have been thinking about putting my cricket database online. It wouldn't be pretty, and any professional coders out there may be horrified at my code, but there are enough of you out there with good ideas that I think it would be worthwhile. In the long term, I would like there to be a cricket equivalent of Retrosheet, which now has over 50 years' worth of play-by-play data for Major League Baseball.

I'm sort of thinking out loud on this at the moment. My database uses CricketArchive player and match ID's, so if they changed their numbering system (which they did a few years ago), that'd screw things up. I might exchange emails with them and see what comes out of it.

Feel free to share your thoughts on this.

Form

2009-01-04T02:38:00.003+01:00

Russ in a post here suggests a way of measuring form (go and read that post, it's full of goodies). The broad question is, if there are two batsmen with the same true talent, and one has better recent form, will that batsman tend to average more in his next innings?

To start with, Russ weights more recent innings more heavily than earlier innings. Specifically, the k-th innings for a batsmen with N innings in his career is weighted 0.95^N-k. (This is, I believe, similar to what the ICC rankings do.)

Now, one problem with assessing "form" statistically is that a batsman will usually play a series against one team, followed by a series against another, etc. Since one of those two opposition teams can have a much stronger bowling attack than the other, what may appear to be good form and bad form may simply be a result of playing against weak bowlers and then strong bowlers. So, for everything I do in this post, I'll adjust the batting averages by the quality of the attack, as explained here.

So, when I talk about a regular average, I really mean an adjusted average. When I talk about a weighted average, I'll mean Russ's weighting by how recent the innings was (each innings also being adjusted for the quality of the attack).

Before continuing about form, I'll just look at the weighted average as a predictive tool. For all batsmen with at least 50 innings, I calculated career-to-date averages and weighted averages, as well as a 10-innings moving average. Then, from the 11th innings of each batsman's career, I calculated the absolute difference between his next innings and each of those three measures. (If the innings was a not-out, I used the not-out score as the absolute difference for each measure.) Then I averaged these errors. I did the same for all batsmen, and then found the "average average" error. The regular average was the best predictor, about 1% better than the weighted average, and 4% better than the moving average. The weighted average becomes more accurate if the 0.95 in the formula is increased towards 1, but it is always worse than the regular average.

So, as a measure of true talent of a batsman, I'll use the regular average rather than the weighted average.

Now to the question of defining form. Russ does this by taking a weighted log average. Defining R_i as the runs scored in the i-th innings, and w_i as the weight of that innings, this weighted log average is:


     / SUM w_i log(R_i) \
exp |  -------------  |
     \     SUM w_i    /

I've actually modified this a bit. If the i-th innings is a not-out, I didn't include it in the sum in the denominator. I hope this isn't too great a crime against statistics.

The measure of form is then the ratio of the weighted log average to the weighted average. Now, if scores are distributed exponentially, then this ratio is about 0.56 (well, it is with equally weighted innings at least). If a batsman makes the same score every innings (and gets out!), the ratio is 1. If a batsman recently has one big score and a bunch of little scores, the ratio is down towards 0.3. So, good form is a high ratio, bad form is a low ratio.

Because I exclude not-outs in the denominator, it's possible to get ratios greater than 1. I'm not really sure how to interpret these, but let's carry on anyway.

Russ's hypotheses are (I hope I've got this right):

a) If there are two batsmen with a similar average, one with a typical ratio and one with a low ratio, then the one with the low ratio will tend to average more in his next innings. The logic here is that the batsman with the low ratio is capable of larger scores, whereas the other batsman is just not so good.

b) Given two batsmen with the same average, one with a high ratio will tend to do better in his next innings than one with a typical ratio.

Both of these are correct, somewhat to my surprise. I went through all batsmen, and for each innings (after the tenth in their career), calculated the career-to-date average, and the ratio-to-date, and binned them as in the table below. I then calculated the overall average for each bin.

Ratios are down the left-hand side, averages across the top. The figures are the low end of the bin. So, e.g., the '5' means that the bin is for averages 5 to 9.99, the '10' is for averages 10-14.99, etc. Only bins with at least 50 innings are shown; bold is used for at least 100 innings.


r/a   5     10    15    20    25    30    35    40    45    50    55
0.35              23.4  28.4  28.7  40.8  41.3  
0.40  12.6  18.8  21.4  31.7  30.5  34.9  40.3  40.7  55.3  
0.45  7.4   12.1  15.9  20.9  26.7  30.1  34.0  46.9  39.9  50.2
0.50  9.6   11.1  17.6  22.2  26.4  31.7  36.3  34.9  43.5  46.0  
0.55  8.4   11.9  16.5  21.0  26.1  31.6  35.2  40.1  42.6  53.3  
0.60  7.6   12.2  18.3  24.4  27.9  33.5  38.2  42.7  46.6  58.0  40.4
0.65  7.8   12.3  18.1  24.7  28.0  33.4  39.2  43.3  48.7  48.1  46.6
0.70  8.2   12.4  19.3  25.7  27.5  34.4  40.4  46.1  44.0  56.7  
0.75  9.3   15.3  17.4  23.1  29.3  38.9  41.0  47.4  51.8  50.1  
0.80  9.8   12.0  17.7  24.4  30.5  35.1  48.2  54.2  51.8  58.9  
0.85        15.0  22.1  25.8  38.8  40.5  44.6  51.3  61.9  46.8  
0.90        15.6  26.7        42.1        42.6  59.2  54.1  
0.95                                            46.5
1.00        16.6        35.9  32.5  44.7  53.9  68.5  73.2  62.9

When the ratio is very low, the batsman does indeed tend to average much more in his next innings. (Since I've used regular averages to define the true talent, the top row may be full of players early in their career. I'm not sure.) Going down each column, the minimum is usually somewhere around 0.5 to 0.6, which seems to correspond to the 0.56 that you'd expect from the exponential distribution.

Really good recent form seems to give a 20% boost and sometimes more. This is a lot more than I had expected.

(My thinking on this issue seems to have been confused — in my last post I said that Johnson was good because he kept getting starts, which is consistent with this analysis.)

Johnson's batting

2008-12-20T04:32:00.002+01:00

I've only seen Johnson bat once (at the Gabba against NZ), and he looked very good. Looking through his Test scores to date, he's only been dismissed in single figures in a quarter of his innings. If you assume an exponential distribution of scores, that is consistent with an average of about 35. Currently he's averaging low 20's.

So, there are two main possibilities — he's been lucky to make so many starts, or he's been unlucky to get out for 20odd so often. Batting at number 10 probably isn't helping him.

I'd like to see him moved up to number 8. At worst, he'll be about as good as Lee. At best, I think he could be a very good bowling all-rounder.

Sorry for the lack of posting lately. I'm trying to finish off my Masters thesis (transferred down to PhD), I will start a proper job in February, and I'm spending much of my spare time playing and studying chess.

Can Dan Cullen bowl a doosra?

2008-11-15T00:08:00.002+01:00

I ask because I saw him bowl balls spinning from leg to off during the warmups last night (the warmups was the only time I was in line with the bowler). Some of them I picked as leg breaks, but others I didn't, making me wonder if he can bowl a doosra now. It's equally possible that I wasn't paying close enough attention to his hand.

Other notes from last night's game:

- What would be a good name for a team which includes Michael Dighton, Dan Marsh, and Ryan Harris? I know, All Stars.

- Brendan Drew was bowling nice little outswingers in the warmups.

- When Xavier Doherty was fielding at long off a few metres from me, I resisted the temptation to tell him that he's got the worst first-class average of any bowler ever. It's only just false. With the qualification of 5000 balls, the only people with worse averages are a guy who only played for Oxford University, Sachin Tendulkar, and Stuart Saunders, who at least averaged 25 with the bat for Tassie. But Doherty's pretty good at limited-overs cricket. Weird.

- When Magoffin (I like Magoffin — he was the only guy who stuck in my memory from the Queensland Academy side I saw play India in 2003/4) was bowling to Sean Marsh, Gilchrist set a 7-2 field. It was a disgustingly negative tactic. Captains have a responsibility to make sure that the game is entertaining for the spectators. It's because of people like Gilchrist that people are turning away from Twenty20 cricket.

Keepers

2008-10-21T02:16:00.002+02:00

I've got what I think is an interesting post up at it Figures. It ranks wicket-keepers by the rate at which they let byes through, adjusted by country. It doesn't work for keepers who kept up to the stumps to fast bowling a lot (i.e., keepers from the olden days), but I'm happy with how well it works for modern keepers. Here is the full list (qualification: 20 Tests) for those who want to just see the results.

I must admit I hadn't heard of some of those guys near the top. Kirmani rang a bell, but Tamhane was new to me. I was happy to see in his Cricinfo profile that Wally Grout compared Tamhane to Don Tallon. It's unfortunate that Tallon (and Grout, and the rest) kept up to the stumps so often. It makes cross-era keeper comparisons difficult. (You can't just go by the prevailing rate of byes in world cricket at the time, otherwise modern keeper-batsmen would come up as the equal of Knott, Taylor, etc.).

Ideally, we'd be able to take this one step further and get a well-founded measure of a keeper-batsman. But the main difference between a good keeper and a bad one is the number of dismissals effected, and it would be close to impossible to get accurate estimates on, e.g., how many dismissals Knott would have had if he'd kept for Pakistan in the 1990's.

Free hits

2008-10-08T12:26:00.000+02:00

We're on the eve of what should be another fantastic battle of Test cricket, but this evening I finally got around to a simple study of the effect of the free hit rule in the IPL. The IPL's the only ball-by-ball database I have, otherwise I'd see if the results are the same in ODI's as well.

Anyway, I went through the tournament and found the run rate on the ball immediately following a no-ball (there might be some non-front-foot no-balls in there, but whatever...). The result was 140 runs off 82 balls, a run rate of 10.2 per over.

To work out how many runs you'd expect to have been scored if it wasn't a free hit, I used the overall run rates by over (actually I did a bit of smoothing first, fitting quadratic trends to the first six overs and remaining 14 overs). The graph looks like this:

So, e.g., if there was a free hit in the 10th over, you'd assume that if it was a normal ball, 7.0/6 = 1.17 runs would have been scored off that ball. Doing this for each of the free hits, you end up with an expected score of 111 runs coming off those free hits, a run rate of 8.1 per over. So the batsmen are scoring more than usual off the free hits, but not by a lot. An extra third of a run per free hit, on average.

It is a nice check that the average run rate in the 20th over (when you're batting like they're all free hits anyway) was 10.1.

Presumably free hits in ODI's also go at about 10/over (or 1.7 per ball), so proportionally speaking, the punishment is greater in the 50-over form of the game.

That graph above is pretty interesting. There's a bit of noise, but the end of the fielding restrictions is very clear. It's interesting that the acceleration is gradual.

The batsmen are circumspect (relatively speaking) in the first over, and the effect certainly muhc larger than just the first over being just bowled by the best bowler (no-one had an economy rate as low as 5.2 in the IPL). Perhaps it is worth opening with your "fifth" bowler. I don't know.

Queensland v Kolkata

2008-09-28T11:10:00.000+02:00

In a post some time back, Nestaquin told me that the Kolkata Knight Riders would be playing some games in Brisbane, and that I should go and report on what I see. Today they played the first of three double-headers against Queensland, and this gives me an easy way to end the neglect of this blog. The games were played at Allan Border Field, which is a lovely ground which doesn't have enough matches to warrant over-corporatisation — there's still a white picket fence, a hill, and trees.

The first Kolkata player I saw was Ajit Agarkar (at least I'm 90% sure it was him — I only saw him for a second or two), who was going for a jog as I was walking to the ground. But unfortunately he didn't play, as the coach John Buchanan brought his squad here to give the young guys some experience.

Boy do they need it. They batted first in the morning game. I had a pen and notepad, and decided to keep a tally of balls where the batsman was beaten (i.e., played and missed, miscued, hit on pad). I don't know of a good word to describe this, so I'll borrow the French football term occasion (which in football means a scoring opportunity). In 120 balls plus a few wides, there were 58 of these occasions, which makes one every other ball. The batsmen, mostly sub-first-class standard, were completely unable to cope with the pace and bounce of the quick bowlers. Then Chris Simpson came on to bowl some offies, and they weren't all that good against him either.

Sharma (not sure of his first name) gave up 12 occas on his way to 5 runs, and Patel gave up 10 on his way to 3 runs.

They crawled their way to 8/79 from their 20 overs. The only three batsmen to make double-figures were Michael Buchanan (John's son; he's been a fringe player for Queensland) with 16, their other opener Prashantha (I think that was his name — I didn't see a match programme, haven't seen a scorecard, and had to go by the ground announcer), who was steady-ish but slow and who scored 12 before getting out in the 8th over, and Adam Hollioake. I didn't even know he was still playing, but Cricinfo tells me that while he retired from first-class cricket in 2004, he kept playing T20's till 2007. Anyway, he looked pretty overweight, but he looked a class above most of his teammates.

There really wasn't much positive to take out of the batting. Even with only four fielders inside the circle, they were incapable of nudging singles. Symonds bowled two consecutive maidens with the field set deep!

I couldn't be bothered taking notes after that.

Their bowlers in the morning game were no better than the batsmen. Too short, too full, too much width, .... They might have been a bit unlucky, in the sense that every time the batsmen swung at the ball it found the middle of the bat, but it was still carnage. The Queensland openers whacked 80 runs in about 7 overs, and then they decided to keep playing, with a revised target of 180. They made it with about four overs to spare, with seven wickets in hand. Symonds failed, scoring just 2, skying a catch to long on with his first real swing of the bat.

I stuck around for the afternoon game, and I'm glad I did. Queensland batted first (shuffling their batting order), and this time the KKR bowlers stuck to a much better length. Short balls got to the head rather than the lower chest. And they had more luck, as batsmen started miscuing. Chaudry (I think) took three wickets. Queensland kept batting aggressively but lost wickets. Siddarth Kaul was one of the bowlers. Iqbal Abdullah bowled some left-arm orthodox darters and looked OK. I can't remember the others. They brought on a legspinner who struggled to find his length for a while, which meant that most of his spell was bad — when you only get four overs, there's not much time to get your rhythm. Queensland got to 8/96 after 12 and finished all out 129.

Michael Buchanan led the chase with 49 from 31. The batsmen at the other end were not so good, and when Buchanan was dismissed they were 4/80ish. They looked ready to collapse, but the lower-order kept hanging on, Hollioake again making double figures. I wasn't taking notes or tallying, but my impression was that they were much more solid, making contact with the ball and middling it a lot more often.

They needed 29 off the last two, and Daniel Doran bowled the nineteenth over (I think it was the nineteenth...). Doran must have good figures in grade cricket, because he keeps getting picked for Queensland, despite averaging over 70 with the ball in the last two years. Anyway, one of the KKR tailenders hit him for 16 runs in three balls (all straight hits I think), and they then needed 11 off the last over with one wicket in hand.

Grant Sullivan bowled it — he had looked sharp but lacked control and Buchanan had hit him around earlier. His first ball was short and gave too much width, and Kaul managed to get some bat on it and got a boundary to third man. 7 off 5. Dot ball. Third ball was good length, Kaul hits it over long off for 6, and momentarily thinks that he's won the match, but then realises that scores are tied. Field comes in. Dot ball. Dot ball. Last ball, defended to Andrew Symonds, they run, Symonds throws and misses, Kolkata win by one wicket.

The KKR guys were all happy with a win.

Simulating one-day cricket and batting orders

2008-09-07T11:14:00.001+02:00

I spent some of today reading through this paper (you'll probably need a University subscription to read that) by Swartz et al. It's called 'Optimal batting orders in one-day cricket', and it is useful because it gives a way of simulating one-day innings.

(The paper itself looks at the Indian batting order in the 2003 World Cup. The best they came up with went Dravid, Tendulkar, Ganguly, Sehwag, Mongia, Y Singh, Khan, Kaif, H Singh, Agarkar, Srinath. Their second-best lineup swapped Dravid and Ganguly, sent Kaif to 7. They reckon it would have done better by about 6 runs, on average, than their actual lineup for the World Cup final. Not many matches are won and lost by less than six runs, but I suppose you want to squeeze out every run you can. It's interesting that the simulations reckoned that Kaif was best left to come in and slog at the death. A full run of their simulations takes a long time — there are a lot of batting lineups to go through, even when you do clever tricks and make the search much smaller. But they say that it would be much quicker if you had only a limited number of options, such as during a match when you've lost a couple of wickets. Using a computer to find the optimal batting order based on the situation of the game meshes well with Rob Smyth's belief that batting orders in one-day should be fluid.)

The way they do it is to work out 'baseline' characteristics for each batsman in the team. That is, they get the probability that a batsman will play a dot ball, score a single, a 2, a 3, a 4, a 6, or get out. But they don't just take their overall career numbers, they take into account the match situation when they batted.

So, given the number of wickets fallen w, balls bowled b, Duckworth-Lewis percentage resources used R(w,b), what they actually did was fit parameters to a loglinear model that looks like this (the subscript k denotes what happened on the ball, so k = 0 is a dot, etc.; the subscript j refers to the jth batsman):

log(q_jwbk) = μ_jk + α_k*w/9 + β_k*b/299 + θ_k*R(w,b)/100.

The μ_jk's give the baseline probabilities for each type of ball-result (dot, single, etc.) for each batsman at the start of the innings. (Well, not directly probabilities — the probabilities p_jk are given by p_jk = q_jk / &Sigma_k q_jk.)

The other parameters (α, β, θ) describe how the probabilities change as the game situation changes. It is assumed that all batsmen change in the same way.

So that's all well and good. You throw the paramaters into the computer, generate a bunch of random numbers and you end up with 50 overs of simulated cricket. The results are pretty close to what real cricket scores are, at least at the team level. I tried running the same algorithm and one of the openers scored a double-century on the second run, so it's probably not perfect, but on the average it seems to do a good job. (I'm getting a slightly higher result for the team average — 253 against 250 — than the authors of the study did. Some minor bug in my code somewhere, I guess.)

Anyway, I think that this paper could be useful for me. What I want to do is see how to properly assess batting average and strike rate. So what I hope to do is get the relevant parameters for batsmen as a whole in the 2000's. Unfortunately, I don't have ball-by-ball ODI data, so I'm going to have to estimate it somehow. I've asked S Rajesh for the overall numbers (i.e., total dot balls, singles, 2's, etc.), and hopefully with some fiddling I'll get the weird loglinear parameters to match them.

The α, β, and θ parameters I'll leave unchanged. Hopefully India's batsmen from 1998 to 2003 (the period that the study looked at) are representative of how batsmen generally change over the course of an innings.

Then, once I've got a good simulator of an average batting lineup against average bowling, I'll be able to vary the parameters of one of the batsmen, tweaking average and strike rate (indirectly — I'll be tweaking probability of dismissal on each ball, and probability of each type of scoring shot). Then you see what effect this has on the average team score. So it'll be like the post below, only accurate. Hopefully.

Averages and strike rates in ODI's

2008-08-25T11:49:00.001+02:00

Rating batsmen in one-day cricket is difficult because both average and strike rate are important, and it's not clear how they should be weighted.

I can't see a theoretical solution (multiplying the two measures might be good enough, but it seems arbitrary), and I think that the actual answer will come from simulations. In this post, I show some results from some woefully inaccurate simulations. But hopefully even though the total scores were below what they should be, the equivalences of various averages and strike rates should be reasonably accurate.

Here's what I did. I took the overall average and strike rate since 2000 for each batting position (I think using the top eight teams). I ran a largish number (20000) simulations to get what the "average" total score is, and it turned out to be 208 (I told you it was inaccurate...).

Then, I replaced one of the openers with a batsman with an average of 1, strike rate 50, then average 2, strike rate 50, average 3, strike rate 50, and so on, doing 20000 simulations each time to get the average total score. I did this until I had a grid of average total scores for strike rates from 50 to 130, and averages from 1 to 60. Then I made contour plots with curves of equal value.

The simulations assumed a constant run rate and exponentially distributed scores. Not realistic, but it was straightforward to do and avoided doing ball-by-ball simulations.

There's a bit of noise in the results. Here's the contour plot for openers:

The number 3:

The number 4:

Because I was exceedingly lazy, the contours in the separate plots may not correspond to the same total team scores. But you will agree that the pictures are colourful.

Now for some numbers. In each of the following little tables, the rows are equivalent. So, an opener with an average of 50 and a strike rate of 73 is worth the same as an opener with an average of 25 and a strike rate of 101. According to the simulations, at least.

Making an average score of 210, as an opener:

210, number 3:

210, number 4:

I didn't get past number 4. I'll do the rest tomorrow.

I imagine that the curves would change with more accurate simulations, but this is at least a start.

Non-boundary strike rates

2008-08-24T06:10:00.002+02:00

Near the end of the discussion here, there's a comment from me about the changing nature of the way runs are scored in ODI cricket. Most of it, of course, is coming from boundaries, which are much more common today. But it is interesting that there's been no real changes in the rate of non-boundary scoring since 1990.

Here's a graph showing the yearly overall "non-boundary strike rate", that is the runs that are actually run, divided by the number of balls not hit to the boundary (times 100). Top eight sides only. (There's some missing boundary data, especially before 1990. The actual non-boundary strike rates for these years are lower than those in the graph.)

(I've called it "run sr" in the graph, run as in running.)

I would have thought that batsmen today are more adept at "milking" bowlers, but the non-boundary strike rate has never got above 3 runs per over for any long period.

If you suppose that there are three types of balls:

- good balls that can't be scored off
- OK balls that can be worked around
- bad balls that can be hit to the boundary

then ~~it seems that batsmen these days are able to hit the OK balls to the boundary more often, but can't do much about the good balls~~. (Edit: No, wait, that's not right. They're getting better at milking the good balls at the same rate as they're getting better at hitting the OK balls for four. Roughly. The constant non-boundary strike rate with an increased frequency of boundaries means that the percentage of dot balls is getting lower.)

At a team level (now including all teams, since 2000, but I've forgotten what I did with Kenya, probably I discarded them because they play against minnows too often):


team          sr    run sr
Australia     83.6  50.6
Sri Lanka     75.5  47.4
South Africa  78.3  47.0
Pakistan      77.3  46.7
India         78.8  45.7
England       74.4  45.4
New Zealand   75.2  44.7
West Indies   74.2  43.9
Zimbabwe      67.0  43.3
Bangladesh    63.4  38.6

Bangladesh are really bad at working the ball around for singles, etc. There's a clear gap between Australia and Sri Lanka, then a gradual progression down through to Zimbabwe. Then there's a huge dropoff to Bangladesh.

India are a bit of an anomaly, with their high overall strike rate coming more heavily from boundaries than the other teams. It's probably not just a factor of their grounds — opposition teams in India have the highest non-boundary strike rate of away teams anywhere.

Top eight sides since 2000, individuals, average at least 30, at least 1000 runs.


Player          mats  inns  runs  avg   sr    run sr  diff
JN Rhodes       71    66    1994  41.5  86.1  59.7    26.3
DS Lehmann      44    39    1219  42.0  78.8  56.0    22.8
MEK Hussey      77    60    2079  54.7  85.6  55.3    30.4
L Klusener      91    73    1592  33.9  87.6  54.9    32.7
A Symonds       157   133   4300  40.2  94.1  54.1    40.1
MJ Clarke       117   105   3486  42.5  81.0  53.9    27.2
MS Dhoni        101   91    3064  44.4  89.0  53.6    35.5
SR Waugh        46    38    1134  40.5  79.9  52.6    27.2
PD Collingwood  124   114   2956  31.1  74.6  51.5    23.1
RP Arnold       136   123   2984  32.8  72.1  51.5    20.6

Symonds and Dhoni are best known for their hitting, but they're good at the less flashy stuff as well.

Unhonourable mention goes to Chris Gayle: overall strike rate of 80.3, non-boundary strike rate of 37.3, lowest of all the players who made the qualification. Lazy.

Bowlers as they get more experienced

2008-08-13T11:56:00.002+02:00

There was a comment from Gary Naylor here saying that Monty Panesar should improve on his average of 32 as his career goes on, since he'll learn more about how to bowl.

I'm not convinced. I took all spinners with 149 or more wickets since WWII, and found split their wickets into which Test it was in their careers, so I could find the overall average in debut Tests, second Tests, third Tests, etc.

To get rid of some noise I actually took a five-Test moving average, so the first data point in the graph below is the overall average in the spinners' first to fifth Tests, the next the average in their second to sixth Tests, etc.

Also I weighted wickets by the average of the batsmen dismissed.

Note that there's a bit of a selection effect going on. I'm only looking at spinners who were good enough to play enough Tests to take 149+ wickets. Towards the right-hand end of the graph this is also a factor — if you imagine it continuing further on out, you'd eventually just be plotting Warne, Murali, and Kumble. There are, for those interested, 24 bowlers going into Tests 1-36, then 23 in Test 37, 22 in Tests 38-43, 20 in Test 44, 19 in Tests 45 and 46, and 18 in Tests 47-49.

I don't know how much I want to read into the graph, though I'm happy in saying that spinners improve after their first ten Tests or so. After that there may or may not be a trend — batsmen working them out? Certainly there's no strong evidence that Panesar will improve significantly (he's played 33 Tests), though of course it's possible.

Here's the corresponding graph for pacemen:

There are 46 bowlers going into all of those data points. There's a downward trend — pacemen tend to get better with experience, at least for a few dozen Tests.

One thing to try in future is age rather than Test experience. The people who've done this sort of analysis in baseball say that age is a better thing to use than Major League experience. But of course baseball is not cricket, so I'm not sure what will come out of it.

The modern lack of tour games

2008-08-06T13:46:00.000+02:00

People often complain (myself included!) about the lack of tour matches these days.

Here's a breakdown of home side victories by Test number in the series (excl. Bangladesh and Zimbabwe):

1st: 0.40 +/- 0.02
2nd: 0.38 +/- 0.02
3rd: 0.38 +/- 0.02
4th: 0.39 +/- 0.03
5th: 0.39 +/- 0.04
6th: 0.35 +/- 0.12

That's over all Test history. There might be an extra slight advantage for the home side in the first Test of a series, but it's within error bars. But perhaps that small advantage is due to a bigger advantage that's come about recently? Here are the numbers since 2000:

1st: 0.42 +/- 0.05
2nd: 0.46 +/- 0.05
3rd: 0.50 +/- 0.06
4th: 0.55 +/- 0.11
5th: 0.60 +/- 0.15

That it's such a neat little increasing sequence is probably luck. That the home side is winning more Tests is not surprising, since we're in a very result-heavy era. But the main point to take home is that there's no evidence that touring teams get better as they get more used to the foreign conditions or whatever. Perhaps those stingy boards are right not to schedule extra tour matches.

This is a result that surprised me.

It Figures

2008-07-27T13:09:00.000+02:00

I have joined Cricinfo's stats blog It Figures. My first post there is a more sophisticated version of the IPL bowling analysis that I did earlier.

I should be posting to It Figures once a week, which is the rate I've been posting here. I'm not sure what will happen to this Blogspot blog. Perhaps only analyses that are too technical or too minor for Cricinfo. Boring and complicated — that'll bring in the visitors. We'll see how it goes.

Bowleds, LBW's, and a little quiz

2008-07-27T12:30:00.000+02:00

Whenever people study umpiring bias, they almost always look at LBW's. There's not a lot else you can do from looking at scorecards — other dismissal types are much more clear-cut.

A paper by Trevor Ringrose in 2006 ('Neutral umpires and leg before wicket decisions in test cricket', J. R. Stat. Soc. A 169, 903) considered LBW rates by country and the presence of neutral umpires, and found that the neutral umpires made no difference to the home-side bias of LBW decisions that affects some sides.

That paper's too technical for me to be bothered wading through this evening, so instead I'll talk about what Charles Davis did in The Best of the Best. For each team X, he calculated the difference between X's LBW percentage (that is, number of LBW's divided by number of wickets) and their opponent's LBW percentage, first for X's home Tests, then for X's away Tests. Find the difference of those two values and you get the home-side bias in LBW decisions for team X.

Pakistan is the major side that apparently gets favoured the most by home umpiring, clearly ahead of Australia. But Davis points out that in addition to having lots of LBW's go their way, Pakistani bowlers (I'm sure we know which ones) in the 1990's also got a very high number of bowled wickets. Correcting for this, the apparent umpiring bias in Pakistan becomes comparable to that in other countries. It just seems worse because they hit the pads so often.

So, following thinking along these lines, I took all bowlers with at least 100 Test wickets since World War II and plotted their LBW to caught ratio (i.e., number of LBW's divided by number of wickets caught) against bowled to LBW ratio. There's no fancy regressions to the mean or anything, these are raw numbers.

There's a bit of a trend there, but plenty of scatter. Ian Johnson (109 wickets) has one of the highest bowled to caught ratios (just over 0.7) but an LBW to caught of less than 0.2.

The two W's are fairly easy to spot — they're the ones fairly close together with LBW to caught ratios above 0.6, and bowled to caught ratios above 0.5. So they indeed got plenty of bowled and plenty of LBW's. But there are several bowlers with lots of LBW's and not many bowleds.

Now for that little quiz. Waqar has the highest LBW to caught ratio at 0.68. Wasim is third at 0.62. Who's second? He's the other data point quite high on that scatterplot, with a bowled to caught of 0.29.

Wickets broken down by ball in the over

2008-07-20T12:57:00.000+02:00

Quick one today, I've been busy for reasons that will become clear in a couple of days.

Here's the breakdown of wickets by ball in the over, in Tests since 1998 or so.

1: 2448
2: 2443
3: 2537
4: 2464
5: 2639
6: 2413

Ball five is about 3.3 standard deviations above the mean, which is interesting and significant at p=0.003. (Usually 3.3 standard deviations would correspond to p=0.0005, but there are six tests going on, which increases the likelihood that one of them will turn out significant. So I multiplied that 0.0005 by 6, which I hope is the correct thing to do.) I can't think of any obvious reason why the fifth ball in the over is relatively wicket-prone, so I'm leaning towards it just being a blip. Perhaps those stalemates in which the top-order batsman bats with the tail-ender and holds the strike for the first four balls? I don't know.

Now for the IPL:

1: 122
2: 131
3: 104
4: 104
5: 104
6: 124

The numbers are pretty small, but it's something to think about for when I gather more T20 data. Perhaps batsmen take a couple of balls to get their eye in against new bowlers. In Test cricket, these bowler changes happen less frequently, and also the batsmen are more watchful. In T20, they might be slogging from ball one. Just a thought, nothing concrete.

Michael Vaughan looks funny when he gets bowled, but that is all.

2008-07-12T07:44:00.004+02:00

(Edit: I just fixed a problem with the regression to the mean. No major changes to the batsmen that made the extremes of the tables, but the regressed estimates of bowled proportions are now much more accurate.

Edit: I should say that the methods used in this post have been either inspired by or directly copied from the baseballers, particularly from the authors of The Book - see their blog here.)

Some people in comments here (starting with MacMillings) have been discussing Michael Vaughan getting out bowled a lot. I was asked to have a look at it.

Charles Davis, in The Best of the Best, produced a graph similar to this one:

That's a plot showing the proportion of various dismissal types over time, looking only at batsmen from 1 to 6 in the order. Though I've started that graph at the end of World War II, the decline in bowled dismissals has been going on since the start of Test cricket. Why that should be so is a bit of a mystery. The slack's been taken up by catches and (sometimes) LBW's, the latter being influenced somewhat by changes in the Laws.

It's not entirely accounted for by keepers standing back and taking more catches &mdashes; though more and more wickets are coming from catches to the keeper, adding them to the bowleds still gives a clear decreasing trend.

So, rather than wondering where Michael Vaughan stands in relation to batsmen from history in terms of getting bowled, we'll consider only batsmen from 1990. The trend in bowleds from 1990 to the present is close enough to flat.

The next thing to think about is whether or not differences in bowled proportions for batsmen is an inherent characteristic of the various batting styles, or simply due to random chance.

I took all batsmen with at least 50 dismissals since 1990, and an adjusted average of at least 35. Across all these wickets, about 15% were bowled. Now, any wicket is either bowled or something else. If this is random, then the proportion of bowleds for a batsman will follow a binomial distribution, with mean 0.15 and standard deviation sqrt(0.15*(1-0.15)/outs). Here and below, 'outs' is the number of times a batsman is dismissed.

Plugging those numbers in to get z-scores for each of the batsmen in the dataset (59 of them), we find 6 with a z-score more than 2 standard deviations from the mean (from random chance, you'd expect about 3), and 27 more than 1 standard deviation from the mean (you'd expect about 19). The standard deviation of the z-scores is about 1.2 instead of 1.

Now, the observed variance comes from two terms — random luck, and the inherent 'true' differences between the players. Since luck is independent of the actual differences, we have that var(observed) = var(true) + var(luck). The observed variance is about 0.04²; the variance due to luck is roughly 0.15*0.85/120 = 0.033² (the denominator 120 being the average number of outs across the batsman in the dataset). The var(true) is the difference, and so the standard deviation of the inherent differences is sqrt(0.04² - 0.033²) = 0.025.

So, there are genuine differences between batsmen in terms of how often they get out bowled, and it's sensible to start comparing them. But before I start doing so, we should regress each player's observed bowled proportion to the mean. We have an estimate of the player's bowled proportion as p +/- sqrt(p*(1-p)/outs), and the player's coming from a distribution that goes like 0.15 +/- 0.025. The estimate of the batsman's 'true' bowled proportion is calculated using the same formula as given here.

First, does a high proportion of bowled dismissals make a bad batsman?

There's no trend at all amongst good batsmen. Tail-enders (not shown on the graph) do get out bowled more often though.

Now for the batsmen who get out bowled the most and the least since 1990. The 'b' column is the number of bowled dismissals. The last two columns are the observed proportion of bowleds and that figure regressed to the mean.


                                        bowled prop
name            outs  b   avg   adj avg obs   reg
HH Gibbs        147   35  42.0  36.9    0.238 0.179
JH Kallis       168   37  57.0  49.8    0.220 0.176
VVS Laxman      132   30  43.8  39.7    0.227 0.175
RS Dravid       182   35  55.4  47.9    0.192 0.168
AJ Stewart      214   40  39.5  39.7    0.187 0.167
AR Border       62    15  43.3  39.9    0.242 0.166
RA Smith        83    18  42.6  42.3    0.217 0.166
SR Waugh        170   31  53.2  47.9    0.182 0.164
SR Tendulkar    207   37  55.9  48.6    0.179 0.164
ME Trescothick  133   25  43.8  41.1    0.188 0.164
---
Saeed Anwar     89    9   45.5  41.8    0.101 0.132
ML Hayden       152   17  53.0  45.7    0.112 0.132
RR Sarwan       121   13  40.4  36.6    0.107 0.132
S Chanderpaul   163   18  49.1  45.9    0.110 0.131
KC Sangakkara   111   11  55.2  46.6    0.099 0.129
Younis Khan     98    9   49.1  45.5    0.092 0.126
JC Adams        73    6   41.3  38.7    0.082 0.126
CD McMillan     81    6   38.5  35.4    0.074 0.119
PA de Silva     119   10  45.3  39.6    0.084 0.119
CL Hooper       133   11  38.5  37.2    0.083 0.116

I wouldn't have picked Border to be near the top. Though he was on the decline in his last few years (which is all the above table considers), his high bowled proportion was a feature throughout his career.

Where's Michael Vaughan? At an observed proportion of 0.157 (now 0.164 after his latest dismissal), regressed to 0.153. Just above above the mean, nothing special or unusual at all.

His technique does lend itself to jokes though.

Lastly, there was some talk about whether or not bowleds are more common at lower scores. Since 1990, dismissal proportions by score, amongst top six batsmen:

The regression lines from top to bottom are caught by non-keepers, caught by keeper, LBW, bowled.

Bowleds in fact stay pretty steady. Catches at the wicket and LBW's decline, and catches to non-keepers become steadily more prevalent as the innings goes on.

Bradman Day

2008-07-10T13:27:00.001+02:00

As some of you may be aware, Don Bradman would be turning 100 this year if he were still alive. He was born on 27 August 1908, and the simple thing to do, if we wanted to have a day to celebrate and remember Bradman, would be to do so on 27 August 2008.

But Andrew Samson suggested last year a much more appropriate date: 6 August 2008. Why the sixth? Because Bradman would have been 99.94 years old on that day.

Rugby and the ELV's

2008-07-06T08:06:00.000+02:00

As the heading indicates, this post is not about cricket.

Last night's rugby Test between Australia and France was won convincingly by Australia (40-10) despite the French having much more possession (I haven't seen a figure since mid-match, but it was somewhere around 65%). This got me wondering about the relation between possession, territory, and winning in rugby. I downloaded the last two seasons' worth of data for the Super 14 from Rugby Stats to see what it said. The Rugby Stats site gives all sorts of data (unfortunately not going back further than the last couple of years), but for this post I've just used taken the home team possession and territory for each game, along with the fraction of points scored by the home team. So, eg, if the home team won 20-10, they had 0.667 of the points scored.

I'll start with the 2007 season, which of course was played under the traditional rugby laws. Here's some of what gretl had to say:


Model 1: OLS estimates using the 94 observations 1-94
Dependent variable: h_score_percent

      VARIABLE       COEFFICIENT        STDERROR      T STAT   P-VALUE

  const                -0.379896         0.355244     -1.069   0.28772
  h_poss                1.88078          0.688795      2.731   0.00759 ***
  h_terr               -0.0305528        0.182781     -0.167   0.86762

  Mean of dependent variable = 0.558623
  Standard deviation of dep. var. = 0.194866
  Sum of squared residuals = 3.26391
  Standard error of residuals = 0.189386
  Unadjusted R-squared = 0.0757645

So, having the ball helps — for each extra percentage point of ball, you got almost two percentage points worth of the final score. On average, about 44 points were scored (in total) each game, and 2% of 44 is 0.88 points. Of course, when the home side gets a bigger slice of the points, the away side must lose the same amount, so it's really about a 2-point swing. (If you work with raw scores and not fractions of total points, you get a similar result.) There's a lot of scatter in the data — the R-squared is only 0.076.

So, all other things equal, if the score is 27-17 with equal possession, it'd be (on average) 28-16 with 51-49 possession.

Territory, on the other hand, doesn't make a difference.

Now let's look at 2008, played under the ELV's.


Model 1: OLS estimates using the 94 observations 1-94
Dependent variable: h_score_percent

      VARIABLE       COEFFICIENT        STDERROR      T STAT   P-VALUE

  const                -1.70310          0.333894     -5.101  <0.00001 ***
  h_poss                4.26355          0.656012      6.499  <0.00001 ***
  h_terr                0.228343         0.111618      2.046   0.04367 **

  Mean of dependent variable = 0.539166
  Standard deviation of dep. var. = 0.172713
  Sum of squared residuals = 1.84663
  Standard error of residuals = 0.142452
  Unadjusted R-squared = 0.334345

The ELV's appear to have made possession much more important — you end up with a 4-point swing in score for each percentage point of possession, rather than 2 points. Also, territory seems to be mildly important and beneficial now. The R-squared is 0.33, so possession and territory are much better at predicting the final result under the ELV's than they are under the old laws.

If any of you are rugby fans, feel free to make any requests for rugby analysis.

Mini-orders

2008-06-30T13:04:00.000+02:00

Samir Chopra asked for a stats post on "mini-orders", and here it is.

A mini-order is defined, for this post, as a block of three players at the same positions in the batting order. So, for instance, you could have Langer-Hayden-Ponting as a mini-order (with positions 1, 2, and 3). Now, I could fill up pages with the various possibilities (123, 345, 456, 567, etc.), but that seems like it might be excessive. So below I've listed the leading mini-orders by runs scored. This is, of course, a list heavily biased towards recent teams.

In the table below, the columns are the number of team innings in which the triple appeared; total runs made in those innings by the batsmen in that mini-order; their average in those innings; the number of runs made in partnerships between those three batsmen in those innings; and the average of those partnerships, adjusted for era and quality of the bowling (relative to an overall average of 31.5). The regular average and partnership average are typically close to each other. The partnership stats are not complete, since I ignore any team innings which look like they involved a retired hurt.

Note that the order is strict — Langer-Hayden-Ponting is considered separately from Hayden-Langer-Ponting. The latter only happened twice, by my count. If you ignore order, then Taylor-Slater-Boon would go into fifth place. Taylor and Slater alternated almost perfectly in which of the two faced the first ball.


pos name1          name2            name3           i   runs  avg   p-runs  adj part avg
123 JL Langer      ML Hayden        RT Ponting      94  15034 60.6  10352   53.8
345 RS Dravid      SR Tendulkar     SC Ganguly      78  11319 55.8  5624    49.5
123 CG Greenidge   DL Haynes        RB Richardson   84  9778  43.8  6611    41.5
123 MS Atapattu    ST Jayasuriya    KC Sangakkara   58  7352  46.5  4807    35.2
456 SR Tendulkar   SC Ganguly       VVS Laxman      54  6742  49.9  2806    53.0
345 JL Langer      ME Waugh         SR Waugh        51  6405  47.1  2340    34.9
123 ME Trescothick MP Vaughan       MA Butcher      49  5956  44.8  4132    41.8
456 ME Waugh       SR Waugh         RT Ponting      43  5466  48.4  2343    54.2
456 PA de Silva    A Ranatunga      HP Tillakaratne 43  4688  40.1  2033    38.2
345 JH Kallis      DJ Cullinan      WJ Cronje       37  4412  43.7  2190    37.6
345 RR Sarwan      BC Lara          S Chanderpaul   35  4281  42.8  1560    40.3
123 SM Gavaskar    CPS Chauhan      DB Vengsarkar   35  4038  40.4  2555    41.3
456 DJ Cullinan    WJ Cronje        JN Rhodes       33  3999  46.0  1678    39.8
345 KC Sangakkara  DPMD Jayawardene TT Samaraweera  32  3992  45.4  1464    30.7
345 HM Amla        JH Kallis        AG Prince       29  3951  52.0  2209    46.4
123 CG Greenidge   DL Haynes        IVA Richards    32  3813  42.8  2990    47.2
345 Younis Khan    Inzamam-ul-Haq   Yousuf Youhana  25  3772  54.7  1778    48.5
123 L Hutton       C Washbrook      WJ Edrich       28  3729  49.7  2407    47.8
345 AP Gurusinha   PA de Silva      A Ranatunga     33  3721  42.8  2081    50.1
123 GR Marsh       MA Taylor        DC Boon         30  3675  43.2  2779    44.3
345 Younis Khan    Yousuf Youhana   Inzamam-ul-Haq  21  3600  66.7  1766    77.0

The constancy of the Australian batting lineup in recent years is well-known, of course, so it's perhaps no surprise to see that the Langer-Hayden-Ponting trio has appeared in more innings in that order than any other. Even allowing for the high scoring these days, they come out easily better than Greenidge-Haynes-Richardson.

Leading mini-order at each position by adjusted average of the batsmen, qualification 10 innings:
123: Woodfull-Ponsford-Bradman, 13 innings, avg 81.7, adj avg 75.4
345: Bradman-Kippax-McCabe, 12 innings, avg 78.8, adj avg 71.7
456: Hussey-Clarke-Symonds, 16 innings, avg 64.3, adj avg 57.8
567: Clarke-Symonds-Gilchrist, 11 innings, avg 53.1, adj avg 48.6

A very Australian affair.

Followup on accuracy of averages

2008-06-29T11:09:00.000+02:00

Russ pointed out a couple of things in the previous post. For those who missed the comments thread, here are the revised formulas for calculating uncertainties.

Batting: 0.9 * average / sqrt(# innings)
Bowling: 0.9 * average / sqrt(# wickets)

So, e.g., Mike Hussey becomes 68.4 +/- 9.5. About 68% of 'true' averages will lie within the range given. You need to double it to get it up to 95%.

I haven't made much of an effort to work out the underlying distribution of Australian players that Hussey comes from. To get a rough idea of what should happen, I found the mean and standard deviation of averages of Australian batsmen at batting positions 1 through 7, over the last ten years. There's a bit of a problem about what to do with players who only played a couple of Tests and averaged (say) 5 — clearly they could have averaged up around 20 or 30 if given more opportunities.

Anyway, I bumped those guys up to 20, and the result was something like mean 42, standard deviation 12. So, carrying on with the Hussey example, we crunch the numbers like this:

regressed average = (68.4/9.5² + 42 / 12²) / (1/9.5² + 1/12²)

uncertainty = 1 / sqrt(1/9.5² + 1/12²)

to estimate Hussey's 'true' average as about 58 +/- 7.

Let's just hope that he can score runs in India.

Accuracy of averages

2008-06-22T08:26:00.003+02:00

Today I would like to relate some horrifying thoughts about averages. I would like to be wrong, so if you think that there are mistakes with what I've done, do comment. (Update: See the comments thread, and followup. The uncertainties I give below for batsmen are about twice as big as they should be. For bowlers they are ~~about three times~~ also about two times too big.)

I started thinking about this as I started working my way through The Book: Playing the Percentages in Baseball (the authors blog here), trying to pick out the bits which can carry over to cricket, so that we don't have to re-invent wheels that the baseballers have already made for us.

One key point that they make is that a player's raw statistics aren't the best estimates of his true talent — you have to regress to the mean. The less reliable the stat, the more you regress. The less data you have, the more you regress. (And vice versa.) We know this intuitively in some cases — much though I love him, no-one really thinks that Mike Hussey is an 80-average batsman, and indeed in the West Indies his average has dropped to below 70.

But the question is, how many innings does a batsman have to bat before we can be confident that his average is accurately reflecting his talent (and not have to worry about regressing to the mean)? The short answer appears to be something on the order of 10000 innings, if we want to nail the average down to within a run or so.

That's an appallingly large number of innings, completely counter-intuitive for me. Averages seem to stabilise for batsmen after a hundred innings or so. But that intuition we have is based on the wrong thing. Career averages are stable because subsequent innings can't change the overall average much. A better way of thinking is, what would happen if the player re-ran his career from the start (so same opponents, etc.) but with different luck? Here, luck could be things like balls that beat the bat actually finding the edge (or vice versa), dropped catches, etc.

At this point I still would have thought that over a couple of hundred innings, you'd get the same average, to within a run. But the numbers are telling me different things.

To take an artificial example, suppose that a batsman's scores are exponentially distributed with mean 50, and no not-outs. I ran a few simulations of such a batsman over 300 innings, and here are the sample means that came out: 51.8, 54.4, 47.1, 48.4, 50.1.

That's quite a wide range, even for a longer career than any in Test history. At 47.1, you're talking about a very good batsman. At 54.4, he's an all-time great (perhaps not in today's batting-friendly world). In practice, we would expect that it would be even worse than this, because batting scores are not exponentially distributed — the standard deviation for real cricket scores tends to be higher than for exponential scores.

So now let's look at some real cricket scores. The way I'll do this is to take a player, and compare one half of his career to the other. Now, you can't take the first half and second half of the career, because there might be a change in talent over that time (developing better technique, losing reflexes, etc.). So instead, I split the innings into odds and evens (further splitting by first and second innings in matches — I didn't do this perfectly, but it should be close enough). This way, any genuine slumps or good years will be split evenly into the two halves for comparison.

Allan Border in his 'even' innings (132 of them) averaged 49.5, and averaged 51.6 in his 133 odd innings. That's not too bad, I suppose. The two are pretty close together.

But what about Steve Waugh, who was almost as prolific in terms of innings? Evens 55.9, odds 46.3. Tendulkar: evens 52.6, odds 58.0. Viv Richards: evens 66.1, odds 36.5.

Those are some hefty differences (Richards' being one of the most striking). Here is a plot of the odds average against the evens average for all batsmen who played 50 or more Tests and averaged at least 30.

That R-squared value drops even further (to 0.18) if you remove Bradman. If there were no luck at all involved, then R-squared would be 1, and the dots would make a nice little y = x line. Cricket is a lot more luck-filled than that.

We would like some kind of estimate of the uncertainty involved in batting averages. As we see from the graph above, they'll be pretty big. I'm not entirely sure if what I did was the best way of doing things, so if any stat-heads amongst you can suggest improvements, please do.

I took the odd averages, guessed an error that went like k * (odd avg) / sqrt(number of odd innings), and fiddled with the constant k until roughly 68% of the even averages fell within that margin. I got k = 1.7 or so. (If anyone could tell me where the 1.7 comes from, I'd be grateful. The average co-efficient of variation for batsmen is about 1.05, so by the Central Limit Theorem I would have expected k = 1.05.)

So, we can use this to estimate the uncertainty over whole careers, by 1.7 * avg / sqrt(innings).

Even for a career as long as Border's, that gives an uncertainty of about +/- 5.3 runs. Mike Hussey comes out to 68.4 +/- 17.9.

Now in Hussey's case, we'd lean much more towards the lower part of that estimated range — he's not an 85-average batsman. Why do we think that? Because only one man in history has been that good, and no-one else has ever got close. It's much more likely that Hussey is like everyone else than he's like Bradman.

To make estimates of this sort more rigorous, we need to know the distribution of the batsmen that Hussey is a part of. This won't be the overall mean and standard deviation of averages across all Test batsmen, because clearly the talent pool in Australia is much stronger than in Bangladesh. Probably what I'll do is use my adjusted averages and work by country (and possibly era — the standard deviation of averages is on a slow historical decline). But this will be for a later post.

I'll finish by saying that the story is similar for bowlers. Here is the even-odds graph for bowlers with at least 3 wickets per Test over 50 Tests:

The uncertainties I make to be about 1.7 * avg / sqrt(wickets). Warne (for instance) becomes 25.5 +/- 1.6.

Clarke when the pressure's off

2008-06-14T06:00:00.000+02:00

Homer broke down Michael Clarke's innings to see what happened when he came in with the score less than 150, and when he came in with the score greater than or equal to 150. Clarke does much better when the going's easy. But that's not a proof that Clarke is special — we would expect that batsmen do better when the bowlers have been struggling to take wickets.

So ran the numbers for all batsmen at 5 or 6. I grouped the innings into those worse than 3/150 or 4/200 (these seem reasonably equivalent), and those better. Then I took the difference of the averages. Then, to get some mileage out of this old monstrosity post of mine, I got an estimate of the probability that the "going's easy" average would arise by chance, given the "going's not easy" average, and the number of innings in each category. To give an example, Michael Clarke below gets a p-value of 0,20 — only about one in five batsmen would have such a rise in average. If there's an asterisk, then it means that the difference was too large for my estimation algorithm, and I got a senseless result.

(In The Best of the Best, Charles Davis defines a 'pressure average', which takes into account the state of the match — 4/50 in the second innings isn't a pressure situation if you've got a lead of 250 on the first innings. I can't be bothered going into this much detail.)

Note that many of the batsmen below spent much of their career higher up the order. Also note that my stats are a couple of months out of date.

Qualification of at least 10 easy innings and at least 10 not-easy innings:


             worse than 3/150    better than 3/150
name          inns  runs  avg     inns  runs  avg   diff    p
SC Ganguly    77    2285  32,2    43    2069  54,4  -22,3   *
MJ Clarke     24    854   37,1    17    1037  74,1  -36,9   0,20
MV Boucher    12    342   28,5    11    599   66,6  -38,1   0,26
DR Martyn     23    619   31,0    14    787   60,5  -29,6   0,35
PH Parfitt    18    632   39,5    11    696   87,0  -47,5   0,37
DB Vengsarkar 16    439   33,8    11    581   72,6  -38,9   0,37
TE Bailey     25    653   29,7    13    543   60,3  -30,7   0,38
DI Gower      35    1262  39,4    16    926   71,2  -31,8   0,45
KR Miller     32    978   34,9    19    1000  55,6  -20,6   0,49
RP Arnold     13    215   16,5    11    331   30,1  -13,6   0,50

Clarke really has been pretty bad (well, sort of — 37,1 is below average). In terms of the raw difference, he's fifth worst (Les Ames is just off this table, difference of -37,3.).

And now those rare batsmen who do worse when the pressure's off:


             worse than 3/150    better than 3/150
name          inns  runs  avg     inns  runs  avg   diff    p
A Flower      80    3761  57,9    10    310   31,0  26,9    *
CH Lloyd      78    3700  52,1    30    987   35,3  16,9    *
ND McKenzie   29    1056  40,6    16    438   27,4  13,2    0,30
A Symonds     10    389   43,2    10    233   29,1  14,1    0,44
SJ McCabe     19    830   48,8    12    397   33,1  15,7    0,46
SE Gregory    37    1015  28,2    12    205   18,6  9,6     0,56
IVA Richards  45    2051  51,3    22    852   40,6  10,7    0,77
RT Ponting    33    1604  51,7    17    570   40,7  11,0    0,81
KD Walters    60    2653  51,0    28    1113  42,8  8,2     0,87
KF Barrington 21    878   43,9    14    409   37,2  6,7     0,90

When the p-value is higher than 0,5, it means that such a 'slump' would occur in the career of one in two batsmen — pretty unremarkable. Clive Lloyd's record is probably the most remarkable of these, given the relatively large number of innings.

In the set of 83 players, 52 have better averages in easy situations, and 31 in not-easy situations.

Sorry for the no-post last weekend. The problem with devoting only one day a week to cricket stats is that if I don't get something working, then it doesn't get done for a while. I will try to return to IPL analysis next weekend.

Rajasthan and Moneyball

2008-05-31T08:02:00.002+02:00

Michael Atherton's column in The Times asks if Rajasthan are the Oakland A's of the IPL. The Oakland A's are a low-budget team in Major League Baseball, who were nevertheless able to make the playoffs and compete well with much richer teams. They did this by exploiting inefficiencies in the player market and clever drafting — batters who earned lots of bases on balls were undervalued by other teams, and other teams tended to draft players straight out of high school, a much riskier strategy than drafting players who had proven themselves at college level. The story behind this is detailed in the excellent book Moneyball. The publication of the book seems to have made life more difficult for the A's — many other clubs now employ the same sort of statisticians as the A's did, using the same ideas.

Rajasthan spent the least amount of money of all the IPL franchises at the player auctions, so in that sense they're similar to the A's. But we shouldn't overstate how prescient they were, because they really weren't.

They flagrantly ignored Moneyball principles early on. Their big success-from-obscurity has beeen Swapnil Asnodkar. Asnodkar, indeed, averages over 40 in List A cricket, and could easily be selected based on that stat. (Quite how he manages to do so with such a loose technique is not something the stats can shed any light on.) But they didn't pick him in the XI until their fifth game. Earlier, they had picked (for instance) Taruwar Kohli. You can't get much less Moneyball than that — he was picked off his under-19 performances, without having even played a first-class or List A match. Selection based on under-19 results! Under-19 cricket is of much lower quality than senior domestic cricket, and you're much safer in going to players with proven senior records. We don't need Moneyball to tell us that.

Of course, there is the requirement to play four under-22's, but there should be some 21-year-olds playing first-class cricket to choose from. Warne also picked the legspinner Salunkhe, who hadn't played a first-class game. Perhaps the experience of playing with Warne on the field was good for Salunkhe in the long term (I don't know), but he wasn't particularly effective and was soon dropped.

Some of the principles of Moneyball should carry over to the player trading in the IPL. The difficulty will be in evaluating the players. It is easy to see how they performed in this tournament, but teams shouldn't be working out their trades purely on this tournament. Some players were lucky (Marsh and Tanvir), and some unlucky (~~Dhoni~~ (Edit: Check stats before posting!), Tendulkar, Misbah). If teams are silly and give excessive weight to IPL stats, then it may be possible for teams to pick up some bargains with the big name stars who under-performed. The converse also applies.

Working out the balance between batting and bowling will also be important. The top bowlers seem to be worth between five and ten runs a game, relative to an average bowler. How much is a top batsman worth? More importantly, what about the middle-level players? I haven't answered this question, though the guys at Rediff might have.

Things to think about.

Rating IPL bowling

2008-05-31T07:59:00.001+02:00

I have been saying in comments around the blogosphere that economy rate is probably much more important than bowling average in T20. I decided to work out just how much wickets are worth.

Once I finished getting some numbers out, I realised that the method I'd used was quite close to Duckworth-Lewis, and I could probably have just adopted the old DL tables for these purposes. Hopefully I'll get around to comparing them to what I got some time. In the meantime, I figured that IPL innings might be different from the last 20 overs of ODI innings, and that you all probably wanted a nine-colour scatterplot.

Each dot represents a wicket in the first innings of the league stage of the IPL. (I ignored second innings, since they don't always last 20 overs.) I've fitted linear curves for each wicket, forcing it through the origin (you can't score any runs with zero balls left). You'll note that the points near the origin tend to be above the best-fit lines — that's because of late-over slogging. That would be important if I wanted to adjust targets for a rain-rule method, but here I'm only interested in the gaps between the best-fit lines, to see what the wickets are worth.

We see that the wickets aren't particularly important. A wicket on the first ball of the innings reduces the final score, on average, by about two and a half runs. This agrees with common sense — with only twenty overs to bat, you can keep batting aggressively with the fall of a few wickets.

The slopes of the regression lines (to more significant figures than are really justified...) are:
0 (extrapolated from wickets 1 to 6): 1,378
1: 1,357
2: 1,329
3: 1,298
4: 1,271
5: 1,249
6: 1,233
7: 1,027
8: 0,459
9: 0,172

Now, we can use this to start evaluating the impact of bowlers. Suppose a bowler takes the fifth wicket on the last ball of the tenth over. With four wickets down with 60 balls left is worth, the batting team should score another 1,271*60 = 76,26 runs. With five wickets down, they should score 1,249*60 = 74,94 runs. The difference of 1,3 runs gets credited to the bowler. Do this for all the bowler's wickets, and you can adjust his runs conceded and get an effective economy rate.

There are a few points worth noting:
- There's no consideration of how high-scoring the pitch/ground is.
- The quality of the batsman dismissed is ignored.
- The same crediting applies in both first and second innings.
- If a team collapses quickly (say six wickets down by 10 or 12 overs), then the bowler who picks up the next wicket gets quite a lot of credit, since the difference between being in the tail and being in the recognised batsmen is large when there are still some overs left to bat. This isn't really fair on the bowlers who took the early wickets, but it doesn't seem to cause too many problems when comparing bowlers who bowl regularly.

The overall economy rate for bowlers during the IPL was about 1.36 runs per ball. By taking the effective economy rate and comparing it to the average, you get a measure of how many runs the bowler was worth. In the table below, I've called this the value-24: the number of runs above average the bowler is over 24 balls (kind of). I'm not good at coming up for names of things. I wanted to do this because I'm hoping to do something similar for batsmen (that is, get a run per game value for them), so that we can put batsmen and bowlers on the same scale.

The top bowlers, qual. 144 balls (i.e., six four-over spells):


name           balls runs  wkts  cred    avg   econ  eff econ  value-24
Sohail Tanvir  211   210   21    -50,34  10,0  5,97  4,54      14,40
GD McGrath     300   319   12    -22,89  26,6  6,38  5,92      8,87
SM Pollock     276   301   11    -25,34  27,4  6,54  5,99      8,59
IK Pathan      294   326   14    -31,11  23,3  6,65  6,02      8,49
MF Maharoof    192   215   12    -21,09  17,9  6,72  6,06      8,32
AB Dinda       234   260   9     -20,40  28,9  6,67  6,14      7,99
DW Steyn       228   252   10    -8,49   25,2  6,63  6,41      6,93
A Nehra        269   348   12    -50,57  29,0  7,76  6,63      6,02
AB Agarkar     156   207   8     -33,11  25,9  7,96  6,69      5,81
M Muralitharan 300   346   8     -8,83   43,3  6,92  6,74      5,59
DJ Bravo       170   232   11    -37,64  21,1  8,19  6,86      5,12
SR Watson      283   344   13    -19,57  26,5  7,29  6,88      5,05
SK Warne       264   349   17    -42,63  20,5  7,93  6,96      4,71
M Ntini        162   198   5     -8,34   39,6  7,33  7,02      4,46
Shahid Afridi  180   225   9     -13,55  25,0  7,50  7,05      4,37

Sohail Tanvir has, of course, been the stand-out bowler of the IPL. McGrath has been much talked about, but fellow metronome Shaun Pollock not so much.

Tanvir's also been lucky, of course. He's almost certainly not that good. I'll pick up this theme a little bit in the next post.

Lastly, some guys over at Rediff (e.g., here) have been doing what look to be good statistical analyses of the IPL. Unfortunately, they seem to sweep all the calculations under the carpet; if anyone happens to know how they calculate the MVP index, please share with us. (Edit: Here's a description of it.)

IPL results bits and pieces

2008-05-31T07:53:00.000+02:00

The league stage of the IPL is over, and so it's time to start looking back at it. This post looks at some overall results.

Firstly, let's re-visit those blog predictions, now with the final league standings:


Actual           Me          Q           Arjwiz
1. Rajasthan     Rajasthan   =Delhi      Bangalore
2. Punjab        Chennai     =Kolkata    =Delhi
3. Chennai       Delhi       Deccan      =Kolkata
4. Delhi         Deccan      =Chennai    =Deccan
5. Mumbai        Bangalore   =Punjab     Chennai
6. Kolkata       Kolkata     Mumbai      Punjab
7. Bangalore     Punjab      Bangalore   Mumbai
8. Deccan        Mumbai      Rajasthan   Rajasthan

In terms of Pearson's rho (1: perfectly right, -1: perfectly wrong), I won with a score of 0,33, followed by Q at -0,33 and Arjwiz -0,76. Arjwiz is the only one of us to get a significant result. Unfortunately for him, it's in the wrong direction.

Now let's compare the two halves of the IPL. There are various ways of doing this, and I'm not really sure which is the best. First up, home and away wins:


team    home  away
Ban     1     3
Che     3     5
Dec     0     2
Del     4,5   3
Kol     4     2,5
Mum     4     3
Pun     6     4
Raj     7     4

If IPL matches are essentially just coin tosses, then the correlation between the two columns should be around zero. The results are actually correlated more strongly than I would have expected — r = 0,49. To minimise any potential differences in home advantage between teams (not that you'd really expect too many; overall, home teams won 29 out of 55 games), I also split the matches into two round-robins, with one group having four home games and the other three (for each team). That gave r = 0,28, though there are many more ways of splitting up the games. Probably I should get the computer to do all of them and find the average.

Anyway, it looks like IPL cricket is not just a coin-toss game, though just how much of the results is luck-based will take a few years to work out properly. The positive correlations that we've seen would happen by chance about once every six or so tournaments.

Of the 55 matches, the team batting second won 32 times. That's a bit more than a standard deviation above the expectation of 50%, so nothing significant. Until a more detailed analysis comes along, it seems safest to bowl first.

Batting well with a batsman

2008-05-25T04:29:00.000+02:00

That's right people, a new post! Now that I'm back at uni, I have less time for cricket analysis, so I'll be aiming to get about one post per week, maybe two if I find something simple and interesting on Statsguru.

Some long time ago at Well Pitched, there was a discussion on great batsmen and how they supposedly "lift" their teammates when batting with them. I was sceptical about this being a real effect. Analysing it properly will take at least a couple of posts, and this is the first one.

Getting data on partnerships from summary scorecards always carries with it the problem of retired hurts. It's not just a question of definition (if an opener retires hurt before the fall of the first wicket, do you have two first-wicket partnerships or a three-way partnership?). The problem is that retired hurts are not always recorded on scorecards if the batsman in question returned to the crease. (Certainly in my lazy database, this is never recorded.) So it sometimes happens that you look at the FOW's to work out which partnerships happened and how many runs each was worth, subtract and you find that a batsman contributed negative runs during some passages of play.

So before I started gathering partnership data, I did my best to get rid of innings where there was a retired hurt. Innings were deleted if:

- a batsman finished retired hurt;

- the number three was the first wicket to fall, etc.;

- reconstructing the FOW's from the minutes batted by each batsman (where possible) disagreed with the actual FOW's;

- any partnerships required negative runs from one batsman to make sense.

Point three is an interesting one, because careful traces through of the minutes batted can identify both the presence of retired hurts and also of errors in the minutes as given. One curious error is in this Test, in which Manoj Prabhakar apparently batted for 304 minutes, while the rest of the batsmen combined for only 274.

Anyway, the above procedure isn't perfect — it won't pick up all retired hurts, especially if the minutes aren't recorded, and there are probably some innings where the anomalous minutes are just scorer/Cricinfo/CricketArchive errors and not actually showing retired hurts. But it seems to do a reasonable job, and about 430 innings were removed.

Now to the analysis proper. For each batsman and each innings, I took the runs in his partnerships and subtracted off his own score, so that we're left with the runs scored by his partners and extras while he was at the crease. Then you count how many times he saw his partners get out, and you have the average of his partners (plus extras) when he was at the crease.

To get an expected average, I added up the averages of all his partners, and divided by the total number of partnerships.

Then divided the actual partner-average by the expected partner-average, and you get a measure of how well people bat with him, relative to their careers.

When you do this, you find that players with short careers have much more variation than players with longer careers. Graph (qual. 20 innings, batsman's average at least 30):

(The average ratio across all these batsmen is about 1,1.)

Now, what I think I should do at this point is to work out how reliable the statistic is (i.e., how much of it is skill, and how much just luck), and then regress each player to the mean appropriately. (I'm learning from the baseballers, who do this sort of thing a lot.) But working out how reliable this stat is will require some thought (you're welcome to do the thinking for me). One problem is that part of what it measures might be called flat-track-bully-ness. If a batsman does disproportionately well on flat tracks, then it might be the case that he is part of many big partnerships which bloat his partner average.

But I will ignore this for now, and instead find z-scores. I ordered the batsmen in order of innings batted, found the moving standard deviation of the next 30 ratios, and then fitted a curve to it. It goes a bit like 1,3/sqrt(no. inns), for those interested. Then for each batsman, you use this as the standard deviation, and find how many standard deviations from the overall mean his ratio is.

(In terms of the reliability, the question is: Does being a standard deviation above the mean after 20 innings mean that you'll probably be a standard deviation above the mean after 200 innings?)

In the table below are the batsman's average, innings batted (having excised team innings probably involving retired hurts), runs by partners (incl. extras), total number of partnerships, expected partner average, actual partner average, ratio, z-score. Note that the partnership average is not just the partner runs divided by the number of partnerships — it's the partner runs divided by the number of times the batsman saw partners dismissed.


                                            partner-avg
name            avg   inns  p-runs  pships  exp   act   ratio z
RT Ponting      58,6  183   9622    308     43,7  63,7  1,46  3,94
RL Dias         36,7  33    1260    54      28,9  54,8  1,89  3,67
DS Lehmann      45,0  42    1733    62      43,8  78,8  1,80  3,66
DJ Bravo        33,0  44    1601    70      33,7  59,3  1,76  3,52
RWT Key         31,0  25    895     36      39,1  74,6  1,91  3,24
RT Robinson     36,4  45    1971    72      37,0  61,6  1,67  3,06
HH Dippenaar    30,1  60    2237    88      43,0  67,8  1,58  2,96
ME Trescothick  43,8  136   5890    234     38,8  54,5  1,41  2,88
Shoaib Mohammad 44,3  65    3673    123     37,1  56,5  1,52  2,73
G Pullar        43,9  44    1898    67      43,7  70,3  1,61  2,70
CL Cairns       33,5  97    2815    158     30,0  42,7  1,42  2,55
FA Iredale      36,7  22    930     42      25,1  44,3  1,76  2,48
V Sehwag        53,8  82    2794    128     39,7  57,0  1,44  2,44
Javed Miandad   52,6  172   8715    335     35,8  47,6  1,33  2,42
MLC Foster      30,5  23    624     26      45,2  78,0  1,72  2,39
GC Smith        49,5  107   4905    191     40,0  55,1  1,38  2,31
Habibul Bashar  30,9  96    2775    189     21,3  29,5  1,38  2,22
M Prabhakar     32,7  57    2064    89      35,8  51,6  1,44  2,06
CG Greenidge    44,7  175   7559    301     41,8  54,0  1,29  2,00
AH Jones        44,3  71    3560    144     31,9  44,5  1,39  1,97

Make of that what you will....

The bottom-end, those who apparently make their partners bat badly:


                                            partner-avg
name            avg   inns  p-runs  pships  exp   act   ratio z
Saeed Anwar     45,5  84    3287    194     34,3  29,3  0,86  -1,92
DJ Cullinan     44,2  111   3965    217     38,3  33,9  0,89  -1,95
WR Hammond      58,5  129   6160    284     40,1  36,0  0,90  -1,98
RA McLean       30,3  66    1049    105     31,0  25,0  0,81  -2,02
FE Woolley      36,1  92    2404    162     37,0  31,2  0,84  -2,10
SM Pollock      32,3  151   3704    249     30,3  27,2  0,90  -2,16
JT Tyldesley    30,8  54    1655    129     29,0  21,8  0,75  -2,16
AG Chipperfield 32,5  20    431     52      24,4  12,3  0,50  -2,19
MA Noble        30,3  70    2277    162     29,7  23,0  0,77  -2,30
WJ Cronje       36,4  105   3764    219     36,7  30,6  0,83  -2,32

Well if Hansie Cronje coming last on this statistic isn't the most appropriate thing I've ever put on this blog, then I don't know what is! Good to see his worshipper Shaun Pollock also down there.

Two names mentioned in the Well Pitched discussion were Steve Waugh and Inzamam-ul-Haq. They are at z = -1,22 and z = -0,40 respectively.

When I next attack this problem, I will also check to see if there are any patterns with batting position, and also look at batting with the tail.

Back in Australia

2008-05-17T23:49:00.000+02:00

Sorry for the interruption to posting. I'm now back in Brisbane, and things should be organised enough to return to cricket blogging in a couple of days.

Learning from baseball: Pitchf/x

2008-05-12T13:19:00.002+02:00

While we're on the subject of baseball, I thought I'd outline a simple idea used in baseball that would be useful and fun in cricket. In short: put Hawkeye data on the web for anyone to download.

In Major League Baseball, they have a system called Pitchf/x, which we can basically think of as Hawkeye. They don't have it at every game (only about a quarter, I think), but since there are over a thousand games a season, that's still a lot of pitching data. The raw data gets put on the MLB website, and you can download big pitch-by-pitch tables, with each pitch described by release point, start speed, end speed, break length, break angle, etc.

Classifying the pitch type can be difficult, but by using enough of the variables in the table, people who've studied this problem are getting reasonable results (for an introduction to it, see here). Here's an example, taken from this article on Jake Peavy by Pitchf/x'er Josh Kalk:

If you have a look at the linked article, you'll see other graphs, plotting different variables.

It's a gold mine for baseball analysis, and it would be the same in cricket. There are all sorts of things you could look at, at the level of an individual bowler, or looking at the characteristics of the ground — length of the ball, amount of swing, amount of turn, how much bounce there is in the pitch, etc.

To get it to work, we'd want something like the following recorded for each ball (it may be possible to make this more efficient with some knowledge of cricket ball physics, but this should give the idea):

bowler, batsman, age of ball, did ball hit bat?, number runs scored off the ball or type of wicket, and then x-, y-, z-components of position and velocity at: release point, just before pitching, just after pitching, contact with bat/batsman, crossing the stumps (projected if necessary).

I will happily plug the first broadcaster that puts this data on the web.

John Buchanan and The Guardian article

2008-05-10T09:57:00.001+02:00

Hello to those of you who've come here from Andy Bull's piece in The Guardian. I hope you find something interesting here.

There are a couple of ideas in that article that I think are worthy of more detailed discussion.

What John Buchanan says is interesting, but it seems to me that he's taking a purely coaching perspective. He says:

1) Ignore existing cricket statistics - these are just the 'outcome numbers' of a process of getting there.

If I were a coach, I would probably agree with this. Buchanan goes on to give the example of strike rate. It would be no good a coach saying to a player, "Hey, you're averaging 35 at a strike rate of 70. I want you to average 40 at a strike rate of 80." You need to break batting down into its parts and make improvements at that level.

That's where the ball-by-ball analysis comes in — what Buchanan calls 'process numbers'. (Buchanan is very big on processes, I gather. I've seen him talk about them elsewhere.) You look at the dot balls, try to improve shot selection on them, etc. You hope that you'll end up scoring more runs at a higher rate.

That's what the coach does. From a selection perspective, the outcome numbers are still going to be important. No-one cares what your percentage of dot balls is if you average 25, and no batsman will hold down a spot in the national side with such a low outcome number. Cricket games are won by the team that scores the most runs, and we shouldn't lose sight of that. All the 'processes' work is no good if it doesn't improve averages (or strike rates, in limited overs cricket).

Now, there are times when process numbers might be useful in selection — if a batsman has bad process numbers, then perhaps with coaching he might improve a lot more than a batsman who's already largely optimised his game. I don't know. Without seeing the figures involved and knowing what improvements are usually made, it's hard to say how useful such an approach would be.

Now onto one of the questions Bull posed at the end of the column:

Could we see teams selected through statistical proof rather than the current woolly combination of gut instinct, vague notions about character and compromised measures such as batting averages?

I will be very surprised if, in the forseeable future, detailed statistics will be better at team selection than human experts with regular stats. In terms of working out when to drop players, they might be. (I said here that selectors are probably best off with their gut on dropping players. Perhaps with detailed process stats you could do better, I don't know.)

But when it comes to finding the best players in domestic cricket, I doubt if a computer would do better than Duncan Fletcher, for example (if you haven't read Andrew Strauss's thoughts on Fletcher, I recommend doing so). Fletcher famously picked Michael Vaughan for the 1998/9 tour of South Africa on 'temperament'. His record in county cricket was not great — his first-class averages in the previous two seasons were 34 and 41. His average for Yorkshire is still well under 40. But despite that, in England colours he turned himself into a good batsman, doing better against Test sides than against county sides.

Now, it's possible that with sufficient process numbers from his county games, you would be able to tell him apart from the rest of the county hacks averaging high 30's. But I'd be surprised if it were so.

Obviously you'll want to be paying attention to stats when picking national sides — you won't consider batsmen averaging under 30, and you'll certainly be looking at those averaging 60 — but since the quality of the players is significantly lower in domestic cricket, you'll want humans watching them, gauging their technique and judging if they'll hold up against 90mph pace bowling or top-class spinners.

They don't always get it right, of course, but I think that they do better than a computer (or a person looking only at numbers) would do.

Australia batting first in ODI's

2008-05-06T12:01:00.001+02:00

There's an interesting comment by Nesta on my rambly post about batting-first strategies. Essentially, Nesta reckons that Australia have come close to perfecting the art of batting first in 50-over cricket.

Since there's much more scope for variation in batting-first strategies than batting-second strategies (in the latter, everyone know how many runs they need), you might conjecture that this will show up in the results. And it looks like it does.

I considered ODI's between the top eight sides in the 2000's. I split them into day games and day-night games, because the two are markedly different (day games strongly favour the team batting second; day-night games favour the team batting first).

In day games, Australia has won 73% of matches when batting first (ignoring no-results). Second is Sri Lanka at 49% — a whopping 24 percentage points! Australia has won 78% of matches batting second, with South Africa second at 71% — only seven percentage points behind.

In day-night games, batting first: Aus 76%, South Africa 63%; batting second: Aus 62%, South Africa and Pakistan 55%. Once again, a bigger difference in batting first results.

So it does look like Australia have an advantage over their rivals when it comes to batting first, above and beyond their general cricket superiority.

Now for some tables. For each team, I give the number of matches (actually this column includes no-results because I was lazy when doing the copy-paste), the win fraction batting first, the win fraction batting second, and the ratio. First up, day games:


team          mats  1st   2nd   ratio
Pakistan      41    0,40  0,37  0,92
Australia     42    0,73  0,78  1,07
Sri Lanka     53    0,49  0,61  1,25
India         47    0,39  0,58  1,49
West Indies   47    0,29  0,50  1,73
South Africa  39    0,38  0,71  1,85
New Zealand   39    0,29  0,62  2,14
England       38    0,19  0,56  2,94

Only Pakistan does better batting first in day games, but that is probably noise, given where Pakistan is on the next table. Australia is second, with only a small improvement when chasing.

Day-nighters:


team          mats  1st   2nd   ratio
Sri Lanka     59    0,58  0,34  0,59
Australia     70    0,76  0,62  0,82
England       42    0,37  0,31  0,85
South Africa  43    0,63  0,55  0,88
India         50    0,42  0,39  0,92
Pakistan      55    0,58  0,55  0,93
West Indies   22    0,25  0,25  1,00
New Zealand   41    0,43  0,43  1,01

Australia once again second — it's interesting to see Sri Lanka in the top three in both tables as well. Only New Zealand have a better record chasing in day-nighters.

It's worth pointing out that this could do with a more detailed analysis — Australian grounds may be more bat-first-friendly in day-nighters than others, which would explain Australia's high position in the second table.

Luck

2008-05-04T22:13:00.001+02:00

I thought I'd simulate a double-round-robin tournament with eight teams, to model the IPL. So Teams A to H each play 14 games. Here is the final ladder, ordered by number of wins:

C: 10
F: 10
B: 9
G: 7
H: 7
D: 6
A: 4
E: 3

Team E'll be looking for a new coach — only three wins out of fourteen.... Anyway, as the title of this post will suggest, the result of each match was decided by a (virtual) coin toss. The point here is that if all teams are perfectly evenly matched and results come down to the luck of the day, you'll still end up with teams at the top of the ladder having much better records, over 14 matches, than the teams at the bottom.

Now of course there is skill involved in cricket, and some teams in the IPL are better than others. But can we tell which team is the best just from the results? Probably not from just one season (unless they put in a really dominant performance — lots of wins, by big margins). And more importantly, it'll be impossible to say how good each team actually is. To explain this point, I'm going to borrow the notation from American sports (since that's how I think of it in my head — much of what I write here can be found somewhere in the archives of this blog and this blog). A .500 team ("five hundred") is a team that wins 50% of its matches. A 0.600 team wins 60%, and so on.

To work out if a team is really a .600 team (say), you'd need an infinite number of matches to prove it. Of course, we could get by with a large number — just how large depends on how much luck is involved in each game. The problem with T20 is that we don't know how much luck there is. So we're going to be fumbling around in the dark somewhat — once we've had a few seasons (to get enough data), we'll be able to look at the win-loss records of the teams and see if the how much greater the variance is than that expected by chance.

I worked out some numbers for ODI's and Tests here; T20 will have more luck involved than fifty-over cricket, but the IPL complicates things as the foreigners are dominant, and there's only four of them per side. If the long-term variance in win percentage is the same in the IPL as it is for ODI's (a big if), then you'll need each team to play about 17 or more games before the skill will demonstrably be playing a part in the results.

One season of IPL isn't going to be enough. In the coin-toss example above, every team was a .500 team. Only G and H ended with .500 records. Teams above them were lucky, teams below (especially E) were unlucky.

If we look at the IPL table today, Rajasthan are at .833. Are they genuinely an .833 team? They could be. Or they could be a .900 team that happened to lose one of their first six matches, or a .500 team that's had a bit of luck.

Let's not forget, Zimbabwe beat Australia not long ago in a T20 game. We should expect bad teams to win matches. And sometimes, mediocre teams will string together a few wins on the trot. Conversely, good teams will lose some. Does anyone really believe that Deccan (Gilchrist, Afridi, et al.) is a .167 team?

One way of seeing how much skill is involved will be to compare the first half of the tournament with the second, and see what correlation there is. Unfortunately, the coming and going of lots of big stars will make this really muddy, but I'll still do it at the end of the tournament.

So my message is, don't read too much into individual results. Don't say that the team on top of the ladder is the best simply because they're coming first — they might be the best team, but they might just be lucky. Go and read this excellent piece by Lawrence Booth at Cricinfo.

The IPL so far

2008-05-03T12:41:00.002+02:00

Each team in the IPL has now played five matches. I thought I'd have a look at the points table. Really the only reason I'm doing this is because I end up looking prescient, and if I let it go too long the results might start turning against me.

Near the end of this post, I came up with some half-baked ratings on how clever each team's bidding was. Contrary to just about everyone else, Jaipur (ie, Rajasthan) came out best. I didn't even really believe it myself, so I don't want you to go back and read the paragraph afterwards in that post. Just pay attention to the numbers.

Q gave his auction ratings here, while Arjwiz gave his here.


Actual           Me          Q           Arjwiz
1. Delhi         Rajasthan   =Delhi      Bangalore
2. Chennai       Chennai     =Kolkata    =Delhi
3. Rajasthan     Delhi       Deccan      =Kolkata
4. Punjab        Deccan      =Chennai    =Deccan
5. Kolkata       Bangalore   =Punjab     Chennai
6. Deccan        Kolkata     Mumbai      Punjab
7. Mumbai        Punjab      Bangalore   Mumbai
8. Bangalore     Mumbai      Rajasthan   Rajasthan

I've got the top three! Albeit in the wrong order, because of net run rate. In terms of Pearson's rho (-1: perfectly wrong, 1: perfectly right), I'm at 0.62, Q's at 0.34, and Arjwiz is -0.27.

Maximising runs or wins

2008-05-01T21:33:00.000+02:00

In a post at 99.94, I took the comments thread off on a long tangent that was only just related to the original post.

It got me thinking about batting strategies (at a conceptual level) in limited-overs cricket. Batting second, it's simple: choose the strategy to maximise your chance of reaching the target. Every team does this instinctively — chasing 350, they go for broke, and often end up losing by a lot.

Batting first, I'm not sure what the optimal strategy is. Instinctively, I at first thought that you should choose the strategy to maximise the expected number of runs that you score. But scoring runs isn't actually the end goal — it's winning the game. And increasing the average number of runs you score won't always improve your win/loss ratio.

To take an extreme example, suppose you're a really bad team like Bangladesh, up against a team like Australia. Whenever Bangladesh bats first, they choose the run-maximising strategy. The results might be a bell curve centred around 180. So a lot of scores around 170-190, a few past 200, a few below 160, etc.

Now Australia has no problem chasing any of those. Australia's only going to have problems when the target's up over 250. So while the Bangladeshi averages will be best-served by going with the run-maximising strategy, they may end up losing every game.

On the other hand, if they play more aggressively, then sometimes their batsmen will have a bit of luck and they'll end up with a big score. In their long series of matches with Australia, they'll have loads of heavy defeats, after making scores like 120 and 150 and so on, but every now and then, they'll make 250 and have a chance at winning. So their averages will suffer, but their win/loss ratio will improve.

It'd be a public relations disaster, of course — all those thrashings.

If you've got two more evenly-matched sides, choosing the win-maximising strategy when batting first becomes problematic. Maybe you've studied the opposition's batting and concluded that you're best-off aiming for 270+. But maybe the pitch is not so good, and you don't know how to adjust that 270 score. You'll probably go back to a run-maximising strategy.

Nevertheless, I think with a very careful analysis, there's scope for improving win/loss ratios. I think it's most applicable in T20, because it's so short. If you bat first and lose early wickets, what do you do? Go for broke (hoping for 140 but probably getting 90), perhaps, rather than slowly batting out the overs (and getting 120)? It'll probably need a few years of IPL before we have enough data to say.

On an unrelated topic, the latest post chez Z-Score has a teaser question: What is the highest Test partnership for a pair who only batted once together in Tests? The hints are that they aren't Australian, and that the partnership is higher than 320. For those who don't want to search for it themselves, feed this into ROT13:

yrauhggbanaqznhevpryrlynaqjuraratynaqznqrbireavaruhaqerq.

WG Grace

2008-04-27T17:14:00.001+02:00

WG Grace had a very long career — he played a long time after his peak. That's why, when looking at his career averages (unadjusted first-class averages from CricketArchive: bat 39,45; bowl 18,14), you don't see why he's such a huge figure in the history of the game. His aggregates are huge, sure, but it looks like he was a great who played for a long time, rather than a rival to Bradman as the greatest ever.

To see where this latter perception comes from, I plotted his cumulative adjusted averages (weighting innings according to the quality of the attack, relative to an overall average of 24,5) against time. I considered only first-class matches in England (since I know that that part of my database works — I didn't want to spend half a week debugging Australian matches that I haven't tested yet).

To give a feel for the batting scale: Bradman is at 98,9; Headley 65,8; Ranji 60,5; Merchant 56,3 (those are the top four); Mike Hussey 52,2; Barry Richards 45,3.

Bowling scale: Murali 13,7; Lindwall 14,1 (top two); Darren Gough 20,9; Eddie Hemmings 25,6.

All references to averages below are adjusted ones.

Grace's (adjusted) batting average peaked at the end of the 1873 season at 92,8; at this time he had scored over 10000 first-class runs. Batting doesn't get much more Bradmanesque. By 1880, he had 19560 runs at 74,5. This also marks the start of his decline as a bowler. By the end of 1880, he had 1335 wickets at 21,9. If Grace had stopped playing then, his ratio of batting average to bowling average (3,4) would have been well clear of second place (Keith Miller at 2,8).

Even in 1886, though (more than 20 years after the start of his first-class career), his batting average was higher than Headley's.

The decline in batting average is very marked — it almost falls to 50 by the end of his career. The rise in his bowling average is much more gentle, because as he got older he bowled less, not bowling more than 4000 balls in a season after 1888.

Bowler workloads

2008-04-26T20:21:00.003+02:00

Over at CFF, there was a debate over spinners' averages. One poster said that spinners bowl disproportionately many overs on flat pitches, while lazy pacemen rotate at the other end. This has the effect of bloating out spinners' averages unfairly. Another poster responded by saying that spinners also bowl disproportionately many overs on raging turners, which would help their averages.

Which factor is the dominant one? To answer this, I considered every innings, and scaled each bowlers' figures so that he effectively bowled a quarter of the overs. So, for instance, if a team batted for 100 overs, and one bowler took 1/80 from 20 overs, that would be scaled up to 1,25/100 from 25 overs. If another bowler took 1/60 from 30, it would become 0,83/50. So each bowler's average in any given innings won't change, but we'll see any effects of bowling or not bowling in tough or easy conditions.

Now, sometimes a bowler might, say, bowl one over in an innings and take a wicket in it, or bowl one over and get hit for 15 runs. Obviously it's not realistic that he would have taken 25 wickets or gone for 375 runs from 25 overs, so if the number of balls bowled by a bowler was less than 60, I didn't do any adjustment. It's a bit arbitrary where you put the cut-off, but pushing it back to 30 balls doesn't change the overall trends.

One very stunning result comes out of this analysis. Every major wicket-taker (at least 100 Test wickets), except for Vanburn Holder, has his average increase. I'm still wondering a little bit if it's a bug in my code, but since I can't find one, and it passes various sanity checks, I'm reasonably confident that these results are true.

Assuming that I haven't made some silly mistake, there's a simple explanation for this phenomenon — captains can tell which bowlers are being effective on a given day and which ones aren't, and they make the effective bowlers bowl more overs. There could also be a bit of luck involved — say a bowler is a bit unlucky and goes for fifteen wicketless overs. If he'd been given another ten, he might have picked up a wicket or two. But since he hadn't, he didn't get to bowl again.

Let's have a look at the top and bottom of the table, ordered by the difference in weighted (by the average of the batsmen dismissed) average. Qualification: 100 wickets. Columns of the table are: wickets, regular average, weighted average, scaled regular average, scaled weighted average, difference between scaled regular average and regular average, difference between scaled weighted average and weighted average.


                          avg      scaled avg    diff
name             wkts  reg   wtd   reg   wtd   reg   wtd
VA Holder        109   33,3  37,0  33,1  36,8  -0,2  -0,2
DA Allen         122   31,0  30,1  32,4  30,6  1,4   0,4
PM Pollock       116   24,2  26,5  24,9  27,0  0,7   0,5
M Dillon         131   33,6  31,4  34,3  32,1  0,7   0,6
AN Connolly      102   29,2  27,1  29,3  27,9  0,0   0,7
CEH Croft        125   23,3  23,3  24,3  24,1  1,0   0,8
NAT Adcock       104   21,1  23,5  22,1  24,3  1,0   0,9
M Muralitharan   724   21,8  24,6  22,5  25,5  0,6   0,9
MW Tate          155   26,2  26,3  26,9  27,3  0,8   1,0
WJ O'Reilly      144   22,6  22,8  23,1  23,8  0,5   1,0
---
C Blythe         100   18,6  23,8  22,5  28,9  3,9   5,1
H Trumble        141   21,8  26,5  25,8  31,6  4,0   5,1
Mohammad Rafique 100   40,8  40,6  45,4  45,8  4,7   5,2
N Boje           100   42,7  38,9  48,4  44,0  5,8   5,2
Intikhab Alam    125   36,0  37,4  41,8  42,9  5,8   5,5
W Rhodes         127   27,0  32,7  31,4  38,3  4,4   5,6
J Briggs         118   17,8  32,4  21,9  38,3  4,1   5,9
AF Giles         143   40,6  37,7  46,9  43,7  6,3   6,0
TE Bailey        132   29,2  30,6  35,6  37,1  6,4   6,4
AL Valentine     139   30,3  33,2  35,9  39,6  5,6   6,4

Those near the top of the table are the ones who bowl in the tough conditions or don't bowl so often in favourable ones; those near the bottom don't bowl so much in the tough conditions but do when things are going well.

The results aren't what I would have expected. Murali's position near the top is easy to explain — he does a huge amount of bowling for Sri Lanka come what may. Generally, though, it's pacemen at the top and spinners at the bottom.

The keen-eyed amongst you will note that, with the exception of Murali, none of the bowlers listed above took more than 155 wickets. There is much more variation for bowlers with lower numbers of wickets:

I've shown a quadratic fit because it looks better than a linear one. The general trend is clearly downward — bowlers who take lots of wickets tend to get hidden less from flat pitches than bowlers who don't.

Now of course, considering only bowlers with large career wicket hauls will pick out good bowlers, but what about great bowlers from the olden days who didn't play so many Tests? If you take the ratio of scaled weighted average to weighted average, and plot against the weighted average, you get only a very very slight positive correlation (y = 0,0008x + 1,07; R-squared = 0,01).

(If you plot the difference, rather than the ratio, you get a strong positive correlation. I think it's more accurate to work with ratios here.)

So, my conclusions so far are:

- Most of the variation is to do with small samples.
- But better bowlers do have slightly smaller differences (or ratios) when you scale their workloads to one quarter of each innings' overs.

Murali definitely belongs near the top of the table — such a small difference after taking over 700 wickets is clearly a genuine (and easily explained) trait of his bowling, and not statistical noise. This looks to me like a good way of trying to answer the question, "How would Warne have done if he had Murali's workload?" I don't think it's reasonable to do this for most pairs of bowlers, but if they have a large number of wickets (as with Warne and Murali), then the difference between a pair of players is likely to be genuine. And in this case, Murali comes out easily the better.


               wtd avg   sc wtd avg
M Muralidaran  24,6      25,5
SK Warne       27,9      30,7

Now to the question that started this all — spinners v pacemen. For pacemen, the average ratio of scaled weighted average to weighted average is 1,08. For spinners it's 1,11. So it looks like spinners getting to bowl on raging turners is a bigger factor for their averages than having to shoulder the workload on flat tracks. Doing the diff v wkts plot as above for spinners and quicks separately clearly shows the quicks (on average) having smaller differences across all lengths of career.

Lastly, if you only scale downwards (i.e., if they bowled more than a quarter of the overs, then scale back to a quarter, else do nothing), then the ratios become 1,05 for quicks and 1,08 for spinners.

More on getting your eye in

2008-04-22T21:44:00.001+02:00

I've left things too late for a comprehensive post, but since I'm disappearing again for a few days (Toulouse and Carcassonne this time), I thought I'd do a little post on those effective average curves without coming to any sweeping conclusions.

Some of the fits seem to have fallen into a local minimum that doesn't describe the effective average curve very well. Most obvious of these is Mark Richardson, who apparently always bats like he averages 57 but really only averages 45 (eyeballing his empirical hazard function, I think his effective average starts high and decreases). Such clearly wrong curves seem to be pretty rare, so hopefully they don't poison the overall trends too much.

Best and worst on nought (qualifications: average of 35, and some minimum number of innings, probably 50):


name          µ(0)  µ(1)  µ(10) µ(30) avg
MH Richardson 57,4  57,4  57,4  57,4  44,8
AN Cook       53,3  49,5  46,5  44,3  43,5
AL Hassett    52,1  52,1  52,1  48,0  46,6
CL Walcott    47,1  47,2  50,1  62,8  56,7
CH Lloyd      42,6  46,7  47,7  48,3  46,7
H Sutcliffe   41,0  73,8  75,0  75,6  60,7
Saeed Ahmed   39,1  41,3  41,3  41,3  40,4
CC McDonald   36,8  41,1  41,1  41,1  39,3
PE Richardson 35,8  36,8  36,8  36,8  37,5
GM Ritchie    35,8  16,9  33,4  34,9  35,2
---
DL Amiss      7,8   29,4  39,6  46,2  46,3
NS Sidhu      7,7   37,0  46,8  52,6  42,1
MW Gatting    7,6   26,2  37,1  44,4  35,6
VL Manjrekar  7,4   42,5  42,5  42,5  39,1
DM Jones      7,1   35,7  46,1  52,2  46,6
FMM Worrell   6,9   37,6  51,5  60,2  49,5
JA Rudolph    6,9   23,0  33,9  41,4  36,2
FE Woolley    6,5   41,1  44,3  46,0  36,1
CG Borde      6,5   36,7  41,1  43,5  35,6
MS Atapattu   6,1   32,5  36,4  38,4  39,0

But as I said in my previous post, much of the variation in how players do on nought can be put down to chance. Not all of it, though, so some of those names in the table above should belong around where you see them.

Here's a plot of effective average at nought against regular average:

Now the effective average on 1 against regular average (there's much less scatter because there's basically no smoothing done at nought, but there is at 1):

Effective average at 10:

Effective average at 30:

The effective average at 10 is very good at predicting the overall average — the scatter is noticeably less than at 1 or 30 (I haven't tried numbers in between to see where the minimum actually is). Part of this might be an artefact of the model, but I think that part of this is a real effect, especially the larger scatter at 30. Good players should have their eye in by 30 and do much better than their overall average. Bad players don't improve so much from how they did earlier in their innings. Why don't they improve? Why do they get as much eye in as possible (which is not very much) early in their innings? Questions for psychologists perhaps.

London trip

2008-04-14T11:12:00.001+02:00

Tomorrow I'll be heading to London, where I'll be until the end of the week. For the first time in almost a year, I'll actually watch a day of cricket. I don't mean that I haven't been to a cricket ground for a year (it's been longer than that) — I'm including television as well. It's been a while. So let's hope that the rains stay away on Wednesday for the first day of the Championship, and in particular Surrey v Lancashire.

So before I disappear for a week, here's a quick run-down on what I've done on getting your eye in in the last couple of days. I worked out how to script gretl, so I've now got effective average curves for pretty much all major batsmen (there are a few with really bizarre hazard functions that refused have my curve type fitted to them).

There doesn't appear to be any substantial differences in effective average on nought between openers, all-rounders, and others.

All-rounders might behave differently in terms of how they perform once they're off the mark, but I need to have a bit of a more careful look.

I'm wondering how much of the variation in effective average on nought is due to luck. Looking at batsmen who average over 40, the average proportion of ducks is about 0,061. Using that and applying the binomial theorem with 120 innings (the average number of innings across the dataset), you get an expected standard deviation of 0,022. The actual standard deviation is 0,024. About 60% are within one standard deviation of the mean, a little less than would be predicted (68,7%) by the normal distribution. (I'm assuming there's enough innings for the normal distribution to be a good approximation.) So it looks like there are real differences between batsmen in terms of ducks, but they might not be so big.

Getting your eye in

2008-04-11T21:57:00.004+02:00

(Update: See these two followup posts.)

An anonymous commenter pointed me to arXiv:0801.4408v1, a paper by Brendon Brewer called "Getting Your Eye In: A Bayesian Analysis of Early Dismissals in Cricket".

Before starting the discussion, I'll define the hazard function. These seem to be used all over the place in the (pretty small) academic literature on cricket scores. The hazard function, written H(x), is defined as the probability that the batsman will be dismissed at score x.

Simple enough. But Brewer points out a very neat interpretation of it (he may not be the first to do so, but it's the first time I've seen it). If the hazard function is constant (i.e., always equal probability of getting out), you get a geometric distribution of scores (or, in the continuous limit, the exponential distribution that I mention every couple of weeks). In particular, a hazard value H is related to a batting average µ by µ = 1/H - 1.

So (here's the important bit), given a particular value of H for some batsman (say H(0) = 0,06 — the probability of making a duck), we can say that, on zero, he bats like someone with an average of 1/0,06 - 1 = 15,67. If you're not convinced that this is useful, you should be by the end of this post.

Technical details follow. Feel free to skip to the tables below.

The methods used in the paper are too technical for me to be bothered understanding them all, but here is a brief summary:

- Assume that the hazard function is of a particular type depending on various parameters.
- Estimate what those parameters are.

I think that the main problem with the assumptions is that they don't take into account how important getting off the mark is. It's assumed that it's a pretty smooth transition from zero to some higher score. But, if you work out the hazard directly (I took all batsmen who average over 40 in Tests), you get this:

There's an almost 20-run jump in effective average from a score of zero to a score of one. (Also, it curve doesn't level off for a long time.)

So, I instead took assumed that the average associated with the hazard goes like:

µ = a + k*(b - a)*x^p.

I originally intended for all of those parameters to have nice interpretations, but the actual results made a mockery of that idea. Anyway, if you fit the graph above up to x = 30, you get the following (fit parameters: a = 15,2; b = 49,4; k = 0,63; p = 0,17):

Technical notes: I did the fits using gretl's non-linear least squares abilities. You do the fits with the hazard function and then convert to an average, since sometimes the hazard function gets close to or equal to zero. If you convert to an average before fitting, you get some data points heading off to infinity and nothing works. I used scores from 0 to 49. The way the equation's set up, the parameter 'a' basically picks out the empirical hazard at zero. I think that this is reasonable, since getting off the mark is so important. But it's debatable.

Unfortunately, I don't know how to automate everything, so I have to do one batsman at a time. Maybe next time I'm listening to football on the radio I'll go through and process a bunch of batsmen.

Now, the parameter 'a' does have a nice interpretation: it's the effective batting average at zero. The parameter 'p' tells you how flat the curve is (near zero: very flat). While I give the values of the parameters for each player, the more important thing is the effective average µ at various scores. I've given µ(0) (which is just 'a'), µ(1) (effective average at 1), µ(10), and µ(30). There's a bit of round-off error (I went gretl -> pen and paper -> Excel), but it's nothing serious. I've also given the regular average in the last column.

To start with, here's the very atypical example of Don Bradman:


player        b     k     p     µ(0)  µ(1)  µ(10) µ(30) avg
DG Bradman    71,3  0,96  0,13  10,4  68,9  89,3  101,4 99,9

Don't let Bradman get off the mark! On zero he batted like someone who averaged 10, but on one he was almost as good as Mike Hussey. Very soon he batted like the best batsmen ever. Bradman's apparent woefulness before he got off the mark seems pretty typical. Let's have a look at some others.


player        b     k     p     µ(0)  µ(1)  µ(10) µ(30) avg
SR Waugh      50,3  0,56  0,22  10,7  32,9  47,5  57,6  51,1
N Hussain     38,2  0,53  0,29  11,3  25,6  39,1  49,5  37,2
JL Langer     56,3  0,63  0,026 16,9  41,7  43,3  44,0  45,7
G Kirsten     61,6  0,24  0,41  12,4  24,2  42,8  60,0  45,3
BC Lara       199,0 0,11  0,25  12,5  33,0  49,0  60,5  53,2
ME Waugh      70,6  0,39  0,18  10,0  33,6  45,8  53,6  41,8

Interestingly, the two best batsmen on nought are Langer and Kirsten — both openers. Unfortunately, this is a (very) limited dataset, so we'll put this aside for further study.

In Brewer's own small dataset, he saw that the two all-rounders had little change from "before eye in" to "eye in". Part of that I think is due to the wrong shape of the hazard function, but all-rounders doing well on nought does seem to be a real effect (again, pending a more thorough study):


player        b     k     p     µ(0)  µ(1)  µ(10) µ(30) avg
SM Pollock    44,2  0,53  0,026 16,2  31,0  32,0  32,4  32,3
CL Cairns     43,2  0,39  0,28  14,2  25,5  35,8  43,5  33,5
GStA Sobers   68,2  0,75  1E-08 12,4  54,3  54,3  54,3  57,8
Imran Khan    48,7  0,55  0,052 14,9  33,5  35,9  37,1  37,7
GA Faulkner   31,7  0,13  1,02  26,4  27,1  33,6  48,5  40,8
JH Kallis     68,2  0,56  0,13  18,6  46,4  56,1  61,8  57,1
KR Miller     50,8  0,48  0,086 16,6  33,0  36,6  38,6  37,0

In terms of flatness of the effective average (once off the mark), I think the most important factor is the regular average. Sorta-all-right batsmen who average 30 will typically get lots of starts but not go on with them. Two examples (again it'd be nice to have more, but I didn't cherrypick — they were the only two I looked at):


player        b     k     p     µ(0)  µ(1)  µ(10) µ(30) avg
MR Ramprakash 52,3  0,49  1E-08 6,7   29,1  29,1  29,1  27,3
RS Mahanama   51,1  0,40  0,037 11,7  27,5  28,9  29,6  29,3

So players like Ramprakash and Mahanama have got their eye in once they're off the mark, but that's as far as it goes. Better batsmen continue to improve, but these ones don't, for whatever reasons.

That's all I have for now. Feel free to make requests, and to make it interesting, say what you think the results will be for each batsman (e.g., terrible on zero, good on zero, etc.).

And I hope you're all convinced that the effective average is a wonderful number for this exercise.

Adjusting ODI averages for not-outs

2008-04-10T12:32:00.002+02:00

In my post on adjusting averages for not-outs in Tests, commenter Rich asked about doing the same in ODI's. At first I thought that this would be too difficult, but I decided that with an hour of mindless copy-pasting from Statsguru, I could at least get it working for individual players.

(Usually I'd rather watch paint dry than copy-paste Statsguru data for an hour, but it's not so bad you're listening to the Champions League football on the radio.)

There's one very important difference for this exercise between ODI's and Tests. In Tests, pretty much all the top batsmen can expect to bat out their innings most of the time. In ODI's, the top order can usually do this, but the middle order often have to slog at the end. So whereas an opener can get a start of 50 and carry on to a century, the number six who gets to 50 will often get out soon afterwards.

So I split the analysis into two parts: one for the top order (1-3), and one for the middle order (4-7). Perhaps 1-4 and 5-7 would have been better, but I can hardly be bothered re-gathering the data.

I only considered batsmen with an average of 35 or more, and only considered innings in the last ten years, since there's been a big explosion in ODI run habits recently.

First up, projected increases for the top order:

This is similar to the Test graph — batsmen clearly get their eye in after scoring some runs — but the downward trend starts much earlier, as is expected. After about 60 runs, the average increases are less than the overall average for this dataset (almost exactly 40). So not-outs tend to deflate averages when the score is below 60, but inflates them afterwards.

And now for the middle order:

I wouldn't pay much attention to the curve out past 100, since there's not much data there. It won't make too much difference, since there aren't that many unbeaten centuries in the middle order.

The curve is quite different from that of the top order, in roughly the way we would expect. Not-outs have a deflating effect on averages only up to 25 runs or so, and after that they inflate averages.

Now, I haven't done a thorough analysis on all batsmen, since I don't have that data handy. I've just done some selected cases. Some caveats: Some of the batsmen played earlier than ten years ago, and perhaps the average increase curves was different then. Also, I've applied either top order or middle order adjustments to each batsman, and not both. This won't have too much of an effect, but to do it properly you'd want to split the innings into top-order and middle-order and do them separately. If a batsman's highest score was a not-out, I added the average increase to it. (For Tests, I added the batsman's regular average, but doing so in ODI's is much less accurate.)

In the table below there are four averages presented: the regular average, one adjusted based purely on the batsman's own scores, one based purely on the relevant graph above (shifted up or down to match the batsman's regular average), and one mixture of the two, giving more weight to the graph when the batsman doesn't have many scores greater than or equal to the not-out being projected. I've called these reg, ind, gph, mix. The latter one is the one I'd go with. There's two openers, and the rest from the middle order.


player        inns  no  reg   ind   gph   mix
SR Tendulkar  407   38  44,3  43,4  43,9  43,5
SC Ganguly    300   23  41,0  40,6  40,6  40,6
---
MG Bevan      196   67  53,6  48,1  52,4  48,8
L Klusener    137   50  41,1  41,7  40,1  41,4
MEK Hussey    64    26  55,6  55,0  54,4  54,9
A Symonds     154   32  39,7  40,1  39,0  39,9
DR Martyn     182   51  40,8  42,1  40,0  41,6
RP Arnold     155   43  35,3  33,5  34,6  33,8

Bevan's average has, contrary to my expectations, been pulled back quite a bit, down below 49. Nevertheless, it's still a lot higher than most Bevan sceptics would have it. I wouldn't want to draw too many general conclusions from what was a deliberately biased set of batsmen (all with fairly high not-out proportions), but it looks like ODI averages can be and sometimes are inflated by not-outs much more than Test averages.

Largest deficits to win

2008-04-06T22:37:00.000+02:00

Something a little less full-on today, back to first-class trivia. The largest first innings deficit leading to a win is 402 runs, in this match. But it was a contrived result: Central Districts made 464, Northern Districts replied with 2dec/62, Central 0dec/26, Northern 8/429 won by two wickets. Ignoring matches where the team batting second declared behind, and also ignoring the forfeited Test, the top five deficits to win are (up to the end of the 2007 English season):

384: Barbados 175 & 7dec/726 def. Trinidad 559 & 217, 1926/7

291: Australia 256 & 471 def. Sri Lanka 8dec/547 & 164, 1992/3

287: Manicaland 9dec/513 & 146 lost to Mashonaland 226 & 506 (f/o), 2001/2

279: Western Province 460 & 3dec/219 lost to South African Universities 181 & 7/500, 1978/9

277: Nottinghamshire 4dec/431 & 0dec/94 lost to Lancashire 154 & 4/372, 1961

I can only find two instances of a batsman being out stumped X bowled Y in the first innings, and stumped Y bowled X in the second. They are Frederick Thackeray in this match (st: Cobbett b: Bayley in the first innings, st: Bayley b: Cobbett in the second) in 1839, and Ron Oxenham in this match (st: Strudwick b: Freemand in the first innings, st: Freemand b: Strudwick in the second) in 1924/5.

Slumps - Is there a problem or is he just unlucky?

2008-04-05T15:55:00.000+02:00

I've had a bit of a think about "form slumps", and how we might go about seeing if they're just due to chance (sometimes a batsmen will be dismissed for a string of low scores) or due to some genuine problem (a technique flaw, or a weakness found by opposition bowlers). This post becomes more technical than usual later on, so feel free to fall asleep after the first table. This may not be the best or fastest way of going about this problem, but it's they way I did go about it — sort of like stream of consciousness statistics. The important thing is that it works, and it doesn't require anything more than Excel.

The first thing we need to know is the distribution of individual innings scores. Actually we don't need that — we just need to know what the standard deviation is, relative to the mean. So, for each batsman with 50 Test innings or more and an average of at least 35, I calculated the coefficient of variation (the standard deviation divided by the mean). This is a measure of the consistency of a batsman. My own opinion is that consistency is over-rated — a batsman who goes 0, 100, 0, 100, etc. is just as useful as a batsman who goes 50, 50, 50, etc. But it is still interesting to see which batsmen are more consistent than others, so here's the top and bottom of the table. A higher co-efficient of variation means less consistency.

(Technical note: for not-outs, I just added the batsman's average and considered it a regular innings. Not the best thing to do, but it's close enough, and we'll be doing far worse later on.)


name            inns  avg     sd      cv
MH Richardson   65    44,77   35,72   0,80
H Sutcliffe     84    60,73   48,52   0,80
JB Hobbs        102   56,95   48,53   0,85
AN Cook         51    43,47   37,83   0,87
PE Richardson   56    37,47   32,91   0,88
A Ranatunga     155   35,70   31,40   0,88
IR Redpath      120   43,46   38,62   0,89
NC O'Neill      69    45,56   40,72   0,89
KF Barrington   131   58,67   52,50   0,90
JB Stollmeyer   56    42,33   37,98   0,90
---
Ijaz Ahmed      92    37,67   46,07   1,22
DN Sardesai     55    39,24   48,54   1,24
V Sehwag        90    53,76   66,76   1,24
VT Trumper      89    39,05   48,57   1,24
Zaheer Abbas    124   44,80   56,86   1,27
Hanif Mohammad  97    43,99   55,81   1,27
DL Amiss        88    46,31   60,59   1,31
JA Rudolph      63    36,21   47,99   1,33
W Jaffer        54    35,68   48,09   1,35
MS Atapattu     156   39,02   52,81   1,35

There is only a very slight trend (R-squared = 0,022) showing higher averages associated with lower co-efficients of variation.

The average co-efficient of variation is about 1,05. From now on we'll ignore individual differences between batsmen and just assume that the scores of each batsmen can be treated as random variables, coming from a distribution with mean µ (i.e., his average) and standard deviation 1,05µ.

No, we're interested in slumps. So instead of considering individual innings, we'll be considering groups of innings. We don't know what the distribution of individual innings is exactly, but the distribution of groups of innings will be approximately normal, by the central limit theorem. In particular, the mean of a group of n innings will be approximately normally distributed with mean µ (the same as the career average) and standard deviation 1,05µ/sqrt(n).

Note that while I said that µ was the career average, in order for things to work out properly, we'll actually use the career average apart from the slump in question.

So, to define how bad a slump of n innings is, we'll calculate a z-score. Let µ_c be the career average (excluding the slump) and µ_s be the average during the slump. Then define z = (µ_s - µ_c) * sqrt(n) / (1,05 * µ_c).

For example, suppose a batsman was averaging 45. Then he had a slump where for 25 innings he averaged 30. The z-score here would be (30 - 45) * sqrt(25) / (1,05 * 45) = -1,59. I'll call this a z = -1,59 slump.

How rare is this sort of slump? We can look up the answer in either a cumulative normal distribution table or use Excel (or something fancier). In Excel, the relevant function in French is LOI.NORMALE.STANDARD(z). For that small minority of you who use computers set to English, I think the function is called NORMSDIST.

Anyway, plug in -1,59 and you get 0,056. So, that tells us that the probability that he'd have a slump that bad (or worse) right now is about 5,6%. Does the theory match reality? For each batsman, I considered blocks of 26 innings (26 and not 25 for reasons that were important to me when I was testing this but don't make any difference now) that didn't overlap. It's important that they don't overlap, because otherwise the probabilities will be dependent. Add up the number of slumps worse than a particular z, divide by the total number of blocks sampled, and you get a probability. Here are the results, for varying z's, with the observed probability and that derived from the normal distribution.


z     obs     loi normale
-3,0  0       0,001
-2,9  0       0,002
-2,8  0,000   0,003
-2,7  0,001   0,003
-2,6  0,001   0,005
-2,5  0,002   0,006
-2,4  0,004   0,008
-2,3  0,006   0,011
-2,2  0,008   0,014
-2,1  0,012   0,018
-2,0  0,018   0,023
-1,9  0,023   0,029
-1,8  0,032   0,036
-1,7  0,041   0,045
-1,6  0,053   0,055
-1,5  0,066   0,067
-1,4  0,080   0,081
-1,3  0,095   0,097
-1,2  0,111   0,115
-1,1  0,134   0,136

That's not too bad. The observed probabilities are low in the tail, but we'd be talking about really really bad slumps out there, so that's not too important. For the z=-1,59 slump in the example above (something reasonably typical), we see that it matches pretty closely. From now on, we'll just use the normal distribution instead of the empirically derived probabilities.

So the theory works well enough. But we can't stop here. The probability of a z = -1,59 slump right now is about 5,6%. But that's not what we're interested in. We'd like to know the probability that at some point during a batsman's career, he'll have a z = -1,59 slump. This is a very different thing entirely! The probability of a batsman making a duck is about 6,5%, but you wouldn't call it a slump if he's just made a duck, because ducks are just going to happen sometimes.

So how do we find the probability, given a career of M innings, that there'll be a z = -1,59 slump in there somewhere. To answer this (perhaps not in the best way), let's review some basic probability.

Imagine you roll a die five times. What's the probability that you get at least one 6? You can't say: "The probability of a 6 on any roll is 1/6, so the probability of getting a 6 after five rolls is 5/6." That would be the expectation of how many 6's you get. What you need to do is find the probability that you won't get a 6 on each roll — 5/6 — raise that to the fifth power (probability that you won't get a 6 in five rolls), then subtract it from 1. The answer is about 0,598.

So, suppose a career is 80 innings long. A slump of 20 innings could start at innings 1, innings 2, ..., innings 61. So, do we take our probability of 0,056 from above, subtract from 1, raise to the power of 61, subtract from 1? No! Because when you take blocks of innings that overlap with each other, you're looking at dependent events. Two rolls of the die won't affect each other. But the average of innings 2 to 21 will be highly dependent on the average from innings 1 to 20 — after all, 19 of the innings are in both blocks.

So we can't just raise that probability to the 61st power. So what can we do? I don't know what the best way is, but I decided to numerically work out what power you should raise (1 - 0,056) to.

So, summary of the procedure so far:
1. Find z.
2. Get associated probability p_now = LOI.NORMALE.STANDARD(z).
3. Raise (1 - p_now) to some power x, to be determined numerically.
4. Find 1 - (1 - p_now)^x. This is the probability of having a z-slump at some point during the career.

But, when I was working through this, I accidentally cut out the last step. Happily, I get much better fits this way (I did try doing following the above procedure afterwards), but the exponent that you get doesn't have the same nice interpretation as the one above.

So, actual procedure:
1. Find z.
2. Get associated probability p_now = LOI.NORMALE.STANDARD(z).
3. Raise (1 - p_now) to some power x, to be determined numerically. This is the probability of having a z-slump at some point during the career.

So, time to work out x. The first thing we'll see is that the ratio of the career length to slump length is important — for a given ratio, it doesn't matter so much what the length of the slump is. So x will be the same for a 40-innings slump in a 120-innings career as for a 20-innings slump in a 60-innings career.

If N is the length of the career and n the length of the slump, then the ratio I actually worked with (just by 'historical accident) was (N + 1 - n) / N.

Let's plot x against z for 20-innings slumps in 79-innings careers:

It's a nice exponential fit. Repeating the procedure for other block sizes (always with length ratio of 3), you get the following table (fit parameters here are the A and k in x = Ae^-kz):


slump length  A       k
20            0,0026  -4,97
26            0,0018  -5,05
30            0,0018  -5,08
40            0,0033  -4,90

The co-efficients can be sensitive to how much of the tail you let in, but basically they don't change too much. You might be thinking, "Hey! There's almost a factor of 2 difference there!" We'll ignore that and see what happens later.

Now we'll hold the slump length constant (at 26) and vary the ratio. Resulting graph of the fit parameter A:

It's another exponential decay.

Resulting graph for fit parameter k:

It's linear.

You can see that I've gone up to a ratio of 5. Much past that and I start running into lack of data problems, though I could probably have kept going with more thought and patience.

So, now we have all we need. Full procedure for finding the probability that a batsmen will have a particular z-slump of length n during his career of length N:

1. Calculate his average over the slump µ_s, and his career average excluding the slump µ_c.
2. Calculate z = (µ_s - µ_c) * sqrt(n) / (1,05 * µ_c).
3. Find p_now = LOI.NORMALE.STANDARD(z).
4. Find q = (N + 1 - n) / n.
5. Find A = 0,25 * e^-1,4*q.
6. Find k = -(0,62 * q + 2,96).
7. Find x = A * e^{k * z}.
8. Find p = (1 - p_now)^x.

So does it work? Let's have a look at observed probabilites and the p as calculated above for a 20-innings slump in a 79-innings career:


z     obs     p
-3,0  0       0,000
-2,9  0       0,000
-2,8  0       0,001
-2,7  0,008   0,003
-2,6  0,008   0,008
-2,5  0,039   0,018
-2,4  0,086   0,038
-2,3  0,117   0,071
-2,2  0,148   0,120
-2,1  0,211   0,186
-2,0  0,289   0,265
-1,9  0,336   0,354
-1,8  0,398   0,447
-1,7  0,531   0,538
-1,6  0,602   0,623
-1,5  0,695   0,699
-1,4  0,734   0,764
-1,3  0,813   0,818
-1,2  0,859   0,861
-1,1  0,969   0,896

Wrong way out in the tail, but pretty good for z greater than -2 or so.

What about a 15-innings slump in a 134-innings career? That's outside the regions we used to derive the fit parameters.


-2,1  0,390   0,353
-2,0  0,542   0,548
-1,9  0,678   0,708
-1,8  0,780   0,821
-1,7  0,814   0,895
-1,6  0,848   0,940
-1,5  0,864   0,966
-1,4  0,966   0,981
-1,3  0,983   0,990
-1,2  0,983   0,994

Pretty good.

Now, if I were a selector who never actually watched the players and only looked at the scores they made, how would I used this in practice? Well, if a particular batsmen was in a slump, I'd want to know how likely it is that that sort of slump would happen in a career as long as his. If it's more than 50%, then I'd let him keep going. If it's less than 50%, I'd drop him.

A practical example: Andrew Strauss. Before his much-publicised recent slump, he averaged 46,39. During the slump, up to the second Test against New Zealand, he averaged 28,10. The slump lasted 29 innings; his career up to then had lasted 83 innings.

So:
1. µ_s = 28,10; µ_c = 46,39.
2. z = (28,10 - 46,39) * sqrt(29) / (1,05 * 46,39) = -2,02.
3. p_now = LOI.NORMALE.STANDARD(-2,02) = 0,0217.
4. q = (83 + 1 - 29) / 29 = 1,90.
5. A = 0,25 * e^{-1,4 * 1,90} = 0,0176.
6. k = -(0,62 * 1,90 + 2,96) * (-2,02) = 8,359.
7. x = A * e^k = 75,1.
8. p = (1 - 0,0217)^75,1 = 0,192.

So slumps as bad as Strauss's should only happen to about one player in five, in a career as long as his. So based on numbers alone, I would have dropped him. Of course, he hit 177 in his next Test.

Now I think that this has been a useful exercise, but I'm not sure how much use it has in practice. You don't pick cricket teams based purely on statistics — you have to watch the players as well. If (say) a batsman is regularly getting out LBW early in his innings, you don't want to let it keep happening until p becomes 0,5 before dropping him. You want to get in early, and either drop him or work on his technique.

The largest winning margins in first-class cricket

2008-03-30T22:16:00.001+02:00

I've done the bulk of the work in putting together a comprehensive first-class database, containing all matches up until the end of the 2007 season. (There are a few holes near the end; some of the 2007/8 seasons started before the 2007 season finished.)

I will no doubt soon go back to statistical analysis, but for the next few posts I'll probably do statistical lists. Usually I find these boring, but since I don't know of such first-class lists online, I thought that they'd be of interest. If they are already on the Internet somewhere and you know where they are, please let me know.

We'll start with the largest margins of victory, first of all by innings.

Railways v Dera Ismail Khan, 1964/5: Railways 6dec/910 def. Dera Ismail Khan 32 & 27 (f/o) by an innings and 851 runs.

I have no idea how Dera Ismail Khan came to be classified as a first-class team. They clearly weren't one. In the 1980's they had another string of losses by an innings before their best performance, a loss by only a handful of runs to Hazara.

Victoria v Tasmania, 1922/3
Tas 217 & 176 lost to Vic 1059 by an innings and 666 runs. This was the match when Bill Ponsford made his 429.

There are quite a few Australian matches from the timeless era in these lists.

Victora v New South Wales, 1926/7
NSW 221 & 230 lost to Vic 1107 by an innings and 656 runs. Victoria's innings remains the highest ever in first-class cricket. Ponsford made 352.

New South Wales v South Australia, 1900/1
SA 157 & 156 lost to NSW 918 by an innings and 605 runs. Remarkably enough, the highest score in NSW's 918 was Syd Gregory's 168.

England v Australia, 1938
Eng 7dec/903 def. Aus 201 & 123 (f/o) by an innings and 579 runs. Hutton 364.

And now the top five by runs.

New South Wales v Queensland, 1929/30.
NSW 235 & 8dec/761 def. Qld 227 & 84 by 685 runs. Bradman 452 not out.

Australia v England, 1928/9.
Eng 521 & 8dec/342 def. Aus 122 & 66 by 675 runs. Bradman's Test debut.

South Australia v New South Wales, 1920/1
NSW 304 & 770 def. SA 265 & 171 by 638 runs.

Muslim Commercial Bank v Water and Power Development Authority
MCBA 575 & 0dec/282 def. WPDA 98 & 150 by 609 runs.

Sargodha v Lahore Municipal Corporation
Sar 336 & 416 def. LMC 77 & 90 by 585 runs.

On an unrelated note, there are a few games from the early parts of the 19th century where either some players, whole teams, or even the team scores are unknown. Such gaps in the first-class record looked to have ended in 1829, but they made a re-appearance in Sri Lanka in the 1990's. 1, 2 (Colombo only had eight players!), 3, 4. That last match was in 1999.

I won't be counting any of these incomplete scorecards for my stats.

Toiling away

2008-03-25T14:17:00.000+01:00

Soulberry asked for the best fast bowlers on flat pitches since 1970. Defining a flat pitch is not easy, so I've taken a short-cut to make my life easier. I've just totted up the averages for bowlers in innings where the opposition scores at least 450. It's a bit artificial, but it should do.

Here we go. This is actually all Test bowlers, qualification 15 wickets in these high-scoring innings. They're ranked by the weighted averages, where wickets are weighted by the average of the batsmen dismissed. This is particularly useful in this exercise, as we're not interested in who picks up cheap tail-end wickets. I've bolded the bowlers who satisfy Soulberry's criteria.


name              runs  wkts  wtd wkts  avg     wtd avg
Mushtaq Mohammad  629   21    21,2      29,95   29,61
WJ O'Reilly       932   24    29,6      38,83   31,53
MW Tate           1124  30    34,2      37,47   32,83
C White           792   17    23,6      46,59   33,63
SM Pollock        1376  40    40,2      34,40   34,20
NJN Hawke         629   18    18,3      34,94   34,32
MHN Walker        552   15    15,3      36,80   36,01
RGD Willis        1386  33    38,4      42,00   36,09
BA Reid           867   24    23,9      36,13   36,31
JN Gillespie      871   17    22,9      51,24   38,07
M Dillon          1413  28    37,0      50,46   38,21
DK Lillee         956   26    24,8      36,77   38,52
B Lee             1191  27    30,4      44,11   39,23
JC Laker          846   20    21,5      42,30   39,29
CEL Ambrose       667   16    16,9      41,69   39,40
CE Cuffy          571   15    14,5      38,07   39,49
DA Allen          966   20    24,4      48,30   39,66
AJ Bell           503   15    12,5      33,53   40,21
DR Hadlee         890   19    22,1      46,84   40,25
FS Trueman        644   15    16,0      42,93   40,38

I'm not sure how much I'd want to read into these numbers, since the wicket tallies are generally quite low. But Shaun Pollock looks like he deserves to be near the top.

And now the bottom end:


EAS Prasanna      1573  21    21,8      74,90   72,02
GS Sobers         1520  21    20,9      72,38   72,73
Mohammad Sami     1507  17    20,6      88,65   73,26
SJ Harmison       1825  24    24,9      76,04   73,32
SP Gupte          1120  18    15,1      62,22   74,17
DL Underwood      1593  18    21,2      88,50   75,11
FH Edwards        1136  15    15,1      75,73   75,38
PCR Tufnell       1339  15    16,3      89,27   82,35
Mushtaq Ahmed     1266  15    15,1      84,40   83,62
RJ Shastri        1719  17    19,4      101,12  88,52

What is a chuck?

2008-03-21T10:11:00.000+01:00

This is a bit different from my usual fare, but I thought it deserved its own post, rather than just being in a couple of comments threads around the place. Thanks to AGB commenter Professor Rosseforp for bringing my attention to the Ferdinands and Kersting paper.

For a long time, the definition of an illegal delivery action was that the ball couldn't be thrown or jerked. (Stuart gave us details here and here.)

A recent paper suggests that we might be able to return to a sort of 'jerkiness' definition, only this time backed up by some science. I'll go through a bit of background first. The key goal that we want is for science to come up with a criterion whereby bowlers who look like chuckers are chuckers. An exception to this is Murali, who can bowl in a brace (so he can't possibly chuck) and still look bad. But the problem of a chucking definition is much bigger than just Murali, and it should be possible to get science to agree with the naked eye, at least most of the time, for bowlers with 'normal' arms.

The ICC's current tolerance of 15 degree elbow straightening (or 'extension'? not really sure of the difference) was based on a study done by Porter, Elliott, and Hurrion during the 2004 Champions Trophy. Unfortunately, the full details of the study haven't been released to the general public. The reason given by the ICC is confidentiality issues: "We do not think it would be correct to release the figures publicly without the prior consent of the individual bowlers and the researchers themselves. This consent has not been obtained."

I don't understand why the researchers, who have published many biomechanics studies publicly, should care. I can see why the ICC might not want to give names and elbow straightenings, because there'd be a torrent of allegations of chucking all over the place.

Nevertheless, the ICC's secrecy over the matter means that we don't really know much about elbow straightenings, except that Sarwan's was zero, and Pollock and McGrath up to 12 degrees or so.

But luckily there was a paper published (in Sports Biomechanics, 'Fast Bowling Arm Actions and the Illegal Delivery Law in Men's High Performance Cricket Matches') by Portus, Rosemond, and Rath in 2006 which does give us some numbers. They also don't name names, but they did study thirty-four deliveries by twenty-one fast bowlers, from Test, ODI, and tour matches. None of the bowlers had had any questions raised over their actions. These sorts of analyses take a long time, which is why so few balls were studied.

The errors in the measurements are +/- 1 degree.

Of the thirty-four balls bowled, three were by two bowlers with hyperextended elbows, so we'll ignore them. Of the remaining thirty-one, six had elbow straightenings larger than 15 degrees. These were spread across four bowlers out of nineteen. If you go by the 15-degree rule, then those are chucks. Many of the bowlers only had one ball recorded, so it's not clear if there were any more chuckers-under-the-15-degree-rule in the sample. Looking at the numbers in the table, you'd guess that at least a couple of them go past 15 degrees sometimes.

So, to summarise: Under the 15-degree rule, one in four or five fast bowlers sometimes chuck, often more than once per over.

These are, remember, bowlers whose actions haven't been questioned.

So here we have the first problem of the 15-degree rule: many bowlers who should be deemed legitimate are breaking the rule.

Now we get onto the other problem of the 15-degree rule: you can chuck without straightening your elbow 15 degrees.

This takes us to the paper by Ferdinands and Kersting, also published in Sports Biomechanics, in 2007 ('An evaluation of biomechanical measures of bowling action legality in cricket'). The technical details are a bit over my head, but this is what they say:

If bowlers adopted a similar action to throwing, where the elbow remains flexed at release, then it may be possible to utilize effective humeral internal rotation in bowling while satisfying the current 15° elbow extension angle limit. For instance, if in bowling the elbow can be flexed 26° at release, which has been achieved in professional Cuban baseball pitchers (Escamilla et al., 2001), then a bowler can theoretically have an elbow flexion angle of 41° and extend 15° before ball release. Any amount of elbow extension is allowed after release. Such a bowling technique would use a bowling arm with a lower absolute elbow angle about the flexion-extension axis (more flexed), which would extend rapidly before and after release. This technique would share some of the characteristics of a throwing-type action, but still remain legal according to the current elbow extension angle limit.

But it's not just speculation about new 'bowling' techniques that would be blatant chucks without breaking 15 degrees. Ferdinands and Kersting studied bowlers (in the lab, not in match conditions) from club level in New Zealand to international level, some of whom had had their actions reported. There were 'fast' bowlers and spinners. One of the limitations of the study is that their 'fast' bowlers weren't fast by international standards, and this is the obvious place where the next bit of research should go.

There were six bowlers studied with suspect actions. Five of these had mean elbow extensions less than 15 degrees, and indeed at least 75% of the balls bowled by those with suspect actions passed the 15 degree test (it's not clear precisely how many from the graphs).

Let's have a look at the box-and-whisker plot for the various groups considered:

The boxes represent the middle 50% of deliveries, in terms of how much elbow extension there is. The horizontal lines in them show the medians.

The suspect box starts higher and finishes higher than the others (so there is some correlation between elbow straightening and apparent chucking), but there's a big overlap with spinners and fast bowlers. So just going by the elbow straightening isn't very good at distinguishing those with bad-looking actions from those with clean actions.

Now (at last!) here comes the key point. In addition to just measuring the total straightening, they also measured the rate of the elbow extension, the 'elbow extension angular velocity'. Now the box-and-whisker plot clearly shows up the chuckers:

The bottom of the suspect box is at around 200 degrees per second, and the top of any of the other boxes is around 100 degrees per second.

The implication is clear: measuring the elbow extension angular velocity gives much, much better agreement with the naked eye than just going by the total straughtening. It's not perfect, and there were some deliveries in the non-suspect groups that went above their suggested threshold of 200 degrees per second, but it's a lot better than what we currently do. And importantly, it gives us an objective definition that generally agrees with what our eyes tell us is a clean action and what is not.

As I said earlier, we need to see this research done on a large group of international-class bowlers before applying it to international cricket. But the results are very promising. At the very least, the 200-degree-per-second cutoff is better than the 15-degree cutoff, which clearly doesn't work.

I'll finish by noting that this gets back to the old 'jerk' definition. A gradual straightening caused by general stress on the elbow during the bowling action isn't jerky, but a rapid bit of straightening just before release is jerky.

Adjusting averages for not-outs, take three

2008-03-20T10:32:00.000+01:00

This is my third attempt to implement a good method to deal with no-outs when calculating batting averages. The first two happened before this blog started (I've backdated one here), but both had flaws. The flaw in the one I just linked to is a subtle one, and I only realised it after reading this post by Charles Davis.

He was interested in calculating the average number of runs scored once you reach a century. This is basically the same question I have in projecting not-outs forward. If a batsman who averages 40 finishes 100 not out, how many extra runs would he have scored?

The way I did this originally was like this:
1. Take all innings greater than or equal to 100.
2. Take their average.
3. Subtract 100.

This seems reasonable, but Davis points out an anomaly. Suppose a batsman has scores of 100 not out, 100 not out, and 100. Then his average calculated by this method is 300 - 100 = 200. But he's never scored a run past 100. So what the procedure should be is:
1. Take all innings greater than or equal to 100.
2. Subtract 100 from each.
3. Take their average.

Now this example is extreme, but the problem is a significant one when you do this over all batsmen at all scores, because there are a lot of not-outs at each score. If you're interested, compare the graph below to the bad one in my earlier post.

For this graph, I took each batsman with a Test average of at least 40, and computed their average increases at each run (up to their highest score), and then took the average over all players at each score. If a batsman's highest score was a not-out, I added the batsman's average to it and turned it into an 'out'.

The average increase from zero (i.e., the overall average) is 47,5. The average increase from 1 is 49,8. So in a sense, your first run is worth three. This, along with the steady increase in the curve that you see until about 85, is just the effect of getting your eye in, and batting becoming easier as you continue to score runs.

The dip either side of 100 is what you might call a psychological feature — it's there because batsmen often drop their concentration once reaching a century and get out soon afterwards. The curve rises again until about 125, and then there's a pretty steady downward trend, with two more psychological dips around 200 and 250. There also looks to be one around 300, but there aren't many data points there.

The curve has a lot of noise in it, and before using it to project not-outs forward, it's worth smoothing out the non-psychological bits. I didn't spend too much time doing this, and there are a couple of ugly splices, and in one place scoring a run actually sends you backwards by one run. That shouldn't be too serious in the grand scheme of things.

Now, you wouldn't want to just use this curve to project not-outs, because obviously some batsmen are better than others at making large scores. Steve Waugh v Mark Waugh is an obvious example. On the other hand, if you're projecting a not-out, and there's only one innings higher to work with, then that higher innings is probably not representative, and it's useful to use the overall average increase given in the graph.

Note that when using the graph on an individual batsman, I move it up or down so that the average increase from zero matches his average.

To compromise between just going by the individual and just going by the graph, I used the following formula, where n is the number of innings larger than the not-out to be projected:
proj = 1/sqrt(n+1) * proj_by_individual + (1 - 1/sqrt(n+1)) * proj_by_overall.

The co-efficients here are arbitrary, but I think they look OK. If there's one innings to work with, it gets about a 30% weight, and the graph gets 70% weight. If there are three innings, it's 50-50.

Now for some results. In the following table I've listed the top 20 batsmen as measured by this adjusted average. There's no adjustment for era or quality of bowling. The 'diff' is the difference between regular average and adjusted average. It's positive is the regular average is higher (i.e., inflated by not-outs), and negative if the regular average is lower (deflated by not-outs). The rank is the rank by regular average, which lets you see how the batsmen have shuffled around. Qualification 20 innings.


name          inns  no  runs  avg    adj    diff   rank
DG Bradman    80    10  6996  99,94  99,38  +0,57  1
MEK Hussey    36    8   2188  78,14  74,25  +3,90  2
RG Pollock    41    4   2256  60,97  61,60  -0,63  3
GA Headley    40    4   2190  60,83  60,95  -0,12  4
WR Hammond    140   16  7249  58,46  60,04  -1,58  10
H Sutcliffe   84    9   4555  60,73  59,62  +1,11  5
GS Sobers     160   21  8032  57,78  59,02  -1,24  11
E Paynter     31    5   1540  59,23  58,80  +0,44  6
RT Ponting    191   26  9676  58,64  58,76  -0,11  8
ED Weekes     81    5   4455  58,62  58,49  +0,13  9
KF Barrington 131   15  6806  58,67  57,83  +0,84  7
KC Sangakkara 114   10  5914  56,87  57,18  -0,32  14
SR Tendulkar  236   26  11851 56,43  57,06  -0,63  18
L Hutton      138   15  6971  56,68  57,06  -0,38  16
JH Kallis     195   32  9394  57,63  56,78  +0,85  12
CL Walcott    74    7   3798  56,69  56,53  +0,16  15
JB Hobbs      102   7   5410  56,95  56,51  +0,43  13
RS Dravid     202   25  10015 56,58  56,45  +0,13  17
Mohd Yousuf   138   12  7009  55,63  55,59  +0,04  19
VG Kambli     21    1   1084  54,20  54,55  -0,35  22

Overall there's not much change. Hammand and Sobers move up several places, but otherwise we're dealing with fairly small adjustments to the average.

Mike Hussey's adjustment is the largest of any batsman with an average over 40. That adjustment will likely come down as his career continues and his stats become more like those of other players.

Considering only batsmen who average over 40 with at least 50 innings, the average difference is -0,16. So on average, not-outs deflate averages by about a sixth of a run. There's a very slight (and noisy) trend saying that batsmen with a high proportion of not-outs have their averages deflated more, which also agrees with the idea that not-outs tend to deflate averages.

As I said, there's a lot of noise. In that latter dataset, there are 52 batsmen whose averages seem to be inflated by not-outs and 75 whose averages are deflated. But in almost all cases the differences are pretty small.

The moral of the story is not to worry about not-outs when looking at a batsman's stats.

One last comment. A paper by Clive Loader in 1996 considered Allan Border's career and the effects of not-outs on his average. It was only one example in the paper, which looked at something called censoring in various contexts. Using some kind of binomial model, he estimated that not-outs had deflated Border's average by between 1 or 2 runs. My numbers say that his average was inflated by about two fifths of a run. An unfortunate disagreement, and I probably won't get to the bottom of it without learning a good deal more statistics, because that paper uses methods beyond my current knowledge.

Teams with good spinners bat well against spin and pace.

2008-03-16T21:28:00.000+01:00

It's useful to check the common wisdom, or things that seem obvious. Often they're true, but sometimes they're not, and even if true, sometimes an analysis reveals surprising related results. This post falls into the latter category.

If a team has good spinners, then their batsmen should do better when playing against spinners. This might be because the country generally produces lots of spinners and so batsmen grow up playing lots of them, or because the batsmen get to practice in the nets against good spinners, or some combination of the two. The same principle should apply to pace bowling.

So let's check. I went to Statsguru and got each of the top eight teams's averages against pacemen, and each team's overall pacemen's average. To start, I considered only the 2000's. The results are plotted below:

That's quite a strong trend, but the direction agrees with the common sense — teams whose pace bowlers have high averages don't score as many runs against pace bowlers, and vice versa. As I said, the trend is very strong, and much of this is due to the luck of world cricket over the past decade. Australia has had excellent batsmen and excellent pacemen, the West Indies the opposite. I repeated the exercise for earlier decades (90's, 80's, and 70's; the results for the 1960's go a bit haywire because three out of the seven teams were weak), and the direction of the trend is the same, but the magnitude varied from a slope of -0.17 to -0.8.

Still, a verification of what we thought we knew, and some ideas of how strong the effect is. Now let's do the same for spinners.

The same basic trend, though with a gentler slope. The results for the earlier decades are similar, with the slopes ranging from -0.3 (the one shown above) to -0.75.

Now let's see if we can get null results. If a team has good pacemen, that should tell us nothing about how well they play spin, right?

Right. No trend at all.

Similarly, if a team has good spinners, that should tell us nothing about how they play pace.

Erm, wrong. Teams with good spinners tend to play pace well. The same trend exists for the earlier decades, with slopes ranging from -0.18 to -0.33 (above). I don't know what the p-value is, but it looks like it's not just luck.

I don't know how to explain this. I have two ideas:
1) learning how to play high-quality spin makes you a better batsman in general;
2) spinners with good pacemen in the team do better than those without, and so the trend is really just "average against pace" v "pace average" in disguise.

I'm inclined to think that the second of these two ideas is a strong factor, but there's no correlation between spin average and pace average in either of the last two decades. For the 2000's:

For the 1980's and 1970's, there is a positive correlation between the two.

From my post on bowler support, a useful rule of thumb in dealing with aggregated data like this is that for every run lower a bowler's teammates' bowling averages are, the bowler's average should go down by a quarter of a run. This works pretty well for the 1980's and 1970's graphs, but doesn't help explain what's going on in the last two decades.

It's a bit of a puzzle, and I don't know what the answer is. Teams that have good spinners tend to play pace well.

(A post-script: I was going to do another post on captaincy today, based on some suggestions from The Atheist, but my regression failed rather miserably. I'll have another try later.)

Evaluating captaincy

2008-03-11T21:35:00.000+01:00

Captaincy is one area of cricket that does not receive much statistical scrutiny. It is not hard to figure out why — about the only thing you could easily compute would be how many wins the team had, and this figure is strongly dependent on the quality of the team. To remedy this, I've come up with a way of estimating how many wins, draws, and losses that a captain would be expected to have, given the strength of his side, the strength of the opponents, and whether they're at home or away. Once we have these, we can compare to the captain's actual record, and see who does better or worse.

First, I'll explain how to get the expected results. This method is a bit rough and can certainly be improved in places, but overall I think it does a good job. For each team, I calculate the average batting average (averages weighted by the averages of the bowlers faced), and average bowling average (with wickets wickets weighted by the batting average of the batsman dismissed). The latter is a little bit tricky — some teams use more bowlers than others. So, for each innings, I weighted the bowling averages by the number of balls bowled by each bowler. Then, if two innings were bowled, I took the average of the two.

Then you subtract the average bowling average from the average batting average, and you get a rating for the team. Do the same for the other side, and you get a measure of the difference in strength between the two sides.

Next you go through all Tests, calculate the difference in strength (to make things consistent, I did home team rating minus away team rating), and find how many wins, draws, and losses there are at various differences in strengths. I did this by binning all Tests into 20 bins. Plotted on the graph below are the expected fraction of wins and draws.

The fractions of wins does basically what we'd expect — it starts out flat and very low for teams that are outclassed, before rising steadily before plateauing. There are always going to be some draws (because of rain), so the fraction of wins won't hit zero or one. Even the weakest of home teams can achieve a draw rate of about 30% (well, maybe not Bangladesh), whereas very weak teams away can only draw about 20% of Tests.

The trend in draws is a bit different. It seems to go gently upwards until the teams are evenly matched, and then more sharply downwards as the home team becomes stronger.

I approximated these curves with piecewise linear functions. For the draws, it's flat for x less than -27, then upwards so that it hits the y-axis at y = 0,424, then downwards until x = 17, and then flat, at a value of 0,185.

For the wins, it's flat at 0,031 below x = -13,7, then upwards until x = 17,2, and then flat at a value of 0,785.

So now, for each Test, I calculate the difference in strength. Then I plug that number into the fitted graphs to get a fraction of a win, draw, and loss. For example, suppose that the teams are evenly matched. Then the home side gets 0,366 wins; 0,424 draws; 0,21 losses. The wins and losses for the away side are flipped: 0,21 wins and 0,366 losses.

You do this for each Test that a captain plays, and add up the expected wins, draws, and losses. Now we can compare to the actual record.

There's a question here about how to deal with draws. I decided to ignore them, for a couple of reasons. The first is that teams which score runs faster should have less draws, but I didn't take strike rate into account when doing the regressions above (I don't have strike rate data for all Test batsmen). Also, all Tests in Australia (as well as some elsewhere) were played to a finish between 1882/3 and World War II — no draws in a major cricketing country for over sixty years!

So instead I calculated the fraction of wins out of matches that ended in a result, that is: wins / (wins + losses). Do this for the actual value, divide by the expected value, and you get a ratio saying how much better or worse the captain's record is compared to what would be expected.

Whether or not it is reasonable to ascribe all the difference to the captain is certainly debatable, but it seems the best thing to do for now. Here are the best captains, as measured by this statistic. Qualification 20 Tests as captain.


                    ----expected----  --actual--  exp   act
name            mat w     d     l     w   d   l   w%    w%    ratio
Abdul Kardar    23  5,1   7,1   10,8  6   11  6   0,32  0,50  1,56
GP Howarth      30  7,9   11,1  11,1  11  12  7   0,42  0,61  1,47
J Darling       21  5,8   7,8   7,4   7   10  4   0,44  0,64  1,44
JM Brearley     31  11,3  12,0  7,7   18  9   4   0,60  0,82  1,37
Inzamam-ul-Haq  33  6,8   11,9  14,3  10  10  13  0,32  0,43  1,35
MP Vaughan      41  15,0  14,5  11,5  21  11  9   0,57  0,70  1,24
RB Richardson   24  8,4   8,2   7,4   11  7   6   0,53  0,65  1,21
GA Gooch        34  8,1   12,6  13,3  10  12  12  0,38  0,45  1,20
CA Walsh        22  5,7   7,6   8,7   6   9   7   0,40  0,46  1,16
DG Bradman      24  11,7  7,7   4,6   15  6   3   0,72  0,83  1,16
SP Fleming      80  23,6  27,1  29,3  28  25  27  0,45  0,51  1,14
RB Simpson      39  10,9  14,2  13,9  12  15  12  0,44  0,50  1,14
IVA Richards    50  21,7  18,2  10,1  27  15  8   0,68  0,77  1,13
CH Lloyd        74  31,3  27,0  15,7  36  26  12  0,67  0,75  1,13
N Hussain       45  14,0  15,5  15,5  17  13  15  0,48  0,53  1,12

Abdul Kardar, Pakistan's first Test captain, comes out on top. You can see that he didn't actually win many Tests, but his team managed to draw a lot that they "should" have lost. It is reassuring to see Mike Brearley so high up. Inzamam and Michael Vaughan are fifth and sixth.

Most of those high in the table did not have extended careers as captain, so perhaps some of them were just lucky and are higher than they should be. Of those with at least 50 Tests, Stephen Fleming is the best, just shading Viv Richards and Clive Lloyd.

At the other end, we have those who took their teams and un-inspired them to ineptitude:


                    ----expected----  --actual--  exp   act
name            mat w     d     l     w   d   l   w%    w%    ratio
M Azharuddin    47  18,5  17,2  11,3  14  19  14  0,62  0,50  0,80
HH Streak       21  5,0   6,1   9,9   4   6   11  0,33  0,27  0,80
MW Gatting      23  5,1   8,9   9,1   2   16  5   0,36  0,29  0,79
CL Hooper       22  5,2   7,8   8,9   4   7   11  0,37  0,27  0,72
BS Bedi         22  7,0   7,8   7,2   6   5   11  0,49  0,35  0,72
DI Gower        32  7,0   12,0  13,0  5   9   18  0,35  0,22  0,62
JR Reid         34  5,3   11,1  17,6  3   13  18  0,23  0,14  0,62
AC MacLaren     22  6,6   7,9   7,5   4   7   11  0,47  0,27  0,57
KJ Hughes       28  7,3   10,3  10,4  4   11  13  0,41  0,24  0,57
A Flower        20  3,5   6,4   10,1  1   9   10  0,26  0,09  0,36

Kim Hughes was pretty lucky to get to captain Australia in 28 Test matches.

A couple of others to finish (feel free to request others).

Mark Taylor 0,93, but probably disproportionately many of his losses were in dead rubbers. He also only had 11 draws (expected 17,5), in the era before Waugh made draws almost extinct for Australia.

Sunil Gavaskar 1,10 is the best Indian captain. He achieved this record by turning five expected wins into draws, and seven expected losses into draws. Not exciting stuff, but it gave him an overall positive record (9 wins, 30 draws, 8 losses).

Imran Khan at 1,08 didn't do that much better than he should have, and was also pretty drawish (14-26-8, expected 17,6-18,0-12,3). He did a bit better than Javed, who scores 0,99.

Wicket-keepers and byes

2008-03-09T12:39:00.000+01:00

In my post on first-class wicket-keepers in England, I pointed out a curious trend — there were quite a lot of keepers from the 1980's near the top of the table of byes as a percentage of team runs. I guessed that it was because of the higher run rates during this time, compared with previous decades (and because of the trend afterwards to choose keepers based on their batting). Using byes per 600 balls gives more reasonable results, but it should still be biased by the prevailing run rate — if the run rate is higher, the batsmen are probably hitting more balls, so the keeper's bye rate should be lower.

Ideally, we'd have a stat for byes per balls that pass the batsman. But since we'd need ball-by-ball data to find this, we have to make do without. Yesterday I had an idea of how to adjust for run rates. I thought it was a good idea, but it didn't work. First, I'll explain what I wanted to do and how it should have worked.

You can't use bye rates and run rates and look for a correlation over all matches — the average standard of wicket-keeping can (and does) change with era. But the standard of an individual keeper should be fairly consistent over his career, and every keeper will keep in innings where the opposition scores heavily, and in innings where the scoring is slow. Since some keepers in England play many hundreds of matches, there should be enough data at an individual level to work out what the trend is.

So for each keeper, I took all innings kept and ordered them by the run rate (when calculating the run rate, I ignored byes). To avoid the problem of what to do with very short innings (and to make the graphs nicer), I aggregated the data into ten bins, and found the overall average number of byes per 600 balls for each bin. The result for Alan Knott is below.

It's not perfect, but the overall trend is pretty clear — at higher run rates, Knotty gave away less byes.

That was the idea. It turns out that not all keepers have this trend. Here's Jack Russell:

It's a nice trend, in the wrong direction.

Taking keepers who kept in at least 100 innings, there are actually more with positive slopes than negative. Some of this effect might be noise, so I did the same calculation but calculated the regression lines using only the middle eight bins (perhaps short innings come up disproportionately often in the first and last bins, and so the data's less reliable).

Then, I considered only keepers whose career began after World War II, and who kept in at least 300 innings. The result? Twenty-nine keepers with positive slope, twenty-eight with negative, one flat. The average slope for this set of keepers was 0,01.

There does appear to be a slight tendency towards negative slopes for those with very long careers (i.e., more than 725 innings), but that might just be noise, and it's still not a hard-and-fast rule — Bob Taylor kept in 976 innings, and has a slope of 0,235.

So that's a bit of a surprising dead end of a conclusion. Wicket-keepers don't generally give away less byes when the batsmen score runs faster.

The meaningfulness of Tests and ODI's

2008-03-08T08:41:00.001+01:00

Today I want to statistically show what is obvious and logical — that there's a lot more luck in ODI's than in Tests. But something not quite so obvious comes up later.

I got the idea for this sort of analysis from this post by Tangotiger, who usually studies baseball statistics. (In that post, he gives a method for finding the standard deviation of the talent distribution in a league. I think that he actually estimates a lower bound for this quantity.)

I took all Tests and all ODI's between the top eight nations since 2003. (I did this in Statsguru. I didn't realise it before, but in the advanced filter, if you click on the 'view' near the right edge of the table, you get checkboxes rather than a dropdown menu. Do this for Team and Opposition, check the major nations, and in eighteen clicks you've excluded Bangladesh, Zimbabwe, and all the weird teams.) I then threw away any draws, ties, or no-results, and calculated the fraction of wins for each team. (So, e.g., a team with five wins, five losses, and five draws will have a fraction of wins of 0,500 — not 0,333.)

Tests:


team         w   l   w%
Australia    35  7   0,833
England      25  16  0,610
South Africa 21  18  0,538
India        14  11  0,560
Pakistan     13  16  0,448
Sri Lanka    10  14  0,417
New Zealand  5   14  0,263
West Indies  4   31  0,114

ODI's:


Australia    93  34  0,732
India        61  69  0,469
South Africa 53  41  0,564
New Zealand  50  52  0,490
Pakistan     50  53  0,485
Sri Lanka    50  56  0,472
England      38  61  0,384
West Indies  30  59  0,337

The gap in winning percentage between top and bottom is much larger in Tests than in ODI's, which is what we would expect — Tests better differentiate between the quality of the teams. The standard deviation of the win percentage for Tests is 0,219; that for ODI's is 0,119.

So far, nothing you wouldn't have guessed. But it's interesting to compare this to what you'd expect from chance. That is, if every match that ends in a result were decided by a coin toss, what standard deviation would you expect? The SD for the number of wins out of n games would be sqrt(n/4), from the binomial distribution. The variance, being the square of the SD, would be n/4. The fraction of wins is the number of wins divided by n. Now, Var(aX) = a²Var(X), so the variance in the fraction of wins would be 1/(4n). So the SD would be sqrt(1/(4n)).

If you take n as the average number (for each team) of result matches in each set (31,75 for Tests; 106,25 for ODI's), you get the SD's expected from chance as 0,088 for Tests and 0,049 for ODI's.

What you'd like in a distribution of winning percentages is that it's clearly wider than what you'd expect from chance (so that you can conclude that the differences between teams are due to differences in the quality of their play, not just the luck of the day). Since the SD you'd expect from chance for ODI's is smaller than that for Tests (because more ODI's are played), you actually don't need the real SD for ODI's to be as large as that for Tests, in order to sort the teams out.

A simple way to quantify this (there may be a better way) is to take the observed SD divided by the SD expected from chance. For Tests, this is 0,219/0,088 = 2,47. For ODI's, it's 0,119/0,049 = 2,46.

Almost exactly the same! That's probably a little bit lucky — those numbers would probably be a bit further apart if I'd picked a different period — but it shows that, in terms of sorting out which the ranking of the teams, the balance between Tests and ODI's is about right. There were, over this period, about 2,4 ODI's played between these teams for every Test.

That doesn't mean I like all these ODI's! Each one of them is, in itself, much more meaningless than a Test match (at least outside World Cups, and except for draws, which are happily a minority of Tests these days). And even though they take up fewer playing days in total than Tests, each match is independent of the others. A bad day for a team doesn't matter — the teams start from scratch again in two days' time. In a Test match, of course, a bad day directly affects the remainder of the match.

Left-handers

2008-03-06T11:00:00.000+01:00

There are a handful of papers in academic journals that analyse cricket statistics. The methods used in these papers tend to be far more sophisticated than what I use (and usually I don't even understand them), but often the results are interesting and/or useful. Unfortunately, they tend to languish in academic journals, unknown by the average cricket fan. To try to remedy this, every now and then I'll have a look at one of these papers and discuss the methods and results.

The first paper I'll look at is by Robert Brooks et al. It's called Sinister strategies succeed at the cricket World Cup, and was published in the Proceedings of the Royal Society Series B (Biology Letters, Supplement) 271: S64. You can get a copy from the website of one of the authors here.

The authors studied the 2003 World Cup, in an attempt to see work out why left-handers are more prevalent in top-level cricket than in the general population. Cricket's not unique in this regard — most sports involving one-on-one contests have a higher proportion of left-handers. Individual sports (such as athletics or golf) do not.

My own feeling was that a large part of left-handed batsmen's success is because the stock ball of right-arm pacemen usually swings into them, and inswingers are easier to play than outswingers. But this paper by Brooks gives strong evidence to suggest that it's more a case of bowlers not being used to bowling to left-handers.

The paper draws out two effects. The first is that weaker countries have a lower proportion of left-handers. The suggested reason is that when the domestic competition is weak, natural talent is the biggest factor in getting selected for the national side — the variation in talent is large enough so that the left-handers' natural advantage is not important. But at a stronger level of competition, where there is less variation in players' ability, the extra advantage that left-handers have becomes more important, leading to disproportionately many of them in national teams. Their figure below shows the trend:

On the vertical axis is the team's net run rate (i.e., how good they are), and on the horizontal axis is the percentage of innings by left-handers. They've fitted a quadratic to the data, which gives a pretty good fit. The interesting feature is that the quadratic peaks at close to 50% left-handers, suggesting that the ideal batting line-up should have an equal number of left- and right-handers.

Now, the obvious explanation for this is that teams with equal numbers of right- and left-hand batsmen enjoy lots of opposite-handed partnerships, and it is an accepted piece of wisdom that bowlers struggle when having to change their line when the batsmen rotate the strike.

But this does not look to be a significant factor. The authors looked at each batsman when they were in partnership with someone of the same hand or with someone of the opposite hand, and found no significant difference. There's some mixed evidence on the usefulness of left-right partnerships. In The Best of the Best, Charles Davis says that left-right opening partnerships (in Tests) average about 15% more runs than would be expected based on the individual averages, whereas same-handed partnerships are about average. My own figures, based on a regression on opening batsmen's averages, puts left-right combinations at 6% better than they should be, and same-handed partnerships 4% worse. But there is plenty of individual variation. It does certainly look like there's a real effect, but you need a large dataset to see it — much larger than just one World Cup — and this is why the authors of the paper didn't find anything significant.

Nevertheless, if there is an advantage to having 50% of the team left-handed, and left-right partnerships are not significant or small, then there has to be something else. The authors show us the following graph.

On the horizontal axis is the percentage of "bowlers' wickets" for each team, and on the vertical axis is the difference between balls faced by left-handers and right-handers by the batting team. Bowlers' wickets are defined as catches from edges, LBW's, and bowleds. They had to do a lot of trawling through Cricinfo's commentary archives to find catches that were at slip!

The trend here is pretty obvious. When there are more bowlers' wickets (suggesting stronger bowling attacks... or really bad fieldsmen), left-handers don't enjoy as much of an advantage over right-handers. The explanation offered by the authors is that weaker bowlers tend to come from weaker competitions, where there are not so many left-handed batsmen. So these bowlers aren't as used to bowling to left-handers so much.

This gives us a reason for the optimum of 50% left-handers. Any more than 50%, and the bowlers would be so used to lefties than right-handers would start to have an advantage.

So it looks like most of the left-handers advantage comes down to bowlers not being used to bowling at them. But the overall story is certainly more complicated. In The Best of the Best, Davis shows that players who bowl right and bat left do better, on average, than players who bowl left and bat left. Is this because the top hand is more important, so that's where you want your dominant hand? Who knows? Players who bowl left and bat right do worse than players who bowl right and bat right. I don't understand.

Partly explaining all these double-centuries

2008-03-02T21:00:00.003+01:00

It's pretty obvious that there are a lot more double-centuries being scored these days than in previous eras. This isn't just because there are more Tests being played now than ever before: Charles Davis calculated the percentage of centuries converted into double-centuries in early 2005 (search for the "Double the fun for Ponting" post). Whereas this was between 7 and 8 percent from the 1960's through to the 1990's, it jumped to 11.4% from 2000 to 2005. That is, more than one in ten centuries were turned into doubles.

Davis said that the overall batting average had rised a little bit, but not enough to account for this rise. He suggested that while bowlers overall were a little bit weaker now, they're particularly weak when a batsman gets well set — "once bowling attacks are beaten down, there is less capacity for comeback."

The key question here is, if the overall batting average rises by some amount, how much should the proportion of centuries-that-are-doubles rise?

The answer to that question depends on the distribution of individual scores. Let's assume that the distribution is exponential. It's not, but if we compare decades, hopefully the errors will roughly cancel out, giving us a meaningful comparison.

If the overall average is µ, then the fraction of scores greater than or equal to 200 is exp(-200/µ). The fraction of scores greater than or equal to 100 is exp(-100/µ). So the fraction of centuries turned into doubles is exp(-200/µ)/exp(-100/µ) = exp(-100/µ). Note that this is just the fraction of centuries — this is the memoryless property of the exponential distribution.

Then we take µ₁ for the 1990's, and µ₂ for the 2000's, and use these to find the expected fraction of centuries that are doubles.

I considered only batsmen in positions 1 to 7, since that's where most centuries come from. The overall average for these batsmen in the 1990's was 35.35, and for the 2000's it was 38.04.

Plug these numbers in, and you expect 5.9% for the 1990's, and 7.2% for the 2000's. Now, we expect that these values will be wrong (and they are) because the distribution isn't really exponential. But dividing one by the other should mostly cancel this out, and so we expect that the proportion of centuries turned into double should rise by a factor of 1.22. A less than 10% rise in batting average leads to a greater than 20% rise in centuries turned into doubles.

The proportion for the 1990's was 7.6% (I lumped not-outs and outs together; Davis says 7.9%). For the 2000's it's 10.5%. The proportion increased by a factor of 1.38. Just based on the averages, you'd have expected it to rise to 9.3%. The difference here is about 9 centuries over the course of the decade to date, or about one extra double-century per calendar year.

That sounds like something you can blame on the minnows, but that's not the case. If you re-do the analysis excluding Bangladesh and Zimbabwe, then the figures become expected 1.24 and observed 1.39.

So there does appear to be a real effect, but it's not that great. The general rise in batting averages is most important factor, but there is this extra double-century a year that "shouldn't" happen.

There's no point stopping here. What's special about 200? I did a similar above analysis for Tests from the 1950's onwards. Then, grouping by decade, I found the fraction of scores greater than or equal to 1, greater than or equal to 2, etc. up to 240. For a reference case, I also did this for all Tests during this period.

Then for each decade, I calculated the observed increase or decrease of the fraction of each score from the reference case (as a factor, e.g., 6% to 9%, an increase by a factor of 1.5), and the expected increase or decrease based on the decade average against the reference average (e.g., 5% to 7%, an increase by a factor of 1.4).

Take the observed increase minus the expected increase (1.5 - 1.4 = 0.1), and you get a measure of how common scores greater than or equal to the given score are, against what they "should" be. A positive value tells you that scores greater than or equal to the given score are more common than you would expect, based on the decade average.

It's graph time.

I hope you can make out the different colours. The most striking feature of the graph is the curve for the 1950's. Despite the overall average being much lower (only 32.43), there were a comparable fraction of large scores to what we see today, when the overall average is 38!

The curve for the 1960's is kind of a damped mirror image of the previous decade — less centuries than you would expect. The curve does start to come back up after 200, but I'd be sceptical about reading too much into the curves much past 200, since those scores are pretty rare and statistical noise becomes more prevalent.

The 1970's is similar to the 1960's, though it was closer to the expected.

The 1980's are almost dead on expected all the way up to 200.

The 1990's are a bit below expected for large scores.

The 2000's are a bit above expected, particularly above about 175. There's an amusing dip just past 200: the number of scores greater than or equal to 204 is almost exactly as expected.

So there you go. We have more double-centuries than we should (not by much), but a bigger phenomenon is the number of scores greater than 175. I blame Michael Vaughan.

Scheduling cricket around the IPL

2008-02-25T08:30:00.000+01:00

I had planned a couple of more blog entries before my holiday to Italy, but I've got distracted. So before I disappear for a week, here are some thoughts on the IPL and cricket scheduling. This is a bit of a change from my usual fare, but since everyone's talking about the IPL and its consequences, I thought it was worth sharing.

I am a big supporter of the IPL, and I hope it's a big success. But there is one big danger that I see, and that's if it expands too much. This year, the IPL teams will play in a double round-robin — fourteen matches each — before semi-finals and final. There's huge scope to make the tournament bigger. Teams in Major League Baseball (where games last about as long as T20's) play 162 games a season. Now I'm not saying that anyone wants a cricket tournament in which each team plays 162 times, but the point is clear: the IPL could get much bigger.

I've done some scribbling, and I think that Test cricket can survive in much its present form as long as the IPL season is not longer than four months. IPL Teams could play, say, five games a fortnight, and so the home-and-away season could be around 35 matches per team, which would be followed by a finals series.

To make it concrete, let's assume that the IPL could fill up February to May.

England's home Tests will have to be between June and September, much as they are now. Most other Tests would be between October and January, though Tests in June or July are possible in, e.g., Australia. (The West Indies never plays Tests before January — does anyone know why? Their domestic season starts in November.)

I see a five-Test tour looking like this:

tour match: days 1-4
tour match: days 7-10
Test 1: days 14-18
Test 2: days 21-25
tour match: days 31-34
Test 3: days 38-42
tour match: days 46-50
Test 4: days 56-60
Test 5: days 63-67

Three-Test tours would be truncated after the third Test. It should be clear what my thoughts on first-class tour matches are! There are no one-day matches, though you could probably squeeze one or two in either at the beginning or at the end. Or you could shorten the tour matches, or get rid of one, to make space.

I would like to see Australia play five-Test series against England, South Africa, and India. Australia's home schedule would look something like:

Eng in summer, WI in winter
NZ, SL in summer, Bd in winter
Ind in summer, Pak in winter
SA in summer
Eng in summer

The Ashes stay on a four-year cycle, and everyone else is on five-year cycles. This could easily be relaxed to six-year cycles. The non-England teams could be shuffled around from winter to summer, depending on other schedules or commercial considerations.

Other countries may also want five-Test series (they'd be good with India-Pakistan, etc.), but they'll have to be able to host Tests outside of the October to January window, have touring teams accept the loss of some tour match days, or the loss of rest days between matches. The other alternative (probably the more reasonable one) is to have six-year cycles.

England's home schedule would look like this (five-year cycle):
Bd (2 Tests to be squeezed in), Aus
WI, SL
SA, NZ
Ind, Pak
Aus

If they had a six-year cycle, it could be:
Aus
WI, SL
SA
Ind, Pak
Aus
NZ, Bd

Or something like that. There are fiddly details that I haven't worked out, but basically if you spend enough time fiddling, everyone should get to play each other, there'll be more five-Test series than there are now, the IPL can go for four months, and Test cricket will survive. The Boards would make less money from international cricket under this proposal, because there aren't any ODI's, but hopefully there will be active and popular domestic T20 competitions to boost the coffers. Your cricket fan in Australia in February watches the NSW v Victoria T20 game before flicking over to watch a Kolkata play Mumbai game afterwards. Something like that. (I'm optimistic about domestic T20 being viable, because I was part of the crowd of over 27000 who watched Queensland play New South Wales at the Gabba in 2006/7. Now, no other Australian domestic crowd recently has come close to that, outside finals, but if it was at the forefront of the cricketing calendar, I think large crowds would be common.)

This is obviously a bit Utopian (and I've ignored the Champions League), but it least it shows in principle that things can all work out.

I should be back blogging next Monday or Tuesday.

The IPL player auction

2008-02-23T17:03:00.000+01:00

Sorry for the delay in updating — I've just got back from a short holiday in Amsterdam. On Tuesday morning I'm heading off to Florence and Rome, so there'll be another break in posting soon.

I had a question on whether Indian players were valued more than non-Indians in the IPL auction. The answer is that they were, by about $250k each. My analysis is a bit rough, since I didn't want to get bogged down in details in the couple of days I have before Italy.

Firstly, it's important to note that it's not a free market — there were requirements on young players, international players, icon players would have distorted the market, etc. But we'll see what the numbers tell us.

I took all the non-icon players who had ODI stats (or, failing that, List A stats) that included batting strike rate. Because it seemed a reasonable thing to do, I gave each player a batting rating, defined as the batting average multiplied by the strike rate, divided by 100, divided by 20 (roughly). For bowlers (and I chose bowlers by looking at them and deciding whether I'd consider their bowling in buying them; there's a grey area of course, but for most players it's pretty obvious) I gave a bowling rating: bowling average times economy rate, divided by 6, divided by 25.

I might be biasing the ratings towards batsmen or towards bowlers, but it shouldn't be too bad. Then I added the batting and bowling ratings for an overall player rating.

I put three other variables into the regression model: number of matches (a bit dodgy in one or two cases, where I used List A rather than ODI's), and dummy variables for Indians and wicket-keepers.

I probably should have done something about the Australians and West Indians, who are only available for half the tournament, but I couldn't be bothered.

Here are the results of the regression:


Modèle 1: Estimation en MCO avec 70 observations 1-70
Variable dépendante: salary

      VARIABLE       COEFFICIENT        ERR. STD         T           p. critique
  const             46927,8         114520             0,410   0,68332
  mat                 673,801          361,921         1,862   0,06716 *
  rating           163109            62931,6           2,592   0,01178 **
  indian           267326            71927,6           3,717   0,00042 ***
  keeper           136852            93199,5           1,468   0,14682

  Moyenne de la variable dépendante = 504357
  Écart-type de la var. dép. = 286130
  Somme des carrés des résidus = 4,25589e+012
  Erreur standard des résidus = 255881
  R2 non-ajusté = 0,246619

Key points:

- There is a slight positive correlation between matches (i.e., experience) and salary. For every hundred extra ODI's, the salary goes up by about $65000.

- My hastily calculated player ratings are positively correlated with salary. Increase the batting average (times strike rate) by 10, your salary goes up by $80000.

- If you're Indian, you get a bonus $265000. Indian cricketers can expect to be part of marketing campaigns.

- Wicket-keepers get an extra $135000, and I'll ignore the p value which tells me that it's not significant. The extra money they get is expected, since I didn't incorporate wicket-keeping skills into the player ratings.

- These factors explain 25% of the statistical variance, which is 50% of the salaries in cricket terms.

Now just for a bit of fun, I decided to use the player ratings to work out how many dollars each team spent per player rating point. I've fixed it so that the teams are on a scale of 3 to 9, so that I can compare with Q.


Team        Me   Q
Jaipur      9    3
Chennai     6,3  7
Mumbai      3    6
Bangalore   5,3  5
Hyderabad   5,8  8
Mohali      3,1  7
Kolkata     3,5  9
Delhi       6,0  9

The conclusion here is that at least one of me and Q has no idea what we're doing. Of course, my analysis is based purely on ODI numbers (possibly out of date — several people have said that T20 is a young man's game, with the play very fast), whereas Q's looked at T20 form and crowd-drawing power. Even so! I suspect the difference of our ratings of Jaipur is that they didn't actually spend much money on players. So they got quality for what they spent, but the overall team isn't all that good. The point of the bidding process is to get the best team (including marketing, etc.), not to get the most player rating points per dollar.

1800's first-class cricket in England: wicket-keepers

2008-02-17T19:36:00.002+01:00

This is Part 9, and also the final instalment, in my series on first-class cricket in the 1800's in England.

1 - data
2 - classification of matches
3 - filling in the gaps
4 - bowlers
5 - batsmen
6 - bowlers across eras
7 - batsmen across eras
8 - all-rounders (across eras)
9 - wicket-keepers

In this post I look at wicket-keepers. I've decided to focus on pure wicket-keeping, and so I've ignored batting. It's a bit boring-listy, but there's a graph below all the tables.

Many of the early scorecards do not indicate who the wicket-keeper was. If there were one or more stumpings, whoever effected the first one was deemed as the keeper. I don't think that this is much of a problem in terms of the tables below, since most of the record-getters played once the scorecards became more complete. Nevertheless, there are probably minor errors, since sometimes teams change wicket-keepers, and I've allocated all the innings byes to the first one (that I know of).

To begin, let's have a look at the leading keepers by dismissal in the 1800's. The last column is the percentage of team runs conceded as byes.


name          start end   mat ct   st  dis   b %
M Sherwin     1876  1896  308 577  205 782   3,05
EW Pooley     1864  1883  289 420  352 772   2,29
H Wood        1881  1899  290 532  114 646   2,87
D Hunter      1888  1899  252 437  180 617   2,98
R Pilling     1877  1889  226 418  187 605   3,04
H Phillips    1869  1891  209 335  184 519   2,97
HR Butt       1890  1899  204 334  130 464   2,46
JH Board      1891  1899  185 329  115 444   2,60
T Lockyer     1849  1866  154 234  112 346   1,78
J Hunter      1878  1888  153 220  118 338   3,91

Mordecai Sherwin tops the list. He was, amusingly, a keeper in both cricket and professional soccer. Both Hunter brothers make the top ten.

The leader in terms of dismissals per match (with at least 20 matches) is Charles Smith at 2,69. Behind him are Pilling and Pooley.

If you look down the right-hand column of the above table, one man stands out — Tom Lockyer. He was easily the best in terms of byes in the 1800's. With a qualification of 50 matches:


name          start end   mat ct   st  dis   b %
T Lockyer     1849  1866  154 234  112 346   1,78
AP Wickham    1878  1899  65  68   42  110   2,03
JA Bush       1870  1890  145 205  93  298   2,13
AE Newton     1885  1899  83  130  50  180   2,21
A Pike        1894  1899  63  99   28  127   2,27
EFS Tylecote  1871  1886  62  90   48  138   2,29
EW Pooley     1864  1883  289 420  352 772   2,29
FH Huish      1895  1899  88  208  23  231   2,36
JP Whiteside  1888  1899  108 159  44  203   2,45
HR Butt       1890  1899  204 334  130 464   2,46

Fred Huish, early in his career, was showing signs of his greatness as a wicket-keeper. He figures prominently in the next table, which shows the leading keepers by number of dismissals for all first-class matches in England. I've added an extra column — byes per 600 balls. Players from the 19th century are in bold.


name          start end   mat ct   st  dis   b %   b/600
RW Taylor     1960  1988  547 1257 155 1412  1,03  2,76
H Strudwick   1902  1927  610 1133 220 1353  2,97  8,63
JT Murray     1952  1975  546 1116 219 1335  2,05  5,50
FH Huish      1895  1914  493 922  377 1299  2,91  8,35
D Hunter      1888  1909  543 910  347 1257  2,59  6,69
B Taylor      1949  1973  520 1036 200 1236  1,54  4,28
HR Butt       1890  1912  543 949  275 1224  3,05  8,89
H Elliott     1920  1947  517 886  292 1178  1,84  4,67
JH Board      1891  1913  482 810  348 1158  2,63  7,82
RC Russell    1981  2004  405 1033 111 1144  0,93  2,92

Huish never played a Test match, mostly because of Bert Strudwick, two places ahead of him on that table. Bob Taylor is the leader all-time, as we would expect (since he holds the overall first-class record).

My last two tables show a curious phenomenon. The first has the leading keepers by byes percentage. Qualification for both: 60 matches.


name          start end   mat ct   st  dis   b %   b/600
DE East       1981  1989  189 479  53  532   0,81  2,40
RJ Turner     1988  2005  233 666  49  715   0,83  2,88
P Whitticase  1984  1995  129 309  14  323   0,84  2,63
GR Stephenson 1967  1980  270 584  77  661   0,85  2,23
BJM Maher     1981  1993  125 279  14  293   0,87  2,85
RC Russell    1981  2004  405 1033 111 1144  0,93  2,92
SJ Rhodes     1984  2004  390 1009 101 1110  0,95  3,02
CP Metson     1981  2001  230 556  51  607   0,95  3,03
CMW Read      1998  2007  151 462  23  485   0,99  3,45
APE Knott     1964  1985  411 1012 101 1113  1,02  2,76

Lots and lots of 1980's, with David East the best. Now again, but for byes per 600 balls:


name          start end   mat ct   st  dis   b %   b/600
GR Stephenson 1967  1980  270 584  77  661   0,85  2,23
DE East       1981  1989  189 479  53  532   0,81  2,40
P Whitticase  1984  1995  129 309  14  323   0,84  2,63
RW Taylor     1960  1988  547 1257 155 1412  1,03  2,76
APE Knott     1964  1985  411 1012 101 1113  1,02  2,76
BJM Maher     1981  1993  125 279  14  293   0,87  2,85
RJ Turner     1988  2005  233 666  49  715   0,83  2,88
RC Russell    1981  2004  405 1033 111 1144  0,93  2,92
BSV Timms     1959  1971  231 456  70  526   1,24  3,00
SJ Rhodes     1984  2004  390 1009 101 1110  0,95  3,02

Now Bob Stephenson moves up to first, part of a general movement of 1960's keepers up the rankings. It would appear as though he was unlucky not to play a Test, but with his career coinciding with Taylor's and Alan Knott's, he was kept in county cricket.

I'm guessing that the difference between byes percentage and byes per 600 balls is due to batsmen hitting the ball more often in the 1980's, so that less balls got through to the keeper. Ideally, we'd have a "byes per balls that passed the batsmen". It should be possible to come up with a correction factor based on the run rate (so that you'd use run rate as a proxy for balls hit), but I haven't tried to do so, and in the absence of ball-by-ball data I don't know how accurate it would be.

We know the number of overs in about 95% of the innings that Tom Lockyer kept. In these, he averaged just under 10 byes ber 600 balls, almost exactly the same as Paul Nixon.

I don't want to do era adjustments for wicket-keepers. The balance between bat and ball can change, and so averages should be adjusted accordingly to get comparisons of talent levels. But letting through byes shouldn't change much with eras of low scoring. Still, you might want an idea of how far away from the average keeper of the era someone like Lockyer was, so here's a graph showing the overall byes percentage for each season. There's a lot of early noise because of the low number of matches. There's a huge peak in the era of very low scoring around the 1830's. This isn't just because the byes were constant and the runs were decreasing — there's a peak in the byes per match as well, suggesting that keepers had just as much trouble with the round-arm bowling on those pitches as batsmen did.

Noteworthy are the jumps that follow the World Wars, telling us that keepers were out of practice and skill levels had dropped. The general trend after World War II was downward until about 1990, when it starts to rise again, presumably when teams started giving more importance to keepers' batting ability.

Looking pre-World-War-I, there are a couple of phases whose causes I don't know. There is a clear rise from the 1860's to the 1880's, before it starts to fall again and then noisily flatten out in around 1900.

And that's the end of this post and this series on 1800's first-class cricket in England. Thanks to anyone who actually read it all.

FIN

Bowler support

2008-02-14T22:45:00.001+01:00

One of the problems in comparing bowlers from different teams is that they often have different levels of support. This is a recurring theme in Murali v Warne debates (in between expletive-laden rages) — Murali took more wickets per Test, but that was because Warne followed McGrath and Gillespie, and Murali only ever had Vaas. Warne had it easier, and that makes Murali's low average more remarkable. But maybe the batsmen didn't try to score as much against Murali, because they could pick off runs easily at the other end.

The debate can go on and on, and it's not clear which factors are the most important. So I asked myself the question, if you swapped the two bowlers between the two teams, what would their records be?

You can't answer this question perfectly, of course, but you can try. For each innings in which the bowler bowled, I defined the support average as the mean of the averages of the four bowlers who bowled the most overs in that innings (three bowlers if the bowler himself would have been one of the four). Note that, unless otherwise stated, the averages used are averages in which each wicket is weighted in proportion to the batting average of the batsman dismissed. I use end-of-career averages (to make my life easier and the numerics more stable).

I'll give an example of what I mean by the support average. Suppose that in one particular innings, the bowlers used were:

bowler (bowler's average): # of overs
A (25): 30
B (24): 34
C (33): 23
D (31): 15
E (45): 6

The support average of bowler A is (24 + 33 + 31)/3 = 29,33. The support average of bowler E is (25 + 24 + 33 + 31)/4 = 28,25.

To do the analysis, for each bowler I took all innings and sorted them by support average. I then binned them into quartiles (to reduce the noise and make for easier interpretation), that is, the quarter of innings with the lowest support averages, the quarter with the next lowest support averages, and so on.

Then for each quartile I calculated the bowler's average, and also the average support average, with the latter weighted by the number of balls bowled in each innings (so that, for instance, an innings where the bowler only bowled one over would barely be counted).

Then you can make tables like these ones:


SK Warne      q1     q2     q3     q4     overall
supp avg      26,32  27,81  29,61  35,24  29,92
bowl avg      26,58  29,01  26,59  29,52  27,91

M Muralidaran q1     q2     q3     q4     overall
supp avg      33,86  37,19  39,26  47,28  39,45
bowl avg      23,01  24,26  27,98  23,15  24,43

You can see that Murali's support average is indeed much higher than Warne's, as you would expect. I don't know how much I want to read into individual trends — four data points, even aggregated ones, aren't a lot. That won't stop me trying. The overall trend for Warne is for his average to increase as his support gets weaker. In particular, for the upper quartile (the only one near where Murali has to bowl) his average is the highest, getting close to 30. Murali seems to turn it on when he has no-one to support him at all.

How about another pair, this time from the 1980's?


RJ Hadlee     q1     q2     q3     q4     overall
supp avg      34,51  36,63  38,49  45,75  38,81
bowl avg      24,80  19,69  26,15  25,80  23,96

MD Marshall   q1     q2     q3     q4     overall
supp avg      24,96  27,13  30,01  36,55  29,73
bowl avg      25,01  19,29  20,75  23,22  21,83

When Marshall didn't have Croft, Garner, Roberts, and/or Holding around him, he was still awesome.

And since I've apparently made comparing pairs of bowlers a theme for this post, here are the rather surprising results for McGrath and Gillespie:


GD McGrath    q1     q2     q3     q4     overall
supp avg      28,02  29,32  30,61  33,38  30,28
bowl avg      22,19  20,41  21,92  25,78  22,36

JN Gillespie  q1     q2     q3     q4     overall
supp avg      26,47  27,43  28,49  34,30  29,28
bowl avg      32,81  28,75  30,62  22,23  28,05

It seems that Gillespie did actually do pretty well when McGrath wasn't around, and it was McGrath who got worse (a little bit) when he didn't have support. Well, perhaps — it could just be an artifact of McGrath's career trajectory, I haven't checked.

These are all well and good, but we'd like to do a bit more serious analysis with them. If you fit a regression line to an individual player, you can get a rough guide of how their average will change when the support gets better or worse. While it might be dubious to do this for just one player, if you do it for all players, the noise should largely cancel out and we'll be left with some solid numbers.

So, I took all bowlers with 100 Test wickest at at least 3 wickets per Test, calculated the slope of the regression line for each, and then took the mean of the slopes. The result was 0,50. That's a pretty hefty figure. It means that, on average, if the support average goes down by a run, then the bowler's average will go down by half a run. But it's inflated by the presence of some outliers. If you exclude players with less than 50 Tests (remember that we're dealing with quartiles here, so you need a large number of Tests to get reasonable quartile results), that figure drops to 0,25. So if the support average drops by four runs, the bowler's average drops by one run, on average.

You might be wondering if there's a correlation between average and the regression slope. There isn't. The scatterplot is equal parts scatter and plot.

No trend emerges if you make the qualification 50 Tests to get rid of the outliers. I tried a few other variables, but I couldn't find anything with an R-squared of better than about 0,004. Even in cricket terms, none of them explained more than 7% of the data.

1800's first-class cricket in England: all-rounders (across eras)

2008-02-12T13:07:00.002+01:00

This is Part 8 in my series on first-class cricket in the 1800's in England.

1 - data
2 - classification of matches
3 - filling in the gaps
4 - bowlers
5 - batsmen
6 - bowlers across eras
7 - batsmen across eras
8 - all-rounders (across eras)
9 - wicket-keepers

In this post, I look at all-rounders. As I did for Test cricketers, I'll be ranking players by the ratio of batting average to bowling average, where the averages are weighted as in Parts 6 and 7.

Let's start, as always, with the 1800's. The averages below are with respect to 16,6, the overall average for the period in question. The +/- percentage figure applies both to the bowling average (well, technically it applies to the regular bowling average; I fondly hope that it's accurate for the weighted average) and (by a trick of mathematics) to the ratio as well. I've given the wickets per match for those interested; recall that these are underestimates for bowlers whose wicket tallies are estimated. Qualifications: 2000 runs and at least two (regular) wickets per match.


name             start end   mat  runs  avg     wkts   avg   wpm   ratio +/- %
W Lambert        1801  1817  62   2961  37,05   318,1  16,3  5,1   2,3   10,0
Lord F Beauclerk 1801  1825  94   4319  37,28   406,4  18,5  4,3   2,0   10,0
WG Grace         1865  1908  838  46792 37,41   2495   18,91 2,98  1,98  0,0
AG Steel         1877  1895  142  6184  28,28   699    14,53 4,92  1,95  0,0
J Broadbridge    1814  1840  90   2368  26,82   407,6  14,2  4,5   1,9   9,9
CT Studd         1879  1884  85   3928  30,35   426    16,89 5,01  1,80  0,2
A Mynn           1832  1859  200  4749  27,02   1059,9 15,9  5,3   1,7   7,0
CG Taylor        1836  1859  122  3020  33,56   292,0  20,5  2,4   1,6   7,0
EH Budd          1803  1831  68   2597  30,74   285,8  20,5  4,2   1,5   10,0
W Caffyn         1849  1873  180  5405  24,26   564    16,17 3,13  1,50  0,3
T Hayward        1854  1872  108  4487  27,00   237    18,00 2,19  1,50  0,6
J Wisden         1845  1863  175  4020  19,77   1037,5 13,9  5,9   1,4   3,4
RG Barlow        1871  1891  321  10074 18,43   879    13,06 2,74  1,41  0,0
G Giffen         1882  1896  158  5621  20,23   502    14,81 3,18  1,37  0,0
GA Lohmann       1884  1896  256  6495  16,17   1590   11,99 6,21  1,35  0,0
CTB Turner       1888  1893  93   2118  13,15   610    10,34 6,56  1,27  0,0
GA Davidson      1886  1898  155  5338  18,45   605    15,35 3,90  1,20  0,0
W Bates          1877  1887  257  8651  19,09   746    16,13 2,90  1,18  0,0
WE Midwinter     1877  1884  127  3533  17,90   330    15,14 2,60  1,18  0,0
W Flowers        1877  1896  409  12035 17,61   1085   15,25 2,65  1,15  0,0

Lambert's pretty clear at the top. Beauclerk is mildly ahead of WG Grace and Allan Steel, but the uncertainty means that all wa can say is that he's likely to be somewhere between second and sixth.

Before I started on this extended exercise in analysing old English players, I didn't know much at all about the cricketers of the era, apart from WG Grace. One name I did know was Alfred Mynn, rated by John Woodcock as the fourth-greatest cricketer of all time. Now, Woodcock's list has lots of problems (most notably, WG Grace is number one, ahead of Bradman), but I was interested to see how Mynn would fare after adjusting for eras. He comes in at number seven (plus or minus one) on the table above. But if (as might have happened) Woodcock ignored cricket before 1830, then you can see what his method was — he chose near the top all-rounders with huge aggregates. Mynn was not a special batsman, but he was a prolific wicket-taker, even if his bowling average wasn't remarkable for his time. Add in his popularity, and his dominance of single-wicket matches, and you can see where Woodcock was coming from, even if number four is too high.

Now let's move onto all first-class cricket in England. Players whose career began in the 1800's are in bold. Averages are with respect to 24,5.


name             start end   mat  runs  avg     wkts   avg   wpm   ratio +/- %
KR Miller        1945  1959  75   4253  49,22   164    17,59 2,19  2,80  0,0
WW Armstrong     1902  1921  124  5641  41,87   407    16,36 3,28  2,56  0,0
W Lambert        1801  1817  62   2961  54,68   318,1  24,01 5,13  2,28  10,0
RJ Hadlee        1973  1990  187  6887  30,48   780    14,13 4,17  2,16  0,0
GStA Sobers      1957  1974  209  13491 48,01   548    23,38 2,62  2,05  0,0
FE Woolley       1906  1938  886  54535 40,98   1893   20,11 2,14  2,04  0,0
WG Grace         1865  1908  838  52043 51,85   2675   25,62 3,19  2,02  0,0
Lord F Beauclerk 1801  1825  94   4319  55,02   406,4  27,3  4,3   2,0   10,0
FA Tarrant       1903  1914  295  15925 36,93   1327   18,92 4,50  1,95  0,0
AG Steel         1877  1895  142  6184  41,74   699    21,45 4,92  1,95  0,0
J Broadbridge    1814  1840  90   2368  39,58   407,6  21,0  4,5   1,9   9,9
JM Gregory       1919  1926  77   2869  34,26   281    18,49 3,65  1,85  0,0
GH Hirst         1891  1929  801  35378 35,52   2687   19,26 3,35  1,84  0,0
CT Studd         1879  1884  85   3928  44,80   426    24,92 5,01  1,80  0,2
MJ Procter       1965  1981  264  14733 32,27   848    18,31 3,21  1,76  0,0
W Rhodes         1898  1930  1007 35015 30,35   3960   17,43 3,93  1,74  0,0
GA Faulkner      1907  1924  74   3046  29,83   267    17,42 3,61  1,71  0,0
FS Jackson       1890  1907  301  15626 38,88   744    22,81 2,47  1,70  0,0
JW Hearne        1909  1936  593  34438 41,25   1687   24,22 2,84  1,70  0,0
A Mynn           1832  1859  200  4749  39,88   1059,9 23,5  5,3   1,7   7,0
TL Goddard       1955  1962  48   2549  32,85   140    19,39 2,92  1,69  0,0
SG Smith         1906  1914  143  7575  33,87   606    20,48 4,24  1,65  0,0
CG Taylor        1836  1859  122  3020  49,52   292,0  30,3  2,4   1,6   7,0
R Kilner         1911  1927  389  13722 29,48   917    18,53 2,36  1,59  0,0
Imran Khan       1971  1988  240  11679 31,80   733    20,17 3,05  1,58  0,0
JR Mason         1893  1914  324  16619 35,92   817    23,71 2,52  1,52  0,0
EH Budd          1803  1831  68   2597  45,37   285,8  30,2  4,2   1,5   10,0
W Caffyn         1849  1873  180  5405  35,81   564    23,87 3,13  1,50  0,3
T Hayward        1854  1872  108  4487  39,85   237    26,58 2,19  1,50  0,6
IJ Harvey        1999  2007  75   4044  28,43   219    19,11 2,92  1,49  0,0

Keith Miller comes out on top, ahead of (surprisingly) the Big Ship Warwick Armstrong. Lambert leads a host of 19th century players, who are vastly over-represented in the table — almost half of the top thirty spots! Given the number of players since 1900, you'd expect only about five or six from the 1800's. Alfred Mynn is a long way down the table (20th place), but if you give more weighting to wickets per match, he would be higher.

At number nine is Frank Tarrant, someone I'd never heard of. He never played a Test, which, at first glance, is extraordinary for someone with his first-class record. His lack of Test cricket is explained by his being Australian and playing for Middlesex, which barred him from playing for Australia (though he did play for the MCC at times).

The abundance of 19th century all-rounders tells us something about the nature of the game and/or its players. I'm not sure exactly what factors contributed to it, but I would suggest the following. When cricket was less developed, and had fewer top-level players, a talented athlete was more likely to dominate with both bat and ball. As batting and bowling techniques became more sophisticated, and the number of players increased, there were more specialists in both disciplines, making it harder for the talented cricketer to be good (relative to his peers) with both bat and ball.

Next up (and the last instalment in this series): wicket-keepers.

All-rounderness

2008-02-10T22:12:00.000+01:00

The statistical judging of all-rounders is usually not done in what I would consider a satisfactory way. I'm not really going to remedy this problem in this post, though I will present some ranked lists. It's more a case of looking at the stats and seeing why it's hard to get them to agree well with common sense (without making some arbitrary decisions).

I'll only be considering batsman-bowler all-rounders. If you want to comment about wicket-keepers, it should be about Tim Zoehrer.

The base of this analysis will be the bowling averages with wickets weighted by the batting averages of the batsmen dismissed, and the batting averages with runs weighted by the strength of the bowling attack. This gives a comparison across all eras, and rewards those players who performed well against stronger sides. Both the averages I use in this post are normalised to 31,48, which is the overall batting average for all Tests. All references to averages below are these weighted averages.

My main ranking tool will be the batting average divided by the bowling average. I prefer this to the difference, which is more commonly used, because I think it gets closer to a definition of "all-rounder-ness". So, for instance, a batting average of 60 and a bowling average of 30 gives a ratio of 2. A batting average of 40 and a bowling average of 20 also gives a ratio of 2. I think this is fair — in the first case, you have an all-time great batsman who was a good bowler, and in the second you have an all-time great bowler who was a good batsman. You might think that one is a better player than the other, but I'm trying to get at the all-rounder-ness. I hope that's clear.

So, let's think of what qualities we'd like and qualifications we'll use in ranking the best genuine all-rounders of all time.

1. 20 Test innings
2. At least 2 wickets per Test (I don't use weighted wickets here; I just want to make sure that they bowled regularly)
3. A batting average above average (i.e., higher than 31,48)
4. A bowling average below average (i.e., lower than 31,48)

The top seven all-rounders of all time are then as follows. (Runs and wickets are the regular runs and wickets; wpm is wickets per match; ratio is the ratio of batting average to bowling average).


name             mat runs  avg     wkts avg   wpm   ratio
KR Miller        55  2958  35,37   170  23,20 3,09  1,52
Imran Khan       88  3807  36,07   362  24,01 4,11  1,50
W Bates          15  656   36,05   50   26,96 3,33  1,34
TL Goddard       41  2516  35,60   123  27,06 3,00  1,32
IT Botham        102 5200  33,43   383  30,51 3,75  1,10
TE Bailey        61  2290  32,23   132  30,33 2,16  1,06
JM Gregory       24  1146  32,89   85   31,24 3,54  1,05

Why only seven? Because that's all there is. No other players satisfy those four conditions above.

I'm actually pretty happy with that list. It's obviously not the list of the best all-rounders ever, but as a list of the most all-rounder of all-rounders, I think it works. Keith Miller just beats Imran Khan as the best ever.

Now let's remove requirements 3 and 4 above and see what we get.


name             mat runs  avg     wkts avg   wpm   ratio
GStA Sobers      93  8032  54,62   235  34,63 2,53  1,58
KR Miller        55  2958  35,37   170  23,20 3,09  1,52
Imran Khan       88  3807  36,07   362  24,01 4,11  1,50
AG Steel         13  600   48,93   29   35,16 2,23  1,39
W Bates          15  656   36,05   50   26,96 3,33  1,34
TL Goddard       41  2516  35,60   123  27,06 3,00  1,32
AK Davidson      44  1328  27,75   186  21,51 4,23  1,29
GA Faulkner      25  1754  45,89   82   36,35 3,28  1,26
SM Pollock       108 3781  30,31   421  24,18 3,90  1,25
AW Greig         58  3599  39,76   141  33,45 2,43  1,19
RJ Hadlee        86  3124  26,09   431  23,76 5,01  1,10
IT Botham        102 5200  33,43   383  30,51 3,75  1,10
TE Bailey        61  2290  32,23   132  30,33 2,16  1,06
W Barnes         21  725   30,57   51   28,81 2,43  1,06
JM Gregory       24  1146  32,89   85   31,24 3,54  1,05
A Flintoff       66  3331  31,06   190  29,89 2,88  1,04
MA Noble         42  1997  33,12   121  32,61 2,88  1,02
CL Cairns        62  3320  32,53   218  32,07 3,52  1,01
G Ulyett         25  949   32,25   50   31,93 2,00  1,01
C Kelleway       26  1422  37,08   52   37,45 2,00  0,99

Now Sobers returns to number one, which is where most judges would put him. You can see why he missed out on the previous list — his bowling wasn't that good. Despite an average in the mid-30's (both weighted and regular), he was actually a very economical bowler. A high average and low economy rate (2,22) means that his strike rate was appallingly bad, over 90. Not the go-to man if you need a wicket! But he's generally considered the second-greatest ever player because no other great batsman could bowl so well. Even if "so well" is not so well.

The cut-off of 2 wickets per Test is pretty arbitrary, and it would be unfair to stop here, because it would exclude Jacques Kallis. For my last table, I've lowered the bar to just 1 wicket per Test. This means that a bunch of part-timers are included. While it would be silly to consider them as being as good as the more regular wicket-takers given here, the stats must tell some story — perhaps it suggests that they were underbowled, or perhaps they were just lucky and dismissed a few good batsmen from time to time.


name             mat runs  avg     wkts avg   wpm   ratio
JH Kallis        113 9331  50,52   223  31,50 1,97  1,60
RM Cowper        27  2061  44,43   36   28,14 1,33  1,58
GStA Sobers      93  8032  54,62   235  34,63 2,53  1,58
FS Jackson       20  1415  55,40   24   35,89 1,20  1,54
KR Miller        55  2958  35,37   170  23,20 3,09  1,52
Imran Khan       88  3807  36,07   362  24,01 4,11  1,50
CG Macartney     35  2131  44,52   45   30,30 1,29  1,47
AG Steel         13  600   48,93   29   35,16 2,23  1,39
A Symonds        19  1031  38,34   22   28,16 1,16  1,36
W Bates          15  656   36,05   50   26,96 3,33  1,34
TL Goddard       41  2516  35,60   123  27,06 3,00  1,32
AK Davidson      44  1328  27,75   186  21,51 4,23  1,29
EJ Barlow        30  2516  40,24   40   31,36 1,33  1,28
GA Faulkner      25  1754  45,89   82   36,35 3,28  1,26
SM Pollock       108 3781  30,31   421  24,18 3,90  1,25
BM McMillan      38  1968  37,59   75   30,46 1,97  1,23
ER Dexter        62  4502  43,58   66   35,48 1,06  1,23
FMM Worrell      51  3860  49,45   69   40,78 1,35  1,21
AW Greig         58  3599  39,76   141  33,45 2,43  1,19
JDP Oram         25  1380  37,47   49   33,21 1,96  1,13
RJ Hadlee        86  3124  26,09   431  23,76 5,01  1,10
IT Botham        102 5200  33,43   383  30,51 3,75  1,10
TE Bailey        61  2290  32,23   132  30,33 2,16  1,06
W Barnes         21  725   30,57   51   28,81 2,43  1,06
JM Gregory       24  1146  32,89   85   31,24 3,54  1,05
NWD Yardley      20  812   23,53   21   22,63 1,05  1,04
A Flintoff       66  3331  31,06   190  29,89 2,88  1,04
Mushtaq Mohammad 57  3643  37,45   79   36,35 1,39  1,03
MA Noble         42  1997  33,12   121  32,61 2,88  1,02
CL Cairns        62  3320  32,53   218  32,07 3,52  1,01

And Kallis actually slots in at number one! Bob Cowper, with his part-time offies, will probably suprise most of you (it surprised me, even though I was vaguely aware of his handy bowling). Andrew Symonds' career is definitely on the improve. His regular bowling average is now under 35. That his weighted bowling average is just over 28 tells us that he's dismissing some good batsmen.

So there you go. I don't know what features you'd want in an ideal ranking of all-rounders. You could set boundary of 2 wickets per Test, and penalise players (such as Kallis) who take less wickets, but setting the boundary would be arbitrary.

Charles Davis, when rating bowlers, actually gives equal weighting to wickets per Test and bowling average. While I see the arguments for doing so (and it would eliminate the problem of setting that boundary), I still like to fall back on the average, so as not to unduly reward bowlers with no support.

1800's first-class cricket in England: batsmen across eras

2008-02-09T12:10:00.001+01:00

This is Part 7 in my series on first-class cricket in the 1800's in England.

1 - data
2 - classification of matches
3 - filling in the gaps
4 - bowlers
5 - batsmen
6 - bowlers across eras
7 - batsmen across eras
8 - all-rounders (across eras)
9 - wicket-keepers

In this post, I do a comparison of batsmen across eras, by weighting each innings by the strength of the bowling attack. The latter is taken as the "average average" of the bowlers in an innings, weighted by the number of balls bowled by each. (You can see the results for Test matches here.) The same effects (only now for batsmen) occur here as in the weighted bowling averages in Part 6 — batsmen are rewarded for scoring runs against better bowling attacks, and batsmen in low-scoring eras are rewarded because typically the bowlers will have correspondingly low averages.

For innings where the bowlers' overs are not recorded, I've instead used the overall batting average for that season. I couldn't see an easy way of getting an unbiased estimate of the bowling strength, when we don't know who bowled. Using the season average will in general inflate the modified batting average (since typically a batsman will score more heavily against weaker attacks, but using the season average counts that as the same as scoring against a strong attack). But for the players near the top of the tables below, this is not so likely — these batsmen tend to "rise to the occasion" and perform disproportionately better against stronger bowling attacks. Nevertheless, perhaps you might want to put a mental asterisk next to players whose careers included matches from before 1855.

Also, some of the bowling averages are estimates (even where we know the overs bowled), so some of the weighted averages should only be given to one decimal place. But I'm getting lazy.

It's interesting to see a graph of the season averages:

There's a lot of noise in the early years because not many first-class matches were played (sometimes only one). The lowest-scoring season was 1831, when the average of runs off the bat (that is, excluding extras) was just 7,35. You can have a look at the scorecards for that season here.

To start our comparisons of batsmen, we look at just the 1800's. I've given the weighted runs, regular average, and (once again) two weighted averages, one with respect to 16,6 (the overall average for the 1800's) and one with respect to 24,5 (the overall average from 1801 to 2007). One is just a scaling of the other. Qualification: 2000 runs.


                                                                   wtd avg
name             start end   mat inns  no  runs  wtd runs  avg   wrt 16,6 wrt 24,5
KS Ranjitsinhji  1893  1899  291 232   28  10411 8042,8    51,03  39,43  58,19
RM Poore         1898  1899  42  47    6   2277  1584,2    55,54  38,64  57,03
F Pilch          1820  1854  213 389   30  6797  13668,9   18,93  38,08  56,20
WG Grace         1865  1899  838 1250  89  46792 43431,9   40,30  37,41  55,21
Lord F Beauclerk 1801  1825  94  172   14  4319  5890,3    27,34  37,28  55,02
W Lambert        1801  1817  62  112   5   2961  3964,3    27,67  37,05  54,68
N Wanostrocht    1830  1852  134 242   12  4392  8027,7    19,10  34,90  51,51
CG Taylor        1836  1859  122 222   11  3020  7080,2    14,31  33,56  49,52
G Parr           1844  1870  187 321   26  6116  9137,6    20,73  30,97  45,72
EH Budd          1803  1831  68  119   9   2597  3381,4    23,61  30,74  45,37
CT Studd         1879  1884  85  145   23  3928  3702,8    32,20  30,35  44,80
A Shrewsbury     1875  1899  459 654   66  20837 17552,8   35,44  29,85  44,06
J Guy            1837  1854  136 244   11  3090  6723,5    13,26  28,86  42,59
AG Steel         1877  1895  142 227   21  6184  5826,0    30,02  28,28  41,74
W Ward           1810  1845  116 210   21  3517  5341,6    18,61  28,26  41,71
EG Wenman        1825  1854  135 241   15  3088  6382,4    13,66  28,24  41,68
CB Fry           1892  1899  381 209   8   7364  5597,2    36,64  27,85  41,10
R Robinson       1801  1819  57  111   9   2039  2811,1    19,99  27,56  40,68
TW Hayward       1893  1899  671 283   26  9558  7014,5    37,19  27,29  40,28
W Gunn           1880  1899  505 716   65  21520 17612,3   33,06  27,05  39,93

Ranji's high average wasn't just because batting was getting easier towards the end of the century — even allowing for that he still comes out on top. Once again, Robert Poore's lucky, since his career went downhill after he fought in the Boer War.

Fuller Pilch was described a few years after his retirement as the best batsman ever, and he kept this tag until WG Grace came along. From the little I've read about him, he seems to be the first man to consistently get his foot to the pitch of the ball. In an era where pitches were of very low quality, smothering any turn or uneven bounce was very important.

It's worth commenting on the discrepancy between the rankings in the table (i.e., Pilch ahead of Grace) and the opinion of the time (Grace ahead of Pilch). Grace was considered a better batsman than Pilch because he could play attacking shots off a wider range of deliveries. But since Grace's innovations to batting technique spread to the other cricketers of the time, he didn't stand out as much as Pilch — scoring for most batsman improved after Grace.

It is sad that Nicholas Felix is so called, since his actual surname was Wanostrocht. He wanted to be known as Felix, but Wanostrocht is such a cool name for a cricketer that I've gone against his wishes in these tables. In addition to being an excellent batsman in a low-scoring era, he also invented a type of bowling machine.

Now let's compare batsmen in England across all eras. I was unsure as to how useful this would be — we all know of batsmen who have excellent records in domestic cricket but do terribly in Tests. But weighting runs by the strength of the bowling attack does a pretty good job in discarding that breed of batsman. Of course, it also doesn't allow for players such as Marcus Trescothick, who have mediocre county records but respectable Test numbers. Players who played in the 1800's are in bold.


                                                                   wtd avg
name             start end   mat inns  no  runs  wtd runs  avg   wrt 16,6 wrt 24,5
DG Bradman       1930  1948  92  120   18  9837  6835,1    96,44  67,01  98,90
GA Headley       1933  1954  47  74    9   4460  2897,8    68,62  44,58  65,80
KS Ranjitsinhji  1893  1920  291 473   58  23341 17015,4   56,24  41,00  60,51
VM Merchant      1936  1946  49  81    15  4130  2517,8    62,58  38,15  56,30
F Pilch          1820  1854  213 389   30  6797  13668,9   18,93  38,08  56,20
WH Ponsford      1926  1934  67  86    12  4110  2812,8    55,54  38,01  56,10
WM Woodfull      1926  1934  72  87    9   4374  2956,3    56,08  37,90  55,94
NC O'Neill       1961  1964  44  71    8   3350  2379,6    53,17  37,77  55,75
AF Kippax        1930  1934  42  55    11  2412  1648,0    54,82  37,46  55,28
DR Martyn        1991  2005  30  44    11  2549  1231,1    77,24  37,31  55,06
Lord F Beauclerk 1801  1825  94  172   14  4319  5890,3    27,34  37,28  55,02
CB Fry           1892  1921  381 635   42  30490 22042,7   51,42  37,17  54,86
W Lambert        1801  1817  62  112   5   2961  3964,3    27,67  37,05  54,68
AR Morris        1948  1953  46  66    5   3224  2249,2    52,85  36,87  54,42
J Cook           1989  1991  71  124   19  7604  3863,8    72,42  36,80  54,31
RB Simpson       1961  1966  49  84    14  3702  2574,0    52,89  36,77  54,27
W Bardsley       1909  1926  126 175   17  7866  5603,0    49,78  35,46  52,34
MEK Hussey       2001  2005  60  105   13  6710  3253,5    72,93  35,36  52,19
SR Waugh         1987  2002  75  109   28  5290  2855,4    65,31  35,25  52,03
WR Hammond       1920  1951  515 828   88  40733 26039,2   55,04  35,19  51,93
WG Grace         1865  1908  838 1428  97  52043 46760,0   39,10  35,13  51,85
N Wanostrocht    1830  1852  134 242   12  4392  8027,7    19,10  34,90  51,51
SG Barnes        1938  1948  34  46    5   2074  1417,3    50,59  34,57  51,02
DS Lehmann       1991  2006  89  139   8   8894  4525,7    67,89  34,55  50,99
AL Hassett       1938  1953  73  100   11  4684  3063,3    52,63  34,42  50,80
CL Walcott       1950  1957  49  78    12  3271  2268,0    49,56  34,36  50,72
WM Lawry         1961  1968  65  105   12  4590  3182,2    49,35  34,22  50,50
G Boycott        1962  1986  492 814   127 38981 23425,2   56,74  34,10  50,33
JB Hobbs         1905  1934  740 1178  98  53843 36714,9   49,85  34,00  50,17
L Hutton         1934  1960  425 676   75  32306 20354,7   53,75  33,87  49,99

The top spot should be pretty uncontroversial. Vijay Merchant is at number four — he is perhaps not famous enough for having the second-highest first-class average of all time (71,64). Pilch slots in at five, followed by a string of Australians. You'll note that WG Grace fell significantly in the years between 1899 and his retirement in 1908. It's that old story of a player hanging on too long. Thirty-five years of first-class cricket and he still wanted another decade.

Beauclerk and Lambert, those all-round giants of the first quarter of the 19th century, just miss out on the top ten.

Jimmy Cook is perhaps the oddest name in the list. He was a South African who played most of his career during isolation in South Africa, but played three seasons with Somerset before retiring. He did get to play three Test matches.

Next up: all-rounders.

Increasing or decreasing scores

2008-02-07T14:25:00.000+01:00

Imagine a scorecard in which each batsman's score is less than the previous one. So, for example, the first opener scores 131, the second 82, the number three makes 75, the number four makes 56, and so on. What would you guess is the longest such sequence of declining scores in a Test innings? What would you guess is the longest increasing sequence?

These are, I admit, not the most important questions in cricket, but I found them fun. I would have thought that, while a long decreasing sequence would be rare, once you hit number 7 and the tail it should common enough to get down to number eight or nine. The problem is that once a batsman makes a duck, the sequence has to stop, so it'd be pretty unlikely to get all the way to number eleven.

An increasing sequence should be rarer, but perhaps when a captain reversed his batting order on a sticky wicket, you could get a sequence up to seven or so. Certainly I thought the longest increasing sequence would be shorter than the longest decreasing sequence.

As it happens, the longest increasing sequence goes to number six, but so does the longest decreasing sequence!

Decreasing sequences: Eng v Aus, 1905, Eng v SA, 1955, Aus v Eng, 1990/1, Zim v NZ, 1992/3.

Increasing sequences: Aus v Eng, 1884/5, Aus v Eng, 1932/3, Eng v WI, 1957.

If you allow a score to be equal to the previous score, the longest non-decreasing sequences still only go down to number six, but there are two non-increasing sequences to number 7: SA v Aus, 1949/50, Eng v Aus, 1977.

What about first-class matches? I only have a database of matches played in England (which is about 28000, more than half ever played), but the largest sequences (not allowing equal scores) in this dataset are of length eight:

Decreasing: Lancs v Middlesex, 1913, Lancs v Glamorgan, 1928, Wales v Minor Counties, 1930, Yorks v Lancs, 1952, Kent v Surrey, 1988.

Increasing: Lancs v Kent, 1988.

The Kent v Surrey innings (Surrey's first) is actually non-increasing right the way down to number eleven (it finishes with three ducks and a nought not out). There are two other cases of non-increasing sequences from one to eleven: Yorks v Lancs, 1888 (finishes with five ducks and a nought not out), Gloucs v Leics, 1929.

The longest non-decreasing sequence is still eight.

Is there a way we could have guessed how rare these sequences are? Any maths-phobes may wish to stop reading now. There's some calculus below.

A naïve approach would be to say that, since each score is either higher or lower than the previous one, it's just like flipping a coin. (Not exactly: it could be equal, but it'd be pretty close.) What's the probability of getting seven heads in a row? One in 128. But there have been over 1800 Test matches, so sequences of length seven should have occurred dozens of times!

The error in the above reasoning is at the start: it's not like flipping a coin. The first step is (the second opener could be higher or lower than the first, with a close to 50% chance each way), but after that, it's no longer 50-50. Suppose the first opener makes 40. It might be 50-50 as to whether the second opener scores less than 40. He might make 30, say. But then the probability that the number three will score less than 30 is less than 50%. If he makes 20, the chance that the number four will make less than this is even smaller than the previous probability, and so on.

A better way would be to assume that individual innings follow an exponential distribution, which says that scores of 0 are more common than scores of 1, which are more common than scores of 2, etc. (This isn't real distribution — in reality it's quite skewed towards zero — but it's a reasonable approximation for these fun purposes. Also, the runs are scored in discrete units — 1, 2, 3, etc. — but the exponential distribution allows for any positive real number of runs, such as sqrt(2) or 4.9.) Assume further (to make the maths easier) that each batsman has the same average.

The probability that a batsman with average 1/k makes a score less than x is given by:


x                 
/                 
| ds k*exp(-k*s)             (*)
/                 
0

Sorry for the ugly formatting. That's the integral from 0 to x of k*exp(-k*s) with respect to s.

Now let the first n batsman's scores be called s1, s2, ..., s(n-1), sn. We want the probability P that the sequence goes s1 > s2 > ... s(n-1) > sn. To start, we use (*) on the last link in the chain of inequalities:


s(n-1)                 
/                 
| dsn k*exp(-k*sn)
/                 
0

Once we have that, we carry that on to the next inequality, and so on, until we have all of them. The last integral is from 0 to infinity, since the first score s1 can be anything:


oo                 s1                     s(n-1)
/                  /                      /
| ds1 k*exp(-k*s1) | ds2 k*exp(-k*s2) ... | dsn k*exp(-k*sn)
/                  /                      /
0                  0                      0

Now we evaluate these integrals! Trust me when I say that, when you expand it all out, most of the terms cancel, and you're left with a term that comes from integrating the exponentials which multiply together, so that the probability is 1/n!.

So, a decreasing sequence of length six should happen about once every 720 innings, a sequence of length seven about once every 5040 innings, and a sequence of length 8 about once every 40320 innings.

Essentially the same argument gives the same probabilities for increasing sequences.

It's not perfect (and we wouldn't expect it to be so, given that the real distribution isn't exponential, and batsmen's averages aren't all equal), but it gets the right order of magnitude at least. It gives us a good idea of why we haven't seen a sequence of length seven in Test cricket yet, though we should get one eventually, perhaps in the next ten or twenty years.

1800's first-class cricket in England: bowlers across eras

2008-02-05T21:34:00.001+01:00

This is Part 6 in my series on first-class cricket in the 1800's in England.

1 - data
2 - classification of matches
3 - filling in the gaps
4 - bowlers
5 - batsmen
6 - bowlers across eras
7 - batsmen across eras
8 - all-rounders (across eras)
9 - wicket-keepers

In this post, I do a comparison of bowlers across eras, by weighting each wicket that a bowler takes by the batting average of the batsman dismissed. (You can see the results for Test matches here.) This has two effects. The first is that bowlers who dismiss the best batsmen are rewarded for doing so (and bowlers who just pick up tailenders are punished). The second is that, since in an era of low scoring, most batsmen will have proportionally lower averages (and vice versa), the resulting weighted averages will be comparable across eras.

For scorecards where the caught etc. wickets are estimated rather than known, each "fractional wicket" is weighted by the batting average times the fraction. For these scorecards, the bowler will still be rewarded for bowling the best batsmen, but after that, he is simply rewarded for doing well against teams with good batsmen who he might have dismissed.

In each of the tables below, I give the wickets, weighted wickets, runs conceded, the usual average, and then two resulting weighted averages (one being just a scaling of the other). One is with respect to 16,6 (the overall average for the 1800's), and one is with respect to 24,1 (the overall average from 1801 to 2007).

First up, the top bowlers from the 1800's, as ordered by the weighted average. Qualification: 200 wickets.


                                wtd                     wtd avg
name          start end   wkts  wkts    runs    avg   wrt 16,6 wrt 24,1
CTB Turner    1888  1893  610   805,2   7869    12,90  9,77   14,19
E Jones       1896  1899  256   490,0   4789    18,71  9,77   14,19
T Richardson  1892  1899  1455  2183,6  22980   15,79  10,52  15,28
W Rhodes      1898  1899  333   495,0   5311    15,95  10,73  15,58
JT Hearne     1888  1899  1635  2406,6  25986   15,89  10,80  15,68
W Mead        1892  1899  746   1229,8  13500   18,10  10,98  15,94
GA Lohmann    1884  1896  1590  1963,4  21968   13,82  11,19  16,24
AE Trott      1896  1899  453   715,7   8102    17,89  11,32  16,43
G Freeman     1865  1880  288   250,2   2849,2  9,9    11,4   16,5  +/- 0,2%
FR Spofforth  1878  1897  675   807,9   9204    13,64  11,39  16,54
W Attewell    1881  1899  1809  2453,7  27955   15,45  11,39  16,54
AW Mold       1889  1899  1486  1980,2  23044   15,51  11,64  16,89
H Trumble     1890  1899  450   676,9   7883    17,52  11,65  16,91
HF Boyle      1878  1890  259   299,8   3600    13,90  12,01  17,44
A Shaw        1864  1897  1881  1912,0  23108,4 12,29  12,09  17,55 +/- 0,01%
WH Lockwood   1886  1899  902   1277,2  15484   17,17  12,12  17,60
S Haigh       1895  1899  364   552,2   6722    18,47  12,17  17,67
WR Cuttell    1896  1899  324   523,6   6419    19,81  12,26  17,80
J Briggs      1879  1899  1907  2386,0  29384   15,41  12,32  17,88
AW Hallam     1895  1899  207   322,8   3995    19,30  12,37  17,96

Australians take places one and two! It's always good to pass such sanity checks. Charlie Turner and Ernie Jones have remarkably similar weighted averages — they only diverge at the fourth decimal place. (On a slightly more serious note: we would expect that international bowlers come near the top of these tables, since they are Test-class bowlers, and should generally be as good as the best English bowlers.)

It's also good to see that this analysis puts Tom Richardson as the top Englishmen of the 1800's. He was considered the best fast bowler of all time at his peak, which lasted from 1893 to 1897. A young Wilfred Rhodes gets into fourth spot, based on his spectacular first two seasons of county cricket.

The absence of bowlers from the first part of the century is explained by there not being many matches played. Only a handful of players played a decent number of matches (that is, enough to reach 200 wickets), and most of these were batsmen or all-rounders.

That James Broadbridge, James Cobbett et al. are missing from the top of the table is more of a surprise. It seems that, while the round-arm era saw the lowest scoring in first-class history, none of the bowlers really stood out. They had spectacular averages, but so did everyone else at the time. (It is also possible that some of the bowlers would be higher if the wickets were known — perhaps some dismissed more top-order batsmen than the estimations give them). Broadbridge is 49th, with a weighted average of 14,2 with respect to 16,6. Alfred Mynn is a long way down the table (94th; weighted average 15,9 wrt 16,6).

Now let us move on to the comparison for all time. Players whose first-class careers started in the 1800's are in bold.


                                wtd                     wtd avg
name          start end   wkts  wkts    runs    avg   wrt 16,6 wrt 24,1
M Muralidaran 1991  2007  306   556,4   5163    16,87  9,28   13,47
RR Lindwall   1948  1956  218   385,0   3667    16,82  9,53   13,83
RJ Hadlee     1973  1990  780   1327,1  12707   16,29  9,57   13,90
WJ O'Reilly   1934  1938  213   367,9   3584    16,83  9,74   14,14
H Verity      1930  1939  1732  2534,0  24816   14,33  9,79   14,22
J Garner      1977  1986  426   728,4   7428    17,44  10,20  14,81
CTB Turner    1888  1893  610   764,4   7869    12,90  10,29  14,95
WE Bowes      1928  1947  1591  2533,9  26201   16,47  10,34  15,01
MD Marshall   1979  1994  994   1773,1  18369   18,48  10,36  15,04
JB Statham    1950  1968  1999  3007,7  31533   15,77  10,48  15,22
R Appleyard   1950  1958  664   979,1   10309   15,53  10,53  15,29
CEL Ambrose   1988  2000  447   852,8   9301    20,81  10,91  15,83
CV Grimmett   1926  1934  358   587,0   6441    17,99  10,97  15,93
TM Alderman   1981  1989  370   679,7   7477    20,21  11,00  15,97
IR Bishop     1988  1995  225   417,7   4609    20,48  11,03  16,02
WW Armstrong  1902  1921  407   620,5   6880    16,90  11,09  16,10
ST Clarke     1979  1988  591   1009,3  11226   18,99  11,12  16,15
DW Carr       1909  1914  334   499,8   5585    16,72  11,17  16,22
H Larwood     1924  1938  1336  2023,9  22766   17,04  11,25  16,33
G Freeman     1865  1880  288   249,9   2849,2  9,9    11,4   16,6  +/- 0,2%
SF Barnes     1894  1930  461   708,2   8080    17,53  11,41  16,56
S Ramadhin    1950  1965  399   581,2   6662    16,70  11,46  16,64
Waqar Younis  1990  2003  436   801,7   9251    21,22  11,54  16,75
CA Walsh      1984  2000  1013  1839,4  21241   20,97  11,55  16,76
FR Spofforth  1878  1897  675   792,1   9204    13,64  11,62  16,87
H Trumble     1890  1902  587   837,8   9804    16,70  11,70  16,99
HL Jackson    1947  1963  1730  2557,9  30066   17,38  11,75  17,06
CIJ Smith     1930  1939  824   1322,9  15565   18,89  11,77  17,08
GA Faulkner   1907  1924  267   374,7   4423    16,57  11,80  17,14
W Rhodes      1898  1930  3960  5489,5  64836   16,37  11,81  17,15

A good cross-section of eras and countries is represented, with Murali leading the pack. Hedley Verity is, suprisingly enough, the top English bowler, and George Freeman sneaks into the top 20. It is curious that SF Barnes is so far down the list (number 21). When I did this exercise for Test bowlers, he was the best for anyone with 100 wickets.

In amongst international stars, there are a couple of surprising names. Bob Appleyard, who bowled both off-breaks and fast-medium, is the first of these. He may well have become a great of the game had it not been for a terrible run of illness (including being diagnosed with TB) and injury. His rise for Yorkshire was spectacular, as he took 200 wickets in his first full season. But after just nine Tests (in which he took 31 wickets at under 18), his fall was just as spectacular.

Douglas Carr played just one Test for England in 1909. His career ended with the outbreak of war, but it surprises me that he didn't get any more Tests.

Les Jackson was also unlucky, being selected only twice for England. In an era when England had less pacemen, he probably would have played more international cricket.

Big Jim Smith is best remembered for his slogging (he once hit a half-century in eleven minutes), but his numbers suggest that he could have done well as a Test bowler. As it is, he got just five Tests, taking 15 wickets at 26,2. I suppose that if he had played some Tests against Bradman, his figures might not look so good.

Next up: cross-era comparisons of batsmen.

1800's first-class cricket in England: batsmen

2008-02-03T21:22:00.002+01:00

This is Part 5 in my series on cricket in the 1800's in England.

1 - data
2 - classification of matches
3 - filling in the gaps
4 - bowlers
5 - batsmen
6 - bowlers across eras
7 - batsmen across eras
8 - all-rounders (across eras)
9 - wicket-keepers

(Edit: I've fixed a number of typos in the tables. The runs and averages were correct, but somehow the innings and not-outs had got all mixed up.)

This post is nothing statistically special — just a few lists of batsmen. Batting data for first-class cricket is essentially complete, so there's no need for fancy estimation techniques for calculating averages etc.

The leading batting averages for the 1800's come exclusively from players who played near the end of the century, when batting became easier. Qualification (for all tables in this post): 2000 runs.


name            start end   mat  inns  no  runs  avg     wkts   runs  avg     +/- %
RM Poore        1898  1899  26   47    6   2277  55,54   0      13    0,00    0
KS Ranjitsinhji 1893  1899  128  232   28  10411 51,03   67     2340  34,93   0
WG Grace        1865  1899  732  1250  89  46792 40,30   2495   43960 17,62   0
WG Quaife       1894  1899  120  203   39  6122  37,33   55     1792  32,58   0
TW Hayward      1893  1899  191  283   26  9558  37,19   364    7367  20,24   0
CJ Burnup       1895  1899  77   137   12  4633  37,06   22     855   38,86   0
PA Perrin       1896  1899  76   131   13  4336  36,75   2      109   54,50   0
CB Fry          1892  1899  114  209   8   7364  36,64   149    4046  27,15   0 
A Shrewsbury    1875  1899  400  654   66  20837 35,44   0      2     0,00    0
J Darling       1896  1899  67   109   10  3496  35,31   1      38    38,00   0

Robert Poore's pretty lucky that I stopped at 1899. In his last season of cricket before fighting in the Boer War, he scored a triple century and averaged over 90. When he returned to first-class cricket in 1902, he did not return to those heights, and his first-class career finished with an average under 40. Ranjitsinhji, on the other hand, was able to sustain his average, and indeed it finished over 56 (with a Test average in the mid-40's as well).

Joe Darling is the first Australian to make any of these lists. His presence arises from his scores in the Australian tours of England in 1896 and 1899.

Now let's look at the leading batsmen by runs scored.


name            start end   mat  inns  no  runs  avg     wkts   runs  avg     +/- %
WG Grace        1865  1899  732  1250  89  46792 40,30   2495   43960 17,62   0
R Abel          1881  1899  468  743   48  22846 32,87   246    5644  22,94   0
W Gunn          1880  1899  437  716   65  21520 33,06   74     1660  22,43   0
WW Read         1873  1897  450  723   50  21408 31,81   101    3339  33,06   0
A Shrewsbury    1875  1899  400  654   66  20837 35,44   0      2     0       0
G Ulyett        1873  1893  499  862   39  19031 23,12   598    11765 19,67   0
AN Hornby       1867  1899  422  687   41  15752 24,38   7      179   25,57   0
H Jupp          1862  1881  375  686   48  15244 23,89   7      316   45,14   0
W Barnes        1875  1894  421  666   54  14108 23,05   803    13935 17,35   0
AE Stoddart     1885  1899  255  449   54  13799 31,72   206    5553  26,96   0

We see that WG Grace, as well as taking comfortably more wickets than anyone else, scored more than twice as many first-class runs in England than any other batsman in the 1800's! Of course, he did play many more matches (second on that list is George Ulyett with 499), but you can see how he was such a giant of the game, and why he remains famous to this day.

Henry Jupp, who represented England in the first two Test matches, is the only player in that table to have ended his career before 1890. Here are some other players to have passed 2000 runs, ordered by the start season of their careers:


name             start end   mat inns  no  runs  avg     wkts   runs    avg   +/- %
Lord F Beauclerk 1801  1825  94  172   14  4319  27,34   406,4  5106,9  12,6  10
W Lambert        1801  1817  62  112   5   2961  27,67   318,1  3960,3  12,5  10
W Beldham        1801  1821  69  127   7   2265  18,88   96,0   1193,0  12,4  10
R Robinson       1801  1819  57  111   9   2039  19,99   34,7   802,4   23,1  10
EH Budd          1803  1831  68  119   9   2597  23,61   285,8  4200,8  14,7  10
W Ward           1810  1845  116 210   21  3517  18,61   73,0   1511,4  20,7  10
J Broadbridge    1814  1840  90  163   21  2368  16,68   405,6  3699,7  9,1   9,9
F Pilch          1820  1854  213 389   30  6797  18,93   169,5  1666,3  9,8   9,4
EG Wenman        1825  1854  135 241   15  3088  13,66   62,2   485,7   7,8   10
FW Lillywhite    1825  1851  220 390   84  2203  7,20    1599,8 14181,1 8,9   8,5

Beauclerk and Lambert we met in Part 4. Billy Beldham (whose photograph you can see here) perhaps played his best cricket in the late 18th century, in matches that are not classified as first-class. He is credited with being a founder of what might be called proto-modern batting technique. Batting has evolved a long way since then! While Beldham and his contemporaries stepped forward to meet the ball, it would take about another half-century before batsmen played attacking strokes off either the front foot or the back, and strokes off the pads were developed by Ranji late close to 1900.

Robert Robinson is a rather anonymous figure as far as the Internet is concerned. I haven't managed to find much out about him, despite him having been one of the leading batsmen of the day. He also played in the 18th century, and scored a century in his first important match for which CricketArchive has a full scorecard (Kent v Hampshire, 1792).

Next up: Adjusting bowling averages for era and the quality of wickets taken.

Falls of wicket

2008-01-31T22:36:00.000+01:00

This is a follow-up to my post on the "fow-average" of openers. Soulberry wanted to see what the average fall of wicket was for non-openers. To make things easier for me, I've split it up by position.

Note that I found a bug in my earlier code, so the list of the top openers has shuffled around a little, though Russel Arnold remains on top. In the tables below I give the number of innings (less any not-outs which didn't see a wicket fall) and the fow-average. Note that for the non-openers, I've subtracted off the wicket that the batsman came in at. Qualification: 15 innings. (Edit: Richie Richardson's figures below are wrong, and Michael Clarke's might be as well. My lazy code got them mixed up with Viv Richards and Stuart Clark.)


opener                            number 3    
Russel Arnold           15  3,22  George Headley          32  3,71
Raman Subba Row         16  2,97  George Gunn             16  2,94
Ravi Shastri            26  2,95  Allan Border            37  2,92
Bill Woodfull           43  2,90  Wally Hammond           50  2,91
Glenn  Turner           66  2,87  Richie Richardson       174 2,89
Bruce Mitchell          48  2,78  Alvin Kallicharran      25  2,84
Arthur Shrewsbury       18  2,78  Rahul Dravid            143 2,84
Jackie McGlew           58  2,74  Ken Barrington          40  2,78
Dennis Amiss            69  2,66  Eric Rowan              20  2,71
Chris Tavaré            33  2,63  Lindsay Hassett         19  2,68

number 4                          number 5    
Dean Jones              18  3,20  Michael Clarke          31  3,12
Rahul Dravid            18  3,11  Jimmy Adams             29  3,04
Rajin Saleh             23  2,91  Shivnarine Chanderpaul  80  2,97
Jacques Kallis          96  2,84  Kevin Pietersen         25  2,84
Geoff Howarth           17  2,76  Dilip Vengsarkar        30  2,77
Vijay Hazare            35  2,69  Yashpal Sharma          29  2,70
Brian Hastings          29  2,58  Steve Waugh             138 2,69
Monty Noble             23  2,57  Andy Flower             80  2,69
Herbie Taylor           23  2,57  Ken Viljoen             16  2,69
Richie Richardson       19  2,53  John Crawley            15  2,62

number 6    
Joe Solomon             16  3,41
Trevor Bailey           38  3,37
Imran Khan              22  3,23
Nawab of Pataudi        21  2,86
Hashan Tillakaratne     72  2,86
Jimmy Adams             31  2,81
Shivnarine Chanderpaul  39  2,75
Allan Border            58  2,74
Les Ames                17  2,68
Dattu Phadkar           20  2,68

There's a rather conspicuous absentee amongst the number threes. I went through Bradman's innings, and when he made a big score he was often part of a large partnership, so that he actually didn't see too many wickets fall. His fow-average at number three is 2,45.

Samir also wanted to see average team runs scored while the batsman is at the crease. I did mean to calculate this, but I forgot until after I'd made my spreadsheets. I think the above tables are interesting enough as it is though. There are several players that you would expect to have "held the innings together" often.

1800's first-class cricket in England: bowlers

2008-01-29T18:07:00.002+01:00

This is Part 4 in my series on 1800's cricket in England.

1 - data
2 - classification of matches
3 - filling in the gaps
4 - bowlers
5 - batsmen
6 - bowlers across eras
7 - batsmen across eras
8 - all-rounders (across eras)
9 - wicket-keepers

(Edit: My code at first counted "absent" as a nought not out. This has been fixed. All it does is decrease of new innings and not-out tallies.)

In this post I apply the method detailed in Part 3 to all first-class scorecards with missing data. But first I have to make a small confession — the method I've used is surely not the best one. The scorecards with missing data come in (mostly) two types. The earliest scorecards only credit bowlers with bowled dismissals, and do not record the runs conceded by bowlers (this is a typical example). Later scorecards give full credit to bowlers for their dismissals, but don't record the runs conceded (this is a typical example). There are also five matches where the runs conceded are recorded but bowlers aren't given credit for catches, etc.

The method in Part 3 dealt only with the first type of scorecard. With the second type of scorecard, you should be able to get better estimates of the bowling averages, since you have more data (namely, how many wickets each bowler took). But when I tried to apply a similar method to these scorecards (finding the average percentage of team runs conceded by bowlers who took 1 wicket, bowlers who took 2 wickets, etc.), I got results that were biased in favour of regular wicket-takers. The top 18 wicket-takers in the test dataset had estimates of bowling averages that were too low, with the errors ranging from 0,2% to almost 23%. The (justified) fudge factor used in the previous method makes the estimates even lower!

I don't know (yet?) how to fix this. There must surely be a better, more sophisticated model to estimate runs conceded — you shouldn't get worse results with more data! But since that's what's happening for me, I've instead ignored all the non-bowled dismissals for these scorecards, and applied the method used on the early scorecards. I've then scaled up the estimated runs conceded and estimated wickets so that the wicket tally matches reality.

So, onto the results! In the various tables that follow, I give the start and end years of the career, matches (these may not agree with the usual sources, since I exclude matches that weren't eleven-a-side), wickets, runs conceded, bowling average, +/- %; and then batting stats (for which we have complete data): innings, not-outs, runs, average.

Note 1: If there is a decimal comma in the wickets tally, then it is almost certainly an underestimate. How big an underestimate I don't know. In my test dataset, one bowler's estimated wicket tally was 47% below what it should have been. Despite this, the estimate of the average was only out by just over 7%. For other bowlers, the wickets estimate was within 2% of reality. The lesson here is not to rely on my wicket estimates.

Note 2: One of the columns is called +/- %. About 80% of the estimated averages should fall inside the estimated averages, plus or minus the given percent. If the bowler only ever had bowleds credited to him, this value is 10%.

The first table gives the leading bowlers of the 1800's in England by bowling average. Qualification (for this table and all that follow): 200 wickets.


name          start end   mat wkts    runs    avg   +/- %   inns  no  runs  avg
J Cobbett     1826  1841  94  556,3   4598,7  8,3   9,7     162   16  1437  9,84
FW Lillywhite 1825  1851  220 1599,8  14181,1 8,9   8,5     390   84  2203  7,20
S Redgate     1830  1846  74  414,0   3775,2  9,1   8,0     133   23  957   8,70
J Broadbridge 1814  1840  90  405,6   3699,7  9,1   9,9     163   21  2368  16,68
J Bayley      1822  1850  81  358,7   3500,5  9,8   9,3     140   17  905   7,36
G Freeman     1865  1880  44  288     2849,2  9,9   0,2     70    3   918   13,70
WR Hillyer    1835  1853  216 1407,3  14061,5 10,0  7,1     386   62  2544  7,85
J Wisden      1845  1863  175 1036,5  10356,9 10,0  3,4     305   29  4020  14,57
T Nixon       1841  1859  50  250     2503,5  10,0  5,0     83    17  300   4,55
A Mynn        1832  1859  200 1059,9  10940,1 10,3  7,0     372   24  4749  13,65

Note that this doesn't mean that James Cobbett had the lowest average of the 1800's — if the estimate was particularly bad, it might be up around 10. This would still be one of the lowest ever, of course. Cobbett was a round-arm spin bowler.

Second on the table is William Lillywhite, a medium-pace round-arm bowler. His wicket tally is enormous.

Third is Samuel Redgate, a fast bowler who we can thank for batting pads, along with Alfred Mynn (tenth on the table). These two were the fastest bowlers of their day, but Mynn was also a pretty good batsman. They squared off against each other in the North v South game of 1836. Mynn had hurt his ankle before play started, but nevertheless batted at 5 in South's second innings. Redgate repeatedly hit Mynn on his unprotected legs, damaging them to the point where amputation was considered. In what must be one of the most courageous innings of all-time, Mynn struck an unbeaten century (the only century of his first-class career), before being sent to London for medical treatment. After this, batsmen started wearing leg guards. You can read about this innings in more detail here.

James Broadbridge comes in fifth. This average-estimating exercise is particularly useful for the Sussex round-armer — in the standard sources his average is given as 18,62. This very wrong figure is based on just 14 of his career wickets, which total over 400!

The ninth player in the table above is Thomas Nixon, a round-arm slow bowler whose first-class career comprised mostly matches for the MCC. You'll note that the +/- % figure is given as 5,0; this means that roughly half of his runs conceded came in matches where this was recorded. This gives us a useful check: we know that his average in these matches was 10,12. Since the estimated average is 10,0, it looks like the estimate is pretty good.

For what it's worth, the next table shows the leading bowlers by wickets taken. Since the amount of first-class cricket increased over the course of the 19th century, the top of the list is dominated by people who played close to 1900.


name          start end   mat wkts    runs    avg   +/- %   inns  no  runs  avg
WG Grace      1865  1899  732 2495    43960   17,62 0       1250  89  46792 40,30
J Briggs      1879  1899  446 1907    29384   15,41 0       686   44  11593 18,06
A Shaw        1864  1897  377 1881    23108,4 12,29 0,01    582   92  6244  12,74
W Attewell    1881  1899  399 1809    27955   15,45 0       600   60  7577  14,03
J Southerton  1854  1879  282 1674    24171   14,44 0       474   128 3136  9,06
JT Hearne     1888  1899  258 1635    25986   15,89 0       390   118 3029  11,14
R Peel        1882  1899  397 1606    25233   15,71 0       630   56  10837 18,88
FW Lillywhite 1825  1851  220 1599,8  14181,1 8,86  8,5     390   84  2203  7,20
GA Lohmann    1884  1896  256 1590    21968   13,82 0       371   36  6495  19,39
T Emmett      1866  1888  405 1493    20081   13,45 0       664   87  8641  14,98

WG rather stands out in this list. Not only did he take more than 500 more first-class wickets than anyone else in England in the 1800's, but he did it while averaging over 40 with the bat.

Lillywhite's wickets estimate is almost certainly low, and he should be at least one rank higher. He might deserve to he higher still, but we can't know for sure.

To have a look at some more early bowlers, here's a table with players ordered by the starting year of their careers.


name             start end   mat wkts    runs    avg   +/- %   inns  no  runs  avg
Lord F Beauclerk 1801  1825  94  406,4   5106,9  12,6  10      172   14  4319  27,34
W Lambert        1801  1817  62  318,1   3960,3  12,5  10      112   5   2961  27,67
J Wells          1801  1815  44  271,1   3090,2  11,4  10      85    9   615   8,09
TC Howard        1803  1828  81  462,3   5712,4  12,4  10      149   16  1454  10,93
EH Budd          1803  1831  68  285,8   4200,8  14,7  10      119   9   2597  23,61
W Ashby          1808  1830  37  209,5   2236,8  10,7  10      64    21  213   4,95
J Broadbridge    1814  1840  90  405,6   3699,7  9,1   9,9     163   21  2368  16,68
J Bayley         1822  1850  81  358,7   3500,5  9,8   9,3     140   17  905   7,36
FW Lillywhite    1825  1851  220 1599,8  14181,1 8,9   8,5     390   84  2203  7,20
W Clarke         1826  1855  129 714,1   7588,7  10,6  5,2     220   35  1966  10,63

William Lambert was, along with Beauclerk, one of the stand-out all-rounders of the early 19th century. These two have similar averages, both for batting and bowling. The bowling average of around 12,5 is about typical for the era, which was very low-scoring. That should put a batting average of over 27 into some perspective. Lambert was, however, banned for life for match-fixing.

Lord Frederick Beauclerk is perhaps my favourite character in cricket history. Not only was he a Lord, a title sadly absent from modern English cricketers, but he was the golden boy of the first part of the 19th century (see his picture here). Not only was he an outstanding all-rounder, but he embodied the spirit of cricket so lacking in today's players. A clergyman, he claimed to make £600 a year from betting on cricket. He was unassuming when batting — (according to his Wikipedia article at least) he used to place an expensive watch on the middle stump. He was a "foul-mouthed, dishonest man who was one of the most hated figures in society ... he bought and sold matches as though they were lots at an auction".

You may have noticed that, along with the leading wicket-takers being from near 1900, the leading averages are mostly from around the second quarter of the century. Adjusting the bowling averages for era will be the subject of Part 6. A suivre !

If your favourite 19th century bowler with missing data has been omitted from the tables above, you can find him in the table below, which lists all bowlers whose averages needed some estimating. They are ordered by the starting year of their first-class careers.


name               start end   mat wkts    runs    avg   +/- %   inns  no  runs  avg
Lord F Beauclerk   1801  1825  94  406,4   5106,9  12,6  10      172   14  4319  27,34
W Lambert          1801  1817  62  318,1   3960,3  12,5  10      112   5   2961  27,67
J Wells            1801  1815  44  271,1   3090,2  11,4  10      85    9   615   8,09
TC Howard          1803  1828  81  462,3   5712,4  12,4  10      149   16  1454  10,93
EH Budd            1803  1831  68  285,8   4200,8  14,7  10      119   9   2597  23,61
W Ashby            1808  1830  37  209,5   2236,8  10,7  10      64    21  213   4,95
J Broadbridge      1814  1840  90  405,6   3699,7  9,1   9,9     163   21  2368  16,68
J Bayley           1822  1850  81  358,7   3500,5  9,8   9,3     140   17  905   7,36
FW Lillywhite      1825  1851  220 1599,8  14181,1 8,9   8,5     390   84  2203  7,20
W Clarke           1826  1855  129 714,1   7588,7  10,6  5,2     220   35  1966  10,63
J Cobbett          1826  1841  94  556,3   4598,7  8,3   9,7     162   16  1437  9,84
T Barker           1826  1845  70  241,0   2543,2  10,6  9,0     128   12  1236  10,66
S Redgate          1830  1846  74  414,0   3775,2  9,1   8,0     133   23  957   8,70
FH Hervey-Bathurst 1831  1861  83  310,7   3676,5  11,8  7,5     142   19  755   6,14
A Mynn             1832  1859  200 1059,9  10940,1 10,3  7,0     372   24  4749  13,65
WR Hillyer         1835  1853  216 1407,3  14061,5 10,0  7,1     386   62  2544  7,85
J Dean             1835  1861  296 1118,8  13358,0 11,9  4,9     533   63  4794  10,20
CG Taylor          1836  1859  122 292,0   3281,1  11,2  7,0     222   11  3020  14,31
W Martingell       1839  1860  170 516,3   5722,1  11,1  3,5     290   45  2258  9,22
T Nixon            1841  1859  50  250     2503,5  10,0  5,0     83    17  300   4,55
D Day              1842  1852  41  204,2   2253,5  11,0  6,4     71    14  352   6,18
J Wisden           1845  1863  175 1036,5  10356,9 10,0  3,4     305   29  4020  14,57
T Sherman          1846  1870  78  322     3986,8  12,4  3,6     133   32  704   6,97
RC Tinley          1847  1874  113 287     4239,1  14,8  0,5     191   23  1890  11,25
J Lillywhite       1848  1873  178 223     2573,4  11,5  0,4     312   26  5084  17,78
W Caffyn           1849  1873  180 564     7654,1  13,6  0,3     314   20  5405  18,38
E Willsher         1850  1875  247 1209    15600,8 12,9  0,3     435   60  4699  12,53
J Grundy           1850  1869  282 1063    13202,8 12,4  1,9     477   37  5600  12,73
D Buchanan         1850  1881  56  359     5552,6  15,5  1,0     96    34  224   3,61
T Sewell           1851  1868  149 315     6161,4  19,6  0,1     250   51  2422  12,17
FP Miller          1851  1868  134 253     5129,4  20,3  0,5     230   20  3053  14,54
T Hayward          1854  1872  108 237     3890,9  16,4  0,6     182   11  4487  26,24
FR Reynolds        1854  1874  65  208     3530,6  17,0  1,4     106   26  444   5,55
J Jackson          1855  1867  107 613     7132,8  11,6  0,1     176   30  1821  12,47
VE Walker          1856  1877  135 328     5039,3  15,4  0,9     213   31  3186  17,51
T Hearne           1857  1876  165 287     4120,0  14,4  0,4     277   19  4807  18,63
GF Tarrant         1860  1869  63  365     4539,6  12,4  0,4     106   8   1467  14,97
G Wootton          1861  1873  175 904     12080,3 13,4  0,2     282   61  2343  10,60
RD Walker          1861  1877  113 318     5468,0  17,2  0,5     186   7   3521  19,67
ID Walker          1862  1884  269 208     4634,8  22,3  0,2     466   39  10470 24,52
A Shaw             1864  1897  377 1881    23108,4 12,3  0,0     582   92  6244  12,74
G Freeman          1865  1880  44  288     2849,2  9,9   0,2     70    3   918   13,70
F Morley           1871  1883  212 1184    15748,8 13,3  0,0     324   84  1292  5,38
A Hill             1871  1883  188 722     10392,8 14,4  0,0     303   33  2346  8,69
CT Studd           1879  1884  85  426     7427,5  17,4  0,2     145   23  3928  32,20

1800's first-class cricket in England: filling in the gaps

2008-01-28T21:01:00.001+01:00

This is Part 3 in my series on first-class cricket in England in the 1800's.

1 - data
2 - classification of matches
3 - filling in the gaps
4 - bowlers
5 - batsmen
6 - bowlers across eras
7 - batsmen across eras
8 - all-rounders (across eras)
9 - wicket-keepers

In this post I detail a method of filling in all the gaps in those early scorecards. By doing so, we can get realistic estimates of bowling averages, despite only knowing about bowled dismissals and team totals. This will mostly be a geek interest post. Though the maths isn't technically hard (it's really just the four basic arithmetic operators), it does go on for a bit.

To begin, let's recall what the important gaps in the early scorecards are. First, bowlers were only credited with wickets when they bowl a batsman — catches, LBW's, stumpings, and hit wickets were not counted in bowler's wicket tallies. Second, the number of runs conceded by bowlers was not recorded.

To fill in these gaps, I took a set of scorecards (as old as possible, to try to match the characteristics of the earlier eras) which do contain the relevant information. For each card, I broke the dismissals down into three types:

A. bowled
B. other wicket credited to the bowler (catches, etc.)
C. wicket not credited to the bowler (run outs, etc.) or not-outs.

For each bowler who took 1 wicket bowled, I counted how many other wickets he took, out of the possible remaining (ie, type B above). Similarly for each bowler who took 2 wickets bowled, 3 wickets bowled, and so on.

If you do this for all the scorecards in the sample and add up the corresponding numbers, you can get the probability that a batsman dismissed by a type B wicket was dismissed by a bowler who took 1 wicket bowled, or by a bowler who took 2 wickets bowled, etc.

Put another way: you can get the average fraction of type B wickets taken by a bowler who took 1 wicket bowled, or 2 wickets bowled, etc.

The actual numbers (based on matches with the relevant data until part-way through 1863) are as follows:


wkts bowled       1      2      3      4      5      6      7
frac other wkts   0,300  0,363  0,417  0,432  0,423  0,461  0,525

(Tthe last value here was adjusted by hand, based on later matches.) In this particular dataset, there was never a player who took 8 wickets or more in an innings bowled; I set the fractions for 8 and 9 wickets mildly arbitrarily at 0,5 (based on the equivalent numbers for later matches).

Now comes the estimate of the wicket tally. Suppose in a scorecard that Smith took 1 wicket bowled, and Jones took 3 wickets bowled. There are four catches with bowler unknown, and there was one run out.

There are four type B wickets, and Smith gets 4*0,302 = 1,208 of them, giving him 2,208 for the innings. Jones gets 4*0,428 = 1,712, giving him 5,712 for the innings.

Of course, that means that the total wickets don't add up to 10. If a bowler only took wickets caught, then he's going to be ignored by this analysis. This means that the estimated wicket tallies will be significantly lower than what they really were. But bowlers who didn't get any wickets bowled will also not have any runs conceded estimated for them, as we will see shortly. We will hope that, by ignoring both wickets and runs conceded in these situations, the bowling averages over a career will be largely unaffected.

(It is also possible, if three bowlers each took 3 wickets for instance, that the estimated wicket tally for an innings could be greater than 10. This isn't a serious problem.)

To estimate the runs conceded by each bowler, I followed a similar procedure to that for type B wickets, finding the average fraction of runs (ignoring byes etc.) that bowlers who took 1 wicket conceded, bowlers who took 2 wickets conceded, and so on. The resulting table looks like this (the wickets now are total wickets, caught, bowled, the lot):


wkts 1      2      3      4      5      6      7      8      9      10
frac 0,164  0,223  0,277  0,322  0,359  0,368  0,405  0,401  0,424  0,5

(The last entry in that table was adjusted by hand, based on the corresponding number for later matches.)

This tells us that, for instance, a bowler who took 4 wickets, on average, conceded 32,2% of the batting team's runs in an innings.

So, for each scorecard, we estimate the number of wickets taken by each bowler, and then use this tally and the second table to estimate the number of runs conceded (based on the batting team's score). We now have wickets and runs, so we can calculate an average!

But there's a rather large assumption in this model, and that is that the characteristics of wicket-taking and conceding runs don't change much. This is definitely not true in general: by taking a sample of matches from later, the fractions in the first table all decrease (suggesting that more bowlers were used in the latter part of the 19th century than in the 1850's). This could cause a systematic error in the estimates. To fudge my way around this, I take the overall bowling average (which we know from the team totals and the total number of wickets lost) and compare it to the overall estimated bowling average. The estimated bowling averages are scaled up or down according to the ratio of the overall average to its estimate. If that's not clear, I'll come to an example shortly.

Before we dive in and start estimating averages from 1812, it would be prudent to check to see if the method actually works. I took a set of about 950 matches from 1888 to 1896 (well after the dataset I used to generate the fractions above), and pretended that I didn't have data on type B wickets or runs conceded. I do the estimates, and then compare the averages with the actual averages, which can be calculated exactly (since there's no missing information).

When I did this (before implementing the fudge factor), there was a clear systematic error: the estimates of the averages were almost always lower than the real averages. According to the estimates, the overall average was 15,07. In reality it was 18,23. So I multiplied all of the estimated averages by 18,23/15,07 = 1,21.

Here are the results, with players ordered by wickets taken (in real life). Note that these are not career figures — they are solely based on the sample of about 950 matches. The headings are estimated and actual.


                   wkts         runs           avg
name          mat  est    act   est     act    est    act    % error
J Briggs      198  754,4  1172  9243,2  15930  15,16  13,59  +11,5
R Peel        237  744,3  1158  9830,5  17281  16,34  14,92  +9,5
AW Mold       166  1063,1 1107  10992,9 15884  12,79  14,35  -10,9
W Attewell    214  786,4  1087  10714,9 15960  16,86  14,68  +14,8
GA Lohmann    147  766,1  1011  8168,1  13227  13,19  13,08  +0,8
JT Hearne     149  887,9  956   11805,6 14476  16,45  15,14  +8,6
F Martin      199  773,4  950   10852,5 15128  17,36  15,92  +9,0
T Richardson  97   727,2  765   8332,7  10647  14,17  13,92  +1,8
E Wainwright  199  666,6  730   8538,3  11870  15,85  16,26  -2,5
SMJ Woods     156  617,6  729   9084,8  13795  18,20  18,92  -3,8
WH Lockwood   156  561,1  618   7358,1  10067  16,22  16,29  -0,4
JJ Ferris     163  402,0  616   5895,8  11155  18,14  18,11  +0,2
CTB Turner    90   539,2  585   5830,0  7607   13,38  13,00  +2,9
W Wright      149  498,9  577   6835,2  10637  16,95  18,44  -8,1
EJ Tyler      95   274,2  522   4534,0  9947   20,45  19,06  +7,3
JT Rawlin     116  431,2  487   6345,1  8806   18,20  18,08  +0,7
FG Roberts    127  372,5  458   6046,0  9627   20,08  21,02  -4,5
W Flowers     179  336,7  447   4739,3  8006   17,41  17,91  -2,8
WA Humphreys  127  313,1  445   6196,0  9148   24,48  20,56  +19,1
GH Hirst      107  353,5  418   4685,3  7171   16,40  17,16  -4,5
A Hearne      172  358,5  399   5587,0  7641   19,28  19,15  +0,7
FW Tate       103  328,8  362   5409,6  7836   20,35  21,65  -6,0
FS Jackson    122  288,2  359   4412,3  6571   18,94  18,30  +3,5
WG Grace      220  206,8  358   4062,1  8022   24,29  22,41  +8,4
W Mead        53   254,7  351   3705,0  5605   17,99  15,97  +12,7
FJ Shacklock  100  306,4  349   4053,6  6615   16,37  18,95  -13,6
A Watson      87   328,4  332   3663,2  4928   13,80  14,84  -7,0
JW Sharpe     75   312,1  321   3647,3  4922   14,45  15,33  -5,7
AD Pougher    72   223,5  312   3279,3  5260   18,15  16,86  +7,7
GA Davidson   74   273,6  309   3793,4  5241   17,15  16,96  +1,1

It's not spectacular, but it's pretty good considering the paucity of the data that went into the estimates. Of the top 30 wicket-takers in the sample, only 6 have estimates of the bowling average wrong by more then 10%. And while I've truncated the table at 30 entries here, the good estimates keep going for another 30odd players. The first really wild estimate is for Stephen Whitehead, who took 121 wickets (in the dataset) at an actual average of 21,39, but at an estimated average of 14,95.

It is unfortunate, though understandable, that three of those six entries with errors of over 10% are caused by the top four wicket-takers. The model used for the estimates was based on overall averages, and we would not expect that the best bowlers would follow the same trends, in general.

I repeated this exercise for a similarly-sized dataset containing matches from between 1877 and 1888. The results were similar to those above — again 6 errors of more than 10% in the top 30 players, including the third- and fourth-highest wicket-takers. But further down the table the results are better, perhaps because the era in question is closer to that used to generate the parameters in the model. The first wild estimate was for a bowler who took only 71 wickets.

While I'm emphasising the uncertainties in the estimates for the top bowlers, the estimates are still pretty useful. Suppose that you knew that a modern-day Test bowler had an average between 17 and 23 (that is, 20 plus or minus 15%). He could be one of the greatest of all-time or merely very good. But you know that he's at least very good, and he's not someone like Brett Lee, taking plenty of wickets (until recently), but at an average of 30.

Now we're almost ready to do the estimates for the first half of the 19th century!

1800's first-class cricket in England: classification of matches

2008-01-26T18:48:00.002+01:00

This is Part 2 of my series on first-class cricket in England in the 1800's.

1 - data
2 - classification of matches
3 - filling in the gaps
4 - bowlers
5 - batsmen
6 - bowlers across eras
7 - batsmen across eras
8 - all-rounders (across eras)
9 - wicket-keepers

I think that if the match isn't played between two sides of eleven, then it is not first-class. Unfortunately (for people who share this opinion of mine), this principle was not obeyed when drawing up the list of first-class matches that we have today. There were 149 matches played in the 1800's, classified as first-class at CricketArchive, in which one or both teams had more than eleven men.

While some people might want a little flexibility on the size of the teams (at least for the early days), surely no-one can seriously suggest that a match between a Gentlemen XVIII and a Players XI should be classified as first-class, no matter how amusingly long the Gentlemen's batting card looks.

Also on the first-class record are two Gentlemen XVII v Players XI matches (1, 2), seven matches of XVI v XI, three of XV v XI, eighteen of XIV v XI, eight of XIII v XI, three of XII v XI, and 107 twelve-a-side matches.

There are also seven matches (1, 2, 3, 4, 5, 6, 7) classified as first-class in which one team played with eleven men and one team with less. Of these, three were odds games (one by Players against Gentlemen; two by the Australians in their 1880 tour), two were caused by player injuries, and two are unexplained by the CricketArchive scorecards. The most amusing of these is the last one, Hampshire v Somerset in 1885. The CricketArchive page simply says, "Somerset only brought nine men ...". One of the Somerset players in that match was EW Bastard. It is perhaps fortunate that India did not tour England during his brief first-class career.

Since I don't believe that these any of these matches should count as first-class, I will ignore them for my statistics.

Note that while first-class matches should be XI v XI, full substitutes are permitted. These have always been pretty rare, but are still seen in modern times — a full substitute is permitted when a player gets called up to or released from England duty during a county game. The most recent example in Australia that I know of is Brad Williams, who was replaced by Ben Edmondson during a match in 2003/4.

I do not, however, think that, in the absence of a particular player, another can bat twice. This is what happened in Hampshire v Nottingham in 1843. One of the Notts players was injured, and so Francis Noyes was allowed to bat twice in each innings. I will ignore this match for my records as well.

Openers and falls of wicket

2008-01-25T09:09:00.000+01:00

Samir Chopra asked me a question about openers: what is the average wicket that they're dismissed at? For example, suppose an opener is the first wicket to fall in one innings, the second in another, and the first again in a third innings. His fow-average would be 1,33. (I can't think of a better name for this; it's not really the fow, since that refers to the runs the team has scored when the wicket falls.)

You'd expect that a player would do well on this statistic if they bat slowly or if they're a good batsman in a bad top order.

There's a tricky question here about what to do with not-outs. The way I treated them is as follows.

Suppose the batsman was not out, with the team n wickets down. If he'd never been not out at so many wickets down, I assigned him n+1 for that innings. In particular, this means that an opener who carries his bat gets a "score" of 11.

If the batsman had lasted longer than n wickets, then I replaced the not-out with his fow-average for all the times he lasted longer. An example:

A batsman is dismissed at wickets: 1, 1, 3, 5, 9.
A batsman is not out with the team have lost: 3, 6 wickets.

The "6 not out" is replaced by a 9. Now the two rows of data look like:
FOW's: 1, 1, 3, 5, 9, 9
nots-outs: 3

The 3 is now replaced by (5 + 9 + 9)/3 = 7,67.

So, the opener's fow-average is (1 + 1 + 3 + 5 + 7,67 + 9 + 9) / 7 = 5,1.

Right! With that out of the way, here are the openers with the highest fow-averages, the lowest, and some selected examples in between the two extremes. Qualification of 15 innings. (Edit: The original version of this table had some errors. These have been fixed.)


name              inns  fow avg
Russel Arnold     15    3,22
Raman Subba Row   16    2,97
Ravi Shastri      26    2,95
Bill Woodfull     43    2,90
Glenn Turner      66    2,87
Bruce Mitchell    48    2,78
Arthur Shrewsbury 18    2,78
Jackie McGlew     58    2,74
Dennis Amiss      69    2,66
Chris Tavaré      33    2,63
Jack Robertson    15    2,60
Billy Zulch       28    2,57
Geoff Boycott     188   2,56
Desmond Haynes    191   2,54
Alec Bannerman    46    2,53
----
John Wright       145   2,29
Mark Taylor       186   2,27
Mike Atherton     197   2,25
Graham Gooch      184   2,18
Matthew Hayden    164   2,18
Herbert Sutcliffe 83    2,09
Jack Hobbs        97    1,97
Gordon Greenidge  183   1,94
Justin Langer     113   1,89
Michael Slater    131   1,85
Trevor Franklin   37    1,68
----
JJ Lyons          16    1,50
William Shalders  18    1,50
George Ulyett     15    1,47
Bob Catterall     18    1,44
Mushtaw Ali       16    1,44
Boeta Dippenaar   18    1,39
Syed Abid Ali     21    1,38
Bruce Pairaudeau  16    1,38
Alan Turner       26    1,35
Saleem Elahi      19    1,21

I would have set the qualification at 20 innings, but I think that Russel Arnold deserves a moment in the sun. He started his Test career as an opener, and really did nothing wrong. Indeed, he averages over 50 as an opener (where he scored all three of his Test centuries), compared to under 30 overall. He carried his bat once in a low-scoring draw against Zimbabwe. But those muppets headed by a joker decided that Atapattu was a better opener instead. And he did all right, of course, six Test double-centuries.

Anyway, make what you will of the list above. It's a bit of a mixed bag.

1800's first-class cricket in England: the data

2008-01-24T10:27:00.001+01:00

This is Part 1 in a series of posts analysing first-class cricket in England in the 1800's. The long-term goal is to compare first-class cricketers (in England) from all eras.

1 - data
2 - classification of matches
3 - filling in the gaps
4 - bowlers
5 - batsmen
6 - bowlers across eras
7 - batsmen across eras
8 - all-rounders (across eras)
9 - wicket-keepers

But before we can start calculating averages and so forth, we run into the problem of missing data. The CricketArchive website has the most comprehensive scorecard database on the Internet, but there are some gaps, of varying importance.

- One match (Kent v Sussex, 1829) has only a result — no record of which individuals played, what they scored, or even what the teams scored.

- Four matches (1, 2, 3, 4) contain only team scores, and no individual player details. The last three of these scorecards involve only Cambridge teams.

- Four matches (1, 2, 3, 4) lack the names of players who did not bat. The second of these matches was a Gentlemen v Players game (from 1845).

- There is one further match, as late as 1877 (here), in which one player who batted is unknown. It is known that the player was a full replacement, and that he scored 7 not out, but who he was is a mystery.

- One match (here) does not contain the dismissals in the fourth innings.

While these gaps are mildly annoying, their overall effect is not serious — they are only 11 matches out of almost 4500 that were played in England in the 1800's.

More serious are gaps resulting from changes in scoring style. This concerns only the bowlers — the batting scores are complete, apart from the examples listed above.

The most serious problem is that, for a long time, catches were credited to the fieldsman but not to the bowler. Only bowled dismissals counted towards a bowler's wicket tally. The earliest match where bowlers did get credit for catches was in 1836, and it was only from the 1838 season that it became common practice. It was not always the case, however. Even in 1847 there was a match where bowlers did not get credit for catches.

Making calculation of bowling averages even more difficult is that runs conceded by bowlers were not regularly recorded until about 1854. For the next decade or so, about 8% of matches contain gaps of this sort. After 1867, these scores are almost always recorded, but there is still a trickle of gaps, with the last gaps appearing in a match in 1882.

Recording the number of overs bowled follows a very similar pattern to that of runs conceded, but there are 50 matches, mostly from the early 1840's, in which overs bowled were recorded but not runs conceded.

The plan, then, is to try to fill in the gaps with estimates. I'll start by making estimates of wickets taken, and then do likewise for runs conceded.

No no-balls?

2008-01-17T22:10:00.001+01:00

Talking about the poor over-rates in the current Australia-India Test, Sambit Bal suggests run penalties (a move I strongly disagree with), giving as justification: "See how no-balls have become scarce in Twenty20 after they introduced the free hit."

There have been 50 T20I's, and in these matches, the average rate of no-balls has been 2,45 per 300 balls. In all ODI's, the average rate is 2,94 per 300 balls. So it does appear that the threat of a free-hit is causing at least some bowlers to stop pushing the popping crease. (Note that those figures aren't just front-foot no-balls, but also include illegal bouncers and so on.)

A more detailed look at no-ball rates in ODI's is revealing, however. Here is a graph showing a 49-match moving average no-ball rate (per 300 balls).

(Every match classified by the ICC as an ODI is included, even the silly Asia v Africa games, etc. Some of the spike around February 2007 is caused by the associate nations, whose bowlers lacked some front-foot discipline in their lead-up tournaments to the World Cup.)

A dramatic dip started about a month before the World Cup, and now we're at the lowest level of no-balling in ODI history — it's lower than the rate in T20I's. Is it just a random blip that will right itself in the next year or two, or is it something else? I'd like to think that, as bowlers started becoming more conservative with the position of their front feet (from playing T20 matches), they decided that any small advantage gained from getting really close to the popping crease is outweighed by the risk of a no-ball.

In Test matches, the effect is not so dramatic, but we do seem to be close to a minimum for the front-foot no-ball era.

Something to keep an eye on, anyway. It could just be a blip.

Opening partnerships, and a Kiwi record

2008-01-13T10:26:00.000+01:00

This entry is inspired from a line from The Best of the Best. On Hobbs and Sutcliffe, Charles Davis writes that, "[e]ach was a great batsman in his own right, but even that is not quite enough to account for their performances together".

Given the individual averages of two openers, how much would we expect their partnerships to average? And which opening pairs do the "most better" that you would expect?

To answer these questions, I took all opening pairs who opened the batting at least 15 times together. I ordered each pair so that the first had the lower average of the two (so that, in the tables and equations below, avg1 is the lower individual average, and avg2 is the higher). Common sense suggests that the average partnership should be more determined by the lower individual average, since that batsman is more likely to get out first.

Note that I've used individual averages as openers when doing this analysis.

I then threw the data into gretl, an econometrics program. Since there are two independent variables (one for each opening batsman), I can't easily make a pretty graph. You'll just have to cope with equations and tables. Here is some of the output:


Modèle 1: Estimation en MCO avec 97 observations 1-97
Variable dépendante: avg_part

      VARIABLE       COEFFICIENT        ERR. STD         T           p. critique
  const                -7,60135          5,30561      -1,433   0,15526
  avg1                  0,484575         0,144117      3,362   0,00112 ***
  avg2                  0,766951         0,157437      4,871  <0,00001 ***

  Moyenne de la variable dépendante = 41,6219
  Écart-type de la var. dép. = 12,6076
  Somme des carrés des résidus = 7632,06
  Erreur standard des résidus = 9,01067
  R2 non-ajusté = 0,499844

You'll note that my computer is French. The word moyenne is 'mean', écart-type is 'standard deviation', and the other words are close to their English counterparts. If you don't know what they mean, that is not important.

The table tells us that, "on average", we expect that the average opening partnership (avg_part) should obey the following equation:

avg_part = 0,484575*avg1 + 0,766951*avg2 - 7,60135.

The R² value says that roughly half of the variance in the data-set is explained by this model.

Obviously the equation isn't valid everywhere — if both openers average zero, you would not expect them to score negative runs! But roughly 47 of the 97 opening pairs in the sample do better than the equation, and 50 do worse, so it appears to be pretty much "in the middle".

It is surprising (to me, at least) that the co-efficient of avg2 is so much higher than that of avg1. This says that it is the opener with the higher average who more determines the size of the average partnership. I'm at a bit of a loss to explain this. Perhaps openers with lower averages have lower strike rates (so while they don't score as many runs, they don't get out first)?

Now we get onto the pairs who do better than they should. In the following table, I've given the individual averages-as-openers, the runs scored together, the number of partnerships, 'obs' the observed average partnership, 'exp' the expected average partnership based on the equation above, and the ratio of the observed to expected.


opener1         opener2       avg1    avg2    runs  inns  obs     exp     ratio
T Franklin      J Wright      23,00   38,12   1543  28    55,11   32,78   1,68
Javed Omar      Nafees Iqbal  22,08   25,60   665   19    35,00   22,73   1,54
P Roy           V Mankad      31,71   40,74   868   16    57,87   39,01   1,48
J Stollmeyer    A Rae         41,94   46,18   1349  21    71,00   48,14   1,47
B Murray        G Dowling     23,92   31,55   786   20    39,30   28,19   1,39
C Cowdrey       G Pullar      42,42   43,84   906   15    64,71   46,58   1,39
C McDonald      A Morris      39,40   45,69   949   15    63,27   46,53   1,36
J Hobbs         H Sutcliffe   56,37   61,11   3249  38    87,81   66,58   1,32
Imran Farhat    Taufeeq Umar  33,10   39,30   754   15    50,27   38,58   1,30
Sadiq Mohammad  Majid Khan    34,93   42,23   1391  26    53,50   41,71   1,28

And it's a Kiwi pair who finish first! I suppose that if you analyse enough Test data, you'll eventually find New Zealand coming first in something.

My guess that openers with lower averages score slower certainly applies to Trevor Franklin, who is the fourth-slowest batsman of all-time according to Davis's list (qual. 1000 runs or 2000 balls faced; average over 20).

It may just be coincidence that some opening pairs do well in that table — perhaps they both had a good run of innings while batting together, or maybe they batted against weaker teams (I haven't tried adjusting for strength of bowling attack). But it may also be that they bring out the best in each other. Or, as Davis suggests in the case of Hobbs and Sutcliffe, that they held a psychological edge over their opponents when together.

When Stuart sees the other end of the table, he will be happy to see Graeme Wood coming dead last.


opener1         opener2       avg1    avg2    runs  inns  obs     exp     ratio
M Elliott       M Taylor      35,32   43,50   721   23    31,35   42,88   0,73
M Dekker        G Flower      15,86   29,30   357   22    16,23   22,56   0,72
B Woodfull      B Ponsford    50,90   54,18   860   22    40,95   58,62   0,70
G Gooch         T Robinson    43,88   44,97   621   19    32,68   48,15   0,68
R Simpson       L Hutton      25,92   56,48   477   15    31,80   48,28   0,66
E McMorris      C Hunte       26,86   45,07   548   21    26,10   39,98   0,65
B Pocock        B Young       22,93   32,13   378   21    18,00   28,15   0,64
Wasim Jaffer    V Sehwag      35,82   51,29   619   21    29,48   49,09   0,60
Hannan Sarkar   Javed Omar    20,66   22,08   207   18    11,50   19,34   0,59
A Hilditch      G Wood        31,56   33,61   354   18    19,67   33,47   0,59

It is also interesting that Javed Omar comes both second and second-last. Mark Dekker has easily the worst average of any opening batsman who's opened the innings 15 times.

For all the hugging, Langer and Hayden did slightly worse than would be expected, with a ratio of 0,92. Their average opening partnership of 52,08 is quite good (22nd on the list), but they each have excellent individual opening averages (48,94 and 52,66). The Langer/Hayden and Boycott/Amiss pairs are the only ones to have an average partnership of over 50 and a ratio below 1.

Figures are based on Tests 1 to 1858, that is up to the first Test between New Zealand and Bangladesh.