A Real Statistical Analysis of Jimbo Fisher v. Jeff Bowden: A Response to Andrew Carter's Article

What happens when a Statistician tries to be a comedian on a Florida State blog?

What happens when he lets other guys do the stats and just tries to make funny posts? 

What happens when he doesn't hold other FSU writers to a higher level of statistical analysis and lets dumb posts go?

This happens.  (I should clarify that Andrew Carter has been a good beat writer, has a good sense of humor, and generally does good work)

Inside this story, you find the reaction of TomahawkNation's premier Statistician-turned-comedian to this obvious well-researched and well thought-out article from our friend Andrew Carter.

My initial reaction?

Mr. Carter should probably leave the statistics to the statisticians.  But hey, if he can pass himself off as a Stat guy, I guess I can pretend to be a real writer.  So what would happen if I analyzed his writing line by line, FJM style?

"Florida State fans greeted Jeff Bowden’s resignation during the 2006 season with both relief and jubilation."

I also bought a pony.  It died.  Thanks for reminding me Andrew.  Jerk.

I believe it was also greeted by a temper tantrum from a really old head coach.

"It was Bowden, the former Seminoles offensive coordinator, who’d received the brunt of blame from fans and media for the Seminoles’ decline from dominance earlier this decade. And his departure at the end of the ’06 season – a departure that led the arrival of almost an entirely new offensive coaching staff – was supposed to solve the Seminoles’ problems."

I'm hooked Andrew.  This article looks like it's going to be good.  Nothing scary about this one.  Nope. Nope. Nope.

"Or so many thought."

Ok, that's just a well-constructed sentence right there.

This is the point in the ride where Andrew takes us off the main track down the "spooky" trail.  The Disney puppets are yelling at you to turn back, AC!! Listen to them!!!

"Since Jimbo Fisher’s arrival in January 2007, the Seminoles’ offensive has improved. But has it improved as much as you think?"

Ancient Chinese Proverb about blogs: "Man who straddles fence usually has problem with posts." 

This philosophy has the potential to revolutionize your every day life. 

• I just had the most delicious spaghetti dinner I've ever had.   But could it have been more delicious?

• Jessica Alba is insanely hot.  But is she hot enough?

• I just won the Powerball.  But can I really make due with only $20 million?

"Fisher has been at Florida State now for 31 games. And his first 31 games bear a resemblance to Bowden’s final 31 games."

Steps in the Sentinel Scientific Method

1. Get marching orders from Terry Bowden
2. Make ridiculous claim
   
3. Hope and pray to find something that backs you up
4. Pat yourself on the back

Let me see, there are several similarities between Jeff's last 31 games and Jimbo's first 31.  First of all, there are 31 of each.   Incredibly convenient.  Secondly, in each case the sport of football was played.  So we've got that going for us.  Lastly, did you know that during this time period Bowden had a secretary named Fisher and Fisher had a secretary named Bowden?!?!*   Freaky, right?!!

*Facts are subject to change without notice and may or may not be reliable.  Facts may not be reproduced, reused, rewritten, or re-disseminated without the express written consent of MattDNole, Inc and the Office of the Commissioner of Baseball.

"In Fisher’s first 31 games, FSU is 18-13. In Bowden’s final 31 games, FSU was 18-13."

You got me AC, I'm hooked.  I mean, they played the exact same teams so if they finished with the same record it must mean they are similar.  Oh wait.  You mean they didn't play the same teams?

Bowden's last 31 games: 20 opponents with winning or .500 records, 20 bowl teams

Fisher's first 31 games: 23 opponents with winning or .500 records, 22 bowl teams

About 10% more of Fisher's games were played against teams with winning records.

(How's THAT for soft statistical analysis?!)

"In Fisher’s first 31 games, FSU has a 9-9 record in the ACC. In Bowden’s final 31 games, FSU had a 10-9 record in the ACC – 11-9 if you include FSU’s victory against Virginia Tech in the 2005 ACC Championship."

You spin me right round baby right round.....

Again with the quality reporting.  Nevermind the fact that three of Bowden's ACC wins came against Duke and one against a 4-7 Wake Forest team (that won 30-0 the following year).  If we're throwing those games out because the teams were terrible, we have to be fair.  Bowden's team also managed to lose to 3-9 NC State.  So that game should be expunged from the record as well.

"Some improvements have been noticeable. In Fisher’s first 31 games, for instance, the FSU running game is averaging 147.6 yards per game. In Bowden’s final 31 games, the FSU running game averaged 99.7 yards per game. And Fisher’s quarterbacks – especially Christian Ponder – seem to have played better than those Bowden coached."

Other improvements include leading the top scoring offense in the conference last season, after Jeff's attack never finished higher than 4th over his last three years.  Or that Fisher accomplished that feat with the youngest offensive line in the country, for which he was named the ACC Offensive Coordinator of the year.  A quick check reveals that Jeff Bowden was not named the Offensive Coordinator of the Year in his final 31 games. 

Or that Jeff faced 20 ACC opponents in his last 31 games, and produced 30 or more points only 4 times.  Twice in 2005 (over the aforementioned 4-7 Wake Forest squad and of course, Duke), and twice in 2006 (shockingly against Duke, and Virginia).  That's right, in his last 31 games, Jeff's offense failed to produce a single 30 point performance against any ACC team that would make a bowl.  But if you needed 50+ points for parents weekend against Duke, he was your man.

Compare that to Fisher who has faced 18 ACC opponents so far, and has produced 30 or more points 5 times.  In Fisher's first year, he didn't accomplish the feat even once.  But in his second year (2008), the year in which his offense was the best in the conference and he was honored with ACC Offensive Coordinator of the year honors, Fisher's offense accomplished the feat 4 times, and all of those outbursts occurred against bowl teams.  From none of the conference games to half of the conference games, which by Carter's method, must show improvement.  This year, Fisher is 1 for 2 in reaching the 30 win plateau, and everyone will count Miami as a bowl team.

We've officially reached the "Please don't kick me out of the program when you take over, Mr. Fisher" portion of the article.  Feel free to grab a soda and come back in a minute.

...   

"Let’s talk about ineffective offensive performances. Let’s say an ineffective offensive performance would be one in which the team had less than 325 yards of total offense. An effective offense should generate more than 350 yards per game, at least, but let’s be conservative."

Sure, and let's talk about "Oh my God I'm going to vomit- this can't possibly be football- did our coach get this job via the Make-a-Wish Foundation?" offensive performances.  We'll say those happen whenever your team looks absolutely lost on national television or fails to reach 150 yards of total offense.  Care to count those occurrences, Mr. Carter?

"In Fisher’s first 31 games, the FSU offense had less than 325 yards of total offense nine times. In Bowden’s final 31 games, the FSU had less than 325 yards of total offense 12 times."

*This portion of the article left blank to represent the level of analysis demonstrated by this sentence.* 

Yeah I can't do this anymore.  Let's just take a look at why I believe Andrew's conclusions are weak.

• Carter never adjusts for opponent or strength of schedule
• He doesn't account for changes in clock rules that have significantly reduced the number of plays (and thus drives) per game
• He chooses the weakest possible response variables to draw his conclusions
• He makes no attempt to measure trends in offensive performance
• He assumes each coach had equal amounts of talent to work with

Yards and points per game are a weak measure to compare different eras because of the different rules across years.  A much more effective statistic, and the statistic that we will use today, are yards and points per PLAY.  Through this measure, the variation in number of plays per game has been eliminated.

Yesterday, TNation contributor SteveNole posted a great Fanpost looking at the trends of offensive performance over time.  I highly recommend that everyone read his post.  I believe the most effective measure of a coach is "are his teams improving or regressing?"  So, through the use of longitudinal data analysis and a simple regression model we can examine the pattern of offensive performance over time.

Let's treat each of the 31 games of interest as a separate time point.  We will have to make the assumption that each of these games occurred an equal amount of time apart for simplicity's sake (accounting for offseasons will complicated the analysis beyond the level of a sports blog).  Additionally, we will not account for injuries, suspensions, or strength of schedule (See Andrew?  When a real statistician makes an assumption that will influence the results of the analysis, they at least mention it as a possible source of bias.)

• By fitting a linear model to the 31 timepoints, we will be able to analyze two things.
• Which coach's offense, if either, is performing better?

The numbers are as follows (Coach 0 = Jeff Bowden, Coach 1= Jimbo, Time = Game # in sequence) :

Here is the model we will be using for the data:

Response = B0 + B1*Coach+ B2*Time+B3*Time*Coach

If B1 is significant, then the average value of the response for the two coaches is different.  A positive value indicates Fisher's offense is better, a negative value indicates Bowden's was superior.

If B2 is significant, then the slope of the graph is significantly different from zero.  If the B2 is positive, then the offense is improving.  If it is negative, then the offense is getting worse.

If B3 is significant, then the two offenses have performed differently over time in the given response variable.  The magnitude of the value will determine which offense is improving (or regressing) at a faster rate.

The data are analyzed using PROC GLM in SAS.  Four separate models are created, each using one of the following as a response for the above predictors: Yards Per Rush, Points Per Rushing Play, Yards Per Pass, Points Per Passing Play.

The results are as follows (We are merely looking at the mean model here, so I will not analyze the residuals to check for correlation, etc.):

Yards Per Rush

proc glm data=matt;

model rushing=time coach time*coach;

run;

                                               Standard
               Parameter          Estimate           Error    t Value    Pr > |t|

               Intercept       3.350967742      0.70821288       4.73      <.0001
               Time           -0.007016129      0.03863618      -0.18      0.8565
               Coach           0.716967742      1.00156427       0.72      0.4770
               Time*Coach      0.006258065      0.05463981       0.11      0.9092

At the alpha=.1 level there does not seem to be any significant difference in the time trend of the two coaches (.9092 > .10).

This interaction is removed and the model is analyzed again.

                                                 Standard
               Parameter         Estimate           Error    t Value    Pr > |t|

               Intercept      3.300903226      0.55253902       5.97      <.0001
               Time          -0.003887097      0.02709045      -0.14      0.8864
               Coach          0.817096774      0.48460877       1.69      0.0971

Notice now that at the alpha=.1 we can see that Jimbo Fisher's offense rushes for about .81 yards per play more than Jeff Bowden's offense.

Rushing TDs

                                                  Standard
               Parameter          Estimate           Error    t Value    Pr > |t|

               Intercept      0.0426416506      0.01260494       3.38      0.0013
               Time           0.0000970092      0.00068766       0.14      0.8883
               Coach          -.0183541957      0.01782607      -1.03      0.3075
               Time*Coach     0.0011220481      0.00097249       1.15      0.2533

There is no significant difference in the time trend.

                                     Standard
               Parameter         Estimate           Error    t Value    Pr > |t|

               Intercept     0.0336652655      0.00994531       3.39      0.0013
               Time          0.0006580333      0.00048761       1.35      0.1823
               Coach         -.0004014254      0.00872261      -0.05      0.9634

There is no significant difference between coaches in the number of TDs scored rushing.  The difference in TDs per game is not statistically significant.

Passing Yards

                                                  Standard
               Parameter          Estimate           Error    t Value    Pr > |t|

               Intercept       6.945161290      0.74036869       9.38      <.0001
               Time           -0.007459677      0.04039042      -0.18      0.8541
               Coach          -0.039354839      1.04703944      -0.04      0.9701
               Time*Coach      0.006693548      0.05712068       0.12      0.9071

Again, the coaches are not trending differently.

                                                 Standard
               Parameter         Estimate           Error    t Value    Pr > |t|

               Intercept      6.891612903      0.57762964      11.93      <.0001
               Time          -0.004112903      0.02832062      -0.15      0.8850
               Coach          0.067741935      0.50661471       0.13      0.8941

There is no significant difference in passing points per play between the two coaches.

Passing TDs

                                                  Standard
               Parameter          Estimate           Error    t Value    Pr > |t|

               Intercept      0.0305258299      0.01286568       2.37      0.0210
               Time           0.0004758789      0.00070188       0.68      0.5005
               Coach          0.0136674437      0.01819482       0.75      0.4556
               Time*Coach     -.0009266638      0.00099261      -0.93      0.3544

There is no significant difference in the time trends.

                                                 Standard
               Parameter         Estimate           Error    t Value    Pr > |t|

               Intercept     0.0379391406      0.01011164       3.75      0.0004
               Time          0.0000125470      0.00049576       0.03      0.9799
               Coach         -.0011591777      0.00886849      -0.13      0.8965

The difference in passing TDs here is not significant.  Fisher's offense is not performing any worse statistically than Bowden's despite the difference in scores.

What can we conclude from the data? 

Carter claims that Fisher's offense is throwing for fewer yards: No statistically significant difference in the objective measure

Carter claims that Fisher's offense is rushing for more yards: Moderate evidence of a significant increase in rushing yards under Fisher.

Carter claims that Fisher's offense is scoring less than Bowden's offense: No statistically significant difference in rushing or throwing scores between the two offenses.

The difference in clock rules and the increase in Florida State's strength of schedule could account for the overall total differences in yards and scores.  Personally, I was surprised at the lack of a significant time trend across the data, but the huge variation from game to game is probably the culprit.  Once Fisher has coached at Florida State for a few more years, enough data points will be available to analyze these offenses across seasons, not games.

The point of this article is simple: It is bad logic to craft numbers to fit your agenda.  Obviously I have my own bias, but for actual ANALYSIS I entered the data and examined the results as they were.

Articles like Andrew Carter's are counterproductive to objective sports observation through statistical analysis.  Casual readers will look at his work and says to themselves "oh it has average in it, so it's statistical."  People will believe that you can make up statistics to support anything.  Let me state this unequivocally:

NOT ALL STATISTICS ARE CREATED EQUAL.

The appropriate use of quality data can lead to the improvement or analysis of any process, including sports.

Using Excel on your laptop to calculate a few averages does not represent quality statistical analysis.  Using biased response variables and shoddy inferential techniques does not mean you are the next Statistical Sports Guru.  It means you are a lazy writer.

While far from complete, this analysis is ten times more substantial than Carter's attempt, which is incredibly misleading and incomplete.

At Tomahawk Nation we try to find the most unbiased sources and measures of data to present an accurate view of what happens, because a good statistic is nothing more than a measure of what happens on the field.  A bad statistic is misleading and detrimental to quality analysis of sports-related information.

In the future, when an editor or "whomever" directs Mr. Carter to conduct an "analysis," I would hope he puts a little more thought into his methodology.