FanPost

Strength of Schedule in the BCS

The best way to structure an out-of-conference opponent slate is commonly debated. Recently, with the release of the ACC schedules, I have noticed an abundance of comments discussing the BCS formula and the extent to which it considers strength of schedule. I have to admit that didn't know exactly how the process worked, so I did some investigation and thought I would share my findings. Most of this information is taken from the BCS website and each poll's website.

Brief History: When the BCS was formed in 1997, its formula was comprised of four elements: polls, averaged computer rankings, losses, and strength of schedule. As a point of reference for later, the original strength of schedule component was based on the win/loss record of opponents (weighted by 66.6%) and the win loss record of opponents' opponents (weighted by 33.3%). Therefore, the original strength of schedule measurement was a strength of opponent measurement only and did not consider home field advantage or the timing of the schedule (where BYE weeks were placed, bunching of tough games, short rest, etc.)

From 1997 to 2001 nothing changed with the formula, other than the addition/subtraction of certain computer rankings and whether the highest/lowest computer rankings were thrown out. In 2001, a bonus was added to the final BCS numbers for wins over highly ranked teams (scaled based on rank). In 2002, the BCS decided to lessen incentives for running up the score and removed margin of victory as a consideration for computer rankings. Then came the 2003 LSU/USC split championship. As a result, in 2004, the forumla was revamped to place more emphasis on human polls and less emphasis on objective measurements. The new system, which is still in place today, calculates the average of the AP poll (replaced by the Harris poll in 2005), the coaches poll, and the computer rankings.

Because of the restructuring, the original strength of schedule component was removed. However, this is not to say that strength of schedule is no longer relevant - it is still a part of the formula. Not only do voters have the ability to consider each team's schedule while casting their ballots, but all of the computer rankings also calculate their standings based on the quality of opponents played.

There are 6 computer models currently in use by the BCS: Jeff Sagarin, Anderson & Hester, Richard Billingsley, Colley Matrix, Kenneth Massey, and Dr. Peter Wolfe. The models with the highest and lowest rank are ignored and a team's ranking in the other four are averaged together and placed on a 100 point scale. This then comprises 1/3 of the BCS formula since it is averaged with the two human polls. But to say that the computers consider schedule strength is insufficient to understanding its significance. Each system weighs it differently, so let's take a closer look at each one.

Jeff Sagarin: Jeff Sagarin is believed to use an Elo model, which rates performances of teams based on their relative skill level to their opponents. In other words, a win in his system is rewarded more when it is against a stronger opponent and rewarded less when it is against a weaker opponent. Similarly, a loss is penalized less when it is against a stronger opponent and penalized more when it is against a weaker opponent. Included in the strength of opponent is also the location of the game (so an opponent is considered stronger at home and weaker on the road). Additionally, because such a model requires a starting rank and because there are only 12 regular season games, he weights the initial rankings using a Bayesian network (in other words, there is a preseason rank). However, when the teams are sufficiently connected, the weighting is no longer used and he considers the rankings to be unbiased.

Anderson & Hester: I don't know the specifics of this system (and had a hard time finding information about it), but I do know that it incorporates strength of schedule. Here are two of the four claims made by the system: "These rankings compute the most accurate strength of schedule ratings. Each team's opponents and opponents' opponents are judged not only by their won-lost records but also, uniquely, by their conferences' strength" and "These rankings provide the most accurate conference ratings. Each conference is rated according to its non-conference won-lost record and the difficulty of its non-conference schedule." So it appears that this system not only rates schedules based on strength of opponents and opponents' opponents, but also based upon the respective conferences of the opponents and opponents' opponents.

Richard Billingsley: This system is dubbed a "power ranking" by Billingsley. There are a lot of odd variables present in this model, such as a pre-season rank, specific BYE week rules, and even a deduction for losses (your record determines the amount of credit you can get from each win going forward). Regardless of the nuances, his system uses it's own rankings to adjust for strength of opponent. Another unique feature, this system freezes the opponent's difficulty at the moment the game is played. Thus, if a team is ranked 1st going into the first game and ends the season ranked 120th, the team that beat it first gets more credit for the win than the team that beat it last. In other words, the strength of opponent does not dynamically change as the season goes on. This might do a good job of compensating for injuries to key players, which would justify using a different strength of opponent before and after, but it doesn't compensate for poor initial ranking (ex. he will have Auburn ranked #1 going into 2011, which we know is wrong, and teams that play it earlier will get more credit for beating it). Another very interesting thing to note: he also calculates home field advantage into the strength of opponent, but it is not uniform and is instead based on a 5 year average of crowd size as a percentage of stadium capacity.

Colley Matrix: Unlike most of the other rating systems, this one actually has some excellent literature that explains the formula used. I suggest giving it a quick read, but only if you have a strong understanding of math. The only variables considered are record and opponent. Home field advantage is irrelevant to this model, but it uses opponent records to compensate for strength of opponent. The only nuance I noticed is in regards to non-FBS teams. Because non-FBS teams are so poorly connected, this model originally excluded non-FBS games from consideration - so playing a FCS team did not change your ranking. However, this changed in 2007 and it now incorporates non-FBS games in an interesting way. Side Note: He briefly mentions FSU in his very well done explanation .pdf

Kenneth Massey: This model also has a lot of information available and again I encourage you to take a look (a strong understanding of math helps here too). However, there is a disclaimer on the description page that says the information is outdated, so I don't know whether it is reliable. Regardless, the information above the ratings themselves says that they "are based on win-loss outcomes relative to schedule difficulty," which is all that really matters for my purposes. I did want to point out an interesting method for rating strength of opponent described in the potentially incorrect description of the ratings. According to that, the difficulty of the schedule is not an absolute number, but is dynamically based on the rank of the team playing it. For example, a "great" team is given relatively similar weight for scheduling an "average" team, a "bad" team, or a "pathetic" team because it is assumed that the "great" team's chances of winning against each of those types of teams is similar. Similarly, a "pathetic" team is given relatively similar weight for scheduling a "great" team, a "good" team, or an "average" team because it is assumed that the "pathetic" team's chances of winning against each of those types is similar. In other words, there is a diminishing marginal return for scheduling increasingly bad/good teams. I am unsure whether location is a factor because nothing is mentioned in the description above the ratings, while the potentially incorrect description says that the system incorporates it. There is also conflicting information on the use of pre-season rankings. Above the ratings it claims "There is no preseason bias; all teams begin at zero." However, the potentially incorrect description says that it includes pre-season rankings initially, but "their effect gets damped out completely" during the season - so I am not sure what to go with.

Dr. Peter Wolfe: Finally, this system uses a Bradley-Terry model and considers only win/loss records and game locations. I don't fully understand the model, but I believe there is an emphasis on opponent strength. There is a brief explantion given, but not much else is available in terms of info. (I found a website that claims the Wolfe system accounts for "sportsmanship" but I think this is incorrect. Wolfe's explanation mentions sportsmanship as a "significant but hard-to-measure factor in comparing teams." However, he only mentions this in reference to the BCS decision to use only wins/losses and not scoring margin, so I don't think it was meant to imply that his system has a sportsmanship component.)

Some Observations:

  • Each computer ranking adjusts for strength of opponent (though differently)
  • At least half of the systems adjust for home field advantage (and at least one, RB, uses a different HFA for each team)
  • None of the systems seem to adjust for schedule timing (so the order of opponents and amount of rest are irrelevant)
  • By using record to determine the reward for winning, RB's system is favorable to schedules that place easier games earlier in the season and harder games later in the season. Also, because it uses the prior season's final rankings as pre-season rankings and uses static strengths of opponents, it favors early matchups against teams that declined in the offseason and disfavors early matchups against teams that improved in the offseason.
  • A&H also incorporates conference strength in strength of schedule, which is interesting
  • Some of these systems ignore (or ignored) FCS wins. I'm not sure of the effect this has, but I would expect it to be minimal if these systems really are adjusting for strength of opponent.

So now we have a better picture of how each poll works, but this still doesn't really tell us how strength of opponent is weighted by each system. Because few of the methods divulge their inner workings, let's take a look at how each of the computer systems ranked their top 10 in 2010 (bcsfootball.org's final is through 12/5/10 only, so I looked at each poll's site):

Sagarin A&H Billingsley Colley Matrix Massey Dr. Wolfe
1 Auburn
(#13/#5:14-0)
Auburn
(#7/#5:14-0)
Auburn
(#5:14-0)
Auburn
(#13/#5:14-0)
Auburn
(#3/#5:14-0)
Auburn
(#5:14-0)
2 TCU
(#76/#99:13-0)
TCU
(#55/#99:13-0)
TCU
(#99:13-0)
TCU
(#76/#99:13-0)
Oregon
(#11/#17:12-1)
TCU
(#99:13-0)
3 Oregon
(#10/#17:12-1)
Oregon
(#35/#17:12-1)
Oregon
(#17:12-1)
Oklahoma
(#2/#39:12-2)
TCU
(#54/#99:13-0)
Oregon
(#17:12-1)
4 Stanford
(#9/#31:12-1)
Ohio State
(#36/#43:12-1)
Boise State
(#86:12-1)
Stanford
(#47/#31:12-1)
Stanford
(#16/#31:12-1)
Stanford
(#31:12-1)
5 Ohio State
(#70/#43:12-1)
Stanford
(#39/#31:12-1)
Ohio State
(#43:12-1)
Oregon
(#52/#17:12-1)
LSU
(#5/#3:11-2)
Ohio State
(#43:12-1)
6 LSU
(#15/#3:11-2)
Oklahoma
(#5/#39:12-2)
Stanford
(#31:12-1)
LSU
(#8/#3:11-2)
Ohio State
(#40/#43:12-1)
LSU
(#3:11-2)
7 Alabama
(#14/#10:10-3)
LSU
(#8/#3:11-2)
LSU
(#3:11-2)
Ohio State
(#65/#43:12-1)
Arkansas
(#6/#2:10-3)
Boise State
(#86:12-1)
8 Boise State
(#81/#86:12-1)
Boise State
(#57/#86:12-1)
Wisconsin
(#64:11-2)
Boise state
(#69/#86:12-1)
Alabama
(#9/#10:10-3)
Alabama
(#10:10-3)
9 Arkansas
(#11/#2:10-3)
Arkansas
(#3/#2:10-3)
Alabama
(#10:10-3)
Oklahoma State
(#35/#85:11-2)
Oklahoma
(#13/#39:12-2)
Oklahoma
(#39:12-2)
10 Wisconsin
(#66/#64:11-2)
Alabama
(#4/#10:10-3)
Oklahoma
(#39:12-2)
Michigan State
(#59/#46:11-2)
Boise State
(#59/#86:12-1)
Arkansas
(#2:10-3)

 

The #s after each team list the SOS rank given by the respective system first and then the SOS rank given by FEI. The team's record is listed after the colon. RB and Wolfe did not provide SOS data, so only FEI is shown for them.

More Observations:

  • Although all systems do not consider margin of victory (or at least are not supposed to) and all systems claim to account for strength of schedule, TCU comes in 2nd in 5/6 polls (Massey has it 3rd) even though it has one of the worst SOSs. Therefore, winning is more important than schedule
  • Boise State has the worst strength of schedule of the non-Nevada 1-loss teams and has the worst ranking of the non-Nevada 1-loss teams in 5/6 of the polls (RB discussed below). Therefore, schedule seems to matter when there is no difference in record.
  • Under RB's system, Boise State, Ohio State, and Stanford all have the same record, but are ranked in reverse order of schedule strength. How is this possible? There are so many factors to his rankings, but my guess is his odd rule that each loss reduces the reward for winning future games. My support? Stanford's loss was on October 2nd to then #9 (RB) Oregon, Ohio State's loss was on October 16th to then #28 (RB) Wisconsin, and Boise State's loss was on November 26 to then #21 (RB) Nevada. See a pattern?
  • Massey's system seems to be the most sensitive to strength of opponent, while RB's seems to be the least.
  • Nevada is a 1-loss team, but didn't end up in the top 10 in any of these systems. Why? Probably because it's SOS was really bad (107 in the FEI).
  • While most of the systems rank teams with idential records in accordance to their schedule ranking, some odd results exist. For example, Sagarin has Oregon over Stanford, even though they have identical records and Stanford was given a marginally higher schedule ranking. I think this might be an effect of his Elo model

Conclusion: Strength of schedule is still very relevant to the BCS because of the computer polls and voter perception; however, TCU proves that winning is still more important, even to computers. Since timing of the schedule is irrelevant to ranking, teams/conferences should schedule with the best possible timing to maximize rankings. There is marginal benefit (from RB) in scheduling in an increasing order of difficulty (so no big season openers). There is also marginal benefit (from RB and potentially other polls) in scheduling early matchups against teams that regressed in the offseason.

Hope that clears some things up. If I missed something important or stated incorrect info, let me know and I will update this post. Also, I am not a stat guy (even if I pretend to be), so more detailed explanations of the inner workings of these systems is appreciated.

X
Log In Sign Up

forgot?
Log In Sign Up

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join Tomahawk Nation

You must be a member of Tomahawk Nation to participate.

We have our own Community Guidelines at Tomahawk Nation. You should read them.

Join Tomahawk Nation

You must be a member of Tomahawk Nation to participate.

We have our own Community Guidelines at Tomahawk Nation. You should read them.

Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_9341_tracker