Bubble Talk with Crashing The Dance’s Andy Cox

110306120723_Morris-Dozier[1]

Bracketology is a relatively saturated internet market but Andy Cox has carved out his own niche. Cox has created a computer system that uses statistical machine learning techniques to predict how the selection committee will select and seed teams for the tournament. The system has improved for five consecutive years, most recently selecting 33 of 34 at larges, and it provides a different and more educational perspective on tournament selection.

As someone that studies the past behavior of the selection committee, we like to probe Andy’s mind come March and see what he thinks about the current bubble situation. The most recent Crashing the Dance projections (as of Tuesday morning) have Michigan listed as the third to last team in. Now here’s what Andy had to say about Michigan, the bubble, and more. (Photo: Freep)

What are the basic strengths and weaknesses of Michigan’s resume?

Strengths:

  • Good road wins, including Michigan State (likely in) and Clemson (bubble)
  • Nine wins over RPI top 100 teams, and not the RPI-fooling kind of top 100 wins. Eight of them are against the top 67. That’s pretty good among bubble teams.
  • Four good road wins against teams ranked 67 or better.

Weaknesses:

  • No “signature” wins. Their best RPI win (Harvard – RPI #35) was at home over a team that may not make the field
  • Only one game over .500 against the RPI top 200.
  • Most bubble teams have at least one loss to a team outside the top 100, but their loss at Indiana (RPI #176) is lower than you’d like to see.

Then the real question: what does Michigan still have to do to make the Dance? Is a win over Illinois necessary in the 4-5 game of the Big Ten Tournament?

As usual, the answer is that it depends. Obviously, it depends on what other bubble teams do this week. The bottom of the bubble is so fluid that every little bit you can do to set you apart from the mire will help.

I wouldn’t say a loss would necessarily be the end. I don’t know that committee members believe in “good losses,” but just about every team on the bubble will lose this week. For some on the committee, it could come down to the quality of this loss, which for most will be on a neutral court.

Another thing to think about is that any committee members who consider Illinois and Michigan relatively even could use this game as a tiebreaker.

I understand that the committee doesn’t look at specific conferences but how would you compare the resumes of the four Big Ten teams hovering around the bubble: llinois, Michigan, Michigan State, and Penn State.

Illinois probably has the strongest non-conference profile of the four, though it’s much closer than I expected. Michigan went 5-1 against the other three, best among the four-way head-to-head. I think our selection model is slightly overvaluing Penn State. In my opinion, they have more work to do than the other three.

image-1
Photo: Michigan Daily

The common refrain among Michigan fans is “if Michigan State makes the tournament, Michigan has to make the tournament because they swept the Spartans”. My impression is that this isn’t quite true. What are your thoughts?

While every committee member has their own opinion about head-to-head results, my general sense is that they’re mostly used as tiebreakers when the teams under consideration are very similar. The fact is those were just two games in a 30-plus game profile.

That said, the profiles of Michigan and Michigan State are similar, so it would be difficult for the committee to ignore the head-to-head results.

You spend a lot of time studying the selection committee and what they’ve done in the past. Obviously this all goes into your model, and I’m not sure if it is proprietary and confidential information, but what would you say are some of the most important factors of an NCAA tournament resume?

One theme that comes out every year during the CBS post-selection interviews with the selection committee chair is to schedule as many good non-conference teams as possible and win as many of those games as possible. I have noticed some inconsistency from year to year about whether winning more games against a good-but-not-great schedule is more important than merely being competent against a great schedule. That is to be expected with subjective judgement by a constantly changing group of people, but it’s frustrating for everyone trying to forecast this stuff, and I’m sure it’s frustrating for the coaches who are left out.

As far as specific attributes, I don’t know that it can be narrowed down to just a few. When this started as a grad school project in 2005, I used this great article by Luke Winn to help select the attributes I included in the model. After that, the model does all the work crunching the numbers. Each year I review the teams the model handled poorly (and there always are some) and look for trends. I’ve tweaked the attributes I feed to the model and it seems to have helped. (See this page and its links for details if you’re interested.)

Can you think of examples of how the committee has handled teams with similar resumes to Michigan in the past?

One thing I’d like to add to the site in the future is a tool to search for most similar profiles in the 10-year archive I’ve built up. Until then, the best I can do is try to identify some meaningful criteria and look at the history. It can be arbitrary at times, but it’s better than nothing. For example, I looked at all teams that:

  • Were ranked between 25th and 50th in the Sagarin ratings
  • Had no RPI top 25 wins
  • Had 3 or fewer RPI top 50 wins

There were 63 matches over the last 11 seasons, and 37 of these (59%) were at-large selections or would have been had they not won their conference. Matching teams last year ranged from California (#8 seed) to Virginia Tech (out). Is that the best comparison to use? Probably not, but if you narrow the criteria too much, you end up with only a handful of matching teams that don’t provide a meaningful comparison.

Michigan lacks the top 50 wins – just Michigan State (2x) and Harvard – but the Wolverines have beaten a number of teams on and around the bubble. How many “good” top 100-ish wins does a team need to make up for the lack of signature wins?

When I added having 7 or more RPI top 100 wins to the above criteria, the result was 24 total teams, 18 of which were at-large selections. The problem with binning win/loss totals into 1-25, 26-50, etc. is that it loses some of the subtlety that the committee certainly investigates, especially with the last few at-large teams. A win over RPI #55 goes into the same bin as a win over #95, but it’s obviously a better win.

That also explains some of the odd movement you’ll see in the model. This actually happened from Sunday to Monday with Michigan. Michigan State was just inside the top 50, so Michigan’s two wins over the Spartans counted as top 50 wins. On Monday, the Spartans slipped just a few spots outside the top 50, which cost Michigan two of its three top 50 wins. For bubble teams, that could make all the difference.

  • Kenny

    I am so sick of hearing RPI top 50 wins and how teams RPI moves from 49 to 51 matters.

  • CB2009

    I’d like to hear what he said about our “good losses”. I wonder how many other bubble teams lost 1-possession games to high ranked teams like ‘cuse, OSU, Kansas, Wisco… I know you can’t quantify that in a positive way, but I’d like to think that this, as well as out upward trajectory towards the end of the year would put us in over a lot of the other bubble teams.

  • Mattski

    I find the fact that there is a heart beating–people, a bunch of them–at the heart of the process reassuring. So I am inclined to believe that they have their pool of bubble teams under consideration and then begin to dig into their records. And ourhaving lost by just a handful of points to four or five of the nation’s best teams DOES matter, as does coming on strong at the end of the season.

    It’s a different discussion, but I would like to hear frank talk from some of these eminences about the relative strength of the conferences–how it comes to look as they sift through. I like to think the B1G is a tough, tough conference, but when I watched NC and FSU go at it in Tallahassee the other nigh, I have to admit that the level of play–the speed of play–was breathtaking. It made me damn Tom Izzo and Bo Ryan under my breath one more time. (On the other hand, it is very exciting that a Beilein team is out-toughing a taller Izzo team here at the end of year four of his tenure as our coach. People wondered if he could adapt. . .)

    • Beast1530

      Close losses to top teams are still losses. In the end, Michigan has to show the ability to beat top teams and they haven’t done it yet. This is why I think Michigan needs to beat Illini just to be safe. Even better beat OSU in 2nd round and that will lock them up.

      • Dylan Burkhardt

        Judging by the bubble now, I think an Illinois win would lock things up.

  • Agreed

    I’d agree that losses to top tier teams by close margins has some merit when Michigan’s full picture is considered. It must say something about the strength of this team, and when comparing to equivalent squads it will play a factor. When those humans deciphering our chances think upon the day they watched Michigan loose a buzzer beater on SportsCenter’s night cap, some sympathy will be revealed.

  • Bird

    If we make the tourney, Sparty doesn’t, and Tressel goes down on top of it, I don’t know if I’ll be able to handle it.

    • Mith

      LOL, yeah that’s definitely too much joy for me to process at once.

    • JimC

      We gotta have a dream!

  • skipper

    Don’t forget about momentum and a team showing they are getting stronger at the end of the year. The committee has stated that is important.

    • Tweeter

      I gotta imagine that it still is important, but didnt the committee say that the criteria of “Last 10 games” is no longer used?

  • Tweeter

    I am not a big fan of using a pure numbers system to predict the field, since in the end, it could be just one factor for a particular team that stands out to the committtee above all other factors that make them put that team in the field or leave them out.

    I also agree with Kenny in regards to using top 25, top 50, top 100 as a criteria. To me, there are times that beating a team rated 27 is more impressive than beating a team 16. If you are going to use these numbers, I wish they would use them in a more accurate fashion.

    One thing they could do, is to present each team for consideration by using their best possible resume. So for instance, for Michigan, you would not present them as going 0-7 versus top 25, but instead as going 8-8 against top-67 or whatever. Then you have to compare that to other teams that maybe get presented as going 6-3 versus top 82, and decide which you think is more indicative of a better team. If you use a strict set of criteria for every team, than certain teams, unfairly so, lose out on showing some good parts of their resume. However, if you get rid of the strict formula and just use the best light possible for each team, how can you really argue with that?

    • maxwell’s demon

      I don’t think crashing the dance is saying going purely by numbers is the best way to approach selection. He even says he thinks that PSU is overrated by his own system. I think he’s interested in trying to make the best mathematical model possible, which if nothing else provides a great reference and alternative to human speculation.

      • Tweeter

        yea that is what i am saying. That I personally, do not like using that system to predict the field because in the end the selection is not based on pure math. It is interesting to look at, just not something I base my feelings on where Michigan stands.

  • http://umhoops.com KAB

    We better beat ILL..

  • AG2

    Well, the more I think about it the more it seems like Michigan and MSU have similar profiles. The only thing that MSU really accomplished in Maui was they beat Washington on a neutral court. Michigan beat Clemson on the road. Both teams lost to Syracuse on neutral courts in the NY/NJ area. MSU lost at home to Texas, we lost at home to Kansas. Both teams went 9-9 in conference play. MSU got blasted @ Iowa, Michigan got blasted @ Indiana. MSU beat Wisconsin at home thanks to a miracle finish led by a player no longer with the team, Michigan lost on a miracle banked 3pter by a freshman sub-30% 3pt shooter. Neither team beat Ohio State or Purdue. Both beat Minnesota on the road. Frankly the only reason I can think of for MSU being ahead of us in the RPI is that our win vs. Concordia doesn’t count toward the RPI.

    • Kenny

      Concordia is not the reason. The reason are those “where are they universities” we played earlier in the season. We use to schedule good mid-majors like Cleveland State, Weber State, Butler but last couple of years the non-conference schedule is either Kansas, UConn, or someone I never heard of.

      • Jeff

        That’s not the reason either. What more do you want than a top-20 schedule? The reason is Wisconsin banked in a 3 pointer at the buzzer, our two last minute threes at Illinois didn’t go in, Kansas beat us in overtime, Syracuse and OSU pulled out close wins over us. If any one of those outcomes had gone our way, we’d be comfortably in right now. Such is life.

    • Azad

      MSU got blasted by Texas and Syracuse, we lost by 3 to Syracuse and took Kansas to OT, fwiw.

  • Kenny

    Selecting committee’s are humans, and they do not look at a team’s record in a mechanic way when make decisions. There is always guts feeling factor. I am sure that most will all see Michigan better than Michigan State and Penn State at current time. When people say that they don’t look at conference as a whole when selecting teams, it is simple not true. They don’t assign certain numbers to each conference to begin with and don’t argue that certain conference should get more or less teams in either, but they will find reasons to balance it so that a strong conference like big ten this year will get its fair share of representations. At the end, every team on the bubble has its strength and weakness on the resume to provides points for articulating to either direction. Certain things like those “good loss” may not serve an argument itself, but it provides reasons for members of selection committee to be sympathetic and find other reasons to support our cause.

  • JimC

    I find this comment today from Eamonn Brennan (my new favorite bracketologist) comforting:

    The Wolverines are now 18-12 with a 9-9 record in the Big Ten and an 8-3 record in their past 11 games, and that might be enough to get them in the tournament almost regardless of Friday’s second-round matchup with Illinois.

  • MikeSal

    You almost have to take what bracketologists say with a grain of salt. Win Fri and nobody else has to talk about bubble anymore

  • Alex

    ESPN makes us think of this process as “conference dominated” by having the Bubble Watch feature. When I say “conference dominated”, I mean we as fans think that Big Ten bubble teams should get the nod over other conferences’ Bubble Teams because they are in the one of the toughest conferences. Also, listing conference wide RPIs and SOSs don’t help matters. Neither does Joe Lunardi separating out bids by conference on his Bracketology. Maybe they should modify these features to make it easier for us to remember that the Selection Committee doesn’t really look at conference affiliation when making their at-large pick.

    /r

  • billiam

    Hey Dylan,
    Could you make a bubble watch thread with all the games?
    I know that this is because I’m lazy tho, so shoot me down if you want to.

    • Mith

      Pull up the schedule at ESPN! Come on, billiam, you can do it!

  • Jeff G

    I too wonder how to the committee considers close loses. Michigan is 15 pts away from wins over Syracuse, Kansas, OSU, Wisconsin, and Illinois but I have no idea if that is significant at all. How would that compare to other teams, since I’m sure many teams have a few close losses. Also, if you use some sort of close loss metric for bubble teams you’d presumably have to use it as a seeding metric as well.

  • Tweeter

    Pat Forde picks Michigan as dark horse in BTT and eventual winner.

    Dark horse: Michigan. The Wolverines’ only two losses in the past eight games were by a total of three points, and one of them came courtesy of a fluke banked-in 3 at the gun. If Michigan gets a semifinal shot at Ohio State, it might have a chance — if it can make enough shots.

    Minutes pick: Michigan (17). The Minutes billed these as lunatic picks, right? Well, this would qualify. But John Beilein has never lost his first Big Ten tourney game, and if the Wolverines can win one, hey, maybe they can win three.

    http://sports.espn.go.com/ncb/columns/story?columnist=forde_pat&id=6192475

  • Aaron

    We have a tournament-worthy resume with or without an Illinois win. We have a freshman stud named…Hardaway. We have almost half a million living alumni and the committee knows we’ll be bleeding the Maize and Blue and buying tickets if we dance. I think we’re fine.

Previous post:

Next post: