“Hello.”
“Hi, this is the NCAA calling. We read your recent blog post about how we should improve the selection process for our D1 baseball tournament. The idea sounds good, but your post doesn’t say anything about how we could make things better. Could you tell us how we could make the tournament selection process for teams receiving at-large tournament bids transparent, logically consistent, and, perhaps, more fair?”
“Sure. There are two things to consider. First, what an ideal selection process would look like and second, how to help the selection committee come as close as possible to implementing the ideal selection process given the time and resource constraints they are working with.
To me, an ideal selection process would consist of a set of well-defined, objective criteria that can be measured accurately for every D1 team. Using this method, the general framework for the selection process would be defined and look something like this:
The teams would then be ranked separately for each criterion based on how well they did on that one measure. Then the relative priorities of the selection criteria would be determined. If they are all considered equally important, each criterion would be weighted equally. If not, the committee would determine the relative priorities and decide how much weight should be given to each in making a tournament selection. Finally, the criteria weights and team rankings would be combined to create a summary score for each team. The teams with the highest 33 summary scores would then be invited into the tournament. Below is an example how this would work, with Team B receiving a higher tournament ranking than Team A.
Using this method, the selection announcement would include both the names of the teams given at large bids and a explicit description of how the selections were determined. I think this system would go a long way toward addressing the concerns Coach Hartleb raised as well as those voiced by others.
“That sounds pretty complicated. Is it really possible to do something like that in the real world?”
“Yes it is. The tournament selection process is just one example of a multi-criteria decision making situation. People in a myriad of situations face similar decisions every day and need to cope with challenges similar to those faced by the tournament selection committee. Recognizing how common this type of decision is, and the important consequences that many such decisions have, decision science researchers have developed a number of methods for helping people make better decisions in these situations. A wide spectrum of approaches are available ranging from relatively simple to quite sophisticated. Many are easy to use and do not require special training or expertise. They are particularly helpful in group decision making situations, like the tournament selection process. Some even accommodate both objective data and subjective ratings so you can still adopt this approach even if good objective measures are not available for every criterion deemed important for making a tournament selection.
I hope you find this information helpful. Thanks for calling and good luck with this year’s tournament.”
mdm787 