This is where watching other college games should give you more prospective. USC v. Tenn. While the game was still close, Tennessee got called for an OPI on a play half as obvious as the play by Michigan. They also got called for a hold on a critical play, Those calls may have cost the SEC a second team in the playoffs, but SEC refs made those calls.
Now certain folks would say those were called because knocking Tennessee out helps Bama, but that is another issue.