Stephen Colbert's repeated jabs at Prince William, Kate Middleton, and the royal family may have contributed to the show’s downfall ...