|Michigan sits on the throne because of consistency. They were great right from the beginning, tying Wisconsin for #2 in my 1901 top 25,
and their down periods have been few and of short duration since then.
Harvard was #1 in 1901, so they took the initial lead in the all-time
race. Harvard barely held on to that lead after 1902, when they finished #3 and Michigan #2, leaving Harvard just half a point ahead for both seasons combined.
Yale finished #1 in 1902, and they took the all-time lead away from Harvard in 1903 when they finished #2 and Harvard finished #8. Michigan finished tied with Minnesota for #3 that year, so they passed Harvard up as well, but they were 1 point behind Yale for 1901-1903 combined. Yale was the Team of the Aughts, finishing #2 in 1904, then #1, #1, #1, #3, and #1 over the following 5 seasons. That amazing run enabled Yale to remain the top all-time program through 1913.
Harvard recaptured the throne in 1914. Their comeback began with a #1 finish in 1908, and was followed by finishes of #2, #1, #7 (tied), #1, #1, #3, and #2 over the following 7 seasons. Meanwhile, Princeton passed up Michigan in 1913 for 3rd place over the 1901-1913 period. Harvard, Yale, and Princeton were known as the "Big Three" back then, so it is little surprise that they were the top 3 programs in ranking points early in the century. They would remain the top 3 through 1924.
Harvard remained #1 in the all-time race from 1914 to 1925, but Michigan overtook them and captured the throne for the first time in their history in 1926. The old guard Eastern powers had started to fade not long after World War 1. Harvard finished unranked every season 1923-1928, and was basically done as a football power. Meanwhile, Michigan finished #4 in 1923, then #13, #2, and #2 over the next 3 seasons.
Michigan remained the top all-time football program from 1926 to 1936, but in 1937 they were briefly overtaken by Minnesota. Minnesota had also been powerful from the beginning, finishing #8 in 1901, but for decades they lagged behind Michigan and the Big Three Eastern powers. But Minnesota had a tremendous decade in the 1930s, including 3 straight national championships 1934-1936. They passed up both Princeton and Harvard in 1934 to secure 3rd place all-time, and they passed up Yale in 1935 to put them in 2nd behind Michigan.
Michigan, meanwhile, sunk into their worst down period to that point in history, finishing unranked for 4 straight seasons 1934-1937. But Minnesota's stay at the top didn't last long. They were #1 all-time for just 2 seasons, 1937 and 1938. In 1939, Minnesota finished unranked and Michigan finished #2, putting the Wolverines back on the all-time throne. And Michigan held on to the throne a very long time, from 1939 to 1966.
The team that passed them up was, of course, Notre Dame. Another Michigan down period had opened the door for this coup, as the Wolverines finished unranked 3 straight seasons 1965-1967. Notre Dame had been a strong minor and mid-major type team early in the 20th century, but in the 1920s they emerged as a major football powerhouse under legendary coach Knute Rockne. Still, the program had a lot of catching up to do, and it took subsequent eras of greatness under Frank Leahy and Ara Parseghian for Notre Dame to finally become the all-time top program.
Notre Dame's stay at the top lasted from 1967 to 1974. After 1974, Ara Parseghian retired, and Notre Dame finished unranked in 1975, Michigan #10, putting the Wolverines back on the throne for good. They have held on to the #1 all-time spot ever since, the longest period of time by far that any team has held the throne. And their lead over most top programs is now so huge, as you can see in the list on the left, that it will likely be decades before anyone overtakes them again, if ever.
Princeton is the top all-time program that is not in the FBS today, and they are still holding on to a spot in the all-time top 25. Yale was knocked out of the all-time top 25 just this year (2016), by Stanford.
The top 6 non-FBS teams of all time are all Ivy League teams. The 7th is the University of Chicago, which was a Big 10 team until they abolished their football program in 1939.
Six teams that I placed at the bottom of the list have zero poll points. These are teams that had been ranked in AP polls, but which I tossed out when fixing those AP polls. These teams are Utah State, Northern Illinois, Central Michigan, Western Kentucky, George Washington, and Boston University.