Well, originally, all the Justices “rode Circuit,” sitting as District Court judges part of the year in various parts of the country. They only came together in order to hear appeals from the District Courts to the whole Supreme Court. The country was small, and the number of cases to be heard was small, so we needed fewer Justices. Originally, I believe there were five of them.
As time went on, and federal courts saw more action, the Congress created District Courts with their own judges. The Supremes sat as an appellate court from these. Eventually, Appellate Courts were created, too, though Justices remained active in assigned parts of the country. The number of Justices was raised to seven.
Eventually, the number was raised to nine, to keep up with the legal work. The strain of doing so, however, was beginning to show, since anybody with the means to keep appealing could demand the Supreme Court finally hear their case. Eventually, in the early 20th Century, the Congress allowed the Supreme Court to pick and choose the cases they wanted to hear. Appellants no longer get an automatic hearing by SCOTUS; rather, they have to petition for a Writ of Certiorari. If the Court “grants cert,” then they agree to hear the case; if the Court denies cert, then the Appellate Court’s disposition is final.
So, the Supremes hear only the cases they think are worth their time. Could we do with fewer Justices? Or More? Yeah, I guess. But since the Court remains in control of its docket, it is able to adjust its workload to match what they’ve got to work with. Nine seems to be a comfortable number, though there’s nothing magic about it.
That said, the Democrats’ muttering about “packing the Court” – creating more Justiceships in order to be able to fill them with reliable progressive votes – is a dangerous game to play. The Republicans have so far restrained themselves to arguments about what is fit and proper, but they should also say, loud and clear, that two can play the court-packing game. If I were Mitch McConnell, I would say that if the Dems proceed to create new Justiceships in order to pack the Court, then the next time the GOP is in control of both Houses of Congress – and there will be a next time, then they will immediately create enough new Justiceships to restore the balance – and maybe more so. You want fifteen? We’ll see your fifteen, and raise it to twenty-five.
Once the Dems realize that their strategy could backfire, their ardor for packing the Court might subside a bit. Then again, maybe not: they usually don’t recognize the stupidity of their actions until they come back to bite them (witness: eliminating the filibuster for judicial nominees).
Anyway, if the Supreme Court just did the work it was assigned -- deciding cases -- it wouldn't matter what the governing philosophy of the Justices was; indeed, for most of our history, it didn't. It was only when the Court got into the business of making the law up out of whole cloth that it mattered who sat on the Court.