Jeremy Lott -- Why We Should Get Rid of the Vice Presidency: makes the case that the office of Vice President is superfluous and ought to be eliminated. His argument is specious. He quotes Federalist 68 to bolster his case, to wit, " '[the office] has been objected to as superfluous, if not mischievous' but... was needed to break tie votes in the Senate. But some state legislatures have a clear alternative rule: if it ties, it fails."
He conveniently omits Hamilton's response to this argument, also in 68: "[T]o secure at all times the possibility of a definite resolution of the body, it is necessary that the [Vice] President should have only a casting vote. And to take the senator of any State from his seat as senator, to place him in that of President of the Senate, would be to exchange, in regard to the State from which he came, a constant for a contingent vote."
There's another important reason to preserve the Bucket of Warm Spit that Hamilton did not properly foresee. Factional politics and the rise of political parties, contra Washington, came into being in the Age of Jackson with the result that we had to endure our first impeachment hearings in the early 1840s, when the House actually debated the removal of President John Tyler --his only real crime being to have offended the Whigs who put him on the 1840 ticket as VP under Wm. Harrison (and the Democrats, whom he angered because he used to be one). As it is the legislature that defines the order of succession beyond that prescribed within the Constitution (as modified by the 25th Amendment, naturally), there is a very clear conflict of interest in eliminating a President via impeachment when his replacement would come from the legislature. To me this is a implicit violation of the principle underlying the whole "separation of powers" doctrine. And we've seen where this could lead: Andrew Johnson's non-removal from office by the Senate was in no small part contingent on the knowledge that he would be succeeded by the President pro tempore of the Senate, a Radical Republican whom certain of the dissenting Republicans believed would abuse the Presidency. (Recall that at the time there was no provision for replacing a Vice President who became President; Johnson inherited the #1 position when Lincoln was assassinated.)
Like it or not, the Vice Presidency now stands as insurance against politically motivated impeachment and removal. Without it, a partisan Congress could be more able to seize control of the country. Given their absolutely stellar record of late, I'm happy to let them cool their heels, even if this means putting up with Joe Biden (or Dick Cheney or Al Gore or Dan Quayle).