He was a member of the Michigan State House of Representatives from 1850-1852 and a member of the Michigan State Senate in 1856. On January 26, 1857, Ferry, with his father William Montague Ferry, platted the village of Ferrysburg, Michigan, which his father had first settled in 1834.

He was a delegate to the Loyalist Convention at Philadelphia in 1866. He was elected as a Republican to the United States House of Representatives for the 39th, 40th, and 41st Congresses, serving from March 4, 1865 to March 4, 1871. He was re-elected to the U.S. House for the 42nd Congress in the general election of November 8, 1870. The Michigan Legislature subsequently elected him to the U.S. Senate on January 18, 1871 and Wilder D. Foster was elected in a special election on April 4, 1871 to fill the vacancy in the House. Ferry was re-elected to the Senate in 1877, and served from March 4, 1871 to March 4, 1883. He was an unsuccessful candidate for reelection in 1882. Ferry was the first person from Michigan to have served in both houses of the Michigan State Legislature and in both houses of the United States Congress. The second person to do so is Debbie Stabenow.

While Senator, Ferry was chairman of the Committee on Rules (43rd through 45th Congresses) and the Committee on Post Office and Post Roads (46th and 47th Congresses), as well as President pro tempore of the Senate during the 44th and 45th Congresses. In this capacity, he became, upon the death of Vice President Henry Wilson on November 22, 1875, next in the line of presidential succession, and remained so until March 4, 1877.