Mance began playing professionally in his early teens and left college in 1947 to join Gene Ammons' band, where he began his recording career. Mance also worked with Lester Young before being drafted into the U. S. Army in 1951. In the 36th Army Band, he served with another notable Jazz musician: Julian "Cannonball" Adderley.

After his discharge from the Army in 1953, Mance worked as a sideman and accompanist with musicians such as Dinah Washington, Cannonball Adderley, and Dizzy Gillespie before recording his first album as a leader and forming his own trio in 1961. To date he has recorded over thirty albums, for labels such as Verve, Jazzland, Riverside, Capitol, Atlantic, Milestone, Polydor, Inner City, JSP, Nilva, Sackville, Bee Hive, and others.

Since 1988 he has been a member of the faculty of the Jazz and Contemporary Music Program at the New School University in New York City. In 1997, Junior Mance was inducted into The International Jazz Hall of Fame.