Goldfinger was formed by former Electric Love Hogs member John Feldmann on vocals and guitar, Simon Williams on bass, Rem osano on drums, and Charlie Paulson on guitar. At the time the band was formed, Feldmann and Williams were working at the same shoe store. Before they were signed to a major label, they released an EP titled Richter, which received favorable reviews and a substantial amount of airplay on college radio, leading to Goldfinger signing a major-label contract with Universal Records.

Many of the songs on Richter are demo versions of songs on the band's full length debut, Goldfinger, which was released on February 29, 1996. The song "Here in Your Bedroom" from the album was especially popular and helped the band gain a solid fanbase. In 2006, Alternative Press listed the self titled Goldfinger album as one of the "10 Albums That Shaped 1996" alongside No Doubt, Weezer and others. The band released its second album, Hang-Ups, on September 9, 1997. Although the album was less successful than the band's debut, the band still maintained a devoted fan base in the hardcore community. Founding bassist Simon Williams left the band following Hang-Ups. During this time, Goldfinger released a popular cover of the song "More Today Than Yesterday" by Spiral Starecase, which also helped the band stay out of obscurity.