John Williams was born on 24 April 1941 in Melbourne, Australia, to an English father, Len Williams, who was later the founder of the London Guitar School, and Malaan (née Ah Ket), an Australian-Chinese mother (a daughter of Melbourne barrister William Ah Ket). In 1952, the family relocated to England. Williams was taught initially by his father and educated at the Friern Barnet Grammar School, London. From the age of eleven he attended summer courses with Andrés Segovia at the Accademia Musicale Chigiana in Siena, Italy. Later, he attended the Royal College of Music in London from 1956 to 1959, studying piano because the school did not have a guitar department at the time.

Upon graduation, he was offered the opportunity to create such a department. He took the opportunity and ran the department for its first two years.

Williams has maintained links with the college (and with the Royal Northern College of Music in Manchester) ever since.