Vaughan's father, Asbury "Jake" Vaughan, was a carpenter by trade and played guitar and piano. Her mother, Ada Vaughan, was a laundress who sang in the church choir. The Vaughans lived in a house on Brunswick Street in Newark for Vaughan's entire childhood. Jake was deeply religious. The family was active in New Mount Zion Baptist Church at 186 Thomas Street. Vaughan began piano lessons at the age of seven, sang in the church choir, and played piano for rehearsals and services.

She developed an early love for popular music on records and the radio. In the 1930s, she frequently saw local and touring bands at the Montgomery Street Skating Rink. By her mid-teens, she began venturing illegally into Newark's night clubs and performing as a pianist and singer at the Piccadilly Club and the Newark Airport.

Vaughan attended East Side High School, then transferred to Newark Arts High School, which opened in 1931. As her nocturnal adventures as a performer overwhelmed her academic pursuits, she dropped out of high school during her junior year to concentrate on music