Robbins was born Harold Rubin in New York City, the son of Frances “Fannie” Smith and Charles Rubin. His parents were well-educated Jewish emigrants from the Russian Empire, his father from Odessa and his mother from Neshwies (Nyasvizh), south of Minsk. Robbins later falsely  claimed to be a Jewish orphan who had been raised in a Catholic boys’ home. Instead he was raised by his father, a pharmacist, and his stepmother, Blanche, in Brooklyn.

Robbins dropped out of high school in the late 1920s to work in a variety of jobs, including errand boy, bookies’ runner, and inventory clerk in a grocer’s. He was employed by Universal Pictures from 1940 to 1957, starting off as a clerk and rising to an executive.