Whittaker was born in Aldershot, Hampshire, though spent his early football career in the North East of England as a youth player, whilst training as an Army ordnance engineer. After serving his country in World War One, Whittaker joined Arsenal in 1919, first playing as centre-forward then as wing-half, and became a regular in the side in the 1920s, playing 70 times for the club and scoring 2 goals.

However, Whittaker's playing career was brought to an end in 1925 after a severe injury to his knee. Resolving to carry on in football, he entered study as a physiotherapist and became Arsenal's first team trainer under Herbert Chapman in 1927 (whilst still younger than many of the players on the pitch). Whittaker had an important role under Chapman in reforming the training and physiotherapy regime at the club, and played an essential part in the club's successes of the 1930s.

After Chapman's death in 1934, Whittaker continued to serve under his successor, George Allison, and also became a trainer for the England national team. The Second World War saw Whittaker work as an ARP warden, before becoming a pilot in the Royal Air Force, achieving the rank of Squadron Leader. For his service on missions on D-Day, he was awarded an MBE.

With the end of the war, Whittaker resumed his role as trainer at Arsenal. After Allison's retirement in 1947, Whittaker became the club's new manager; under him the club won the League in 1948 and 1953 and the FA Cup in 1950. However, the club's success waned in the 1950s, with Whittaker unable to attract any major stars to the club. The strain of looking after an entire club took its toll on Whittaker, with fatal consequences. He died of a heart attack in 1956, at the age of 58.