Born in Darlington, Allison started out as a journalist in his native North East, before moving to London in 1905. He became associated with Woolwich Arsenal, becoming the club's programme editor, and later commentating on the very first FA Cup final to be broadcast on the radio, between Arsenal (as they had since been renamed) and Cardiff City in 1927.

Allison later became the club's secretary and then managing director, before taking over as first-team manager in June 1934, after the sudden death of legendary Arsenal boss Herbert Chapman earlier that year. Under Chapman and caretaker manager Joe Shaw, Arsenal had already won the League Championship twice in a row (1933 and 1934), and Allison made it a hat-trick, winning a third successive title in 1935. He also won the FA Cup in 1936 and the League again in 1938.

With the advent of the Second World War, official competition in England was suspended; after hostilities had ended, many of the players that had made Arsenal a success (such as Cliff Bastin, Alex James and Ted Drake) had retired from playing. Arsenal finished a disappointing 13th in 1946-47, and Allison decided to step down and retire from the game.

Allison famously appeared in a movie set at Highbury, The Arsenal Stadium Mystery, where he had a speaking part as himself. Amongst his lines included one uttered at half time: "It's one-nil to the Arsenal. That's the way we like it.", a line which had resonance with the team's penchant for 1-0 scorelines many decades later.