Growing up working on his family's tobacco property in Myrtleford, Victoria, Ross Milne started skiing at Falls Creek when he was 13. A natural talent, he went on to win the North-Eastern District Ski Association Junior Championships for three years in a row and by 1961 he was a member of the Australian team facing New Zealand in the Inter-Dominion Championships.

After breaking his ankle in a fall, Ross came back to win the 1963 NSW Slalom title, only a few weeks after his injury. He then grabbed the Downhill and Alpine Combined national titles at the Mt Hotham National Championships later that year, coming second in the Giant Slalom.

After his national titles, Milne was selected in the Australian team that participated in the 1964 Olympic Winter Games at Innsbruck, Austria. A few days before the opening of the Games, on 25 January 1964, he tragically lost his life when he crashed into a tree during a training run for the men's Downhill. He was only 19 years of age. He was the best downhill skier in Australia at the time and had also competed in Europe the season before, touring the international circuit with Snow Australia Medal recipients Simon Brown and Peter Wenzel, among others.

His younger brother Malcom Milne followed in his tracks and competed at the 1968 and 1972 Winter Olympics. Malcom also became the first non-European skier to win a men's World Cup Downhill race (at Val-d'Isere, France on 14 December 1969), receiving Australia's first World Cup medal at the same time.