Milne was born near Paisley, Ontario. A childhood interest in art revived while he was teaching led him to take a correspondence course, and eventually he travelled to New York City to continue his studies. This was somewhat of an exception in the early twentieth-century Canadian art scene as the majority of artists went to Europe to study. While in New York City, Milne worked as a commercial illustrator for several years before deciding to give up this work and devote his time to painting. Shortly after making this decision, he moved to Boston Corners in New York. Throughout his life Milne sought the peace and solitude of a rural life.

In his paintings, Milne explored different viewpoints. He greatly admired the work of Tom Thomson but had little interest in the nationalistic approach of the Group of Seven. His themes range from landscapes to views of towns and cities, still lifes and imaginary subjects. His experiments with different media and changing viewpoints show his interest in the process of painting itself. In 1929, Milne returned to settle permanently in Canada, stopping for brief periods in Temagami, Weston, and Palgrave. He built a secluded cabin at Six Mile Lake, north of Orillia, and spent the next six years painting, for the most part alone.

Milne was interested in 'pure' painting, in "adventures in shape, colour, texture and space" as he called his watercolours of the 1930s and 1940s. The change from the less vibrant drybrush "adventures" to the fantasy watercolours is often attributed to the birth of his only child, David Jr., born to Milne's second wife when Milne was sixty. His young son encouraged him to adopt a new, vibrant and often whimsical approach to his art. Milne spent the rest of his life in Uxbridge, north of Toronto, exploring the Haliburton and Bancroft areas, as well as the city of Toronto.