Brownless was born and raised in Jerilderie, a rural town in New South Wales. He first played football for his local team, the Jerilderie Football Club, when it was part of the Murray Football League. He boarded for three years in Kilmore, Victoria, at Assumption College, one of Australia's most renowned "football nurseries", a school which has a strong tradition of producing footballers who have gone on to play in the AFL. In 1984, his final year at the school, Brownless kicked 155 goals for the school team.