The wizards represent all that the true 'muggle' most fears: They are plainly outcasts and comfortable with being so. Nothing is more unnerving to the truly conventional than the unashamed misfit!