Milk has a considerable amount of fat. So while in a weight-loss regiment, should milk be avoided completely?
Eating Fat Does Not Make You Fat
Excess fat on the body is an entirely different thing from the fat that you eat. Science and experience agree on this: here's a study entitled Dietary fat is not a major determinant of body fat, and here's Gary Taubes' overview on the subject, appropriate for the layperson. It's a common misconception that eating fat somehow turns into adipose tissue.
Milk might be bad for weight-loss (or it might not), but it wouldn't be because it has a significant proportion of fat. In fact, since fat is satiating--it makes you feel full--I am of the opinion that if one's goal is losing body fat, then fat (like whole-fat milk, or even cream) should be one of the things you eat.