No you will not experience over-training. What you may experience is fatigue though.
I do not believe they will cause damage despite what others may say. I find many people complain about how deteriorated their body is and get unnecessary surgeries when there was nothing wrong with them in the first place besides aches and pains from physical exertion. I know for a fact that bones, tendons, vessels, nerves, and so forth can be in pain from training and some people just go "crazy" from it. Really.
With that in mind go ahead and take it easy with excessive ballistic movements until you've worn the weights daily for a couple months.
I am currently wearing 10 lb ankle weights, 5 lb wrist weights, and a 32 lb weight vest. I don't walk long distances but I move around throughout the day. I sleep with them on.
Avoid ballistic movements until you have worn them for over a month and you will not experience much pain or discomfort. Your arms and shoulders may grow if you wear them ever day all day and increase the weight or start punching the air with them on. It's not a waste of time either way as it will always help to have a 24/7 resistance component in your training.