Is it healthy or is it damaging to exercise a muscle while it's still sore from a previous workout?
In general, training while sore is perfectly fine and usually beneficial, subject to a few caveats discussed below. Almost every serious athlete frequently trains while still a bit sore from the previous workout.
The first thing to understand is supercompensation: a short while after training, your muscles end up a bit stronger. If you train again during this period, your next supercompensation period will be slightly stronger again. Repeat this cycle with proper timing over and over again and you'll gain strength in the most efficient way possible. Wait too long and the muscle returns back to a "normal" state and strength gains are slowed.
The second thing to understand is that soreness has nothing to do with the effectiveness of your workout. It's more of a side effect, which is actually not well understood. Depending on your training level and workouts, you may still be sore even when your muscles are in the supercompensation period. If this happens, you just need to put up with it and train sufficiently hard to keep the gains coming.
BTW, this is also a reason why split routines - where you train each muscle once a week - are very inefficient for the vast majority of beginner & intermediate lifters. Your muscles hit the supercompensation period in far less than a week and have started to return to normal by the time you train again. In this respect, fully body routines 2-3 times per week are going to produce strength gains much faster.
Some caveats: if you are sore to the point that it interferes with the workout (ie, you can't do the motion correctly), then you won't gain much from exercise and are better off resting. A rule of thumb to determine this is to do a thorough warm-up: if everything loosens up and you feel ok, train on. If you are still stiff and it hurts to move, rest. If you are new to a certain type of exercise, soreness levels will be much higher, so the first few weeks may require skipping more workouts than normal (don't make it a habit though). Finally, more advanced lifters who are moving a lot of weight will take longer to recover, which will impact things like workout frequency and volume.
Should you workout, my answer is an unequivocal yes, while I do not know the frequency of your workouts or the intensity. I workout each muscle type every 48 hours even if the muscles still ache. I have been doing this for many years and had no issues. Most trainers recommend it.
Working out often is great for building muscle and maintaining a routine is just as important even if you do not feel like you should. Do it.
In the gym this feeling is called DOMS (delayed onset muscle soreness), and is caused by muscle tearing brought on by your previous workout session. Its also a sign you had a good exerting workout.
You will not feel DOMS often, once you get used to your workouts or reach a plateau with your routine you will stop feeling the DOMS and its a sign you need to make the routine more challenging.
Your muscles need time to recover. Thats why you don't work on your arms every day for example.
Mon = arms
Tue = legs
Wed = back
However, stretching and light workouts can actually help soreness and stiffness.
I would say it isn't a good idea to do heavy training consecutively on the same muscle group.
sometimes when you're starting out a new exercise like a cardio routine your muscles can get sore because they're not used to being worked. In that case I would say that it would be just fine (with moderation).
Either way you need to let your muscles recover properly and it's a good idea to change up your routine day-to-day or to have a staggered schedule.
In any case, overworking can hurt your results.