If depends on your regular exercise routine. If you only do light exercise then it's not a problem to skip exercising for a week or longer. People who do heavy exercise almost every day may experience problems. Heavy exercise burns a lot of energy (1000 Kcal in my case), it is burden on the body that has become part of the regular routine. You are used to eating and sleep while exercising this hard. If you suddenly withdraw the exercise part of this routine then the eating and sleeping part may suffer. If you are very ill, then that illness will force you to do this, you may not be able to eat much due to the illness and the need for sleep will increase anyway, so in that case there isn't an issue.
The problem is when you are only a bit ill, like the cold you are suffering from. Then withdrawing the exercise part completely may overcompensate things, after a few days you may not have a good appetite and not sleep well. This may then feed back on your recovery of the cold, as not sleeping well and eating less well won't help. Obviously it's no good to become unwell during exercise either, so taking things easy during exercise is the best option.