I have heard that drinking room temperature water is easier for you body to absorb but I'm not sure if that is true or not.
Also, I am wondering, after a hard workout, will ice water hydrate you more effectively because it will also cool you down?
There isn't any significant difference between both options.
That being said, cold water hydrates more effectively than room temperature. According to this source,
During intense physical activity, the body's core temperature rises above the normal 98.6°F (37°C). Drinking cool water lowers the body's temperature and helps it settle back to its normal range. Studies have also shown that cold water 41°F (5°C) is absorbed more quickly from the stomach than warm, abating dehydration and allowing you to play harder and enjoy your game of soccer even more.
Another source conducted a study with participants drinking cold and room temperature water; the conclusion being
The participants who had consumed the cold water during rest periods had a significantly smaller rise in core temperature compared to those who consumed the room temperature water.
However, while cold water hydrates better, warm water relaxes the muscles better and makes it ready for more exercises:
Subjects participating in the room temperature condition were able to perform significantly more bench press reps to failure than when they participated in the cold condition.
So, cold water caused a slight performance hit, according to the study. Also, given that Olympic swimmers/divers shower in warm water before their races in order to prepare their muscles, I consider the study to be accurate.
That being said, I would still choose cold water over room temperature while exercising.