Swim Outlet has a nice discussion on swim caps. Different swim cap materials for swimming include: latex, silicon or neoprene. For water jogging or water aerobics where your face and head are not necessarily in the water, they suggest that a lycra cap is cooler.
For medium or long hair they recommend:
If you have medium to long hair, then a silicone cap will suit you best. Silicone caps outlast their latex counterpart, are more tear resistant, and don't tug at your hair. Typically softer to the touch, a silicone cap is gentler on the hair and easier to get on and off after a workout.
For open water swims, they suggest bright colors to help make you more visible. They also suggest that neoprene and silicone caps give more "insulation and floatation".
If you have any sensitivity or are allergic to latex, be sure to opt for an alternate material.