Look into adrenoceptors – specifically alpha-2 and beta-2. These are receptors that control blood flow into/out of the fat cell, as well as cell metabolism (and a host of other biological functions not related to fat storage). Alpha-2 receptors are the nasty ones – areas around the hip and thighs (especially for woman), and the abdomen and low back (for men) have up to 10x the amount of alpha-2 receptors compared to beta-2. The alpha-2 receptor distribution is why it is extremely hard to lose fat around these areas, and why they are the last to go when losing weight; the alpha-2 receptors just won’t let the fat be released from the cell (and the lack of blood flow makes it hard to get the necessary fat-loss (really, fat-release) hormones (epinephrine/norepinephrine, catecholamines, and others) into the fat cell in the first place.) There are most likely evolutionary reasons for the distribution of these receptors (e.g., woman to support child bearing).
There are also biological reasons fat is not stored evenly around the body; but that’s beyond the scope of this answer.
Also, during normal activity during the day (and weeks/months/years), we are constantly storing and/or burning fat to support normal metabolic function (e.g., at night when we’re sleeping and in a fasted state we’ll burn fat). So, at least part of the uneven distribution of body fat is due to the fact that we store fat around the body when in a calorie surplus, but when we burn fat, it’s being taken from the face, arms, chest, legs, etc. preferentially, and not from the hip/thighs or abdomen/low back. Over time, this exacerbates the uneven distribution of fat around the body.
For a comprehensive examination of the above concepts, look into the work of Lyle McDonald. He goes into excruciating biological detail into the above concepts.