Ah, perhaps one of the most philosophically challenging aspects of physiology. This is almost a religious test within the strength training community.
I'm not sure there is a single factor. Where we can always say "This is the reason for the limit." However, I can give a few factors which may wet your appetite.
Surface Area to Volume Ratio
Cells are spheres.

This ratio of surface area to volume is important when it comes to cell function. The more volume, the more nutrition required. However, as a cell gets bigger there is a disproportionate increase in volume relative to surface area.
Credit.
Therefore, a cell can't get the required nutrition after a certain point. It's like an animal getting bigger and bigger, without increasing the size of their mouth. At some point, they can't get enough food in anymore, putting a limit on their size.
A natural position then is "So what? We can just make more cells rather than making our current ones larger." This is the difference between hypertrophy (current cells bigger) and hyperplasia (new cells). This has been debated for a long time, but last I checked no significant (skeletal muscle) hyperplasia has been found in, for lack of a better term, regular humans.

The caveat here is drugs. If you're a bodybuilder taking drugs for muscle growth, then hyperplasia may be on the table. This is believed to be one of the biggest advantages of steroids and the like.
Again, hyperplasia is an endlessly debated topic in the strength and conditioning world. It's been that way for decades. Part of the issue is counting muscle fibers is apparently something no researcher wants to do / can do precisely. But that should say something. If it's that hard to find a difference in fiber number, it's unlikely to matter much, if at all. (Whereas finding a difference in fiber size is rudimentary.)
Testosterone
Think of testosterone as workers who help hold up a building. If you have more testosterone flowing through your body -more workers- OR more room for testosterone to be housed (receptors), you can hold up more muscle- more of a building.
This is a commonly thrown out theory for why people have individual limits when it comes to how much lean mass they can hold. The amount of testosterone flowing through people tends to vary quite a bit, hence, the amount of muscle a person can hold tends to vary quite a bit. Sure, one can take drugs to increase this natural limit, but increasing the amount flowing through the body is futile without increasing how much can be used.
This is also believed to be why some respond better to steroids than others. Charlie Francis, sprint coach of the infamous Ben Johnson, said the irony with drugs is the better athletes are the ones who will get more out of them, due to more androgen receptors. You can see Lance Armstrong bring this up nowadays when asked whether he was cheating. He'll tell you everybody was doing it, but he'll honestly remark it's possible he responded to drugs better than his competitors, giving him an unfair advantage. What he's missing is that advantage is thus inherent. Where he was already ahead of them, drugs or not. (He was a teenage endurance phenom.)
Receptor wise, what you start off with is what you end up with. (At least theoretically. Not sure if this has been examined.)
I understand that might not be a satisfactory answer, but for many asking why is like asking why our genes can't reprogram themselves to grow a new finger or make ourselves taller. It's just not something humans can do. Epigenetics is not my strong suit, but I wouldn't be surprised if certain parts of the genome are forever locked away after a certain age.
Tradeoffs
When it comes to regular humans, I think it's fairly clear we have limits, and what some of those factors are. However, what's limiting one person is not guaranteed to limit another.
My personal theory for endurance limits is the amount of heat a person can dissipate. That said, two runners may be have the same limitation here, but if one has less volume on their body e.g. smaller calves, with the same surface area, then they will be able to dissipate more heat, increasing their performance. This is one reason Kenyans are so dominant. More surface area; less volume. Their limbs are long and skinny; their torsos short and narrow. But you can only decrease your volume so much. (Interestingly, you notice African-Americans dominate more as the event distance goes up. Further distance => heat becomes a bigger concern, where African-Americans have an inherent advantage?)
The tradeoff is those who are better at dissipating heat will not be able to handle cold weather as well. Whenever you get something in biology, you lose something.
In Survival of the Sickest, it's proposed type 1 diabetes is a positive adaptation to cold weather. Diabetics pee more => have less water in their body => less likely to freeze to death.
More mobile joints are less stable ones, and vice versa. This is why world class powerlifters look so stiff and why world class ballerinas look like they'd break in half with a barbell on them.
Those who try to be great at everything end up mediocre at everything. The CrossFit games are fun to watch, but none of those guys have impressive lifts or running times compared to sole lifters and runners.
The Rabbit Hole
I'd also point you to these two videos on asking "why?":
The technical: https://www.youtube.com/watch?v=36GT2zI8lVA
The funny: https://www.youtube.com/watch?v=4u2ZsoYWwJA