The main factor holding one back is the fear response. Our bodies are trying to keep us safe so we should listen. This article describes the effect you speak of:
Under acute stress, the body's sympathetic nervous system prepares the body for sustained, vigorous action. The adrenal gland dumps cortisol and adrenaline into the blood stream. Blood pressure surges and the heart races, delivering oxygen and energy to the muscles. It's the biological equivalent of opening the throttle of an engine.
Vladimir Zatsiorsky, a professor of kinesiology at Penn State who has extensively studied the biomechanics of weightlifting, draws the distinction between the force that our muscles are able to theoretically apply, which he calls "absolute strength," and the maximum force that they can generate through the conscious exertion of will, which he calls "maximal strength." An ordinary person, he has found, can only summon about 65 percent of their absolute power in a training session, while a trained weightlifter can exceed 80 percent.
Under conditions of competition a trained athlete can improve as much as 12 percent above that figure. Zatsiorsky calls this higher level of performance "competitive maximum strength." This parameter is not a fixed number—the more intense the competition, the higher it can go, as the brain's fear centers progressively remove any restraint against performance.
If you are talking about people who get massive adrenaline dumps in extreme emergencies, these folk are able to push past their fear response but will likely damage themselves in the process. But as the article notes, our body can recognize high-importance events (competitions and emergencies) and can operate at much closer levels to its theoretical maximum.