The problem with body weight exercises
In the world of free weights, you have something called a one-rep max (ORM) which defines the maximum amount of weight you can lift at your current level of strength. Using the ORM you can calculate the intensity of any workout with a simple formula, which if I recall correctly is
(reps * weight) / orm. What's noteworthy here is that if you want to adjust the intensity of your workout, you can either adjust the reps or the weights (or both). This makes tweaking and fine-tuning the intensity of a workout quite easy, and by extension it greatly reduces the need for a pyramid program.
However, in the world of body weight (BW) exercises, things are completely different. By removing ORM and weights from the equation, the only way to increase the intensity for any particular workout is to increase the number of reps. If I want to improve my strength and disrupt homeostasis, my only option is to do more reps than I did in the previous workout. At this point the problem should be transparent: you can only increase the number of reps for so long before you hit a plateau.
This is why the pyramid approach exists and is so effective for BW exercises: because it makes it easier for you to increase the total number of reps per exercise and break through plateaus. The specific factors that contribute to this programs effectiveness are:
BW exercises can be extremely anti-novice due to their high starting weight versus FW where you can start with as low as 1 lb weights. A beginner doing push-ups to exhaustion may only be able to do 2-3 reps per set, and as a result an increase in intensity becomes exponentially harder in the beginning stages. In the above example, simply adding 1 rep would end up being a 33-50% increase in intensity. Pyramid address this by dispersing the added reps across many sets, for example 4/1/3/2/2 instead of 4/4/exhaust and give up.
Let's say you do 4 sets of 5 push-ups, where 5 is your exhaustion point. How do you increase intensity? You have to go to 6, but doing so would be a 20% increase in intensity, quite a large gap to jump. Perhaps this gap is so large that you can not jump it by simply staying at 4x5 to build strength. Doing 4x5 might only be enough to increase your strength by 15%, not the 20% needed, and now you're at a point where doing 4x5 is no longer enough to disrupt homeostasis and build muscle; you need to increase the intensity to do so (catch-22!). This is why the pyramid program exists, because it allows you to increase the intensity of BW exercises at a quicker but more manageable pace, and thus build strength faster.
If you are stuck on a certain rep for a long enough time, psychological barriers can come into play that convince your body not to go past that rep, regardless of whether or not you are able to. By taking intensity increases in smaller and more manageable chunks, you will hit fewer walls and be less frustrated with your progress. As a result, psychological barriers will be far less significant of a factor in your progression.