If you are not accustomed to exercise -- and it sounds like you aren't -- then yes, such minimal activity will build muscle.
Up to a point.
At that point, doing 20 pushups will not make you stronger; it will make you good at doing 20 pushups.
To get stronger, you will need to add more weight (like switching to one arm pushups). That will make you stronger.
Up to a point.
At that point, you will need to continue increasing the weight. This trend will continue as your body adapts to each new weight load.
Pushups can only take you so far. Eventually you'll need to start benchpressing (unless you are content to stay with the strength pushups have given you).
P.S. Increasing the number of reps will also build a little strength, but progressive overload (adding a little weight each workout) is much more effective at building muscle.