One way of looking at aerobic function is as the bodies ability to burn fat for fuel at a given pace or speed. Anaerobic function then is the bodies ability to burn sugar for fuel when we move beyond the pace at which we can burn primarily fat. Heart rate is often a good indicator of which mode we're in.
Intervals and speed training, then, burn mostly sugar for fuel. Weight loss when engaged in this type of activity comes from the elevation of the metabolic process extending long after the workout, and is the net result of how elevated the metabolic rate is and for how long, and the caloric intake necessary to provide sugar to fuel this type of workout.
Training at a slightly lower intensity, where we burn primarily fat for fuel, can do two things:
- Train our body to burn fat more efficiently. Fat is actually a much richer energy source than sugar, so the body will happily burn fat unless we provide it with extraordinary amounts of sugar or train frequently at a rate where it's impossible to burn fat.
- Increase the pace at which we can train while still in the aerobic metabolic range. For example: I use a heart rate monitor to avoid entering the anaerobic while training. At first, I had to run maddeningly slow to do so, somewhere around 14 minutes per mile. Six months later I was happily running 9-minute-miles at the same heart rate, for the same length of time. This is referred to in pro cycling as the "base-building" training phase, where they ride hours and hours of long, "easy" miles in the winter early spring to prepare for the summer racing season.
This means that the difference between you and your friend is that you burn more fat than he does at a given pace, which while it would seem strange given your statement that he is more "lean and fit" than you, is at least a hypothesis that shouldn't be rejected outright. If you ride lots of slightly slower miles, your aerobic function can indeed be better than your friend, resulting in your ability to burn fat on your Saturday morning rides, while your friend relies primarily on sugar/carbs for fuel and therefore has to replenish them after the ride.
Note: much of what I've said is based on Phil Maffetone's studies of aerobic function and fitness (example here). Phil takes a somewhat eccentric approach to life, training, and medicine, but the aerobic fitness training works for me, so why throw the baby out with the bath water?