I can swim a mile (66 lengths, 1650y) of breasstroke in about 40 minutes - 45 mins. But I cant swim more than 3 or 4 laps of freestyle. They use the same muscles. Why is this?
Without actually seeing your stroke, I am going to make the guess that it is related to breathing.
Breastroke, it is easy to do a slow stroke (and a 40 minute mile qualifies) and breathe in and out regularly while your face is above the water. In freestyle, the tendency for people that are not competitive swimmers is to hold their breath while their face is in the water, and then try to breathe out and in while their face is out of the water. This leads to hyperventilation and early fatigue.
A proper freestyle is breathing out completely while your face is in the water, then breathing in during the recovery phase when your face comes out of the water.
Lastly, while they do use many of the same groups of muscles, they use them in different patterns which can also contribute to fatigue.
Get someone to watch you and see what you are doing wrong. If it is breathing start with every two strokes (but change sides each length or you will get a bad neck, that might also be an issue) even do as a friend did to begin with get a snorkle. You can work on stroke, such as a pull boy between your upper leg to stop legs wondering out. Start from the beginning, get someone to watch you and don't worry of you go slower to start. When I train I am often mid pack in training and drills but top few on pace, all about the drills and training, speed is the sum total. But do do look after your shoulders/neck as you increase your distance as I managed to lock mine and that hurts!
They don't use the same muscles, and certainly don't use them in the same way, so I'd dispute that premise.
In any case, the breastroke motion is not as naturally explosive than the other strokes in its movements, so it makes sense that you'd be able to go further without getting as fatigued. I can't think of any stroke where you'd be able to keep going as much as breaststroke.