I don't think this question is answerable in the scientific literature. It is fundamentally vague, and once made specific the number of relevant variables is too great to form a simple answer at this time.
What is our goal?
What are we trying to measure? "Mass" is a useful term in casual workout-programming discussions, but are we looking to produce muscle hypertrophy in specific muscles, or all over, or all over but in some areas more than others? Lean mass or total mass? Are we concerned with how we look during this time, or at the end of the routine, or in two years?
"Strength" is easier to measure, but it is quite distinct from mass. It's a fairly simple matter to determine strength differences over time with 1-repetition-maximum tests. I'd be quite surprised if the strength routines (3x5 and 5x5) performed worse than the bodybuilding routines (split isolation) for purposes of strength. (One problem in running the comparison would be specialization in the measured movement. Strength of a person is amorphous and impossible to measure, but any specific test of strength is vulnerable to specialization. For instance, standard measures include a 1RM leg press or leg curl, 1RM squat (what kind? Front, back, and to what depth?), or deadlift or bench or whatever, and grip strength. Powerlifting routines would have a clear advantage in the squat and deadlift simply due to practice of the movement.)
What differences are relevant?
There are so many differences between any given 3x5 routine and any given bodybuilding routine that it's impossible to isolate and measure just one. Is it the use of compound exercises versus isolation routines? Barbells versus machines? Split versus whole-body workouts? Taking or not taking rest days between workouts? Total workout volume, total weekly volume, volume:rest ratio, volume per exercise, volume per muscle, volume per movement? Or is it intensity, not volume, in each of those? What about exercise selection, exercise order, and rest periods?
Each of these matters at least a little bit. Some matter more than others. One of the biggest unmentioned differences is that most split or isolation routines use a different number of repetitions per set. Another is that they do not use a linear progression of weight, which 3x5 novice routines do. That makes quite a difference in terms of strength increases.
The differences between these are possible to determine and in many cases have been studied. However, comparing a 3x5 strength routine to a split-and-isolation bodybuilding routine requires analyzing all of them, and how they may work together or counteract each other in parallel.
It's important to note, also, that the most important factor for success in strength and mass increase has little to do with any of these differences. Compliance is the biggest determining factor of success. Did you lift heavy weights? Did you make all your workouts? Did you eat right? Did you get enough sleep? Did you keep doing these things over several months? Years?
A way forward
Since comparing "3x5 to split" is just not feasible in a single study, we must look at the totality of evidence and form a conclusion based on our judgment. It is also possible to form conclusions based on experiments that use the scientific method but that cannot reach scientific rigor: experiments with oneself, one's peers, one's trainees.
Basically, we're left with heuristics. A 3x5 compound-exercise full-body strength routine works very well for increasing strength in novices, and mass as well if they eat enough (and correctly--which is an entirely separate bag of worms). An isolation-exercise split bodybuilding routine works very well for increasing strength and mass in novices if they eat enough and correctly. Beyond that, we're left with a slew of individual preferences that will tell us how to choose between them: is their priority athleticism or is it looking good? Strength or mass? How often can they work out, and what's their background?
The question gets enormously complex. The only thing I am sure of is that praying to Science for a Single Undisputed Heavyweight Champion True Answer of the World is not the approach that will give us understanding. The truth is nuanced and comes with error bars.