If I take an 'off the shelf' starting strength workout and increase the repetitions to the 8-12 range per set, will it be an effective hypertrophy workout?
Day 1 Squat 3x12 Bench 3x12 Snatch 5x8 Day 2 Squat 3x12 Press 3x12 Dead 1x8 Chins
Starting Strength is developed with a purpose, and it's written the way it is for a reason. You can morph it however you'd like, but then it's no longer SS3x5, and for better or worse, you can't expect the same results.
The heart of Starting Strength is an interdependent synthesis of the following factors:
There are other contributing elements, but I'd argue those are the core of the program. The key is that each element depends on the others. Changing sets of 5 to sets of 12 cannot be done without upsetting the balance: the progressive overload scheme will have to change, since sets of 12 respond differently than fives to increased weight. Total volume will go way up, which might affect later sets in the workout, especially the power cleans.
Do not do snatches in sets of 8. First of all, sets of 8 in the snatch are ridiculous for anything except conditioning or sport-specific training for Olympic lifting. Second of all, snatches aren't in Starting Strength--and power cleans--which are in Starting Strength--are a totally different animal with sets of more than 5, just like snatches.
If you want a hypertrophy program, a hypertrophy program would be a better choice than Starting Strength with a different rep range slapped on the box. Hypertrophy-specific programs add volume and hypertrophy-rep-range work using more appropriate exercise selection and ordering than this modified pseudoprogram. Or, just do Starting Strength and eat a lot. People get bigger doing that.
Progressive overload is considered the most important principle behind hypertrophy, so increasing the weight, repetitions (reps), and sets will all have a positive impact on muscle growth.
That said, SS is designed carefully for new lifters, changing the volume in such a way will not benefit you as well as picking a intermediate hypertrophy programme and going with it.