As anything it depends on your goals. As a powerlifter and strongman, I have to lift heavy weights. For Powerlifting it's the heaviest deadlift, but for strongman it could be one set weight for as many reps as I can within a timelimit. That's very different from a bodybuilder who is looking for a better physique, where weight is irrelevant.
Addressing Discomfort
I notice that I seem to get a dull ache in my lower back the next day, which has caused me to rethink what I am doing.
That's very common. I call it a low back "pump" which is usually what happens when you do enough volume. It can be very uncomfortable, but it is a back pump it goes away within a day or two.
However, to minimize the pain as you add more weight on the bar over time, you'll want to help your body out:
- Concentrate on being and staying tight before you pull. In fact the last thing you do should be to contract your lats to pull your body in to the bar.
- Work on your glutes and your upper body (snatch grip deadlift for 2x10-15 is good for this)
- Above all: do not jerk the bar
Those steps have helped me minimize the pump. Unfortunately it is a part of doing deadlifts, but it's nothing to be concerned about. A sharp pain or a pain that doesn't go away after a couple days is something to be concerned about.
Addressing Set/Rep Schemes
Deadlifts are meant to be done heavy. Many lifters do well with one heavy set for 3-5 reps, and then back off reps that are much lower in weight. I've seen approaches that are all over the map by people with much larger deadlift records than I have.
Your training approach really depends on what you are trying to get out of the deadlift.
Training for 1RM
The goal here is to get your body used to heavy pulls, and put in the volume needed for hypertrophy with much lighter weights. Aim for a 3-5 rep max (you can keep the same weight and add reps or alternate weeks with your current 5RM working to your 3RM), and do back off volume at 100 lbs lighter.
Training for Reps
The goal here is to build up endurance with a heavy weight. You'll only be doing one set, but start with something you can do for 5 reps and work to 8-10 reps. Increase weight and start over at 5 reps.
Physique
Deadlifts aren't even necessary. Most bodybuilders I know opt for variations like Romanian Deadlifts and such that are meant for high reps.
My goals are not to enter competitions etc but rather to increase my overall strength. One specific thing is I'd like to get bigger, more powerful legs to develop a stronger front kick in karate, as well as a stronger core for the upper body exercises I do.
For that purpose, I would recommend front squats. You can supplement the front squats with romanian deadlifts or kettlebell swings (hip hinge exercise).
Front squats have you balancing the bar across your deltoids forcing you to keep your upper back extended and placing more of a load on your core.
Deadlifts are more useful for brute force breaking a hold that someone has you in on the ground with grappling techniques.
While I think deadlifts are a great tool, they just aren't the right tool to develop front kick strength.