First, you'll want to pick a set of exercises to do and learn how to do them with proper form. Then you can start increasing the weight and set a schedule.
In general, I would recommend a full-body workout that hits all the muscle groups with a few compound lifts. However, I'll try to craft a workout for you here that fits your stated goal with the equipment that you have.
For exercises, try:
You can experiment more as you get comfortable, but this might be a good workout to start with. You can do it as often as you feel energetic and not sore, but a good starting goal might be 2x/week; err on the side of an extra day of rest rather than straining yourself because muscles grow when they recover. I would suggest doing 3-5 sets of 8 reps of each with a couple minutes rest between each set.
As far as how much weight to use, you need to first use a very light weight and make sure that you have your form down (pay attention to the tips and the motions in the videos). When you're comfortable with the movements, add weight for each exercise gradually. If you're doing 8 reps per set, then you'll want a weight at which you'd fail after perhaps 10-12 reps.
A few other hints that will help pay off down the road:
- Keep your core tight when doing these exercises
- Squeeze the bar when you're lifting, especially for the dumbell curls.
- Lower the weight slowly; these are called negatives and can be as beneficial as lifting the weight up