Should you switch to something less enjoyable but healthier? That depends on how much you enjoy it, and how integral to your habits this has become. If you find yourself "going to the gym so you can eat a brownie", then the brownie has probably become integral to your exercise as a reward for your workout habit. Exercising regularly is better for you than eating a few brownies a week, and changing habits is hard, so it's likely worth the tradeoff. Just make sure the brownies have natural ingredients: no partially hydrogenated oils, HFCS, etc. (Note: If you're trying to lose weight, you probably won't burn off as many calories in most workouts as the brownie.)
If you're just looking for calories for purely utilitarian reasons, and brownies are a convenient way of doing that, switching to something healthier is a good idea in theory, though it's difficult to find foods that dense in calories. According to NutritionData, a 60g brownie has 240 calories, which as many calories as if it were made of solid sugar! Brownies have more calories per gram than heavy cream, and almost as much as triple-cream brie. Plenty of "energy bars" are that high in calories, but they're frequently not very good (and the ones that are good are pretty much just candy bars anyway). If you're looking for that density of calories, you're going to need to eat something that's either very fatty or very dry (or both): potato chips, fatty cheese, candy bars, buttery pastries, etc. None of it is likely to be very good for you, so go with something you like. Personally, I go for the triple-cream brie, but to each his own...