If you are exercising your abs by doing deadlifts or squats with a huge amount of weight on the bar, then yes it will make your abs stronger.
If you are doing sit-ups, you might notice that you can do more and more sit-ups the more you practice. When I first started sit-ups it was hard for me to do 40, but after doing them 3-4 times per week for a year I was able to do 200, so certainly the effect was more than just physical appearance. Some people might consider this more "endurance" than "strength" though.
If your definition of strength is more aligned with the physics definition (ability to exert force), then doing hundreds of sit-ups with no weight will improve endurance with diminishing improvement (or none at all) to strength. Doing abs exercises that involve weight (sit-ups while holding a heavy weight, resistance training with heavy weight, deadlifts, squats, etc.) will indeed make you stronger and you will see the proof yourself since if you do these exercises regularly you will see yourself being able to do them with heavier and heavier weights as time goes on.