Only if players chance to notice a squeak while seeking a cave may they be even marginally beneficial. Then, players can mine in the direction of the squeaks, which will eventually expose a cave.
Since the lowest nighttime light level is four, bats can only spawn at light levels three or lower, which means they can only be found in caves. They statistically appear to favor flying east over the other destinations for some reason.
Additionally, bats will disappear if players travel too far away, eliminating the chance to keep them as a pet.