As well as the numerous vending machines, all convenience stores stock bottles of water in various sizes. If you eat in a restaurant, most places offer tap water for no extra charge. Just ask the staff. In fact, tap water is a source of pride for many of Japan's mountainous regions as cool, clean water is a product of the surrounding peaks. Make sure to try some if you find yourself in these areas.
It is important to keep hydrated during your travels, especially in the summer. If you are not used to the humidity—and even if you are—it can be very easy to suffer from dehydration. Sports drinks packed full of minerals are a good way to stay hydrated, but drink plenty of water too. Make sure to keep some water with you at all times.