In Cairo tap water is heavily chlorinated and relatively drinkable, but elsewhere it is generally unsafe to drink. Bottled water is cheap and readily available in even the smallest towns, but consider using iodine or a Steripen to reduce the use of plastic bottles, especially if you're on a longer trip.
According to the CDC tap water is generally not safe for consumption.
According to the Numbeo water drinkability and accessibility index, tap water has a score of 41.65 / 100 (100 being best).
Numbeo considers this score to be "Moderate"
1 adult could save about 241$ per year in Egypt by drinking tap water instead of buying bottled water.
๐ณ Drinking tap water would also help with reducing your plastic footprint and improve environmental pollution.
Like in most countries, water quality and accessibility differs from location to location, therefore it's the best to check the information available on that place.
