This really depends on the distribution. If some or all of the people in that bottom 20% are very, very stupid it could actually work out that 80% are above average, because the average is being pulled down by the people at the bottom.

This is why we have different averages like mean, median, mode, and RMS because they each give you different interpretations of the raw data. For example the mean electro motive force of the grid is around 0 volts because it spends as much time in the negative as the positive. We use RMS here because negative numbers become positive when squared.