depends on the sort of trip and your driving style too.
if you know it should be 31, then it should be fine sticking to that.
it only makes a difference if you have a fully loaded tiny car and you need to drive at higher speed. imagine one adult passenger would easily equate to 3 pieces of lugguage.
the handbooks i've seen actually give two (or in some cases three) pressures according to the load in the car - like from a lower pressure with just the driver in the car to a higher pressure with four or five occupants and luggage. as far as i can tell the difference between these extremes is about 4psi.
if the tyres are under or over inflated you get increased wear (costs you money!) and if the tyre pressures are uneven it can make the steering feel a bit heavy and even upset the handling of the car.