Poorest fifth of households have seen a fall in income