Peer's comments show Tories' true colours, say critics