Minister under fire for attack on lawyers and charities over asylum