When you're dealing with difficult clients, protect your reputation first. It's more valuable than what may be happening at the moment. Remain professional and, if you can't walk away, be happy knowing that you've got your integrity even if you made less money than you thought you would.