How do I obtain a refund for a deposit on a mobile home?

A home inspections was done on a mobile home I was going to purchase and revealed no major issues. I had given the realtor a $4,000 deposit for the mobile home. On the walk through the morning of the closing, it was revealed that the wall was damaged by water, as well as water damage to the rug and wood underneath, as well as mold. I cancelled the purchase and asked for my money back, since the contract stated that the property would be in the same condition as was on inspection. The realtor won't give me the money back, and the seller refuses to as well. I consulted an attorney and he said that it would cost more for him to fight it, and that maybe I should go to small claims court, but I would only get $2,000 back. Do I have any other recourse to get all my money back?