Recovery Of Money

I started a committee of rs 200000 for 20 months having 20 members. I got the committee surrender after 3 months but thereafter could not pay. The person got my sign at the time of payment. Now he is demanding money including penalty but I have refused. My question is - if that person can take back the money legally.