Property Case

I am a Sr. Citizen aged 71 years, I am a co-owner of residential property in Delhi. In Jan. this year I signed a collaboration agreement with a builder, sole purpose of which was to persuade another co-owner for an agreement. He paid by a cheque. Now things did not materialize as expected by him, I returned the money by cheque, he did not cancel the agreement as agreed earlier, when I insisted, he sent me a Notice saying that cannot cancel agreement unilaterally, by returning the amount by cheque. He has also said that he paid a big sum to me in cash for which no receipt is available, this was paid in presence of an employee of him. Now because of his this attitude I am not keen to enter an agreement. He wants me to continue as per agreement. I do not want to continue as it was prepared for a different purpose, Terms are not acceptable to me. In reality, I did not receive the money he is talking about. This is a false statement, apparently to force me to agree on his terms. Pls advice