What happens to a prenup if the marriage didn’t occur?

My former fiancé and I had a prenup drafted and we signed it. However, we broke up a few weeks before the wedding. The prenup said I would add my ex’s name to my house and he would have an interest in it. He’s now saying the prenup is still a legally binding contract because we both signed it and it was notarized. He is demanding that I ‘buy him out’ of his interest in my house or he is going to sue me to enforce the contract and get his name added. Is the prenup still valid if we didn’t get married and if so, what do I need to do to keep him from getting his name on the house deed?