You need to draw the line and make it clear to him that this is unacceptable. Instead of retaliating you should discuss with him, not in front of your wife but when alone about the reasons for his being so critical of you and try to address his concerns.
Affectionately,
Alok da.