Indeed: remove the relations prior to refreshing will make the custom relations be replaced with the native ones. Though you don't have to remove the custom ones, for the code they're alike, so it won't make a difference. So to save yourself some time, just keep the custom ones if you can live with the small blue dots in the icons in the designer