Well if you customize the config file to include the connection string specific to each client deployment, then at the same manner you'd customize the overwrite section.
But if you accept the connection string from the user as an input to the application and you set it at code. Which removes the burden of writing connection strings in the config file at each deployment, then solution of having the overwrite section with an empty string should suit you well as this doesn't need to be changed from deployment to another.
So having an overwrite section with an empty string is not specific per deployment