I had not thought of setting the key, then calling the property. Makes total sense, in fact I do it myself.
It seems the best way to handle this would be to not load relationships where the entities PK is used and the entity is new. This should always result in a no-op.
If the new setting could work that way, it would be great. Should cover both cases pretty well.