StackOverflowException in Save

Posts   
 
    
PeterVD
User
Posts: 15
Joined: 16-Jan-2007
# Posted on: 02-Jun-2016 14:40:42   

Hi,

Question related to thread: https://llblgen.com/tinyforum/Messages.aspx?ThreadID=22679 (Sorry, I hijacked that thread. Please ignore/delete my message over there)

Is the Save process still using recursion (iso a loop) to traverse the object graph?

We're currently running into these stack overflow exceptions in the Save-method (recursive save).

We are developing a clinical trial management system and during development we didn't run into this problem, because we were testing with small trials, but the real-life trials can be really big. This in combination with a big object graph (some screens fetch and prefetch more than 20 entities, because this data is needed in the logic) is causing these Stack overflow exceptions.

The first error was in our code. (I traverse the object graph to set a weak reference to the root entity so it can be marked dirty when anything changes in the object graph) This was fixed by changing our code for traversing the object graph with a loop iso recursion. This fixed the issue in our code, but now we run into the error in the LLBLGen Save-method.

So now we are stuck. Unless we do a rewrite of these screens and our framework, but we don't really have the time or resources to do this.

We are currently running version 4.2. (I don't mention the exact version of the DLL's because I don't expect this to be a simple change in a minor version)

Kind regards,

Peter

PS: I admit it, this is an extreme case. The bug I'm currently trying to fix is caused by an objectgraph with more than 30.000 entities!! Obscene, I know, but that is how it was developed.

Walaa avatar
Walaa
Support Team
Posts: 14946
Joined: 21-Aug-2005
# Posted on: 02-Jun-2016 18:57:59   

Is the Save process still using recursion (iso a loop) to traverse the object graph?

Yes it uses recursion, if you instruct it to do so. Otherwise it will just save the first level of entities (entity). Are you passing true for the recurse parameter in the Save method? And the main question would be, do you want to save the entire graph (the dirty objects only) or not?

PeterVD
User
Posts: 15
Joined: 16-Jan-2007
# Posted on: 03-Jun-2016 09:24:26   

Hi Walaa,

I wasn't referring to the recursive behaviour of the save, but to the actual implementation or programming of this recursive behaviour.

Is it still programmed using a recursive method call (and possibly leading to a StackOverflowException) or using a while-loop?

I was hoping this code would have been refactored at some point (in version 5.0 for instance), because we are running into the StackOverflowException at the moment. As I said, it's an extreme case we have. But with a refactoring in LLBLGen, this could be avoided. (I know, that's easy for me to say simple_smile )

Kind regards,

Peter

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39588
Joined: 17-Aug-2003
# Posted on: 03-Jun-2016 11:05:33   

PeterVD wrote:

Hi Walaa,

I wasn't referring to the recursive behaviour of the save, but to the actual implementation or programming of this recursive behaviour.

Is it still programmed using a recursive method call (and possibly leading to a StackOverflowException) or using a while-loop?

This post (in a thread about a different topic) describes it: http://www.llblgen.com/tinyforum/GotoMessage.aspx?MessageID=121313&ThreadID=21465.

I was hoping this code would have been refactored at some point (in version 5.0 for instance), because we are running into the StackOverflowException at the moment. As I said, it's an extreme case we have. But with a refactoring in LLBLGen, this could be avoided. (I know, that's easy to say for me to say simple_smile )

simple_smile As described in the link above, we use Depth First Search to determine the topological order in the DAG made up by entity class instances. We in theory could switch to Kahn's (https://en.wikipedia.org/wiki/Topological_sorting) but in practice there would be a problem: we don't store the graph in a Graph structure with vertices and edges, it's an in-memory object model of entity class instances and references, so to make Kahn's algorithm work, we need to traverse the graph of instances handed to the sorter (where the crash happens). So we first build the graph into an adjacency list datastructure but that precisely goes wrong in your case, so it wouldn't help.

So we then have to rewrite the recursive traversals to non-recursive traversals which gives us a problem: we then have to prove our algorithm is correct (we now can rely on Tarjan and others having done that for us in their papers). And I'm not going to do that, simply because on paper the algorithm is quite simple but in practice implementing this without errors isn't. Believe me when I say there are some mega graphs with all the edge cases you can think of being persisted with this, and we had just 1 bug in it in 10 years (which was caused by one such graph with multiple paths between nodes using inheritance, fun! wink )

So correctness trumps everything in this case, and we'll not sacrifice that here. Of course, there's always the theory of 'use a stack yourself' and that might work, but it's not that simple in practice, considering the sorter uses 2 passes (one to build the adjacency lists, one to sort these).

There's a solution for your edge case however: http://stackoverflow.com/questions/4513438/c-sharp-recursion-depth-how-deep-can-you-go

It means you have to start a thread and give it a stackspace > 1MB (the default .NET one). In general this isn't needed but in your case you need it.

Frans Bouma | Lead developer LLBLGen Pro
PeterVD
User
Posts: 15
Joined: 16-Jan-2007
# Posted on: 03-Jun-2016 11:50:38   

Hi Frans,

Thanks for the link. simple_smile Increasing the stacksize is perfect as a hotfix solution. We will refactor our code to fetch smaller objectgraphs as soon as possible.

Kind regards,

Peter