Big object graphs witt circular references

Posts   
 
    
shennig
User
Posts: 48
Joined: 14-Nov-2006
# Posted on: 24-Jun-2011 10:00:29   

hi.

i'm using llblgen pro 3.0 final with adapter. i transfer entities via wcf and it works fine in general. but sometimes it seems that i get problems with circular relations and the resulting object count caused by backreferences. in detail: i have a DataPoint with several DataValues. the DataValues references their DataPoint via foreign key in db and therefor the DataValueEntiy has a navigator to his DataPoint and the DataPoint has a navigator collection to his DataValues. now i try to transfer round about 1000 DataPoints each with 10 DataValues, but wcf fails with the exception that the maximum data length is exceeded.

but if i delete the collection navigator from DataPoint to DataValue it works! so the problem seems to be the serialization with the circular reference wich causes to much overhead.

i have read that the DataContract Serializer for wcf could be use the IsReference property, so the resulting serialization contains IDREF wich result in a much slender xml format. but i don't now how to turn this form of serialization on?!?

how can i handle great object graphs with circular references?

thank you for help,

shennig

Walaa avatar
Walaa
Support Team
Posts: 14950
Joined: 21-Aug-2005
# Posted on: 24-Jun-2011 11:41:39   

Would you please post the exact exception error and stack trace if possible.

In general you can increase the limits in the binding configuration (maxReceivedMessageSize & maxBufferSize).

shennig
User
Posts: 48
Joined: 14-Nov-2006
# Posted on: 24-Jun-2011 12:09:08   

Walaa wrote:

Would you please post the exact exception error and stack trace if possible.

In general you can increase the limits in the binding configuration (maxReceivedMessageSize & maxBufferSize).

hello walaa.

i have increased maxReceivedMessageSize & maxBufferSize to maximum (2147483647) but it does not help. the exception is "maximum request length exceeded".

i have had this problem before, which result in very poor performance. by looking on the xml serialization i found that the references from the parent to its child-entities in a 1:n relation (via the collection navigator) causes a problem. because every child has a reference to his parent (must have to set the foreign key in DB) but in the resulting xml in the parent xml element is also a list of all children fully specified once more, so the xml gets very big if there are parents with many children. my solution was to delete the parent to children navigator and the transfer via wcf then was very fast.

but now i can't delete the navigator and there are so many parents with children, that i run in the exception you see below.

if the xml serialization would work with idref instead of fully specified xml elements, everything would be fine!


<Exception>
<ExceptionType>System.ServiceModel.CommunicationException, System.ServiceModel, Version=3.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</ExceptionType>
<Message>maximum request length exceeded</Message>
<StackTrace>
bei System.ServiceModel.Activation.HostedHttpRequestAsyncResult.GetInputStream()
bei System.ServiceModel.Activation.HostedHttpContext.HostedHttpInput.HostedInputStream..ctor(HostedHttpContext hostedContext)
bei System.ServiceModel.Activation.HostedHttpContext.HostedHttpInput.GetInputStream()
bei System.ServiceModel.Channels.HttpInput.get_InputStream()
bei System.ServiceModel.Channels.HttpInput.ParseIncomingMessage(Exception&amp; requestException)
bei System.ServiceModel.Channels.HttpRequestContext.CreateMessage()
bei System.ServiceModel.Channels.HttpChannelListener.HttpContextReceived(HttpRequestContext context, ItemDequeuedCallback callback)
bei System.ServiceModel.Activation.HostedHttpTransportManager.HttpContextReceived(HostedHttpRequestAsyncResult result)
bei System.ServiceModel.Activation.HostedHttpRequestAsyncResult.HandleRequest()
bei System.ServiceModel.Activation.HostedHttpRequestAsyncResult.BeginRequest()
bei System.ServiceModel.Activation.HostedHttpRequestAsyncResult.OnBeginRequest(Object state)
bei System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.WorkItem.Invoke2()
bei System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.WorkItem.Invoke()
bei System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.ProcessCallbacks()
bei System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.CompletionCallback(Object state)
bei System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.ScheduledOverlapped.IOCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped)
bei System.ServiceModel.Diagnostics.Utility.IOCompletionThunk.UnhandledExceptionFrame(UInt32 error, UInt32 bytesRead, NativeOverlapped* nativeOverlapped)
bei System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)
</StackTrace>
<ExceptionString>System.ServiceModel.CommunicationException: maximum request length exceeded. ---&gt; System.Web.HttpException: maximum request length exceeded.
   bei System.Web.HttpRequest.GetEntireRawContent()
   bei System.Web.HttpRequest.get_InputStream()
   bei System.ServiceModel.Activation.HostedHttpRequestAsyncResult.GetInputStream()
   --- Ende der internen Ausnahmestapelüberwachung ---</ExceptionString>
<InnerException>
<ExceptionType>System.Web.HttpException, System.Web, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</ExceptionType>
<Message>Maximale Anforderungslänge überschritten.</Message>
<StackTrace>
bei System.Web.HttpRequest.GetEntireRawContent()
bei System.Web.HttpRequest.get_InputStream()
bei System.ServiceModel.Activation.HostedHttpRequestAsyncResult.GetInputStream()
</StackTrace>
<ExceptionString>System.Web.HttpException: Maximale Anforderungslänge überschritten.
   bei System.Web.HttpRequest.GetEntireRawContent()
   bei System.Web.HttpRequest.get_InputStream()
   bei System.ServiceModel.Activation.HostedHttpRequestAsyncResult.GetInputStream()</ExceptionString>
</InnerException>
</Exception>


Walaa avatar
Walaa
Support Team
Posts: 14950
Joined: 21-Aug-2005
# Posted on: 24-Jun-2011 15:48:22   

Max request length can be increased by.

<system.web><httpRuntime maxRequestLength="51200".... 

What you are looking for is setting preserveObjectReferences in the DataContractSerilaizer to true, this is best explained here: http://blogs.msdn.com/b/sowmy/archive/2006/03/26/561188.aspx And an amendment here: https://connect.microsoft.com/wcf/feedback/details/307497/custom-datacontractserializeroperationbehavior-does-not-work-anymore-with-net-3-5-b2

Setting PreserveObjectReferences property, will make your service not interoperable (i.e. expected anyway while sending entities over the wire). Otherwise you'd better use DTOs with DataContracts defined and use DataContract.IsReference property instead.

Please note that generally it's not recomended to object graphs over the wire. By the way, are you using netTcpBinding, this can minimze the request length AFAIK?

shennig
User
Posts: 48
Joined: 14-Nov-2006
# Posted on: 27-Jun-2011 11:06:52   

Walaa wrote:

Max request length can be increased by.

<system.web><httpRuntime maxRequestLength="51200".... 

you are right. after settint maxRequestLength to 1048576 on server and client side it runs. but it took a long time.

Walaa wrote:

Setting PreserveObjectReferences property, will make your service not interoperable (i.e. expected anyway while sending entities over the wire).

interoperability doesn't matter. but i dont't understand how PreserveObjectReferences property for entities could be set. is there a way to set the IsReference property for DataContract? but entities in llblgen pro aren't signed with DataContract. Is there a way, via attributes, to set this property for entities?

Walaa wrote:

By the way, are you using netTcpBinding, this can minimze the request length AFAIK?

no. i'm using mexHttpBinding for wcf.

Walaa avatar
Walaa
Support Team
Posts: 14950
Joined: 21-Aug-2005
# Posted on: 27-Jun-2011 16:47:52   

but i dont't understand how PreserveObjectReferences property for entities could be set

It's set on the Serializer level, please follow the first link I've posted.

You have 3 choices now: 1- Set the PreserveObjectReferences as shown in the previous links. 2- Use DTO (light weight data transfere objects) and you can use the DataContract and IsReference with them. 3- Try using netTcpBinding instead of the HttpBinding, it should be faster and more compact as it uses binary encoding by default as opposed to the Text encodingused by the HttpBinding.