Medium trust and serialization

Posts   
 
    
DenisMono
User
Posts: 10
Joined: 14-Apr-2011
# Posted on: 14-Apr-2011 12:30:15   

Hi, We have a problem with serializing LLBLGen entity collections (adapter-based scenarios) in medium trust Web environments. I went through all forum posts related to this topic, but found no answer: http://www.llblgen.com/tinyforum/Messages.aspx?ThreadID=18893 http://www.llblgen.com/tinyforum/Messages.aspx?ThreadID=17630 http://www.llblgen.com/tinyforum/Messages.aspx?ThreadID=15694 http://www.llblgen.com/tinyforum/Messages.aspx?ThreadID=15446

Our ASP.NET application is placed in Microsoft's Web gallery, and they recently started insisting on medium trust support. In our opinion, the default medium trust policy is way too restrictive, and there are not too many advanced apps that can work without reflection and serialization. So, we have changed the default policy to support it:

<PermissionSet class="NamedPermissionSet" version="1" Name="ASP.Net">
    <IPermission class="AspNetHostingPermission" version="1" Level="Medium"/>
    <IPermission class="DnsPermission" version="1" Unrestricted="true"/>
    <IPermission class="EnvironmentPermission" version="1" Read="TEMP;TMP;USERNAME;OS;COMPUTERNAME"/>
    <IPermission class="FileIOPermission" version="1" Read="$AppDir$" Write="$AppDir$" Append="$AppDir$" PathDiscovery="$AppDir$"/>
    <IPermission class="IsolatedStorageFilePermission" version="1" Allowed="AssemblyIsolationByUser" UserQuota="9223372036854775807"/>
    <IPermission class="PrintingPermission" version="1" Level="DefaultPrinting"/>
    <IPermission class="SecurityPermission" version="1" Flags="Assertion, SerializationFormatter, Execution, ControlThread, ControlPrincipal, RemotingConfiguration"/>
    <IPermission class="SmtpPermission" version="1" Access="Connect"/>
    <IPermission class="SqlClientPermission" version="1" Unrestricted="true"/>
    <IPermission class="WebPermission" version="1">
        <ConnectAccess>
            <URI uri="$OriginHost$"/>
        </ConnectAccess>
    </IPermission>
    <IPermission class="ReflectionPermission" version="1" Unrestricted="true"/>
 </PermissionSet>

As you can see, ReflectionPermission is set to unrestricted, and SerializationFOrmatter flag was added to SecurityPermission.

The problem happens when we try to serialize the EntityCollection via ObjectStateFormatter to be saved in the ViewState.

The following code throws an exception:

System.Web.UI.ObjectStateFormatter objFormatter = new System.Web.UI.ObjectStateFormatter(); 
UserEntity ent = new UserEntity(); 
EntityCollection<UserEntity> coll = new EntityCollection<UserEntity>(); 
coll.Add(ent); 
objFormatter.Serialize(coll);
[SecurityException: Request failed.]
   System.Reflection.CustomAttribute._CreateCaObject(Void* pModule, Void* pCtor, Byte** ppBlob, Byte* pEndBlob, Int32* pcNamedArgs) +0
   System.Reflection.CustomAttribute.CreateCaObject(Module module, RuntimeMethodHandle ctor, IntPtr& blob, IntPtr blobEnd, Int32& namedArgs) +170
   System.Reflection.CustomAttribute.GetCustomAttributes(Module decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType, Boolean mustBeInheritable, IList derivedAttributes) +1356
   System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeType type, RuntimeType caType, Boolean inherit) +184
   System.ComponentModel.ReflectTypeDescriptionProvider.ReflectGetAttributes(Type type) +326
   System.ComponentModel.ReflectedTypeData.GetAttributes() +303
   System.ComponentModel.DefaultTypeDescriptor.System.ComponentModel.ICustomTypeDescriptor.GetAttributes() +74
   System.ComponentModel.TypeDescriptor.GetAttributes(Type componentType) +86
   System.ComponentModel.ReflectedTypeData.GetConverter(Object instance) +379
   System.ComponentModel.DefaultTypeDescriptor.System.ComponentModel.ICustomTypeDescriptor.GetConverter() +81
   System.ComponentModel.TypeDescriptor.GetConverter(Type type) +77
   System.Web.UI.ObjectStateFormatter.SerializeValue(SerializerBinaryWriter writer, Object value) +3144

The following code works, as we are serializing a List instead of our EntityCollection:

List<UserEntity> list = new List<UserEntity>(); 
list.Add(ent); 
objFormatter.Serialize(list); 

We really need this to work, as we used entity collections all over the place. We have tried to enable FastSerialization via SerializationHelper.Optimization property, no luck. All versions of LLBLGen exhibit this problem on both .NET 3.5 and 4.0.

MTrinder
User
Posts: 1461
Joined: 08-Oct-2008
# Posted on: 14-Apr-2011 20:27:35   

As you have seen from the previous threads this is something that we have struggled with in the past - the solution has either seemed to be to work around the issue, or switch hosting provider... simple_smile

Have you had a look at the suggestions in the following article http://www.west-wind.com/Weblog/posts/6344.aspx ?

Do MS themselves have a support team for the web gallery ? - do they have any ideas ?

As the error seems to be coming from Reflection trying to access a custom attribute, are you able to identify which attribute this may be - is it something you could live without ?

Thanks

Matt

DenisMono
User
Posts: 10
Joined: 14-Apr-2011
# Posted on: 14-Apr-2011 22:24:34   

Hi Matt, Unfortunatelly, switching the hosting provider is not an option here - we have an application that should be used by many users on many hosting providers. Personally, I hate the whole idea of giving developers a powerful set of tools like reflection and serialization and than saying that we basically cannot use it at all because some smart guy thinks that it could pose a security threat. However, it is not our call.

We already saw the blog post you are referring to, as well as many more online resources. If you look at my initial message, you will notice that the reflection is actually enabled, and so is serialization. This is a custom policy, as I am aware that it will never work with standard medium trust. The problem is somewhere on your end, and MS guys will not deal with it: a single entity can be serialized, a list of entities can be serialized also, but EntityCollection always throws an error. It is really easy to reproduce, so I would appreciate if you could try to debug it. If this is not related to reflection, what may be the problem?

DenisMono
User
Posts: 10
Joined: 14-Apr-2011
# Posted on: 14-Apr-2011 22:36:37   

Just to add to my previous message, we don't need this for some exotic purpose. The goal is to store the EntityCollection in the ViewState, and the code I sent earlier just isolates the moment when things start to go wrong. Other storage mechanisms (like Session) are simply not suitable for the functionality we need.

daelmo avatar
daelmo
Support Team
Posts: 8245
Joined: 28-Nov-2005
# Posted on: 15-Apr-2011 08:05:42   

We are trying to figure this out. What is you LLBLGen version and Runtime Library version?

David Elizondo | LLBLGen Support Team
DenisMono
User
Posts: 10
Joined: 14-Apr-2011
# Posted on: 15-Apr-2011 10:09:35   

LLBLGen version 2.6, runtime version 2.6.9.616. We have also tried it with LLBLGen 3.1, same results.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39588
Joined: 17-Aug-2003
# Posted on: 15-Apr-2011 11:25:12   

All fine and good, but it's really unclear what exactly is causing this. You didn't post the whole exception message, so all we have is the stacktrace. This looks like to originate on a custom attribute reflection. The strange thing is: why would this fail? I have no idea. We can go into a rabbit hole of trying out all possible things and spend a lot of time trying out things, but that's not really productive. Microsoft can say it's our fault, but frankly, without more knowledge WHAT fails and WHY we can't really do anything to solve it, because we don't know WHAT fails and more importantly: why.

When you simply add the collection to the viewstate (and let viewstate do the serialization) does that work? (like I said, it's a guessing game, and we won't do that, but it's what I can think of). Also, adapter entities have a custom attribute on collection navigator properties (e.g. Customer.Orders). This attribute is defined in the ormsupportclasses, but frankly, I really fail to see why this would fail. The only problem we had in the past for medium trust is reflection permissions.

A workaround for you can be to store the entitycollection in the session using a datasource control and set its cachelocation to session.

Again, if we know what's causing this and why we could look into it. But as it fails on Microsoft's servers (if I understand you correctly) with a cryptic error I don't understand, there's little we can do.

Btw, the stacktrace suggests it ends up in non-managed code. Reflecting over the serialize method, it dives into TypeDescriptor.GetConverter() and eventually dies in a security issue. (which one, unknown!). It does that GetConverter call to see whether it has an ICustomTypeDescriptor. Our code doesn't use that, so it will fail for that path anyway and always has to do the serialization to memory stream anyway. So you can also do that yourself -> Check ObjectStateFormatter.SerializeValue() at the bottom.

What you can do is: // use fast serialization var ms = new MemoryStream(); var bf = new BinaryFormatter(); bf.Serialize(ms, entityCollection); var bytes = ms.ToArray(); ms.Close(); ms.Dispose();

And store 'bytes' into viewstate. This does custom serialization, doesn't follow the path of the serializer you're using and as you're using fastserialization, completely bypasses the serialization of Microsoft, unless you have custom objects in your object graph, in which case the fast serializer will use a memory stream and binary formatter.

Frans Bouma | Lead developer LLBLGen Pro
DenisMono
User
Posts: 10
Joined: 14-Apr-2011
# Posted on: 15-Apr-2011 12:05:03   

Frans, I think that you didn't understand me, we are not talking about Microsoft's or any other particular server environment. It happens everywhere, even on our development machines with the medium trust policy I sent. The problem is not theirs, it is universal.

We have spent a lot of time trying to make our products compatible with the customized medium trust environments that is in place in a vast majority of shared hosting providers. Everything else is solved, now we have to do something with LLBLGen EntityCollections.

The strack trace I sent is everything I got. The issue is very easy to reproduce - you don't even have to pull the data from the database, just instantiate an arbitrary entity, add it to the collection and try to serialize the collection. On the other hand, the session-based approach you are suggesting is a no-go. Session and ViewState both have different uses, and this particular scenario (which is quite often) data needs to be used only accross postbacks. If we start to push this kind of data to the session state, we will get much more memory problems down the road.

DenisMono
User
Posts: 10
Joined: 14-Apr-2011
# Posted on: 15-Apr-2011 12:32:34   

Otis wrote:

Btw, the stacktrace suggests it ends up in non-managed code. Reflecting over the serialize method, it dives into TypeDescriptor.GetConverter() and eventually dies in a security issue. (which one, unknown!). It does that GetConverter call to see whether it has an ICustomTypeDescriptor. Our code doesn't use that, so it will fail for that path anyway and always has to do the serialization to memory stream anyway. So you can also do that yourself -> Check ObjectStateFormatter.SerializeValue() at the bottom.

What you can do is: // use fast serialization var ms = new MemoryStream(); var bf = new BinaryFormatter(); bf.Serialize(ms, entityCollection); var bytes = ms.ToArray(); ms.Close(); ms.Dispose();

And store 'bytes' into viewstate. This does custom serialization, doesn't follow the path of the serializer you're using and as you're using fastserialization, completely bypasses the serialization of Microsoft, unless you have custom objects in your object graph, in which case the fast serializer will use a memory stream and binary formatter.

I didn't notice this part of your answer while I was writing my previous post, it appears to be added later. While in theory this could work, it basically means that every EntityCollection will be serialized two times - once to ByteArray using the code above, and later when this array gets picked up by the ObjectStateFormatter.Serialize. Judging from our previous experiences with serialization, this could easily have both memory and perfomance implications. I will not even start explaining what we need to do afterwards to optimize the ViewState and remove it from the generated HTML. This is why I would not prefer to use this workaround.

The bottom line is that the security problem happens only with LLBLGen EntityCollections. We cannot debug this issue, and although I feel that deep down it is yet another MS programming gem, we cannot solve it without you.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39588
Joined: 17-Aug-2003
# Posted on: 15-Apr-2011 12:34:41   

Sure, but what can we do? it dies somewhere deep in MS' code on some attribute chase which will never succeed anyway. Which attribute, it's unclear, and even if we know which attribute, I have no idea how to fix that. The attributes on the collection class are all needed: they're all simple .NET's own attributes. Frankly I haven't seen this issue before about attributes being reflected which failed in a serialization error and therefore I have no idea what 1) causes it and 2) what fixes it. Sorry.

Anyway, as the serializer method will try this code path of the typedescriptor first (which will fail anyway, as we don't implement a type descriptor) and then will do a simple serialization into a memory stream, you can do this yourself. So instead of calling the MS' serializer, do the serialization in a method which simply serializes to a memory stream and store the byte array in the viewstate.

That would solve it for you as it won't run into the problem with the typedescriptor discovery (which is the one which fails, but I don't know why). Unless you added your own custom attributes to the EntityCollection<T> class and didn't mark the assembly with AllowPartialTrustedCallers.

DenisMono wrote:

Otis wrote:

....use memory stream serializer

I didn't notice this part of your answer while I was writing my previous post, it appears to be added later. While in theory this could work, it basically means that every EntityCollection will be serialized two times - once to ByteArray using the code above, and later when this array gets picked up by the ObjectStateFormatter.Serialize. Judging from our previous experiences with serialization, this could easily have both memory and perfomance implications. I will not even start explaining what we need to do afterwards to optimize the ViewState and remove it from the generated HTML. This is why I would not prefer to use this workaround.

Look into the Serialize method of the ObjectStateFormatter, it does this too: serialize the object (the collection) to a memory stream and then hands off the byte array to the follow up. It has special case code for byte[] so it doesn't have to serialize anything. So in effect this workaround is not giving you extra memory problems. The problem you're facing is that it never reaches that stage of serializing to a memory stream because it dies on a custom attribute witch hunt, for a reason unknown.

The bottom line is that the security problem happens only with LLBLGen EntityCollections. We cannot debug this issue, and although I feel that deep down it is yet another MS programming gem, we cannot solve it without you.

I can repeat it a 1000 times, but I don't have a single clue what it could be, nor can I remove any attributes from the collection class, as they're all needed. Not only that, I have no clue whatsoever why they would fail a weird security issue, which is not described in any way whatsoever (e.g. what right is missing, the core element needed to get anything solved in this case).

Without essential information what exactly fails and why (e.g. which right is missing) I can't do anything. Look for yourself in the sourcecode, it's actually very simple: there's no special attribute defined other than attributes which are simple, public and inside .NET.

Frans Bouma | Lead developer LLBLGen Pro
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39588
Joined: 17-Aug-2003
# Posted on: 15-Apr-2011 12:56:03   

Googling the method it dies: System.Reflection.CustomAttribute._CreateCaObject

(which is a non-managed method) suggests that the attributes which refer to types which might require full trust (read: are MS f*ck ups with respect to security) like toolbox types might be the problem.

As I said, we can't remove those attributes. I'll create a build for you without these attributes for you to test.

Edit Could you please try the attached ORMSupport Classes for v2.6 to see if these fix the problem? The entitycollection class now doesn't have any toolbox /designtime attributes on it. If this works, we know what to fix.

Btw, what I wanted to see is the complete exception MESSAGE, you didn't paste that only the exception stacktrace (a part of it). Reading this: http://www.west-wind.com/Weblog/posts/24566.aspx it suggested the attribute in question was mentioned in the exception message. we can't fix it without this message. It has the format like: [SecurityException: Request for the permission of type 'System.Security.Permissions.UIPermission, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' failed.]

Frans Bouma | Lead developer LLBLGen Pro
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39588
Joined: 17-Aug-2003
# Posted on: 15-Apr-2011 13:23:21   

I think it's the ToolboxItem attribute: NamedPermissionSet for full access to system resources. Demand values: LinkDemand, InheritanceDemand. Associated state: FullTrust

The collection has a ToolboxItemAttribute referencing the type of an entitycollection toolboxitem. This type is specified as a type, not as a string. The type makes it cause the security issue I think, the string would solve it, IF that works for what's it suppose to do. In old(er) vs.net versions this wasn't always the case... deep sigh

Frans Bouma | Lead developer LLBLGen Pro
khorvat avatar
khorvat
User
Posts: 65
Joined: 17-Feb-2011
# Posted on: 15-Apr-2011 13:28:37   

Hi,

here is the complete error message that we get:


Server Error in '/' Application.

Security Exception

Description: The application attempted to perform an operation not allowed by the security policy.  To grant this application the required permission please contact your system administrator or change the application's trust level in the configuration file. 

Exception Details: System.Security.SecurityException: Request failed.

Source Error: 

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

Stack Trace: 


[SecurityException: Request failed.]
   System.Reflection.CustomAttribute._CreateCaObject(Void* pModule, Void* pCtor, Byte** ppBlob, Byte* pEndBlob, Int32* pcNamedArgs) +0
   System.Reflection.CustomAttribute.CreateCaObject(Module module, RuntimeMethodHandle ctor, IntPtr& blob, IntPtr blobEnd, Int32& namedArgs) +63
   System.Reflection.CustomAttribute.GetCustomAttributes(Module decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType, Boolean mustBeInheritable, IList derivedAttributes) +604
   System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeType type, RuntimeType caType, Boolean inherit) +129
   System.RuntimeType.GetCustomAttributes(Type attributeType, Boolean inherit) +65
   System.ComponentModel.ReflectTypeDescriptionProvider.ReflectGetAttributes(Type type) +335
   System.ComponentModel.ReflectedTypeData.GetAttributes() +38
   System.ComponentModel.DefaultTypeDescriptor.System.ComponentModel.ICustomTypeDescriptor.GetAttributes() +58
   System.ComponentModel.TypeDescriptor.GetAttributes(Type componentType) +32
   System.ComponentModel.ReflectedTypeData.GetConverter(Object instance) +95
   System.ComponentModel.ReflectTypeDescriptionProvider.GetConverter(Type type, Object instance) +24
   System.ComponentModel.DefaultTypeDescriptor.System.ComponentModel.ICustomTypeDescriptor.GetConverter() +50
   System.ComponentModel.TypeDescriptor.GetConverter(Type type) +23
   System.Web.UI.ObjectStateFormatter.SerializeValue(SerializerBinaryWriter writer, Object value) +704

Version Information: Microsoft .NET Framework Version:2.0.50727.4455; ASP.NET Version:2.0.50727.5053

We have applied the patch that you have sent and the good news it that we don't get the error message any more.

Can you please specify which attributes did you remove so we could check possible implications ?

khorvat avatar
khorvat
User
Posts: 65
Joined: 17-Feb-2011
# Posted on: 15-Apr-2011 13:31:31   

Could you provide us with the assembly version with "ToolboxItemAttribute" references entitycollection as a "string" so we can test it ?

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39588
Joined: 17-Aug-2003
# Posted on: 15-Apr-2011 13:38:57   

That exception message is indeed useless... disappointed (it was worth a try)

The attributes I removed are: [Designer(typeof(EntityCollectionComponentDesigner)), ToolboxItem(typeof(EntityCollectionToolboxItem))]

The 'Designer' and 'ToolboxItem' attributes are safe. The EntityCollectionComponentDesigner class derives from component designer, which has no requirements. The EntityCollectionToolboxItem derives from ToolboxItem, which has full trust requirements. (who designs these security crap... disappointed )

Anyway, the ToolboxItem is required for winforms design time databinding. But it can be solved with a string representaton of the full type. So we'll try that and I'll attach the dll to this post, just a sec.

Edit. Attached.

It doesn't have any impact on you or your code, the toolboxitem is for winforms databinding. We now added the type as string, we have to test it to see if it indeed works.

If you need a 3.1 fix, we'll provide one too. We'll have to alter the collection class in that assembly as well.

btw, the assembly you have is a debug build. We'll provide a release build once it works simple_smile

Frans Bouma | Lead developer LLBLGen Pro
DenisMono
User
Posts: 10
Joined: 14-Apr-2011
# Posted on: 15-Apr-2011 14:08:38   

"Who designs this security crap" got asked a lot around here in the past few days. simple_smile

Anyway, It appears to be working without problems. We will conduct more tests and get back soon. A version for 3.1 would be appreciated, since that's what we will use for all new apps. Thanks for the excellent support. It makes all the difference.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39588
Joined: 17-Aug-2003
# Posted on: 15-Apr-2011 14:17:36   

DenisMono wrote:

"Who designs this security crap" got asked a lot around here in the past few days. simple_smile

haha I can imagine... This whole mess stinks, not only the medium trust mess, but also the design time attribute mess causing all this.

Anyway, It appears to be working without problems. We will conduct more tests and get back soon. A version for 3.1 would be appreciated, since that's what we will use for all new apps. Thanks for the excellent support. It makes all the difference.

We'll release v2.6, v3.0 and v3.1 builds with the fix later today (likely within the hour or so).

Frans Bouma | Lead developer LLBLGen Pro
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39588
Joined: 17-Aug-2003
# Posted on: 15-Apr-2011 14:50:51   

We've uploaded the new builds.

Frans Bouma | Lead developer LLBLGen Pro