First Linq query very slow.

Posts   
 
    
PatrickD
User
Posts: 65
Joined: 05-Sep-2006
# Posted on: 04-Sep-2009 17:15:35   

I discovered that the very first Linq query takes a long time to process (over 1 second). After some investigation, it seems that this is caused by the fact that for every entity in the DAL an entity factory is created (which is triggered by LinqUtils.HandleMemberMappedOnRelationIsPartOfEntity()). As I've got a few hundred entities this takes some time. It only happens the first Linq query, but I don't know how is determined when the factories should be created. If this is done by for example a static variable, I am wondering what will happen in an ASP.Net application which is stateless. Will I encounter this more than once?. 1 second extra for a query is not acceptable and would make Linq to LLBLGen not usable for me.

Plz your thoughts on this.

Regards,

Patrick

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39588
Joined: 17-Aug-2003
# Posted on: 04-Sep-2009 17:35:47   

This delay is just once, as a static type is initialized. This happens once per AppDomain so in an asp.net application, you barely notice it as it's drowned in the time needed to recompile / start the whole asp.net site, and an asp.net site from then on uses the same appdomain every time so it's not that you run into this penality for every request, only if the app is recycled, which is slow anyway. Also, the profile picture isn't really showing reality, as profiling is slower than real-time. There are more static initializations when the application starts, e.g. inheritance info provider, field info provider, persistence info provider, the dqe is initialized, the first connection created is also taking relatively long, e.g. 100-200ms (as a physical connection has to be established, this time occurs AFTER 'Open()' is called on the ADO.NET connection), though due to connection pooling you don't notice it the next time. I also dont think the factory creation is really the slow factor, as it's just a class instantiation. So if you open the trace below the create call, you'll see what the real slowness is, likely the initialization of other static classes, all happening once.

Frans Bouma | Lead developer LLBLGen Pro
PatrickD
User
Posts: 65
Joined: 05-Sep-2006
# Posted on: 07-Sep-2009 16:34:12   

Otis wrote:

There are more static initializations when the application starts, e.g. inheritance info provider, field info provider, persistence info provider, the dqe is initialized, the first connection created is also taking relatively long, e.g. 100-200ms (as a physical connection has to be established, this time occurs AFTER 'Open()' is called on the ADO.NET connection), though due to connection pooling you don't notice it the next time.

Well, that's the 'strange' thing. It does not occur on application start but on the first Linq query. Before the first Linq call, I have done already many calls to the database with the regular API, so connections are pooled, static classes initialized, etc. I am a bit curious why these factory classes are created at the first Linq Query and not at the first call to LLBLGen. (Of course, I can move it to the startup phase by rewriting my first database call to a Linq call. sunglasses )

Otis wrote:

I also dont think the factory creation is really the slow factor, as it's just a class instantiation. So if you open the trace below the create call, you'll see what the real slowness is, likely the initialization of other static classes, all happening once.

The slowness is caused by the fact that that all entity factory classes are created at once. An average call takes 4 ms., but there are over 200 calls.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39588
Joined: 17-Aug-2003
# Posted on: 08-Sep-2009 12:25:36   

PatrickD wrote:

Otis wrote:

There are more static initializations when the application starts, e.g. inheritance info provider, field info provider, persistence info provider, the dqe is initialized, the first connection created is also taking relatively long, e.g. 100-200ms (as a physical connection has to be established, this time occurs AFTER 'Open()' is called on the ADO.NET connection), though due to connection pooling you don't notice it the next time.

Well, that's the 'strange' thing. It does not occur on application start but on the first Linq query. Before the first Linq call, I have done already many calls to the database with the regular API, so connections are pooled, static classes initialized, etc. I am a bit curious why these factory classes are created at the first Linq Query and not at the first call to LLBLGen. (Of course, I can move it to the startup phase by rewriting my first database call to a Linq call. sunglasses )

That's because the linq provider uses a class called ElementCreator which uses the EntityFactoryFactory class to produce entity factories for the linq provider, which is generic code. This way, the linq provider can work with any generated code base without the need for reflection etc. The thing is that the EntityFactoryFactory isn't used in the rest of the codebase (only in fast serialization), so the static ctor of that class runs when the first linq query is ran, as it is then brought into scope.

Otis wrote:

I also dont think the factory creation is really the slow factor, as it's just a class instantiation. So if you open the trace below the create call, you'll see what the real slowness is, likely the initialization of other static classes, all happening once.

The slowness is caused by the fact that that all entity factory classes are created at once. An average call takes 4 ms., but there are over 200 calls.

I profiled an entity fetch query, and the initial create call on an entity is a bit slow, the rest of the calls are fast (e.g. 1st call to Create takes for CustomerEntity 8.3ms, but fetching 91 entities in total spends in the create method in total 0.3 ms.

However, it's a bit odd that you see this slowdown after a lot of entities have already been created. Anyway, if you create an ElementCreator instance at the start of your application, you should not see the slow down in the first linq query, correct?

For example, when I profile two entity fetch queries after each other, the first takes the hit of the factory creation, the next one is very fast (as in < 1ms). I therefore am a bit puzzled, why you see slow factory creation (which is due to the fact it creates entity types for the first time) way after entities have been fetched already.

Frans Bouma | Lead developer LLBLGen Pro
PatrickD
User
Posts: 65
Joined: 05-Sep-2006
# Posted on: 08-Sep-2009 12:37:45   

Otis wrote:

Anyway, if you create an ElementCreator instance at the start of your application, you should not see the slow down in the first linq query, correct?

I guess, but since there are a number of database queries done at startup, I gonna rewrite some of them (and in the future if time permits all database calls) to Linq queries, so that way the slow down will be hidden in the total time of application startup.

Thanks,

Patrick

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39588
Joined: 17-Aug-2003
# Posted on: 08-Sep-2009 13:46:43   

I still think it's a non-issue, as the 'slow-down' is IMHO hardly noticable compared to the asp.net slowdown at startup (and barely a second). So 'very slow' is IMHO a bit 'overdreven' wink

Frans Bouma | Lead developer LLBLGen Pro
PatrickD
User
Posts: 65
Joined: 05-Sep-2006
# Posted on: 08-Sep-2009 14:17:14   

Otis wrote:

I still think it's a non-issue, as the 'slow-down' is IMHO hardly noticable compared to the asp.net slowdown at startup (and barely a second). So 'very slow' is IMHO a bit 'overdreven' wink

I agree, it will not be noticable once I have moved the slow-down to startup. (And like Google translate would say: 'about floating is also a box' confused )