Insert Performance Issue

Posts   
 
    
Puromtec
User
Posts: 76
Joined: 22-Jun-2005
# Posted on: 03-Oct-2006 20:21:44   

llblgen pro 1.0.2005.1 Final Oracle 10G Express .net 1.1

I am having performance issues with importing data into database.

The process imports many 1000's of records. Each individual record could have a corresponding child row that belongs to 1 of 10 child tables.

In llbl speak: each one of the parent entities has one child collection that receives one entity only.

I am using the UnitOfWork object to speed things up. However, LLBL continues to hit the database the whole time between commit calls.

There doesn't seem to be a way to stop db activity. When I "Add" entities to the colletions, database calls are from GetMulti, etc. To fix that, I set DoNotPerformAddIfPresent = false. That makes adding to collections faster. This doesn't help me overall because for some reason setting this flag to false llbl must call get_childCollection to access the property via parent entity and that causes a database query. I have to set DoNotPerf.. = false for every parent row, which slows things down.

Bottom line, I don't want any database activity until i call unitOfWork.Commit.

I'd appreciate any replies.

-Thanks

Puromtec
User
Posts: 76
Joined: 22-Jun-2005
# Posted on: 03-Oct-2006 21:24:47   

After looking around in the generated code, I found that the collection refetches if the following if statement is true.

if( ( !_alreadyFetchedEngExcursion || forceFetch || _alwaysFetchEngExcursion) && !base.IsSerializing && !base.IsDeserializing && !EntityCollectionBase.InDesignMode) { // LLBL gets the collection from database }

I've come up with a solution, but a dirty one. If it is a bad idea, let me know.

Solution:

Prior to setting the DoNotPerformAddIfNotPresent property and any "Add", set this thing to true.

EntityCollectionBase.InDesignMode = true;

Afterwards, set it back to false.

This is the only part of the above if clause that is public.

Walaa avatar
Walaa
Support Team
Posts: 14950
Joined: 21-Aug-2005
# Posted on: 04-Oct-2006 08:51:00   

Please provide more setails about your setup, and whether you are using SelfServicing or Adapter. And it would be of much help if you post real code (especially of your UOW usage and where exactly does it hit the database)

Please follow the guidelines in the following thread: http://www.llblgen.com/TinyForum/Messages.aspx?ThreadID=7725

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39614
Joined: 17-Aug-2003
# Posted on: 04-Oct-2006 10:58:44   

You're triggering lazy loading.

If you do: myCustomer.Orders.Add(myOrder);

you'll load myCustomer.Orders first, then add myOrder to it.

Instead do: myOrder.Customer = myCustomer;

simple_smile

Tip: data-import is slower if you do it all in 1 transaction. So if you save 1000s of entities as an import, and you do that in a uow, be aware that a transaction will be used which could slow things down a bit, especially if the transaction log of your db is small.

Frans Bouma | Lead developer LLBLGen Pro
Puromtec
User
Posts: 76
Joined: 22-Jun-2005
# Posted on: 04-Oct-2006 15:03:42   

Thanks Otis. That is definitely a better solution.

With regard to data-import tuning, I found that 75 inserts was best for my transaction size.

I'll post code next time.