Newer versions and performance

Posts   
 
    
Rob
User
Posts: 54
Joined: 17-Sep-2004
# Posted on: 09-May-2014 08:39:50   

Hello

I started a project using LLBLGen project back in 2003 or 2004. It is now a rather large solution.

Unfortunately I chose self-serving rather than adapter back then. I have also not upgraded the templates since what came with the 2.6 version and I understand you are in version 4.1 now.

I have tens of thousands of lines of code, so I would like to check with you before considering upgrading. I use entity classes for CRUD and collection classes for many database queries. (AddressCollection.GetMulti(filter) ).

I am having some performance issues and would like to query whether you think it would be a huge project to upgrade to 4.1 and the latest templates, or if it is potentially not too hard?

I realise chances are it is impossible to answer that question. But if you could say something intelligent about it, it would be much appreciated.

Regards

Rob

Walaa avatar
Walaa
Support Team
Posts: 14946
Joined: 21-Aug-2005
# Posted on: 09-May-2014 13:02:47   

It's doable, the amount of work is highly dependent on your project size.

your starting point is to check Migrating your code

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39589
Joined: 17-Aug-2003
# Posted on: 09-May-2014 18:07:53   

Additionally, it's always key to use a profiler (.net profiler, data-access profiler) to see where your application has performance issues.

Frans Bouma | Lead developer LLBLGen Pro
Rob
User
Posts: 54
Joined: 17-Sep-2004
# Posted on: 12-May-2014 05:35:04   

Thanks for the tip. But yes I do use SQL Profiler. It is mostly of use for me when I have my own queries since then I get a better view of what SQL is actually performed. The queries that LLBLGen creates I have hardly any knowledge of at all.

I noticed your home screen claims "•much faster query fetches". Not sure what type of query fetches those are. But if it includes Collections.GetMulti() then it might be worth my while to upgrade.

I find that if I do:


addressCollection.getMulti(someFilter);
foreach(AddressEntity address in addressCollection)
{
  //Do something
}

Then if there are very many rows in my Address table it will run extremely much faster if I instead:


DataTable dt = Generic.getDataFromSQL("SELECT AddressID FROM Address WHERE something to filter on");
foreach(DataRow dr in dt.Rows)
{
  AddressEntity address = new AddressEntity(dr[AddressID]");
  //Do something
}

So using the GetMulti() is way, way slower than using embedded SQL and then fetching the entity.

I am thinking it could be that I am using self-servicing and since there are often foreign keys in the tables (Address is just an example), maybe LLBLGen is fetching all that FK data as well, and that slows me down?

Do you find that most of your users use the LLBLGen way of retrieving resultsets, or is it more common to use LinQ or something else? I find myself thinking I need to make an architectural decision on how I continue on using LLBLGen.

Regards

Rob

daelmo avatar
daelmo
Support Team
Posts: 8245
Joined: 28-Nov-2005
# Posted on: 12-May-2014 07:37:20   

Rob wrote:

I noticed your home screen claims "•much faster query fetches". Not sure what type of query fetches those are. But if it includes Collections.GetMulti() then it might be worth my while to upgrade.

The faster entity fetches features is detailed in the docs: What's new in LLBLGen Pro v4.1:

documentation wrote:

50% faster entity fetches. An extensive refactoring of the internal storage of values inside an entity instance resulted in up to 50% faster entity fetches. As entity instances don't have Entity Field instances to store their values anymore (though from the outside it will appear they will, so no code will break), memory footprints are smaller too. See http://pastebin.com/AdsKitr3 for a benchmark of how much faster the fetches are compared to v3.5 and other ORMs.

Rob wrote:

So using the GetMulti() is way, way slower than using embedded SQL and then fetching the entity.

I'm not sure about your test, you should provide a full detailed benchmark results on that, not only on DB but in the code.

Rob wrote:

I am thinking it could be that I am using self-servicing and since there are often foreign keys in the tables (Address is just an example), maybe LLBLGen is fetching all that FK data as well, and that slows me down?

That's a possibility. you should use an ORM Profiler to know where are your bottlenecks. Remember that SelfServicing uses lazy-loading.

Rob wrote:

Do you find that most of your users use the LLBLGen way of retrieving resultsets, or is it more common to use LinQ or something else?

That totally depends on your preferences. Some people use more LLBLGen API, others LLB2LINQ, others QuerySpec. You can use all of them in the same project if you want. Sometimes maybe a query is more clear in some language than the rest, some other time migth be more flexible to use X, etc. You should try all of them to make a decision.

Rob wrote:

I find myself thinking I need to make an architectural decision on how I continue on using LLBLGen

My advise is that you should evaluate your project and the list of breaking changes to analyze the migration's code effort. Maybe it's more easy to just move to Adapter templaset first. Then migrate to v3.0, then v3.1, and so on.

David Elizondo | LLBLGen Support Team
Rob
User
Posts: 54
Joined: 17-Sep-2004
# Posted on: 12-May-2014 08:24:27   

Thanks guys.

You have given me some food for thought.

I searched my project and in my own code (not generated) I found I currently use .GetMulti() in 1211 places.

I use very little else than the entities and the collections in your framework so chances are that the version upgrades will cause me less headache than the Selfservicing to Adapter change would.

Anyway, now I know a little more and will let that mature in the back of my mind for a while.

Thanks again!

Rob

Rob
User
Posts: 54
Joined: 17-Sep-2004
# Posted on: 27-Dec-2014 10:29:46   

Hi again

It took me 8 months to make up my mind. :-)

I have now purchased version 4.2 and have started taking some tentative steps towards updating my project from version 2.6.

First thing I notice is that the 4.2 application does not open .lgp files as used by the 2.6 version. I tried forcing it to open my .lpg file and it has now been unresponsive for about an hour while using around 18% of my CPU. I'll leave it on for a while longer to see if it is actually doing something, but I am guessing it will not lead anywhere.

So, that leads me to ask you how do I upgrade my project file from 2.6 to 4.2? I would be very disappointed if I need to start from scratch since there are 353 tables and a number of customised other objects in the project file. It would take me a long time to recreate all that.

I was expecting to have to deal with some changes in the generated code, but I thought there would be a way to upgrade the project file.

So, any tips on how to proceed would be greatly appreciated.

Regards

Rob

Rob
User
Posts: 54
Joined: 17-Sep-2004
# Posted on: 27-Dec-2014 13:32:58   

So, I eventually found how to generate a new project file and I now have the project file working in 4.2. So please disregard my last post.

Regards

Rob

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39589
Joined: 17-Aug-2003
# Posted on: 28-Dec-2014 09:16:39   

Good simple_smile Yes it's a bit of a problematic conversion but it's one we had to do as the .lgp file is a binary serialized blob and not really maintainable in the future so we moved to an xml format which is easier to use in sourcecontrol systems and one can fix the odd issue manually if that occurs while with the binary blob... that's not going to happen. wink

The only way to get from a .lgp file to an xml file is to generate the xml file from inside the designer which can load the .lgp file and the lgp file is only loadable with code which serialized it. So to do be able to do that one has to load it into v2.6 and generate the xml file with a template. Loading it in v4 won't work as it doesn't have any v2.6 code inside itself so it can't deserialize the binary data.

It's all documented, including the migration of your v2 code to v4 code which is an important read: http://www.llblgen.com/documentation/4.2/Designer/hh_goto.htm#Migratingv2xProjectTov4.htm and http://www.llblgen.com/documentation/4.2/LLBLGen%20Pro%20RTF/hh_goto.htm#migratingcode.htm

Frans Bouma | Lead developer LLBLGen Pro
Rob
User
Posts: 54
Joined: 17-Sep-2004
# Posted on: 30-Dec-2014 08:36:46   

Hi again

I have made good progress with both my project file and my code using the new generated code in version 4.2.

There might be a stack of errors down the track, but currently I only have one problem occurring over 6 errors.

Cannot access protected member 'SD.LLBLGen.Pro.ORMSupportClasses.EntityCore<SD.LLBLGen.Pro.ORMSupportClasses.IEntityFields>.Transaction' via a qualifier of type 'sevenG.BusinessLogic.EntityClasses.AccountEntity'; the qualifier must be of type 'sevenG.BusinessLogic.EntityClasses.BizTransactionEntity' (or derived from it)

It happens when I reference the entity's Transaction property. For the two entities involved I can access it in dozens of other places, but for some reason not in 6 places. Or maybe it is the way I reference it in those 6 places, I am not sure.

The code looks like this, I have not passed in the whole method:

        private void rollbackBizEvent(int userID, BizEventEntity bizEvent, BizTransactionEntity bizTxn, ref List<long> rolledBackBizEventIDs)
        {
            Transaction dbtxn = (Transaction)bizEvent.Transaction;
            BizEventCollection bizEvents = new BizEventCollection();
            ISortExpression bizEventsSorter = new SortExpression();
            bizEventsSorter.Add(BizEventFields.BizEventID | SortOperator.Descending);
            IPredicateExpression filterBizEvents = new PredicateExpression();
            filterBizEvents.Add(BizEventFields.BizTransactionBatchID == this.BizTransactionBatchID);
            filterBizEvents.AddWithAnd(BizEventFields.ParentBizEventID == bizEvent.BizEventID);
            dbtxn.Add(bizEvents);
            bizEvents.GetMulti(filterBizEvents, 0, bizEventsSorter);

This code is in production and works perfectly when I generate the code from version 2.6. So the problem is from something that has been introduced in later versions.

I have trawled through lots of documentation and googled this for hours so I think I have to give up and turn to you guys. Can you shed any light on the subject?

You may need more info to be able help out, I realise that, but rather than sending in tonnes of stuff I kept it small thinking it is better if you ask for what you need.

Regards

Rob

Attachments
Filename File size Added on Approval
Error.jpg 190,721 30-Dec-2014 08:41.33 Approved
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39589
Joined: 17-Aug-2003
# Posted on: 30-Dec-2014 09:28:57   

The member is now available through an interface, so please do:

Transaction dbtxn = ((ITransactionalElement)bizEvent).Transaction;

this to make the interface of the entity objects less cluttered. Be aware that this member can be null, as it might be your code uses an ambient transaction (TransactionScope) (if you don't use transactionscope, this isn't the case of course, but if someone adds that to your code, the code snippet you showed might fail with a NRE as you don't test whether the member is null wink )

Please consult the reference manual of the runtime libraries for details on this. It's available on the website http://www.llblgen.com/pages/ListAdditionalDownloads.aspx. It can save you a lot of time as more members are now only available through the interfaces (e.g. IEntity or IEntityCore)

Frans Bouma | Lead developer LLBLGen Pro
Rob
User
Posts: 54
Joined: 17-Sep-2004
# Posted on: 12-Jan-2015 11:53:28   

Thanks heaps for the quick help.

I have now managed to put all my various services into production all using the 4.2 generated code. It was a lot less painful than I had feared. I am very glad I did it.

Thanks again

Rob