ORMQueryConstructionException only when compiled in VS2015 (when grouping by multiple fields)

Posts   
 
    
acl
User
Posts: 93
Joined: 28-Mar-2012
# Posted on: 26-Aug-2016 09:30:47   

Hi all,

A very strange phenomenon has come up after we switched our build process from Visual Studio 2013 to 2015. A particular kind of Linq-To-LLBLGen Query will execute fine when built using VS 2013 and fails when built using VS 2015. I can only imagine that the new Roslyn compiler creates a subtly different expression tree, which is not handled well by the LLBLGen Pro Runtime Framework.

The issue described applies to the latest LLBLGen 5.0.5 (as well as LLBLGen 4.0.13, which we were using previously).

Consider the following Linq-To-LLBLGen query:


                Dim linqMetaData = New Linq.LinqMetaData(adapter)

                Dim query = From elem In linqMetaData.TMov
                                    Group By elem.TAcct_Cod, elem.TVat_Cod
                                    Into sumAmount = Sum(elem.TMov_Amount)

                Trace.WriteLine(query.Count)

When compiled with VS 2013, it executes fine. When compiled with VS 2015 executing this produces an ORMQueryConstructionException. The exception message (Grouping on a field in a related entity isn't supported in VB.NET) is strange since we are not grouping on a field in a related entity. Both fields (TAcct_Cod and TVat_Cod are fields of TMov).

FYI: if the Group By clause only contains one field, the query executes fine in both VS 2013 and 2015.

Any help would be greatly appreciated.

Best,

andreas

Here is the full stack trace:

SD.LLBLGen.Pro.ORMSupportClasses.ORMQueryConstructionException was unhandled HResult=-2146232832 Message=Grouping on a field in a related entity isn't supported in VB.NET RuntimeBuild=5.0.5 RuntimeVersion=5.0 Source=SD.LLBLGen.Pro.ORMSupportClasses StackTrace: at SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.PreProcessor.HandleMethodCallGroupBy(MethodCallExpression expressionToHandle) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.0\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Linq\ExpressionHandlers\PreProcessor.cs:line 1021 at SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.PreProcessor.HandleQueryableExtensionMethod(MethodCallExpression expressionToHandle) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.0\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Linq\ExpressionHandlers\PreProcessor.cs:line 1811 at SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.PreProcessor.HandleMethodCallPerType(MethodCallExpression expressionToHandle, Type declaringType) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.0\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Linq\ExpressionHandlers\PreProcessor.cs:line 1681 at SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.PreProcessor.HandleMethodCallExpression(MethodCallExpression expressionToHandle) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.0\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Linq\ExpressionHandlers\PreProcessor.cs:line 766 at SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler.HandleExpression(Expression expressionToHandle) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.0\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Linq\ExpressionHandlers\GenericExpressionHandler.cs:line 296 at SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.PreProcessor.HandleExpression(Expression expressionToHandle) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.0\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Linq\ExpressionHandlers\PreProcessor.cs:line 195 at SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.PreProcessor.HandleAggregateFunctionMethodCall(MethodCallExpression expressionToHandle, AggregateFunction function) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.0\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Linq\ExpressionHandlers\PreProcessor.cs:line 1089 at SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.PreProcessor.HandleQueryableExtensionMethod(MethodCallExpression expressionToHandle) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.0\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Linq\ExpressionHandlers\PreProcessor.cs:line 1776 at SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.PreProcessor.HandleMethodCallPerType(MethodCallExpression expressionToHandle, Type declaringType) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.0\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Linq\ExpressionHandlers\PreProcessor.cs:line 1681 at SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.PreProcessor.HandleMethodCallExpression(MethodCallExpression expressionToHandle) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.0\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Linq\ExpressionHandlers\PreProcessor.cs:line 766 at SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler.HandleExpression(Expression expressionToHandle) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.0\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Linq\ExpressionHandlers\GenericExpressionHandler.cs:line 296 at SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.PreProcessor.HandleExpression(Expression expressionToHandle) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.0\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Linq\ExpressionHandlers\PreProcessor.cs:line 195 at SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProProviderBase.HandleExpressionTree(Expression expression) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.0\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Linq\LLBLGenProProviderBase.cs:line 151 at SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProProviderBase.System.Linq.IQueryProvider.Execute[TResult](Expression expression) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.0\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Linq\LLBLGenProProviderBase.cs:line 774 at System.Linq.Queryable.Count[TSource](IQueryable`1 source) at ....

Walaa avatar
Walaa
Support Team
Posts: 14950
Joined: 21-Aug-2005
# Posted on: 26-Aug-2016 15:02:05   

.NET version?

(Edit) Reproduced, using .NET v.4.6.1

And the following example from the docs:

Dim q = From c In metaData.Customer _
        Group By c.Country, c.City _
        Into Count() _
        Select Country, City, Count

acl
User
Posts: 93
Joined: 28-Mar-2012
# Posted on: 26-Aug-2016 15:39:23   

We can reproduce it using .NET 4.0.

So you'll investigate on your end and let us know?

Walaa avatar
Walaa
Support Team
Posts: 14950
Joined: 21-Aug-2005
# Posted on: 26-Aug-2016 16:02:43   

Indeed, hang on.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39616
Joined: 17-Aug-2003
# Posted on: 26-Aug-2016 16:12:56   

We'll look into it. Keep in mind though that VB.NET generates strange expression trees sometimes so we do 'interpretation' of the trees to see what was meant in the original query. If things changed under the hood, it might not be we can fix it, because a change in interpretation for .NET vN+1 might break a query running fine on .NET vN which then won't run on vN.

the error given isn't suitable for the query, admitted, so we'll look into why it thinks it needs to throw the error. This will take some time though, we hope to have an answer next week.

Frans Bouma | Lead developer LLBLGen Pro
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39616
Joined: 17-Aug-2003
# Posted on: 31-Aug-2016 10:14:28   

Reproduced indeed.

Test:


Dim metaData As New LinqMetaData(adapter)

Dim q = From c In metaData.Customer _
    Group By c.Country, c.City _
    Into Count() _
    Select Country, City, NumberOfCustomers = Count

expression tree:


: Initial expression to process:
value(SD.LLBLGen.Pro.LinqSupportClasses.DataSource2`1[NW26.Adapter.EntityClasses.CustomerEntity])
    .GroupBy(c => new VB$AnonymousType_3`2(Country = c.Country, City = c.City), 
            ($VB$It, $VB$ItAnonymous) => 
                    new VB$AnonymousType_4`3(Country = $VB$It.Country, City = $VB$It.City, Count = $VB$ItAnonymous.Count()))
    .Select($VB$It => new VB$AnonymousType_5`3(Country = $VB$It.Country, City = $VB$It.City, NumberOfCustomers = $VB$It.Count))

Will now check how the expression tree looks when it's compiled with the old compiler and with C# too.

(edit) C# equivalent:


var metaData = new LinqMetaData(adapter);
var q = from c in metaData.Customer
        group c by new {c.Country, c.City}
        into g
        select new {Country = g.Key.Country, City = g.Key.City, Count = g.Count()};

Tree:


: Initial expression to process:
value(SD.LLBLGen.Pro.LinqSupportClasses.DataSource2`1[NW26.Adapter.EntityClasses.CustomerEntity])
    .GroupBy(c => new <>f__AnonymousType137`2(Country = c.Country, City = c.City))
    .Select(g => new <>f__AnonymousType138`3(Country = g.Key.Country, City = g.Key.City, Count = g.Count()))

VB.NET clearly uses a different construct. The main issue is that the aggregates are in the groupby clause in vb.net, but in C# they're in the projection.

When I rewrite the C# code to use lambda's instead, which mimic the VB.NET expression tree, I get the same error:

var metaData = new LinqMetaData(adapter);
var q = metaData.Customer.GroupBy(c=>new {Country = c.Country, City = c.City}, (a, b)=>new {Country = a.Country, City = a.City, Count = b.Count()})
                .Select(a=>new {Country = a.Country, City = a.City, Count = a.Count});

In vs.net 2012 I get the same error btw. It could be because roslyn is still invoked, not sure. It's also a bit moot as it doesn't solve the problem at all, the problem is present in the current framework.

There's a problem though: the error it runs into is in code to work around the issue of the old vb.net compiler, so it interprets the tree produced by that compiler. What I need is to see how that tree looks. I can't make changes to the current code without it, as changing the code might break the code of older vb.net compilers. I'll try on an old VM with the v3.x environment on it, to see whether it works there. No roslyn on that machine.

Frans Bouma | Lead developer LLBLGen Pro
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39616
Joined: 17-Aug-2003
# Posted on: 31-Aug-2016 11:07:07   

I simplified the query a bit, which is now similar to yours. It indeed works on the old compiler, fails on roslyn.

Query:


Dim q = From c In metaData.Customer _
         Group By c.Country, c.City _
        Into NumberOfCustomers = Count()

Tree on old compiler:


value(SD.LLBLGen.Pro.LinqSupportClasses.DataSource2`1[NW26.Adapter.EntityClasses.CustomerEntity])
    .GroupBy(c => new VB$AnonymousType_4`2(Country = c.Country, City = c.City), 
            ($VB$Key, $VB$Group) => new VB$AnonymousType_5`3(Country = $VB$Key.Country, City = $VB$Key.City, NumberOfCustomers = $VB$Group.Count()))

Tree on Roslyn:


value(SD.LLBLGen.Pro.LinqSupportClasses.DataSource2`1[NW26.Adapter.EntityClasses.CustomerEntity])
    .GroupBy(c => new VB$AnonymousType_3`2(Country = c.Country, City = c.City), 
             ($VB$It, $VB$ItAnonymous) => new VB$AnonymousType_4`3(Country = $VB$It.Country, City = $VB$It.City, NumberOfCustomers = $VB$ItAnonymous.Count()))

Tree looks the same, but has a key (pun intended) difference: the parameter name is different under roslyn. We check for the old name to implement a VB.NET specific workaround, but that no longer works, as the name differs.

That's also why the C# query runs into the same issue.

We'll look into implementing a different workaround which doesn't rely on the name generated by the compiler.

Frans Bouma | Lead developer LLBLGen Pro
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39616
Joined: 17-Aug-2003
# Posted on: 31-Aug-2016 13:29:07   

We made a small change to make it work with the roslyn compiled code while also keep the original code as-is so older code also works.

There's still a problem with VB.NET and Select <grouped fields>, <aggregated fields> after a Group By Into. When that started to fail is unclear, but it's likely it didn't work in previous versions too. It's hard to say because the handler of GroupBy isn't a single method but a complex process over several steps and the logic is scattered across the linq provider because a group by statement is handled by multiple parts as it often falls apart in multiple parts (the aggregates refer to a set which is the grouped source but that's elsewhere in the tree and handled elsewhere etc.)

So it might be a change somewhere broke it along the way but when, not sure. We don't have a specific VB.NET test.

The rewrite of the expression tree for this problem will take a lot of time and therefore isn't fixed now, as it isn't required to solve your problem at hand (which was caused by a parameter name check). We've logged the issue and will fix it at a later date, as it hasn't been reported for years so the issue isn't a high priority.

I'll push a 5.0.6 hotfix build to the website so you can download the fixed runtime from there. I'll post back in this thread when that version is available.

Frans Bouma | Lead developer LLBLGen Pro
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39616
Joined: 17-Aug-2003
# Posted on: 31-Aug-2016 13:45:46   

Hotfix build 5.0.6 is now available. Please let us know if this solves your linq issues.

Frans Bouma | Lead developer LLBLGen Pro
acl
User
Posts: 93
Joined: 28-Mar-2012
# Posted on: 01-Sep-2016 11:43:14   

Thank you very much for the quick fix.

We found the download and are testing this. Just a question: is it possible to get the DLLs without the complete installer package?

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39616
Joined: 17-Aug-2003
# Posted on: 01-Sep-2016 12:04:55   

acl wrote:

Thank you very much for the quick fix.

We found the download and are testing this. Just a question: is it possible to get the DLLs without the complete installer package?

Yes they're on nuget as well simple_smile As it's a hotfix you have to enable pre-release. You can also download the package manually and install it locally through nuget: https://www.nuget.org/packages/SD.LLBLGen.Pro.ORMSupportClasses/5.1.0-Alpha-20160830

Frans Bouma | Lead developer LLBLGen Pro
acl
User
Posts: 93
Joined: 28-Mar-2012
# Posted on: 01-Sep-2016 16:25:54   

Ok, didn't know that.

We successfully tested Version 5.0.6, all is well.

Thanks!