Linq to llblgen Expression tree for Contains limitation

Posts   
 
    
nweinit
User
Posts: 21
Joined: 16-Nov-2016
# Posted on: 12-Dec-2017 21:07:38   

The following thread describes exactly the problem I have: http://www.llblgen.com/tinyforum/Messages.aspx?ThreadID=22781

However, I cannot use the solution with a LinqMetaData since (as the error correctly describes):

source isn't implementing ILLBLGenProQuery. Can't execute the query

Therefore my question is - Is there any way to achieve the aforementioned solution with an ILinqMetaData?

Thanks

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39614
Joined: 17-Aug-2003
# Posted on: 13-Dec-2017 17:51:51   

So what's the exact code you tried which fails? (Because if you have the exact same problem the solution provided there should work for your situation too, so I think your situation is slightly different, but don't know for sure until I see your code which fails simple_smile )

Frans Bouma | Lead developer LLBLGen Pro
nweinit
User
Posts: 21
Joined: 16-Nov-2016
# Posted on: 13-Dec-2017 18:45:28   

this is the code i'm using (which is similar to the one in the original thread)


public static IEnumerable<TEntity> InRange<TEntity, TValue>(this IQueryable<TEntity> source, Expression<Func<TEntity, TValue>> selector, int blockSize, IEnumerable<TValue> values)
{
    MethodInfo method = null;
    foreach (MethodInfo methodInfo in typeof(Enumerable).GetMethods(BindingFlags.Public | BindingFlags.Static))
    {
        if (methodInfo.Name == "Contains" && methodInfo.IsGenericMethodDefinition && methodInfo.GetParameters().Length == 2)
        {
            method = methodInfo.MakeGenericMethod(typeof(TValue));
            break;
        }
    }

    if (method == null)
        throw new InvalidOperationException("Unable to locate Contains");

    foreach (TValue[] block in values.GetBlocks(blockSize))
    {
        var keys      = System.Linq.Expressions.Expression.Constant(block, typeof(TValue[]));
        var predicate = System.Linq.Expressions.Expression.Call(method, keys, selector.Body);
        var lambda  = System.Linq.Expressions.Expression.Lambda<Func<TEntity, bool>>(predicate, selector.Parameters[0]);

        foreach (TEntity record in source.Where(lambda))
            yield return record;
    }
}

private static IEnumerable<TValue[]> GetBlocks<TValue>(this IEnumerable<TValue> source, int blockSize)
{
    List<TValue> list = new List<TValue>(blockSize);
    foreach (TValue item in source)
    {
        list.Add(item);
        if (list.Count == blockSize)
        {
            yield return list.ToArray();
            list.Clear();
        }
    }

    if (list.Count > 0)
        yield return list.ToArray();
}

and again, with the notion that: DataSource2<SomeEntity> data; List<long> ids; I call it like so:


ExecuteAsync(new LinqMetaData(adapter).data
        .InRange(x => x.Id, 1000, ids)
        .AsQueryable());

The execute throws a "source isn't implementing ILLBLGenProQuery. Can't execute the query"

daelmo avatar
daelmo
Support Team
Posts: 8245
Joined: 28-Nov-2005
# Posted on: 14-Dec-2017 10:21:04   
  • Try removing the "AsQueryable"
  • What if you don't ExecuteAsync? just enumerate the results as in:
var x = new LinqMetaData(adapter).data
        .InRange(x => x.Id, 1000, ids).ToList();
David Elizondo | LLBLGen Support Team
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39614
Joined: 17-Aug-2003
# Posted on: 14-Dec-2017 18:06:42   

yes I think the ExecuteAsync isn't going to work as the method itself performs several calls to the DB, so it already executes the query multiple times. (inside the InRange(), namely the foreach (T record in source.Where(lambda)) call).

Frans Bouma | Lead developer LLBLGen Pro
nweinit
User
Posts: 21
Joined: 16-Nov-2016
# Posted on: 15-Dec-2017 21:44:09   

daelmo wrote:

  • Try removing the "AsQueryable"
  • What if you don't ExecuteAsync? just enumerate the results as in:
var x = new LinqMetaData(adapter).data
        .InRange(x => x.Id, 1000, ids).ToList();

First of, if you look at my original code, i'm not even looking at the return value so enumerating it is not really the issue here. Execute (as opposed to ExecuteAsync) doesn't make a difference. How can I remove the AsQueryable? The InRange method returns an IEnumerable<Entity> which can't be passed to a LinqMetaData for execution. The bottom line is that the problem stems from fact that LinqMetaData doesn't implement ILLBLGenProQuery which is the main reason why it's failing.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39614
Joined: 17-Aug-2003
# Posted on: 16-Dec-2017 10:33:07   

Did you read my post above? simple_smile LinqMetaData is a class which creates IQueryable elements. It's not an element that is a query itself. the code you're trying to use, InRange, does execute queries inside itself on the line I specified, so passing it to something that executes a query as well is not the way to go.

Frans Bouma | Lead developer LLBLGen Pro
nweinit
User
Posts: 21
Joined: 16-Nov-2016
# Posted on: 18-Dec-2017 17:01:56   

Any suggestions as to how I should go about it then? where would the execute come into place? Given the original premise of: DataSource<SomeEntity> data; List<long> ids; How would I get it to work when ids.Count is > 1000 (with Oracle that is)

daelmo avatar
daelmo
Support Team
Posts: 8245
Joined: 28-Nov-2005
# Posted on: 19-Dec-2017 07:11:20   

Well you could use the .Contains construct directly, like in (using Northwind example):

var orders = new LinqMetaData(adapter).Order
    .Where(x => ids.Contains(x.EmployeeId.Value))
    .ToList();

I understand that the problem in the 1000 values limitation in Oracle. You could add ORs for sublists in order to workaround this. To do so, you could use linq expressions or LLBL2Linq's PredicateBuilder:

int i = 0;
var predicate = PredicateBuilder.Null<OrderEntity>();
while (i < ids.Count)
{
    var toTake = (ids.Count - i) > 1000 ? 1000 : (ids.Count - i);
    var subList = ids.Skip(i).Take(toTake).ToList();
    predicate.Or(x => ids.Contains(x.EmployeeId.Value));

    i += toTake;
}

var orders = new LinqMetaData(adapter).Order.Where(predicate).ToList();

I understand that for Oracle there are other options, like using a SP, or write you ids in form of a query that return such values. Ref: https://community.oracle.com/thread/235143?start=0

David Elizondo | LLBLGen Support Team
nweinit
User
Posts: 21
Joined: 16-Nov-2016
# Posted on: 19-Dec-2017 17:01:17   

Thanks everyone, I guess I should have read the documentation better, specifically the part about the LinqMetaData and the fact that

As a Linq query is self-contained, it also has to know how to execute itself, as it gets executed when it gets enumerated.

(taken from here: https://www.llblgen.com/Documentation/5.1/LLBLGen%20Pro%20RTF/Using%20the%20generated%20code/Linq/gencode_linq_gettingstarted.htm#linqmetadata) So yeah, as soon as I enumerated the original InRange query it ran just fine. Apologies all.