Precision problem when filtering typedview (SQLServer)

Posts   
 
    
BlackMamba avatar
BlackMamba
User
Posts: 34
Joined: 30-Apr-2004
# Posted on: 27-May-2005 15:43:21   

Hi all, LLBLGen is having a problem filtering on a typedview. My real view has 3 fields which are mapped by LLBLGen as decimal , the db type is shown as numeric, length 0, precision 38 and scale 2. I have no problems fetching the typed view but when I tried fething with a filter on one of the numeric fields I get the following error:

Column or parameter #1: Specified column precision 38 is greater than the maximum precision of 28. Parameter '@CreditBalance1' has an invalid data type.

I had found a similar problem on this forum but it regarded oracle. In one of those posts is stated that .net decimals have a maximul scale of 28 and so converting from scale 38 fails. But it should always fail shouldn't it, why am I able to fetch the typed view then?

Thanks for any help

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39861
Joined: 17-Aug-2003
# Posted on: 28-May-2005 12:11:53   

Could you please post some code in which you build up the filter?

If I understand the problem correctly, you specify a CompareValue predicate, to which you pass a System.Decimal value (or C# decimal value) and that decimal has a precision of 38? I'm then confused where exactly the error occurs: IN sqlserver, or in the DQE where the parameter objects are created?

Frans Bouma | Lead developer LLBLGen Pro
BlackMamba avatar
BlackMamba
User
Posts: 34
Joined: 30-Apr-2004
# Posted on: 30-May-2005 10:13:31   

Otis wrote:

Could you please post some code in which you build up the filter?

If I understand the problem correctly, you specify a CompareValue predicate, to which you pass a System.Decimal value (or C# decimal value) and that decimal has a precision of 38? I'm then confused where exactly the error occurs: IN sqlserver, or in the DQE where the parameter objects are created?

Sure, here is the code:


            Dim filter As New RelationPredicateBucket
            filter.PredicateExpression.Add(PredicateFactory.CompareValue(ClientCreditFieldIndex.CreditBalance, ComparisonOperator.LesserThan, LowCreditValue))
            Dim tv As New TypedViewClasses.ClientCreditTypedView
            Adapter.FetchTypedView(tv.GetFieldsInfo, tv, filter, False)

The error I get is this:

Column or parameter #1: Specified column precision 38 is greater than the maximum precision of 28. Parameter '@CreditBalance1' has an invalid data type.

I don't have the chance to test if it comes from SQLServer or DQE at the moment but I will as soon as possible.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39861
Joined: 17-Aug-2003
# Posted on: 30-May-2005 18:33:59   

That's sqlserver's error, but it's weird, as the precision check on the parameter is simply checking for a precision <=38 (in SqlParameter.Precision), and throws an exception if it is > 38. All precisions are capped to 38 by the drivers, so that's never going to happen.

The weirdness is the '28', I've no idea where that comes from. If it's an sqlserver exception, you can catch it in the debugger and check the inner exception object, which contains the SqlException and in there you'll find the SqlServer error code.

Frans Bouma | Lead developer LLBLGen Pro
arschr
User
Posts: 894
Joined: 14-Dec-2003
# Posted on: 31-May-2005 13:56:03   

Isn't there a sql configuration switch that controls the max precision of decimals?

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39861
Joined: 17-Aug-2003
# Posted on: 31-May-2005 14:39:02   

arschr wrote:

Isn't there a sql configuration switch that controls the max precision of decimals?

Now you mention it...

If you execute SELECT @@MAX_PRECISION in QA, it should return the set precision level. Here it's 38 (default).

Though the docs mention that it could be set apparently, I'm not aware of a function/proc how to do that. Only that systypes is altered.

Frans Bouma | Lead developer LLBLGen Pro