Out of memeory

Posts   
 
    
Anthony
User
Posts: 155
Joined: 04-Oct-2006
# Posted on: 12-Apr-2011 18:05:58   

c#, winforms, llblgen v2.6, adapter

Getting out of memory issues when i try to load 88,000 entities into a collection. Is there any limits on how many entities can be loaded?

Have an application that needs to process many , so need to load everything into memory to improve performance. The data is being stored in a db which i load into memory at run-time.

MTrinder
User
Posts: 1461
Joined: 08-Oct-2008
# Posted on: 12-Apr-2011 22:32:24   

There are 3 options you can look at to try to prevent the out of memory errors.

1) Do any of your records have large BLOB or Text fields, and if so, do you need these for your processing ? If not, they can be excluded from the fetch using the Include/Exclude fields functionality.

2) You can look at using paging to allow you to process the data in batches.

3) Does the data need to be fetched as entities? Could it instead be fetched as a typed/dynamic list? There is a significant memory overhead in using entities.

Matt

Anthony
User
Posts: 155
Joined: 04-Oct-2006
# Posted on: 13-Apr-2011 03:31:43   

does the size of the field matter or the data in it? If i have varchar(50) field but all records only have a 2charcater string..is this the same as a table where all the characters are used?

Can you still do the same in memory filtering as with typed/dynamic lists? Sounds like will have to convert to typedview.....

Paging would not appear to be an option

Walaa avatar
Walaa
Support Team
Posts: 14950
Joined: 21-Aug-2005
# Posted on: 13-Apr-2011 10:04:50   

The data is what matters.

TypedList/TypedView are dataTables, so you can filter them using DataViews.