5.3 (5.3.1) RTM adapter

I have a unit of work class that inherits from DataScope: OmzetUnitOfWork Now I have a large task todo that involves millions of entities. So I want to break up the work and reuse the OmzetUnitOfWork. I add for update and add for delete some entities in this and call CommitChangesAsync (which creates a new adapter each time). after that OmzetUnitOfWork.Reset() is called.

but on the second Commit I get an ORMGeneralOperationException

SD.LLBLGen.Pro.ORMSupportClasses.ORMGeneralOperationException: A previous transaction executed on this scope is still pending.
   bij SD.LLBLGen.Pro.ORMSupportClasses.DataScope.BindToTransactionController(ITransactionController controller)
   bij SD.LLBLGen.Pro.ORMSupportClasses.DataScope.<CommitChangesAsync>d__22.MoveNext()
--- Einde van stacktracering vanaf vorige locatie waar uitzondering is opgetreden ---
   bij System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   bij System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   bij System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   bij VA.Pac.Application.Logistiek.Services.OmzetUnitOfWork.<CommitAsync>d__6.MoveNext() in C:\ws\PAC3\VA.Pac\VA.Pac.Application\Logistiek\Services\OmzetUnitOfWork.cs:regel 79
--- Einde van stacktracering vanaf vorige locatie waar uitzondering is opgetreden ---
   bij System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   bij System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   bij System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   bij VA.Pac.Application.Logistiek.Services.OmzetMergeService.<CompressAsync>d__4.MoveNext() in C:\ws\PAC3\VA.Pac\VA.Pac.Application\Logistiek\Services\OmzetMergeService.cs:regel 31
--- Einde van stacktracering vanaf vorige locatie waar uitzondering is opgetreden ---
   bij System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   bij System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   bij System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   bij VA.Pac.Application.Logistiek.Services.OmzetMergeService.<CompressPerDayBatchAsync>d__3.MoveNext() in C:\ws\PAC3\VA.Pac\VA.Pac.Application\Logistiek\Services\OmzetMergeService.cs:regel 25
--- Einde van stacktracering vanaf vorige locatie waar uitzondering is opgetreden ---
   bij System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   bij System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   bij System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   bij VA.Pac.WinService.ServiceInterface.Test.TestService.<Any>d__2.MoveNext() in C:\ws\PAC3\VA.Pac\VA.Pac.WinService.ServiceInterface\Test\TestService.cs:regel 21
--- Einde van stacktracering vanaf vorige locatie waar uitzondering is opgetreden ---
   bij System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   bij System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   bij ServiceStack.Host.ServiceRunner`1.<ExecuteAsync>d__13.MoveNext()

I have created some test code which for the sake of simplicity don't have entities. But in my real program I do have entities in it:

        public async Task CommitAsync()
        {
            base.RefetchStrategy = DataScopeRefetchStrategyType.AlwaysRefetch; // tried all 3 settings
            using (var adapter = _turnoverAdapterFactory.CreateDataAccessAdapter())
            {
                await this.CommitChangesAsync(adapter);
            }
            base.Reset();
            using (var adapter = _turnoverAdapterFactory.CreateDataAccessAdapter())
            {
                await this.CommitChangesAsync(adapter); // <--- ORMGeneralOperationException
            }
            base.Reset();
        }

Why are you calling Reset() between commits? Are you re-fetching data in-between?

I'll be using millions of entities and creating one big transaction for these will suffer from memory and performance problems. So I want to load a batch per day (will be 0-10,000 records), do the work, save (transaction), and clean up (freeing memory). and start with the next batch. By doing a Reset() I expect the entity references in the uow will be cleared to free up memory by the GC at one time. But maybe Reset() does other things and can only be used once (commit).

And sorry, I have entered this thread mistakenly in the wrong section of the forum. I cannot move it myself.

So you are fetching entities you are going to process after the Reset(), right?

Yes, but fetching outside of the DataScope and attaching them to it.

But ignoring that, even in the test above without fetching entities it gives the exception.

I couldn't reproduce it using the following test method and using the CustomerWithOrdersDataScope class example found in the docs

var dataScope = new CustomerWithOrdersDataScope();
dataScope.Add(new CustomerEntity() { CustomerId = "JICEL", Country="Iceland", CompanyName="Company1" });

using (var adapter = new DataAccessAdapter())
{
    await dataScope.CommitChangesAsync(adapter);
}
dataScope.Reset();
dataScope.Add(new CustomerEntity() { CustomerId = "EICEL", Country = "Iceland", CompanyName = "Company2" });
using (var adapter = new DataAccessAdapter())
{
    await dataScope.CommitChangesAsync(adapter); 
}
dataScope.Reset();

The code commits and records are saved to the database.

I don't understand. Please try to run this minified testcase and insert your favorite adapter.

        [Test]
        public async Task TestOmzetUnitOfWorkReuse()
        {
            var uow = _container.GetInstance<TestDataScope>();
            await uow.CommitAsync();
            Assert.IsTrue(true);
        }

   public class TestDataScope : DataScope
    {
        private readonly IDataAccessAdapterFactoryTurnover _turnoverAdapterFactory;

        public TestDataScope(IDataAccessAdapterFactoryTurnover turnoverAdapterFactory)
        {
            _turnoverAdapterFactory = turnoverAdapterFactory;
        }
        public async Task CommitAsync()
        {
            //base.Reset();
            using (var adapter = _turnoverAdapterFactory.CreateDataAccessAdapter())
            {
                await this.CommitChangesAsync(adapter);
            }
            base.Reset();
            using (var adapter = _turnoverAdapterFactory.CreateDataAccessAdapter())
            {
                await this.CommitChangesAsync(adapter);
            }
            base.Reset();
        }
    }

does this stacktrace help:

SD.LLBLGen.Pro.ORMSupportClasses.ORMGeneralOperationException : A previous transaction executed on this scope is still pending.
   bij SD.LLBLGen.Pro.ORMSupportClasses.DataScope.BindToTransactionController(ITransactionController controller)
   bij SD.LLBLGen.Pro.ORMSupportClasses.DataScope.<CommitChangesAsync>d__22.MoveNext()
--- Einde van stacktracering vanaf vorige locatie waar uitzondering is opgetreden ---
   bij System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   bij System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   bij System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   bij VA.Pac.Integration.Tests.Logistiek.TestDataScope.<CommitAsync>d__2.MoveNext() in C:\ws\PAC3\VA.Pac\VA.Pac.Integration.Tests\Logistiek\OmzetUnitOfWorkTest.cs:regel 73

Well, your test case doesn't add any entities or anything to commit. If I implement the Add method in the TestDataScope, and add some entities to it in the test method, it woks as expected.

Try this:

public class TestDataScope : DataScope
{
    private readonly DataAccessAdapter  _turnoverAdapterFactory;

    public TestDataScope(DataAccessAdapter turnoverAdapterFactory)
    {
        _turnoverAdapterFactory = turnoverAdapterFactory;
    }
    public async Task CommitAsync()
        {
            //base.Reset();
            using (var adapter = _turnoverAdapterFactory)
            {
                await this.CommitChangesAsync(adapter);
            }
            base.Reset();
            using (var adapter = _turnoverAdapterFactory)
            {
                await this.CommitChangesAsync(adapter);
            }
            base.Reset();
        }

    public CustomerEntity Add(CustomerEntity toAdd)
    {
        if (toAdd != null)
        {
            this.Attach(toAdd);
        }
        return toAdd;
    }
}

var uow = new TestDataScope(new DataAccessAdapter());//_container.GetInstance<TestDataScope>();
uow.Add(new CustomerEntity() { CustomerId = "JICEL", Country = "Iceland", CompanyName = "Company1" });
await uow.CommitAsync();

To elaborate a bit about the error in question: when calling Commit, it will bind to events of the adapter, and will unbind them when everything is over (commit or rollback). When you issue a reset in between, that doesn't reset any ongoing transaction, it just clears data contained in the scope.

What does cause the issue in the empty scope is that if you call commit on the empty scope the action is a no-op, so no commit is actually issued (the event never fires) so the binding is never undone and the next time you call a commit again on the scope, it sees there's already a binding with the adapter, so it assumes a transaction is still on-going.

While this is a bit of an edge case (2 commits on an empty scope), one could argue it's a bug, but frankly this is so unlikely it will happen at all...

that said, if you get this with entities in the scope then something else is going on (i.e. there's no save happening so the transaction is a no-op). As Walaa's repro shows it's not something related to the 'Reset' method, it's working fine. So I think your repro should be something else, i.e. it has entities which are saved.

My large batch process is centered around reading chunks of entities which are evaluated and if one or more must be updated or deleted in each round of x - entities then they are attached to the DataScope. (i'm using a datascope here because when I AddForDelete then related children get deleted automatically, I dont need any logic here. I tried with a UnitOfWork2 descendant but these don't automatically deleted children - or I am missing something here)

I found it becoming complex to also create another frequency for the commit batch (lets say a counter after x-updates (auto) commit or something). But that's also my lack of knowledge to create a simple and effective strategy for this here.

Now I have two solutions:

• I've created a UoWFactory that gives me a new UoW each round of chunks. So no need for Reset and reuse. That works, but must create new DataScope each time. Didn't time this
• Just added a private boolean _hasChanges that gets set each time an entity is attached (for update or delete). And in the CommitChangesAsync it checks it being false, then quits without doing anything. This also works.

I'll be going for the last option for this scenario for this seems to be working fine.

Maybe you could point me to a better way by not having to use DataScope in this scenario at all? Maybe UnitOfWork2 but how?

I believe you got it on the second solution. Implementing the DataScope class, you need to keep track of entities added for processing before calling the Commit on the base class.

thanks.

Otis, So no other options for using UnitOfWork2?

The Datascope has logic to determine depending entities and help you with the related entities getting deleted as well. A unit of work just does what's been told to do, so if you want cascading deletes then you have to add them first to the UoW (datascope does that for you when it creates the UoW for you).

Another possibility might be to define cascading rules on the FKs in the DB (i.e. 'CASCADE' for delete). This way you can simply delete an entity (say a customer) and the DB will remove all entities depending on it (e.g. orders, order rows (which depend on order)). Not always a good idea, as you could end up wiping your whole db, or it could lead to a problem with multiple paths to the same table, but could be an option as well.

I wouldn't worry too much about re-using the datascope between groups to be honest, just create a new one each time. It's not a very heavy datastructure to instantiate and all data it has is added along the way in your case.

Ok fine, Thanks for your instructions.

And no indeed, I have already some cascading deletes set in DB's, but most must be handled by Business Logic and Validation.

The reuse is working and releases entities just fine.

Ok thanks for confirming so I'm now closing this as I think it's resolved