Merging entities

Posts   
1  /  2
 
    
EdH
User
Posts: 26
Joined: 10-Dec-2003
# Posted on: 09-Jan-2004 18:27:28   

Let's say I have two Entities with the exact same schema but different data. Is their a way to merge the two entities into one entity and update the database table?

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39613
Joined: 17-Aug-2003
# Posted on: 11-Jan-2004 11:01:56   

EdH wrote:

Let's say I have two Entities with the exact same schema but different data. Is their a way to merge the two entities into one entity and update the database table?

I don't have a clear image of what you're trying to do simple_smile Can you give an example?

Frans Bouma | Lead developer LLBLGen Pro
EdH
User
Posts: 26
Joined: 10-Dec-2003
# Posted on: 11-Jan-2004 23:46:08   

Sure, hope this clears it up a little:

Imagine a situation similar to the remoting example you have done with northwind - but instead of remoting a single Entity, it remotes a Entity Collection.

There is a "Master" Server database and one (or many) Client databases. Data is always passed from Master Server to client, never the other way. The Schemas are exactly the same on both databases.

So imagine the client uses remoting to get a CollectionClass generated on the Server - this CollectionClass contains all of the changes since the client last remoted to the server (Let's don't worry about the query logic - for now). These changes could include new rows added, modified rows, or deleted rows to the Master database. The "Deleted" rows are really just modified rows flagged as inactive with a bit column.

The client now must take this CollectionClass and "Merge" the data into it's local client database by looking at each row (Entity) in the collection, and doing an Insert or Update.

Hope this is clear enough now confused Thanks!

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39613
Joined: 17-Aug-2003
# Posted on: 12-Jan-2004 09:30:24   

EdH wrote:

Imagine a situation similar to the remoting example you have done with northwind - but instead of remoting a single Entity, it remotes a Entity Collection.

There is a "Master" Server database and one (or many) Client databases. Data is always passed from Master Server to client, never the other way. The Schemas are exactly the same on both databases.

So imagine the client uses remoting to get a CollectionClass generated on the Server - this CollectionClass contains all of the changes since the client last remoted to the server (Let's don't worry about the query logic - for now). These changes could include new rows added, modified rows, or deleted rows to the Master database. The "Deleted" rows are really just modified rows flagged as inactive with a bit column.

The client now must take this CollectionClass and "Merge" the data into it's local client database by looking at each row (Entity) in the collection, and doing an Insert or Update. Hope this is clear enough now confused Thanks!

If you want to save the entity in another database, you can modify the catalog, schema and table name of each field in an entity. If the catalog name, schema name and table name is the same, you don't have to do anything: just connect to the local db with a correct connection string and save the entities. Don't forget to execute: myEntity.Fields[i].IsChanged = true; for each field in an entity, so it is changed and will be saved simple_smile

Frans Bouma | Lead developer LLBLGen Pro
Charlesh
User
Posts: 14
Joined: 22-Dec-2003
# Posted on: 12-Jan-2004 15:05:58   

Otis,

Maybe a simple question will clear this up since I think I'm confused now as well.

Using LLBL, how do I merge new rows and row updates from a master database to a slave database where the schema matches? Thanks.

Charles

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39613
Joined: 17-Aug-2003
# Posted on: 12-Jan-2004 15:23:41   

Charlesh wrote:

Otis,

Maybe a simple question will clear this up since I think I'm confused now as well.

Using LLBL, how do I merge new rows and row updates from a master database to a slave database where the schema matches? Thanks.

Charles

If you want to do an insert, set IsNew to true, otherwise set it to false. You therefore have to check with the slave database first if the entity exists (try to populate an entity with the pk value(s)). If not, set the IsNew property of the master entity to true, which will result in an insert, otherwise set it to false, which will cause an update.

Frans Bouma | Lead developer LLBLGen Pro
EdH
User
Posts: 26
Joined: 10-Dec-2003
# Posted on: 12-Jan-2004 23:23:57   

Thanks for the reply. Here's what I'm doing...

I've retrieved a collection from a remote database and am going to insert the records into my local database. The local database is empty and the new collection has 5 entities.

Here's some pseudo code explaining what I try to do:

for each entity e in collection
{

 e.IsNew = true;
 e.Save();

}

Is that what you were saying to do? This does not work for me. Nothing gets inserted into the local database.

What am I doing wrong?

Thanks!

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39613
Joined: 17-Aug-2003
# Posted on: 13-Jan-2004 09:20:42   

You're not flagging the fields as changed simple_smile THe fields you want to save (thus the fields with an autonumber, default or computed value don't have to be saved, skip these) you have to set IsChanged to true.


for each entity e in collection
{
    e.IsNew = true;
    for(int i=0;i<e.Fields.Count;i++)
    {
        e.Fields[i].IsChanged=true;
    }
    e.Save();
}

something like this. simple_smile

Frans Bouma | Lead developer LLBLGen Pro
EdH
User
Posts: 26
Joined: 10-Dec-2003
# Posted on: 13-Jan-2004 14:17:30   

Otis-

When we try that, we get an error. It appears that IsChanged is a read only property. Here is the error:

Property or indexer 'SD.LLBLGen.Pro.ORMSupportClasses.IEntityField.IsChanged' cannot be assigned to -- it is read only

Any suggestions?

Thanks

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39613
Joined: 17-Aug-2003
# Posted on: 13-Jan-2004 14:30:45   

Upgrade to the latest Runtime libraries (04-jan-2004). This was recently changed, because it is required for a future upgrade (XML deserialization). (i.o.w.: a 'set' clause was added to the property). You can download these runtimes from the customer area on the website. simple_smile The archive contains 2 folders, DotNet10 and DotNet11. Your llblgen pro installation folder contains a directory 'RuntimeLibraries', which has the same directories. Copy the archive's contents in these directories and then recompile your complete solution. The complete .msi package doesn't contain these runtime libraries yet.

You might run into a CLR issue, the CLR caches assemblies in a 'download cache'. This is behaviour Microsoft has invented and I can't do a thing about it, so if you have installed the latest runtime libraries and you still have the same errors, check the download cache: (on a command prompt)

vsvars32.bat gacutil /ldl

it will then probably list SD.LLBLGen.Pro.ORMSupportclasses.Net11.dll. If so, execute the command: gacutil /cdl

this will clear the download cache (clearling it is harmless).

To avoid this behaviour to happen I found out that if I close all VS.NET instances (and other applications referencing the dll's) and then copy the new dll's into the directory, nothing is cached. If you need help with upgrading the dll's, let me know. simple_smile

Frans Bouma | Lead developer LLBLGen Pro
EdH
User
Posts: 26
Joined: 10-Dec-2003
# Posted on: 13-Jan-2004 19:39:40   

Well, I got the new runtimes and placed them in the correct directories. I then readded references in my project to the new runtimes. I'm still getting that "IsChanged" is read-only.

I also noticed that the readme does not make mention of the fix to "IsChanged". Not sure if that means anything.

Any ideas?

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39613
Joined: 17-Aug-2003
# Posted on: 13-Jan-2004 20:23:03   

The fix was not documented in that readme, because the update was for adapter code.

Did you close vs.net (ALL of the instances you've running) ? after that, check the download cache with the gacutil tool, and clear it. Then restart vs.net, and compile again. Does it still say readonly?

The library contains a set clause for the property, I've checked that 4 times or so today to be sure. simple_smile

Frans Bouma | Lead developer LLBLGen Pro
EdH
User
Posts: 26
Joined: 10-Dec-2003
# Posted on: 13-Jan-2004 20:28:32   

I've done all you've said and unfortunately it's still read-only. Any ideas?

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39613
Joined: 17-Aug-2003
# Posted on: 13-Jan-2004 20:41:51   

Can you download Reflector for me, and check what the dll says in reflector?

http://www.aisto.com/roeder/dotnet/

I load the dll into reflector, and it says: bool IsChanged { public get; public set; }

Assembly SD.LLBLGen.Pro.ORMSupportClasses.NET11 Version: 1.0.2003.1 Name: SD.LLBLGen.Pro.ORMSupportClasses.NET11, Version=1.0.2003.1, Culture=neutral, PublicKeyToken=ca73b74ba4e3ff27

The old version is hanging around somewhere. (it happens to me a lot too, during testing, I really hate VS.NET and the CLR download cache sometimes simple_smile ).

This is the 4 january version, build sunday january 4th.

Frans Bouma | Lead developer LLBLGen Pro
EdH
User
Posts: 26
Joined: 10-Dec-2003
# Posted on: 13-Jan-2004 21:19:11   

Ok, I've downloaded the reflector and pointed to the assembly I'm using. It has the "set" in the assembly. However, I still get that it's read-only. I've removed everything from the download cache, like you requested. I deleted all references on my computer to the older version of the assembly. I'm still having problems...

Any other hints? I'm going to try re-booting right now.

EdH
User
Posts: 26
Joined: 10-Dec-2003
# Posted on: 13-Jan-2004 21:39:16   

re-boot didn't help either. I'm at a loss...

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39613
Joined: 17-Aug-2003
# Posted on: 13-Jan-2004 22:23:47   

Hmm. Can you do a search for SD.LLBLGen.Pro.ORMSupportClasses.NET11.dll on your harddisk? It should be there only once.

I've released a new installer 1 hour ago, you could try uninstalling and re-installing the tool, using the new installer.

Also, remove the ....csproj.user file and clear the bin/obj dirs from dlls and other build left overs, plus remove the .pdb files. Then try to rebuild the solution again.

Frans Bouma | Lead developer LLBLGen Pro
EdH
User
Posts: 26
Joined: 10-Dec-2003
# Posted on: 13-Jan-2004 23:04:39   

Still no love...

I have done all this and it's still read-only. What more can I do?

I appreciate all your help but I'm really stuck.

EdH
User
Posts: 26
Joined: 10-Dec-2003
# Posted on: 13-Jan-2004 23:30:30   

I've tried this on a second machine and it can't compile either.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39613
Joined: 17-Aug-2003
# Posted on: 14-Jan-2004 10:49:50   

I've identified what's wrong.

The interface IEntityField, which is the type returned by myEntity.Fields[x] has for IsChanged just get; defined. The class EntityField has the set defined as well.

This is a mistake on my side, (it's for internal usage only, that set clause anyway, and the code using it is not yet in place).

You can work around this by casting the myEntity.Fields[i] result to EntityField. Like this: CustomerEntity customer = new CustomerEntity("CHOPS"); for(int i=0;i<customer.Fields.Count;i++) { ((EntityField)customer.Fields[i]).IsChanged=true; }

I'm really sorry that I've put you through all these hoops finding for a fault that wasn't there... I thought of this this morning and after checking out the interface definition, it was indeed it. I looked solely at the EntityField implementation, which does have a set clause for the property...

Frans Bouma | Lead developer LLBLGen Pro
EdH
User
Posts: 26
Joined: 10-Dec-2003
# Posted on: 14-Jan-2004 15:14:20   

While I can now set this value, merging of the data still doesn't work. Would it be possible to get an email address from you and send you some code and some projects?

ctadlock avatar
ctadlock
User
Posts: 60
Joined: 12-Feb-2004
# Posted on: 12-Apr-2004 22:00:40   

Back to the merging issue....

I have a similar situation. I have a .NET remoting service and a .NET WinForm application which calls the service for both read and write operations. I have a method

UserCollection GetUsersForAccount(int accountID)

on the service. When this method is called from the client it "caches" the returned UserCollection in memory. I would like to be able to call the same method again and synchronize the UserCollection stored in the client's memory with the UserCollection I just got back from the service. This is because individual User's attributes might have changed or Users may have been added/removed from the account. What is the best way to handle this?

To note, ADO.NET's way of handling this using DataSets is that all rows (entities) and tables (collections) contain a reference to a parent container (dataset) which ensures that only one copy of an row/table exist within a dataset. With this, datasets can be merged together.

One of the issues I'm trying to solve with this is that I want to prevent having multiple copies of a specific object in memory so that updates to an object, (person #4) are reflected everywhere.

Thanks CT

swallace
User
Posts: 648
Joined: 18-Aug-2003
# Posted on: 12-Sep-2004 01:17:07   

I want to update this thread a bit, fixing a couple of issues in case someone else has this need (as I just did) so that it will help them.

The code in adapter, C# is:


// Get them from server
EntityCollection mSyncs = [Your code to get the records from the server...]

// Mark them to be saved
foreach( MessageEntity mMessage in mSyncs )
{
   for(int i=0;i<mMessage.Fields.Count;i++)
   {
      ((IEntityField2)(mMessage.Fields[i])).IsChanged=true;
   }
   mMessage.IsDirty = true;
};

// Put them in local database to be sync'd
DataAccessAdapter mAdapter = new DataAccessAdapter( [Your connection string] );
mAdapter.SaveEntityCollection( mSyncs, false, false );


Specifically, the changes are the IEntityField2 cast for Adapter, the inclusion of the i parameter on .Fields, and the inclusion of .IsDirty = true for the entity.

This code retrieves from the server records to be sychronized onto the local. It does not do the reverse, only from server to remote. In addition, I have Service code (not shown) that fills the mSync EntityCollection with only records changes (via timestamp) since the last sync. I also include a "markedasdeleted" record on both sides that simulates deletions (since, if they were actually deleted, they would not be sync'd onto the remotes).

For updating the server from remote changes, I (personally) invoke a remoting server routine that inserts/updates/deletes records onto the server, then forces a sync of that table. What you do is up to you, but in my experience, two-way table sync's against a single server and multiple clients isn't worth the trouble. Since it's presumed that the client is online with the server (because they are changing records) then this works. If you want them to change records offline, then sync, this doesn't help you. Post some code if you do it.

Dang. Remoting is hard.

swallace
User
Posts: 648
Joined: 18-Aug-2003
# Posted on: 12-Sep-2004 19:57:54   

Hmmm. This only seems to work for records that are being updated, but doesn't work for new records (server) that are being moved to the client.

Putting the .IsNew = True flag onto the Entity before the save takes care of the new records, but messes up the old.

I suppose I could check each entity in the collection and mark it .IsNew if found on the client.

Is there a better way?

netLearner
User
Posts: 150
Joined: 18-Oct-2003
# Posted on: 12-Sep-2004 21:42:55   

This may help:

  1. Identify all the parent and Child collections first.

Suppose you have 2 collections CustomersColl and OrdersCollection:

  1. Load all (parents first) Customers from DB1:

CustomerCollection customers1 = new CustomersCollection(); customers1.GetMulti(null);

  1. Load all children next

OrderCollection orders1 = new OrderCollection(); order1.GetMulti(null);

  1. Change the connection string and load all the Customers and orders from DB2:

CustomerCollection customers2 = new CustomersCollection(); customers1.GetMulti(null);

OrderCollection orders2 = new OrderCollection(); order1.GetMulti(null);

  1. Check to see if each entity in the first collection exists in the second collection:

for Customers:

foreach(CustomerEntity customer in customers1) { if(!(customers2.Contains(customer))) { customer.IsNew = true; customers2.Add(customer); }

customer.IsDirty = true; foreach(EntityField aField in customer.Fields) { aField.IsChanged = true; }

}

customers2.SaveMulti(null);

  1. Repeat the above for Orders and the other child collections you have.

Thanks.

1  /  2