It would be very cool to be able to mimic a lazy-load with adapter. For example, using "OrderEntity" as the parent entity: if the code calls: OrderEntity.CustomerEntity then the code would check to see if the "OrderEntity.CustomerEntity" entity was null and then Fetch it, if appropriate. Since Frans has elected to separate data retrieval code (i.e. adapter.FetchEntity(OrderEntity), it seems that writing this "load-loader" would fall to the developer (gulp... me).

I'm not sure that my skills are up to the task. For example, what event would be triggered by merely calling "OrderEntity.CustomerEntity" that would cause the code to check to see if "OrderEntity.CustomerEntity" was null? For example,

 ... some code...
if (OrderEntity.CustomerEntity.CustomerKey = 10000)
{ do something }

The call to CustomerEntity would cause the code to evaluate OrderEntity.CustomerEntity:

...
OrderEntity.CustomerEntity = (CustomerEntity)adapter.FetchNewEntity(new CustomerEntityFactory(), 
    OrderEntity.GetRelationInfoCustomer());
...

Is this a doable thing? How might you go about implementing this code?

Thanks! Jeff

Jeff wrote:

It would be very cool to be able to mimic a lazy-load with adapter. For example, using "OrderEntity" as the parent entity: if the code calls: OrderEntity.CustomerEntity then the code would check to see if the "OrderEntity.CustomerEntity" entity was null and then Fetch it, if appropriate. Since Frans has elected to separate data retrieval code (i.e. adapter.FetchEntity(OrderEntity), it seems that writing this "load-loader" would fall to the developer (gulp... me).

I'm not sure that my skills are up to the task. For example, what event would be triggered by merely calling "OrderEntity.CustomerEntity" that would cause the code to check to see if "OrderEntity.CustomerEntity" was null? For example,

 ... some code...
if (OrderEntity.CustomerEntity.CustomerKey = 10000)
{ do something }

The call to CustomerEntity would cause the code to evaluate OrderEntity.CustomerEntity:

...
OrderEntity.CustomerEntity = (CustomerEntity)adapter.FetchNewEntity(new CustomerEntityFactory(), 
    OrderEntity.GetRelationInfoCustomer());
...

Is this a doable thing? How might you go about implementing this code?

Thanks! Jeff

Well, I have to ask the obvious question: If you need lazy loading why not just use Self-servicing? Once you decide to put persistence functionality in the entities/collections themselves, you've effectively negated Adapter's primary benefit...

Jeff...

I don't really "need" lazy loads, I just really appreciate them. There are other, more important reasons, that the Adapter model is best for me.

Right now, if you use adapter to make a database call, you have to explicitly make the call. This explicit call is not made within the entity/collection itself. It's made within a separate tier - which is appropriate, in light of separation of the persistance logic. But... wouldn't be nice if, within this persistance layer, is code that mimics a lazy load? Instead of making zillions of explicit fetch calls whenever related entities/collections need to be populates, would it be nice if these entities populated themselves with some clever code?

Make sense, I hope?

Jeff

With the release of 1.0.2004.1 of the runtime libs, on september 24th, we introduced prefetch paths. This functionality offers you to define a graph (with filters) which should be fetched together with the fetch of the entity/entities you want to fetch, in one go (1 query per entity type). So if you want to fetch 10 customer objects with all their orders and all the order details, you define a prefetch path to fetch orders and order details, and pass that in with the FetchEntityCollection call when you fetch the customers. 3 queries are then executed: one for customers, one for orders and one for order details.

Perhaps you haven't looked at them closely, but I think they can help you in great deal

Hello Frans.

Yes, I've looked at prefetchpaths and I've used them. They function very well. However, to use them, you have to anticipate their need ahead of time whereas if a lazy-load model can somehow be employed with the adapter model, you wouldn't have to antipicate which related entities/collections might be called.

AH, ok, I understand. Yes that's a limitation of adapter at the moment. The purpose of adapter however, to have entities completely disconnected from the core persistence logic is destroyed by adding lazy loading as Jeffrey has pointed out.

What will be added in the (near) future is that you can use prefetch paths also on an existing loaded collection or object. This is not entirely the same as lazy loading, but makes working with existing objects in adapter more convenient. (and will probably solve your issue)

Not to belabor this issue, but there is something that I just don't understand. Lazy-loading is a pattern wherein an entity is "loaded" with data when its called. If it's not called then the data is never loaded.

What's so cool about SelfServicing is that to load, all you have to do is call the related entity (i.e., string _accName = Division.Account.AccName.ToString()) ). Assuming that Division is already populated with data (since it's the "parent" entity), this call checks to see if Division.Account is null and then fills it with data if it is. That is invisible, seamless and VERY easy to use. The relationship between Account and Division is built into the generated LLGLPro code and is triggered everytime a call to Division.Account is made. The code know exactly what to do.

It seems to me that this model would work with adapter as well. The generated code would have to check for Division nullness and then call adapter.SomeCall(Account.Division) to populate it. Again, this happens behind the scenes so that there is no need for PrefetchPath.

What am I missing here?

Thanks! Jeff

Jeff wrote:

Not to belabor this issue, but there is something that I just don't understand. Lazy-loading is a pattern wherein an entity is "loaded" with data when its called. If it's not called then the data is never loaded.

What's so cool about SelfServicing is that to load, all you have to do is call the related entity (i.e., string _accName = Division.Account.AccName.ToString()) ). Assuming that Division is already populated with data (since it's the "parent" entity), this call checks to see if Division.Account is null and then fills it with data if it is. That is invisible, seamless and VERY easy to use. The relationship between Account and Division is built into the generated LLGLPro code and is triggered everytime a call to Division.Account is made. The code know exactly what to do.

It seems to me that this model would work with adapter as well. The generated code would have to check for Division nullness and then call adapter.SomeCall(Account.Division) to populate it. Again, this happens behind the scenes so that there is no need for PrefetchPath.

What am I missing here?

Thanks! Jeff

The key point is that an Adapter must be supplied in order for the entities to ask for data. The point of Adapter is that the entities don't know anything about the Adapter. If an "Adapter.FetchEntityCollection" call was added to the entity code, then the benefit of separating out entity and persistence logic is lost.

More specifically, let's say you use entities in your GUI/presentation layer. Ordinarily, you would just need to add a reference to the "DatabaseGeneric" DLL generated by LLBLGen. However, if persistence logic (automatic entity fetches, etc) were added to the entities, then a reference to the "DatabaseSpecific" DLL would have to be added to your GUI in order to access the DataAdapter. This, of course, then allows anyone/anything in the GUI to directly makes calls to the database which is exactly what Adapter was created to prevent.

While preventing this may not be a big deal in some scenarios, it is absolutely not viable in a remoting/web services scenario as one could not guarantee the accessibility of the data source.

At the point that any persistence information is required in the entities, one must default to Self-Servicing. If you're using a tiered approach where the persistence calls are made via a service layer you may be able to construct your own "lazy-loading" wherein the entities make calls to that layer, but that wouldn't be something LLBLGen could provide on it's own.

Hope that helps.

Jeff...

Thanks Jeff. I see what you mean.

If LLBLGenPro were to implement some form of lazy loading using adapter it would have to do so within some self-generated middle tier. That tier would * NOT * be functional until additional manual code was provided to hook it to a DataAccess tier or to provide the adapter code directly. The DataAccess tier or the directly applied adapter code would be trivial.

I guess that's getting pretty far from the expected functionality of LLBLGenPro, although it would be a most powerful (and cool) feature.

Jeff

P.S. Which one are you in the picture?

Thanks for the answers Jeffrey

I've been following this thread and must admit that I agree with some of the points that Jeff M makes.

Using Adapter is necessary in 95% of the porjects I build as I need to have full control over where and when data is persisted.

On the other hand, productivity using Adapter is nowhere near as good as compared with SelfServicing...

I totally agree that Adapter Entities should not have any means by which they can perist themselves, but how about this for a solution. Frans hint hint

1) Adapter entities have an additional event "FieldNotLoaded" added. This event takes as parameters the Entity, and a relation field of that entity which is not loaded.

2) Adapter has new method called "RegisterEntityForLazyLoad(IEntity2 entity)" which hooks up the entity's FieldNotLoaded event.

3) When an entity's fields are accessed, if they are null the entity fires the FieldNotLoaded event which is picked up by the Adapter (which previously registered the entity). The Adapter then loads the data before the event returns and the entity can proceed to access the newly loaded data.

Et Voila... Lazy Loading for Adapter.

Obviously this is needs more thinking, but I think this would anable developers to achieve the SelfServicing productivity using the Adapter model in circumstances where Prefetch is not appropriate.

Marcus

Interesting

There are a couple of issues though, one being that if you keep an adapter around, it will keep all registered entities in memory as well, due to the event handler.

My question is then: why don't you use selfservicing, and what should change to selfservicing to make it interesting?

Otis wrote:

There are a couple of issues though, one being that if you keep an adapter around, it will keep all registered entities in memory as well, due to the event handler.

Hey man... I just come up with the ideas, it's up to you to make 'em work! How about adapter.ReleaseEntity(entity)... I'm sure there's a way

Otis wrote:

My question is then: why don't you use selfservicing, and what should change to selfservicing to make it interesting?

I'm not a fan of SelfServicing from an architectural point of view. .. I want to be able to support mutliple Databases via different adapters. I even believe that it should be possible to use different adapters simultaneously. I know this is not possible at the moment, but there's no reason why not. I think this goes back to the discussion we had previously where I complained about the adapter default namespace / DLL name. Here let me complain again:

[RANT]

<RootNameSpace> & <RootNameSpace>DBSpecific.dll is not great

I really think that you should be able to build mutliple adapters which are loaded via some factory. Again this is required when building products and not often needed when building a project (where the target database type is known).

<RootNameSpace>.SqlServer2000 & <RootNameSpace>.SqlServer2000.dll <RootNameSpace>.SqlServer2005 & <RootNameSpace>.SqlServer2005.dll <RootNameSpace>.Oracle & <RootNameSpace>.Oracle.dll <RootNameSpace>.MySQL & <RootNameSpace>.MySQL.dll

makes much more sense for different adapters and gives us the ability to load different adapter assemblies on the fly via an abstract factory.

Also, the Designer should really allow you to built all adapters during the generation process, rather than building them one at a time.

Obviously you can change the TaskPerformers to do all this... but then when you release a new version its a pain to merge all the changes.

[/RANT]

Marcus wrote:

Otis wrote:

There are a couple of issues though, one being that if you keep an adapter around, it will keep all registered entities in memory as well, due to the event handler.

Hey man... I just come up with the ideas, it's up to you to make 'em work! How about adapter.ReleaseEntity(entity)... I'm sure there's a way

heh, yeah I think there is. The real problem is of course that the philosophy behind adapter is not matchable with selfservicing.

Otis wrote:

My question is then: why don't you use selfservicing, and what should change to selfservicing to make it interesting?

I'm not a fan of SelfServicing from an architectural point of view. .. I want to be able to support mutliple Databases via different adapters. I even believe that it should be possible to use different adapters simultaneously. I know this is not possible at the moment, but there's no reason why not.

You can perfectly use multiple adapters at the same time. You can even create a small factory which creates them for you and solely develop against IDataAccessAdapter

I think this goes back to the discussion we had previously where I complained about the adapter default namespace / DLL name. Here let me complain again:

[RANT] <RootNameSpace> & <RootNameSpace>DBSpecific.dll is not great

I really think that you should be able to build mutliple adapters which are loaded via some factory. Again this is required when building products and not often needed when building a project (where the target database type is known).

<RootNameSpace>.SqlServer2000 & <RootNameSpace>.SqlServer2000.dll <RootNameSpace>.SqlServer2005 & <RootNameSpace>.SqlServer2005.dll <RootNameSpace>.Oracle & <RootNameSpace>.Oracle.dll <RootNameSpace>.MySQL & <RootNameSpace>.MySQL.dll

makes much more sense for different adapters and gives us the ability to load different adapter assemblies on the fly via an abstract factory.

No, this isn't correct, let me explain .

Say you want to support both oracle and sqlserver. You then create 2 projects (LLBLGen Pro projects). One of them will supply the entity code, (the database generic code). You can for example generate that from the SqlServer project and throw away the db specific project. Then you generate from BOTH the database specific project (i.e.: generate code again and specify the right rootnamespace, MyProject.Oracle and MyProject.SqlServer for example). For both you throw away the dbgeneric project and you keep the dbspecific project. Et voila.

You can even create a different generator config by simply copying the adapter config and remove teh dbscpecific tasks, to generate a db generic project only. Generating this in a batch is pretty easy.

In other words: the database generic project doesn't know nor cares about the database specific project. They're generated in 1 go, but this can be easily separated by a simple generator config change.

Also, the Designer should really allow you to built all adapters during the generation process, rather than building them one at a time.

I can't create an oracle project from an sqlserver project for example. There are fundamental differences which eventually result in the same .NET type for a field for example but have different db specific types under the hood, so I can't create all db specific projects from a single project.

What will be added real soon (I've already modified the inner core for this) is multiple catalog support per llblgen pro project. This means you can add multiple databases (sqlserver/db2) or multiple schemas (Oracle) to the project and add entities from all databases/schemas to the project, creating relations between them etc. This will already solve some of your issues I think.

Otis wrote:

You can perfectly use multiple adapters at the same time. You can even create a small factory which creates them for you and solely develop against IDataAccessAdapter

Yes, I guess I was talking about a cleaner default "one time" generation... but I take your points..

Otis wrote:

I can't create an oracle project from an sqlserver project for example. There are fundamental differences which eventually result in the same .NET type for a field for example but have different db specific types under the hood, so I can't create all db specific projects from a single project.

Yes of course... I hadn't considered it from the project perspective.

But then here's a thing... Ins't the "project" an abstraction of the database to some degree... It would be cool if the project could reverse the process and create an Oracle DB from a SQLServer source definition. Okay not everything is portable, but for commonalities LLBLGen could be the translator. One day maybe...

Otis wrote:

What will be added real soon (I've already modified the inner core for this) is multiple catalog support per llblgen pro project. This means you can add multiple databases (sqlserver/db2) or multiple schemas (Oracle) to the project and add entities from all databases/schemas to the project, creating relations between them etc. This will already solve some of your issues I think.

This will be very welcome!

Marcus wrote:

Otis wrote:

I can't create an oracle project from an sqlserver project for example. There are fundamental differences which eventually result in the same .NET type for a field for example but have different db specific types under the hood, so I can't create all db specific projects from a single project.

Yes of course... I hadn't considered it from the project perspective.

But then here's a thing... Ins't the "project" an abstraction of the database to some degree... It would be cool if the project could reverse the process and create an Oracle DB from a SQLServer source definition. Okay not everything is portable, but for commonalities LLBLGen could be the translator. One day maybe...

It is possible, but only to some degree. An sdk utility I still have to finish is a project converter, which converts from one db type to another, based on the db type. It's very complex to get that right, as you need for every from-to combination a conversion routine or I have to cook up some intermediate db format for every type possible (which is probably more efficient). Still, there are a lot of types in one db format which simply don't exist in another. bit types, guid's to name a few...

Otis wrote:

It is possible, but only to some degree. An sdk utility I still have to finish is a project converter, which converts from one db type to another, based on the db type. It's very complex to get that right, as you need for every from-to combination a conversion routine or I have to cook up some intermediate db format for every type possible (which is probably more efficient). Still, there are a lot of types in one db format which simply don't exist in another. bit types, guid's to name a few...

The intermediate database format sounds interesting. A user could design the entire schema in LLBLGen Pro. Datatypes specific to a certain database would not be avaible. Just the basics some text some numbers.

The generated code could also gerneate DDL scipts for each DB type. Now that would be cool for 2005

Bert

bertcord wrote:

Otis wrote:

It is possible, but only to some degree. An sdk utility I still have to finish is a project converter, which converts from one db type to another, based on the db type. It's very complex to get that right, as you need for every from-to combination a conversion routine or I have to cook up some intermediate db format for every type possible (which is probably more efficient). Still, there are a lot of types in one db format which simply don't exist in another. bit types, guid's to name a few...

The intermediate database format sounds interesting. A user could design the entire schema in LLBLGen Pro. Datatypes specific to a certain database would not be avaible. Just the basics some text some numbers.

That format is already there: OleDb's The problem is though, how to map the intermediate type onto the native type and how do you make sure the intermediate type results in the same .net type for all database drivers? that's not possible sadly enough.

The generated code could also gerneate DDL scipts for each DB type. Now that would be cool for 2005 Bert

I'm planning to write some templates for generating DDL scripts for a current project, but not for creating databases. The main issue with these is: maintainability: once you created a database, what then?