Generated views with model first

Posts   
1  /  2
 
    
cerberis
User
Posts: 93
Joined: 20-May-2011
# Posted on: 29-Jul-2014 14:59:45   

Hello,

we are using model first scenario currently. However it looks like we need to support some views on that database. Is this supported? Currently generation adds the schema with the same name in lowercase. It works fine until we want to add new entity in the model - it is added to the new schema. Next generation simply does not find such tables in database and removes mappings.

LLBLGen 4.2 Final MySQL database

any ideas?

regards Mantas

Walaa avatar
Walaa
Support Team
Posts: 14950
Joined: 21-Aug-2005
# Posted on: 29-Jul-2014 18:06:27   

Are you doing catalog refresh? If so, did you apply the updated DDL?

cerberis
User
Posts: 93
Joined: 20-May-2011
# Posted on: 30-Jul-2014 07:26:28   

Walaa wrote:

Are you doing catalog refresh? If so, did you apply the updated DDL?

So you mean, after doing changes to the model we should apply scripts and only then refresh from database? It wont override anything in the models? simple_smile It seems quite wrong for me, as I would like one way changes for models and one way changes for views, not both ways.. simple_smile

cerberis
User
Posts: 93
Joined: 20-May-2011
# Posted on: 30-Jul-2014 07:33:27   

I tried what you say, included view in the same schema and tried to refresh model from database and got this exception:

LLBLGen Pro version 4.2. Build July 1st, 2014 -----[Core exception]-------------------- at System.Collections.Generic.List1.Enumerator.MoveNextRare() at System.Linq.Enumerable.WhereSelectEnumerableIterator2.MoveNext() at System.Linq.Enumerable.WhereEnumerableIterator1.MoveNext() at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.RepopulateIdentifyingFields(GroupableModelElementMapping mapping, LogNode parentLogNode) at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.MigrateElementFields[TElement](TElement element, GroupableModelElementMapping mapping, LogNode parentLogNode, IEnumerable1 unmappedTargetFieldsPreMigration) at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.MigrateElementToNewTarget[TElement](LogNode logNodeToUse, TElement elementToMigrate, String elementTypeName, GroupableModelElementMapping mapping, IProjectElementMapTargetElement newTarget, IProjectElementMapTargetElement oldTarget) at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.MigrateElementToNewTarget[TElement](Dictionary2 allElementsWithMappings, LogNode logNodeToUse, TElement elementToMigrate, Dictionary2 elementToNewTarget, String elementTypeName) at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.MigrateEntities() at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.MigrateProjectToNewCatalog() at SD.LLBLGen.Pro.ApplicationCore.MetaData.DatabaseMetaData.RefreshMetaData(IEnumerable1 populatedCatalogs, LogNode refreshLogNode, Project containingProject, UserConfiguration preferences) at SD.LLBLGen.Pro.ApplicationCore.MetaData.MetaDataStore.RefreshMetaData(String driverID, IEnumerable1 populatedCatalogs, Project containingProject, UserConfiguration preferences) at SD.LLBLGen.Pro.ApplicationCore.ProjectClasses.Project.RefreshMetaData(String driverID, List`1 populatedCatalogs, UserConfiguration preferences) at SD.LLBLGen.Pro.Gui.Classes.GuiController.PerformRefreshMetaDataAction(String driverID) at SD.LLBLGen.Pro.Gui.Controls.CatalogExplorer._commandManager_CommandClick(Object sender, CommandEventArgs e) at Janus.Windows.UI.Internal.JNSAU.OnMouseUp(MouseEventArgs e) at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks) at System.Windows.Forms.Control.WndProc(Message& m) at Janus.Windows.UI.Internal.JNSAU.WndProc(Message& m) at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

daelmo avatar
daelmo
Support Team
Posts: 8245
Joined: 28-Nov-2005
# Posted on: 30-Jul-2014 08:02:05   

Hi cerberis,

I read this thread, but I can't figure out the whole picture. Let's start over:

  • Are you using Database-First approach or Model-Fisrt? (Read this for more info).

  • If you are using DB-First: Did you start your project with a fresh Database retrieval wizard?

  • If you are using Model-First: Did you generated the Create DDL Script and run it into your DB Server?

  • About your posted exception, please include the exception's message.

David Elizondo | LLBLGen Support Team
cerberis
User
Posts: 93
Joined: 20-May-2011
# Posted on: 30-Jul-2014 08:07:15   

Hello,

as mentioned in the first post, we are using model first scenario. Yes we generate create-DLL script and run on the database. We also have some changes in script generation template as LLBL does not allow us to specify DateTime precision for MySQL database.

In addition we need to have some quite complex views, which of course we can implement using LLBLGen features, but it is much more easier to do in SQL.

Here is exception message:


A fatal exception occured during project refresh.

An error has occured during the project refresh and the loaded project data is now unstable. LLBLGen Pro has silently unloaded the project data. It has saved the project before the catalog refresh started so you didn't lose any work. The project has been unloaded to prevent you from accidentally saving the unstable project data. Please re-try with the created backup or the project file on disk.

Do you wish to see the full exception information?

Yes No

When I click Yes, I get this:

Exception information.

LLBLGen Pro version: v4.2. Build: July 1st, 2014

Exception details:

Message: Collection was modified; enumeration operation may not execute. Source: mscorlib Stack trace: at System.Collections.Generic.List1.Enumerator.MoveNextRare() at System.Linq.Enumerable.WhereSelectEnumerableIterator2.MoveNext() at System.Linq.Enumerable.WhereEnumerableIterator1.MoveNext() at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.RepopulateIdentifyingFields(GroupableModelElementMapping mapping, LogNode parentLogNode) at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.MigrateElementFields[TElement](TElement element, GroupableModelElementMapping mapping, LogNode parentLogNode, IEnumerable1 unmappedTargetFieldsPreMigration) at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.MigrateElementToNewTarget[TElement](LogNode logNodeToUse, TElement elementToMigrate, String elementTypeName, GroupableModelElementMapping mapping, IProjectElementMapTargetElement newTarget, IProjectElementMapTargetElement oldTarget) at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.MigrateElementToNewTarget[TElement](Dictionary2 allElementsWithMappings, LogNode logNodeToUse, TElement elementToMigrate, Dictionary2 elementToNewTarget, String elementTypeName) at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.MigrateEntities() at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.MigrateProjectToNewCatalog() at SD.LLBLGen.Pro.ApplicationCore.MetaData.DatabaseMetaData.RefreshMetaData(IEnumerable1 populatedCatalogs, LogNode refreshLogNode, Project containingProject, UserConfiguration preferences) at SD.LLBLGen.Pro.ApplicationCore.MetaData.MetaDataStore.RefreshMetaData(String driverID, IEnumerable1 populatedCatalogs, Project containingProject, UserConfiguration preferences) at SD.LLBLGen.Pro.ApplicationCore.ProjectClasses.Project.RefreshMetaData(String driverID, List`1 populatedCatalogs, UserConfiguration preferences) at SD.LLBLGen.Pro.Gui.Classes.GuiController.PerformRefreshMetaDataAction(String driverID) at SD.LLBLGen.Pro.Gui.Controls.CatalogExplorer._commandManager_CommandClick(Object sender, CommandEventArgs e) at Janus.Windows.UI.Internal.JNSAU.OnMouseUp(MouseEventArgs e) at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks) at System.Windows.Forms.Control.WndProc(Message& m) at Janus.Windows.UI.Internal.JNSAU.WndProc(Message& m) at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

Inner exception: <null>

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39614
Joined: 17-Aug-2003
# Posted on: 30-Jul-2014 10:29:19   

Could you at least try the latest build please to avoid you running into things we already fixed? simple_smile

Frans Bouma | Lead developer LLBLGen Pro
cerberis
User
Posts: 93
Joined: 20-May-2011
# Posted on: 30-Jul-2014 10:39:04   

Still same problem:

Exception information.

LLBLGen Pro version: v4.2. Build: July 25th, 2014

Exception details:

Message: Collection was modified; enumeration operation may not execute. Source: mscorlib Stack trace: at System.Collections.Generic.List1.Enumerator.MoveNextRare() at System.Linq.Enumerable.WhereSelectEnumerableIterator2.MoveNext() at System.Linq.Enumerable.WhereEnumerableIterator1.MoveNext() at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.RepopulateIdentifyingFields(GroupableModelElementMapping mapping, LogNode parentLogNode) at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.MigrateElementFields[TElement](TElement element, GroupableModelElementMapping mapping, LogNode parentLogNode, IEnumerable1 unmappedTargetFieldsPreMigration) at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.MigrateElementToNewTarget[TElement](LogNode logNodeToUse, TElement elementToMigrate, String elementTypeName, GroupableModelElementMapping mapping, IProjectElementMapTargetElement newTarget, IProjectElementMapTargetElement oldTarget) at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.MigrateElementToNewTarget[TElement](Dictionary2 allElementsWithMappings, LogNode logNodeToUse, TElement elementToMigrate, Dictionary2 elementToNewTarget, String elementTypeName) at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.MigrateEntities() at SD.LLBLGen.Pro.ApplicationCore.MetaData.CatalogRefresher.MigrateProjectToNewCatalog() at SD.LLBLGen.Pro.ApplicationCore.MetaData.DatabaseMetaData.RefreshMetaData(IEnumerable1 populatedCatalogs, LogNode refreshLogNode, Project containingProject, UserConfiguration preferences) at SD.LLBLGen.Pro.ApplicationCore.MetaData.MetaDataStore.RefreshMetaData(String driverID, IEnumerable1 populatedCatalogs, Project containingProject, UserConfiguration preferences) at SD.LLBLGen.Pro.ApplicationCore.ProjectClasses.Project.RefreshMetaData(String driverID, List`1 populatedCatalogs, UserConfiguration preferences) at SD.LLBLGen.Pro.Gui.Classes.GuiController.PerformRefreshMetaDataAction(String driverID) at SD.LLBLGen.Pro.Gui.Controls.CatalogExplorer._commandManager_CommandClick(Object sender, CommandEventArgs e) at Janus.Windows.UI.Internal.JNSAU.OnMouseUp(MouseEventArgs e) at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks) at System.Windows.Forms.Control.WndProc(Message& m) at Janus.Windows.UI.Internal.JNSAU.WndProc(Message& m) at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

Inner exception: <null>

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39614
Joined: 17-Aug-2003
# Posted on: 30-Jul-2014 11:06:30   

I see it can happen that the enumerator fails due to altered elements, because one of the changes could raise an event. I've attached a new (debug) build which enumerates over a list, not another enumerator. Could you try this one for me, please?

Attachments
Filename File size Added on Approval
SD.LLBLGen.Pro.ApplicationCore.zip 385,517 30-Jul-2014 11:06.36 Approved
Frans Bouma | Lead developer LLBLGen Pro
cerberis
User
Posts: 93
Joined: 20-May-2011
# Posted on: 30-Jul-2014 11:19:21   

Otis wrote:

I see it can happen that the enumerator fails due to altered elements, because one of the changes could raise an event. I've attached a new (debug) build which enumerates over a list, not another enumerator. Could you try this one for me, please?

Actually that one did not produced any error.. simple_smile However, I see lots of remapping which breaks our models..

Like:

Entity 'Address' Entity re-mapped to table 'Default.addresses'. Entity migrated to target 'Default.addresses'. Type definition of the target 'Default.Addresses.Id' of field 'Id' changed. Type properties Max length, precision and scale of field 'Id' synced with the type properties of target 'Default.Addresses.Id'. The field 'IdAddress' is no longer part of the identifying fields. The field 'Id' is now part of the identifying fields.

For me would be best so that refresh would ignore my created models and refresh only selected objects and not deleting not selected items.. Is it possible?

cerberis
User
Posts: 93
Joined: 20-May-2011
# Posted on: 30-Jul-2014 11:58:32   

Maybe I have to add more background about it. We use identity field in the model which is different from database identity. In that case - refresh from database changes our model identity field, which is not what we want. Also we use ModelOnly relations.. not sure if that wont be removed from entities?

TomDog
User
Posts: 618
Joined: 25-Oct-2005
# Posted on: 30-Jul-2014 12:47:51   

If I can just jump in here and ask a related question: What is the best procedure for doing Model-First and database views? Since the designer can't create a view it seems you have to Database-First for at least the view, but, if your doing Model-First, you presumably don't want that for everything else.

On way might be to have a separate Database-First project with the views and mapped entities reverse-engineered from them. Then use copy and paste to cherry pick the views(and their mapped entity) from the Database-First project to the main Model-First project.

Then you would avoid synchronisation errors.

Jeremy Thomas
cerberis
User
Posts: 93
Joined: 20-May-2011
# Posted on: 30-Jul-2014 13:39:23   

Having another database for views of course solves the problem, but adds more complexity for developers to maintain it simple_smile

TomDog
User
Posts: 618
Joined: 25-Oct-2005
# Posted on: 30-Jul-2014 13:44:16   

cerberis wrote:

Having another database for views of course solves the problem, but adds more complexity for developers to maintain it simple_smile

Why would you need another database? You could connect the Database-First project to any database that had the view you wanted in.

Jeremy Thomas
cerberis
User
Posts: 93
Joined: 20-May-2011
# Posted on: 30-Jul-2014 13:46:25   

Project name is based on database name.. So I cannot have 2 schemas in the same project with the same name. If you mean copy/paste from *.llblgenproj file, that's really not the way to go.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39614
Joined: 17-Aug-2003
# Posted on: 30-Jul-2014 15:53:29   

cerberis wrote:

Otis wrote:

I see it can happen that the enumerator fails due to altered elements, because one of the changes could raise an event. I've attached a new (debug) build which enumerates over a list, not another enumerator. Could you try this one for me, please?

Actually that one did not produced any error.. simple_smile However, I see lots of remapping which breaks our models..

Like:

Entity 'Address' Entity re-mapped to table 'Default.addresses'. Entity migrated to target 'Default.addresses'. Type definition of the target 'Default.Addresses.Id' of field 'Id' changed. Type properties Max length, precision and scale of field 'Id' synced with the type properties of target 'Default.Addresses.Id'. The field 'IdAddress' is no longer part of the identifying fields. The field 'Id' is now part of the identifying fields.

For me would be best so that refresh would ignore my created models and refresh only selected objects and not deleting not selected items.. Is it possible?

If it would, there would be 2 sources of truth, which isn't what will work. The rule of thumb is: you work model first, the script produced is what you run on the DB from which you refresh (if ever, you don't need to refresh from the DB, only if you want to import elements from the DB which you didn't add in model first). If you don't run the script on the DB, the project is in fact out of sync, and the relational model data in the project doesn't reflect how your DB looks like and it can't make decisions based on what it will look like at runtime when the generated code (which assumes the DB looks like the relational model data in the project) is run on the DB.

To me it looks like you refreshed your project with a DB that is totally out of sync with the relational model data in the project.

Frans Bouma | Lead developer LLBLGen Pro
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39614
Joined: 17-Aug-2003
# Posted on: 30-Jul-2014 15:55:54   

TomDog wrote:

If I can just jump in here and ask a related question: What is the best procedure for doing Model-First and database views? Since the designer can't create a view it seems you have to Database-First for at least the view, but, if your doing Model-First, you presumably don't want that for everything else.

On way might be to have a separate Database-First project with the views and mapped entities reverse-engineered from them. Then use copy and paste to cherry pick the views(and their mapped entity) from the Database-First project to the main Model-First project.

Then you would avoid synchronisation errors.

Or another catalog, one with the views ?

I must say this only happens when the DB refreshed from is not kept in sync with the relational model data. If you keep it in sync, nothing is wrong (and as a bonus, you might not make a mistake like running generated code which is mapped to schema version X on a DB which is totally out of sync with that)

Frans Bouma | Lead developer LLBLGen Pro
cerberis
User
Posts: 93
Joined: 20-May-2011
# Posted on: 30-Jul-2014 15:57:12   

As I mentioned: our physical database model is always different from the model. We are using historical tables which keeps history together with current data. That means our identity in model and identity in database are different. So my wish - to have models which are not touched during the refresh. Some kind of flag on the model should be quite easy to implement simple_smile However for now I see that I need 2 databases or implement the view in the code. That's what I was asking simple_smile Thank you.

TomDog
User
Posts: 618
Joined: 25-Oct-2005
# Posted on: 31-Jul-2014 07:42:15   

cerberis wrote:

Project name is based on database name.. So I cannot have 2 schemas in the same project with the same name. If you mean copy/paste from *.llblgenproj file, that's really not the way to go.

I mean have two instances of LLBL running. One containing your existing Model-First project, the other containing a Database-First project completely generated from whatever DB has the view. The catalogs of the two would be much the same. Just copy and paste objects from one LLBL instance to the other (or alternatively import to Database-First project into the Model-First and the pick the objects you want).

No need for 2 DBs.

Though I think you will still have a problem if you have already have a view and you want add another field to it. Apart from deleting and re-adding, you would be looking at editing the *.llblgenproj directly - it's no so baddisappointed

Jeremy Thomas
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39614
Joined: 17-Aug-2003
# Posted on: 31-Jul-2014 09:42:47   

cerberis wrote:

As I mentioned: our physical database model is always different from the model.

That's a little odd. Model first suggests the target DB changes (otherwise, why alter the model to get a different DB?) and at runtime the target DB has to match the relational model data the model is mapped to.

We are using historical tables which keeps history together with current data. That means our identity in model and identity in database are different. So my wish - to have models which are not touched during the refresh. Some kind of flag on the model should be quite easy to implement simple_smile However for now I see that I need 2 databases or implement the view in the code. That's what I was asking simple_smile Thank you.

It's not the technical aspect of it that's bothering me, no worries there wink The thing is: if the target schema the project is refreshed with isn't equal to the relational model and that's deliberately so and the relational model data in the project should NOT be migrated to that schema, why is it different as the model mapped to the relational model data then won't work at runtime as the database is different?

btw: you can have multiple catalogs (databases) in 1 project. You can decide to work model first on DB A (which is kept in sync in the target DB so refresh shows no changes) and have a view and several tables in DB B. You simply don't select the tables, just the view, and it will refresh properly. You map the model to DB A and an entity to the view in DB B. You can then decide at runtime to have everything in 1 DB (e.g. you moved the view to DB A) by overwriting the catalog name at runtime from B to A. Does that help?

Frans Bouma | Lead developer LLBLGen Pro
cerberis
User
Posts: 93
Joined: 20-May-2011
# Posted on: 31-Jul-2014 11:11:57   

That's actually the whole beauty of LLBLGenPro and why I like it: its very customizable and flexible simple_smile . Our model contains all the fields which are in database. However, field definitions are different. For example, we have field 'Id' in each table, which is not PK in model, but it is IDENTITY in database (create/update script is generated using this convention). We also have EntityId column, which is defined as PK in the model, but its a regular field in the database. We have logical relations on EntityId column, which cannot be mapped to physical database relations. To be honest, we should use EntityId, ExistenceStart, ExistenceEnd as PK, but this makes a lot of troubles when creating foreign keys in database, as we need to have 3 columns for foreign key. So we solve this by adding filters automatically at adapter level when creating select/update/delete queries. Everything works fine and very good and we still can use all ORM features. The only thing - we cannot refresh model from database because PK in model and PK in database are different simple_smile So for summary - probably we will use multiple catalogs even if it makes some more work for developers changing the views (or we will move views to the code).

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39614
Joined: 17-Aug-2003
# Posted on: 31-Jul-2014 14:27:19   

cerberis wrote:

That's actually the whole beauty of LLBLGenPro and why I like it: its very customizable and flexible simple_smile .

simple_smile

Our model contains all the fields which are in database. However, field definitions are different. For example, we have field 'Id' in each table, which is not PK in model, but it is IDENTITY in database (create/update script is generated using this convention). We also have EntityId column, which is defined as PK in the model, but its a regular field in the database. We have logical relations on EntityId column, which cannot be mapped to physical database relations. To be honest, we should use EntityId, ExistenceStart, ExistenceEnd as PK, but this makes a lot of troubles when creating foreign keys in database, as we need to have 3 columns for foreign key.

Ok. You could set a setting where PKs don't follow PK constraints in the DB when refreshing btw, so they're seen as 'model only' (like relationships are too). Dont know whether you already use this.

So we solve this by adding filters automatically at adapter level when creating select/update/delete queries. Everything works fine and very good and we still can use all ORM features. The only thing - we cannot refresh model from database because PK in model and PK in database are different simple_smile

Please set 'Identifying fields follow dbprimary key constraints' setting under catalog refresher to 'false' (it's 'default' by default, which means it uses the setting value of the preferences, which is likely set to 'true'). This will make the refresher leave the pk definitions in the model alone.

So for summary - probably we will use multiple catalogs even if it makes some more work for developers changing the views (or we will move views to the code).

If the setting I mentioned above isn't working out, that's indeed the best way (multiple catalogs)

Frans Bouma | Lead developer LLBLGen Pro
cerberis
User
Posts: 93
Joined: 20-May-2011
# Posted on: 31-Jul-2014 16:41:08   

Seems like the setting helped simple_smile

Also I had to disable Sync mapped element names after refresh as it caused lowercase'ing model names.

Thank you.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39614
Joined: 17-Aug-2003
# Posted on: 31-Jul-2014 18:17:53   

So we can close the issue? simple_smile

Frans Bouma | Lead developer LLBLGen Pro
cerberis
User
Posts: 93
Joined: 20-May-2011
# Posted on: 31-Jul-2014 18:23:56   

I would say 'model only' entities would be great and very nice to have feature. But for now we can close it. Thank you.

1  /  2