Custom LptParserEngine only runs in debug mode

Posts   
 
    
basik
User
Posts: 123
Joined: 20-Aug-2012
# Posted on: 29-May-2013 16:22:53   

I've created custom versions of the LptParserEngine, copied them to a location which has been added to the Additional tasks folder under the project settings. This is using the sourcecode available under the Extras section of the 3.5 version in the Customer Area. I've added a key file to sign the assemblies and generated these assemblies: SD.LLBLGen.Pro.LptParser.dll SD.LLBLGen.Pro.TaskPerformers.dll SD.LLBLGen.Pro.TDLInterpreter.dll SD.LLBLGen.Pro.TDLParser.dll

I found I had to remove the default install folder dlls to get it to use my custom versions: C:\Program Files (x86)\Solutions Design\LLBLGen Pro v3.5\TaskPerformers

I have 2 issues: 1. Debug does not seem to work, I attach to the LLBGenPro process from VS2010, set a breakpoint and then run the code generator. The templates process correctly, just can't debug them. 2. The Code generation will only work if I run it using the Start generator(debug) button, even if I build a release version of the assemblies. The Start generator(Normal) button always fails on the line of code in the DotNetTemplateEngine.cs file shown below. This happens even if I build the source code without any of my customisations. The error is Object reference not set

this.PerformResult = this.EmitOutput(parameters, fullPath, templateID, compiledTemplatesAssembly, emitTypeToPerform, filenameFormat, failWhenExistent);

I've output the parameters passed to the progress dialog and none of the arguments to this method are null - see screenshot attached.

Thank you.

Attachments
Filename File size Added on Approval
LptParserEngineIssue3.5.png 62,042 30-May-2013 14:19.49 Approved
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39625
Joined: 17-Aug-2003
# Posted on: 30-May-2013 11:08:45   

basik wrote:

I've created custom versions of the LptParserEngine, copied them to a location which has been added to the Additional tasks folder under the project settings. This is using the sourcecode available under the Extras section of the 3.5 version in the Customer Area. I've added a key file to sign the assemblies and generated these assemblies: SD.LLBLGen.Pro.LptParser.dll SD.LLBLGen.Pro.TaskPerformers.dll SD.LLBLGen.Pro.TDLInterpreter.dll SD.LLBLGen.Pro.TDLParser.dll

I found I had to remove the default install folder dlls to get it to use my custom versions: C:\Program Files (x86)\Solutions Design\LLBLGen Pro v3.5\TaskPerformers

That's because the name of the assembly is the same. Please change the name of the assembly of your lpt engine build (in the vs.net project properties). In the task running the template, it should specify your assembly, not the one we ship. Otherwise it will run ours, not yours. See the code generation configuration dialog, advanced -> open an lpt task.

I have 2 issues: 1. Debug does not seem to work, I attach to the LLBGenPro process from VS2010, set a breakpoint and then run the code generator. The templates process correctly, just can't debug them.

Hmm. Are you sure your assembly is run and not ours? Did you change the task settings or did you keep them as-is?

  1. The Code generation will only work if I run it using the Start generator(debug) button, even if I build a release version of the assemblies. The Start generator(Normal) button always fails on the line of code in the DotNetTemplateEngine.cs file shown below. This happens even if I build the source code without any of my customisations. The error is Object reference not set
this.PerformResult = this.EmitOutput(parameters, fullPath, templateID, compiledTemplatesAssembly, emitTypeToPerform, filenameFormat, failWhenExistent);

I've output the parameters passed to the progress dialog and none of the arguments to this method are null - see screenshot attached.

The exception occurs inside a template. If you look closely, you'll see the error occurs at templatesource.cs, line 2330. The templatesource.cs file is generated from the lpt templates and is placed in a temporary folder as you can see. That line will get you more insight in what's wrong.

Btw, what did you add, if I may ask, to the lpt engine that you need a custom build of it? The setup of lpt is done in such a way that the engine is very thin, and you can add anything to it without the necessity of changing it: if you need processing code inside a template, place it inside an assembly you reference inside the template, this is easier and you don't have to alter the template engine.

Frans Bouma | Lead developer LLBLGen Pro
basik
User
Posts: 123
Joined: 20-Aug-2012
# Posted on: 30-May-2013 13:31:09   

Thank you for the response.

I have created our own custom task performers named assembly in addition to customised versions retaining the original names and created a custom .tasks entry to use in the presets. All that works fine, except our named versions of the LptParser would not work with the default templates under the Entity Framework. I now think this is to do with the CompileTemplates method in the DotNetTemplateEngine.cs which adds name spaces. I can revisit that. I know my assemblies are being called as I temporarily removed the shipped asssemblies from the original install location. Anyway I think I can resolve the issues based on your pointers.

Why do we need a custom version of these? Maybe we don't and it wasn't an option I particularly wanted to follow. This goes back to an earlier thread: https://www.llblgen.com/TinyForum/Messages.aspx?ThreadID=21561

I have successfully created a Domain Driven Design - http://microsoftnlayerapp.codeplex.com/ - set of templates which get rid of the EDMX file and allow a code first Entity Framework project. This creates the 5 layers covered by the DDD template.

For every application module added you would need to generate a set of folders and project files

Solution CompanyName.YourApp.YourModule.sln Projects CompanyName.YourApp.Application.YourModule.csproj CompanyName.YourApp.DistributedServices.YourModule.csproj CompanyName.YourApp.Domain.YourModule.csproj CompanyName.YourApp.Infrastructure.Crosscutting.csproj CompanyName.YourApp.Infrastructure.Data.YourModule.csproj Folders: CompanyName.YourApp.Application.YourModule CompanyName.YourApp.DistributedServices.YourModule CompanyName.YourApp.Domain.YourModule CompanyName.YourApp.Infrastructure.Crosscutting CompanyName.YourApp.Infrastructure.Data.YourModule

The problem we had was really a question of naming conventions. What we need is to be able to name the output elements, folders and project files using these tags. [rootNameSpace].[DomainLayerName].[projectName] and this would represent the convention CompanyName.YourApp.Application.YourModule where CompanyName fits nicely with the Rootname space in the LLBGen Proj. and YourModule can be the [projectName].

The SDK covers the tags available for substitution in the various task performers - http://llblgen.com/documentation/3.5/SDK/hh_start.htm#shipped_taskperformers.htm In spite of the options, I could just not create a Project which would not require a custom set of templates for each module we added to the DDD solution.

The code change was very simple and basically adds this line of code to each of the task performers DirectoryCreator, ProjectFileCreator and the LptParser DotNetTemplateEngine class Perform method:

parameters = CodeGenHelper.ParsedParameters(parameters, this.ExecutingGenerator.ProjectDefinition.OutputSettingValues.CustomProperties);

The method simply allows tags to be used from CustomProperties and to replace them in the parameters of any task performer. And the standard tags like [projectName],[rootNameSpace] are replaced for folders and project files.

For example in the POCOClass folder create task performer I set the folderToCreate property to [rootNameSpace].$(Layer_Domain).[projectName]\POCOClasses

The project has a custom property called Layer_Domain which has a value Domain, so at runtime the $(Layer_Domain) tag is replaced with Domain.

Of course, I accept that this may have been achieved in the template code and I'd gladly do this but creating custom templates is not easy and there is a lot of trial and error. In the end this was the solution that works for us.

Attachments
Filename File size Added on Approval
CodeFirstTemplate.png 39,890 30-May-2013 13:31.22 Approved
basik
User
Posts: 123
Joined: 20-Aug-2012
# Posted on: 30-May-2013 14:23:39   

PS Updated the original screenshot which shows the actual issue when running the generator in Normal and not Debug mode.

The errors in DotNetTemplateEngine.cs of the LptParser project. Line 654

templateObject.___RUN(this.ExecutingGenerator, parameters, outputWriter, currentElement);

Line 461

this.PerformResult = this.EmitOutput(parameters, fullPath, templateID, compiledTemplatesAssembly, emitTypeToPerform, filenameFormat, failWhenExistent);
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39625
Joined: 17-Aug-2003
# Posted on: 31-May-2013 11:04:53   

About the namespaces: you can set additional namespaces in a taskgroup. So if you place task A, B and C in group T, you can click T in the preset in the advanced view of the code generation configuration dialog and in the 'additional parameters' view of the taskgroup you can specify additional namespaces and folders. These can't be formed by a macro, but looking at your setup, I think they're tied to the 5 layers you have so you can define a preset with existing tasks and specify the additional namespaces/folders per taskgroup (so 1 taskgroup per layer for example) which I think should solve your problem without the requirement of a specific lpt parser.

about the line numbers: when running in 'normal' mode, the code generated from the lpt templates is compiled as a release build, so there's no pdb file, no line numbers and no debug info. The code is compiled in-memory so there's no real assembly on disk. I think that's why the method which crashes is the call to RUN, but the code which actually crashes is called by RUN (which is the method of the class created from the .lpt template code). So debug mode is required to obtain enough information to find crashes in your templates.

Frans Bouma | Lead developer LLBLGen Pro
basik
User
Posts: 123
Joined: 20-Aug-2012
# Posted on: 31-May-2013 17:48:51   

Otis wrote:

About the namespaces: you can set additional namespaces in a taskgroup. So if you place task A, B and C in group T, you can click T in the preset in the advanced view of the code generation configuration dialog and in the 'additional parameters' view of the taskgroup you can specify additional namespaces and folders. These can't be formed by a macro, but looking at your setup, I think they're tied to the 5 layers you have so you can define a preset with existing tasks and specify the additional namespaces/folders per taskgroup (so 1 taskgroup per layer for example) which I think should solve your problem without the requirement of a specific lpt parser.

Thank you for the response. Ideally I don't want to customise the generator modules if at all possible. I've been trying to make this work all day. It's so closesimple_smile Please see the attached screenshot of the Additional Parameters tab settings. To support the naming pattern needed for folders, project files and fileformat we need this combination: CompanyName.YourApp.Application.YourModule

The YourModule should be the projectName. And CompanyName.YourApp.Application becomes a combination of the project Root namespace and the Additional root namespace.

The thing I can't work out is that YourModule must be the ProjectName - this is the thing which varies for each DDD module added. We would have as many LLBL Projects as we had modules in the system. However there is no placeholder for projectName accepted by Folder creation tasks. The same issue for the VS project filename.

Sorry if I am missing the obvious.

about the line numbers: when running in 'normal' mode, the code generated from the lpt templates is compiled as a release build, so there's no pdb file, no line numbers and no debug info. The code is compiled in-memory so there's no real assembly on disk. I think that's why the method which crashes is the call to RUN, but the code which actually crashes is called by RUN (which is the method of the class created from the .lpt template code). So debug mode is required to obtain enough information to find crashes in your templates.

Thank you for explaining this bit

Attachments
Filename File size Added on Approval
AdditionalRootnamespace.jpg 151,778 31-May-2013 17:49.06 Approved
basik
User
Posts: 123
Joined: 20-Aug-2012
# Posted on: 03-Jun-2013 13:41:26   

I've been thinking about using the lpt template instead to do this customisation. Is there any way to get hold of the parameters collection for an executing task from within the lpt template code?


//The Perform method used by DirectoryCreator, ProjectFileCreator and CodeEmitter and provided by TaskPerformerBase
public override void Perform(IGenerator executingGenerator, ITask taskDefinition, Dictionary<string, TaskParameter> parameters)
{

}

Or is it too late to try to force folder and filename changes from the template itself?

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39625
Joined: 17-Aug-2003
# Posted on: 04-Jun-2013 10:22:57   

Did you try [containerName] to get the project name injected?

It's not possible to get hold of the parameters passed to the task performer inside the lpt template and even so, it would be too late to manipulate the target filename.

Frans Bouma | Lead developer LLBLGen Pro
basik
User
Posts: 123
Joined: 20-Aug-2012
# Posted on: 04-Jun-2013 11:45:15   

Thank you for the feedback. I thought that would be the case. I've tried the [containerName] but that doesn't work for folder names as described in the SDK.

I would propose an enhancement to the shipped task performers such that they all support the same replacement placeholders. This would allow more flexibility. The ability to put custom properties for a project as placeholders in folders/project files and output elements would be very useful too. Doubt you need my code for this, but I'm happy to share what I learnt from the exercise.

I will create a customised task performer set to provide this functionality for our requirements, but leave the LptParser engine as the shipped one.

Please go ahead and close this thread. Thank you for a super product and support.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39625
Joined: 17-Aug-2003
# Posted on: 05-Jun-2013 14:08:56   

simple_smile

The names to process are easy to add to the task performers we have, so if you have a concrete proposal which ones we should add and for which elements (file name, folder name) we can add them to the next 4.x version.

We rather would see 'frameworksettings' instead of custom properties, at the project level. Would e.g. [$settingname] work for example? The code would then lookup the $settingname in the framework settings known by the project and replace $settingname with the value set.

Frans Bouma | Lead developer LLBLGen Pro
basik
User
Posts: 123
Joined: 20-Aug-2012
# Posted on: 01-Jul-2013 18:21:27   

Otis wrote:

simple_smile

The names to process are easy to add to the task performers we have, so if you have a concrete proposal which ones we should add and for which elements (file name, folder name) we can add them to the next 4.x version.

We rather would see 'frameworksettings' instead of custom properties, at the project level. Would e.g. [$settingname] work for example? The code would then lookup the $settingname in the framework settings known by the project and replace $settingname with the value set.

Sorry for the late response, just so busy! Thank you for the prompt response on this. I'd prefer the framework settings option. That would be easier than the Custom Properties, but I will need to check the SDK on how to use these from the templates and adding our own.

I'll be working on the templates for the next few days and I'll let you if I have any concrete proposals for elements to support, though the framework settings would seem to meet all requirements if they could be applied at folder,project and element level task performers.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39625
Joined: 17-Aug-2003
# Posted on: 02-Jul-2013 10:19:54   

basik wrote:

Otis wrote:

simple_smile

The names to process are easy to add to the task performers we have, so if you have a concrete proposal which ones we should add and for which elements (file name, folder name) we can add them to the next 4.x version.

We rather would see 'frameworksettings' instead of custom properties, at the project level. Would e.g. [$settingname] work for example? The code would then lookup the $settingname in the framework settings known by the project and replace $settingname with the value set.

Sorry for the late response, just so busy! Thank you for the prompt response on this. I'd prefer the framework settings option. That would be easier than the Custom Properties, but I will need to check the SDK on how to use these from the templates and adding our own.

Good place to check is perhaps the linq to sql templates. They're the simplest and use the settings feature.

I'll be working on the templates for the next few days and I'll let you if I have any concrete proposals for elements to support, though the framework settings would seem to meet all requirements if they could be applied at folder,project and element level task performers.

Ok. Just post in this thread and it will pop up in our queue simple_smile

Frans Bouma | Lead developer LLBLGen Pro
basik
User
Posts: 123
Joined: 20-Aug-2012
# Posted on: 02-Jul-2013 15:24:07   

        public static string TokenReplace(string inputValue, Project currentProject)
        {
            try
            {
                OutputSettingValuesContainer outputSettings = currentProject.OutputSettingValues;
                foreach (KeyValuePair<string, string> kvp in outputSettings.CustomProperties)
                {
                    inputValue = inputValue.Replace(string.Format("$({0})", kvp.Key), kvp.Value);
                    inputValue = inputValue.Replace(string.Format("[${0}]", kvp.Key), outputSettings.GetRealStringSettingValue(kvp.Value, currentProject));
                }
            }
            catch (Exception e)
            {
                //this.AddNewLogNode(this.ActiveTask.ElementLogNode, LogNodeType.ActionDescription, "Token replace exception {0} ", e.Message + e.StackTrace);
            }

            return inputValue;
        }

I've added the framework settings as per the SDK and am able to view them in the designer. The code above is just about all I need to implement a custom task performer which would replace the Framework settings tokens based on ${settingname} or [$settingname] tokens in the custom versions of the CodeEmitter,ProjectFileCreator and DirectoryCreator classes.

The only issue I have is that the custom framework settings are not available as a dictionary/array list as far as I can tell from the API - so how could I iterate over the currentProject.OutputSettingValues collection to replace settings without having to know the names of the settings to begin with. Maybe I can add a framework setting that is a list of values which happen to be the settings I need by name - is this on the right track?

I've looked at the Linq to SQL templates and they showed the CustomProperties iteration which I have in the code block above, but retrieve settings by name.


bool emitKnownTypeAttributes = entity.OutputSettingValues.GetRealBoolSettingValue("EmitKnownTypeAttributesForWCF", currentProject);

Any advice would be greatly appreciated. I'm so close to getting this done!

basik
User
Posts: 123
Joined: 20-Aug-2012
# Posted on: 02-Jul-2013 18:19:57   

For now I can just add the settings as a hardcoded list in my custom task performer to progress this customisation. However I've come up against an issue which I can't get to work:

I've created a CustomDotNetTemplateEngine class in my custom task performer assembly which inherits from the DotNetTemplateEngine and overrrides the two Perform methods


    public class CustomDotNetTemplateEngine : DotNetTemplateEngine
    {
        public override void Perform(IGenerator executingGenerator, ITask taskDefinition)
        {
            base.Perform(executingGenerator, taskDefinition);
        }

        public override void Perform(SD.LLBLGen.Pro.ApplicationCore.IGenerator executingGenerator, SD.LLBLGen.Pro.ApplicationCore.ITask taskDefinition, Dictionary<string, SD.LLBLGen.Pro.ApplicationCore.CodeGenerationMetaData.Tasks.TaskParameter> parameters)
        {
            Debugger.Break();
            this.ActiveTask = taskDefinition;
            this.ExecutingGenerator = executingGenerator;


            if (parameters == null)
            {
                throw new GeneratorAbortException("No parameters have been specified. Aborting generator.", taskDefinition);
            }
            if (parameters.Count <= 0)
            {
                throw new GeneratorAbortException("No parameters have been specified. Aborting generator.", taskDefinition);
            }

            parameters = CodeGenHelper.ParsedParameters(parameters, executingGenerator.ProjectDefinition);

            base.Perform(executingGenerator, taskDefinition, parameters);
        }
    }

I had to put the Debugger.Break() line in to get it to actually start the debugger - the SDK documentation instructions would just not work. I've added to a custom .tasks file and got a preset with entries that use this custom task performer. The ProjectFileCreator and DirectoryCreator work just fine.


<task name="CompanyName.Tasks.Base.ConsumeLptTemplateExtended" 
        assemblyFilename="CompanyName.LLBLGen.Pro.TaskPerformers.dll" 
        taskPerformerClass="CompanyName.LLBLGen.Pro.TaskPerformers.CustomDotNetTemplateEngine"
        description ="Consumes a .lpt template and generates code using that template and the meta-data specified with the emitType parameter. Provides some custom naming functionality."
        isOptional="true">

It always fails on this line with an error about missing type or Namespace ITemplateClass - see the attached screenshot and the resulting templateSource.cs.


 base.Perform(executingGenerator, taskDefinition, parameters);

I've tried turning the Debug property on the preset task true and false and running the Standard and Debug generator options.

Any ideas? Thank you.

Attachments
Filename File size Added on Approval
templatesSource.zip 50,595 02-Jul-2013 18:20.56 Approved
ITemplateClassError.jpg 200,981 02-Jul-2013 18:21.48 Approved
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39625
Joined: 17-Aug-2003
# Posted on: 03-Jul-2013 13:41:08   

It's not possible to enumerate the settings, they're stored internally in a dictionary, with calculated keys. You need to specify the setting by name. This is unfortunately by design, as the settings drive the templates, so the setting is of no value other than for templates so the templates know which setting to work on.

basik wrote:

For now I can just add the settings as a hardcoded list in my custom task performer to progress this customisation. However I've come up against an issue which I can't get to work:

I've created a CustomDotNetTemplateEngine class in my custom task performer assembly which inherits from the DotNetTemplateEngine and overrrides the two Perform methods


    public class CustomDotNetTemplateEngine : DotNetTemplateEngine
    {
        public override void Perform(IGenerator executingGenerator, ITask taskDefinition)
        {
            base.Perform(executingGenerator, taskDefinition);
        }

        public override void Perform(SD.LLBLGen.Pro.ApplicationCore.IGenerator executingGenerator, SD.LLBLGen.Pro.ApplicationCore.ITask taskDefinition, Dictionary<string, SD.LLBLGen.Pro.ApplicationCore.CodeGenerationMetaData.Tasks.TaskParameter> parameters)
        {
            Debugger.Break();
            this.ActiveTask = taskDefinition;
            this.ExecutingGenerator = executingGenerator;


            if (parameters == null)
            {
                throw new GeneratorAbortException("No parameters have been specified. Aborting generator.", taskDefinition);
            }
            if (parameters.Count <= 0)
            {
                throw new GeneratorAbortException("No parameters have been specified. Aborting generator.", taskDefinition);
            }

            parameters = CodeGenHelper.ParsedParameters(parameters, executingGenerator.ProjectDefinition);

            base.Perform(executingGenerator, taskDefinition, parameters);
        }
    }

I had to put the Debugger.Break() line in to get it to actually start the debugger - the SDK documentation instructions would just not work.

In the project you have the code in above, place breakpoint at line with 'this.ActiveTask = taskDefinition;', then start llblgen pro designer. In vs.net with the code above, press cntrl-alt-P and attach to the llblgen pro designer. You then generate code, and the debugger should stop at the line you placed the breakpoint. No Debugger.break needed.

I've added to a custom .tasks file and got a preset with entries that use this custom task performer. The ProjectFileCreator and DirectoryCreator work just fine.


<task name="CompanyName.Tasks.Base.ConsumeLptTemplateExtended" 
        assemblyFilename="CompanyName.LLBLGen.Pro.TaskPerformers.dll" 
        taskPerformerClass="CompanyName.LLBLGen.Pro.TaskPerformers.CustomDotNetTemplateEngine"
        description ="Consumes a .lpt template and generates code using that template and the meta-data specified with the emitType parameter. Provides some custom naming functionality."
        isOptional="true">

It always fails on this line with an error about missing type or Namespace ITemplateClass - see the attached screenshot and the resulting templateSource.cs.


 base.Perform(executingGenerator, taskDefinition, parameters);

I've tried turning the Debug property on the preset task true and false and running the Standard and Debug generator options.

ITemplateClass is defined in the LptParserEngine assembly. Although it is specified in the using list, it seems the dll isn't referenced at compile time. Could you add an assembly reference directive to your lpt template (using the <$ full path + filename of assembly $> syntax) to explicitly reference it? I know it sounds odd, but still...

Frans Bouma | Lead developer LLBLGen Pro
basik
User
Posts: 123
Joined: 20-Aug-2012
# Posted on: 03-Jul-2013 17:43:08   

Otis wrote:

It's not possible to enumerate the settings, they're stored internally in a dictionary, with calculated keys. You need to specify the setting by name. This is unfortunately by design, as the settings drive the templates, so the setting is of no value other than for templates so the templates know which setting to work on.

That makes sense really as it is a customisation specific to a set of templates.

In the project you have the code in above, place breakpoint at line with 'this.ActiveTask = taskDefinition;', then start llblgen pro designer. In vs.net with the code above, press cntrl-alt-P and attach to the llblgen pro designer. You then generate code, and the debugger should stop at the line you placed the breakpoint. No Debugger.break needed.

Yes - this works now!

ITemplateClass is defined in the LptParserEngine assembly. Although it is specified in the using list, it seems the dll isn't referenced at compile time. Could you add an assembly reference directive to your lpt template (using the <$ full path + filename of assembly $> syntax) to explicitly reference it? I know it sounds odd, but still...

Yes that was it! Just added this line to an include .lpt template used by all custom templates and it works. The debugger is hit aswell. Thank you so much for the brilliant support as usual! That comes as standard.

<$ "C:\Program Files (x86)\Solutions Design\LLBLGen Pro v3.5\TaskPerformers\SD.LLBLGen.Pro.LptParser.dll" $>