It appears that by serializing an entity collection that had sorting as part of a child collection in the prefetch loses the sort order upon serializing and deserializing using XML.

Is there some reason why this would happen internal to LLBL Gen? I certainly can do a sort on the client after the fact, but I'd sure rather not .

I'm in the C#/VS2005 adapter scenario, using v1.0.2005.1 Final.

Thanks, Paul

paulwhit wrote:

It appears that by serializing an entity collection that had sorting as part of a child collection in the prefetch loses the sort order upon serializing and deserializing using XML.

Is there some reason why this would happen internal to LLBL Gen? I certainly can do a sort on the client after the fact, but I'd sure rather not .

I'm in the C#/VS2005 adapter scenario, using v1.0.2005.1 Final.

The xml serializer code just writes the entities from front to back to the xml. So I'm curious of how it was deserialized in a different order, on ONE exception: as the entities can have circular references, it can happen that at a position of an entity in a collection, it already was processed and at deserialization time, it is processed after the entities are all read, which makes those entities appear at the end of the collection.

In which exact scenario do you experience this: so entity hierarchy, and which entity collection do you serialize first?

Hello, Otis!

Well, the client side sorting was pretty clean, so that solved the issue for me. But here is some more explanation of what I encountered.

I have an entity "Roadmap" with a child collection of "RoadmapMilestones" and another child collection of "RoadmapSteps" (it's an analogy, not dealing with roads directly ).

So, Roadmap<-RoadmapMilestone<-RoadmapStep

Before serializing, the milestones and steps are each sorted by a sequence number, which each entity has as one of its fields.

I have a web method "GetRoadmap" that serializes the roadmap like so:

RoadmapEntity retVal = Impl.GetRoadmap(roadmapId);
string xml = string.Empty;
retVal.WriteXml(out xml);
return xml;

And then a part that consumes the service and deserializes like so (there is no processing in between):

string xml = WebService.Service.GetRoadmap(roadmapId);
RoadmapEntity retVal = new RoadmapEntity();
retVal.ReadXml(xml);
return retVal;

After this, the sort order is not correct. It doesn't appear to even be reverse order (it's actually 1, 4, 5, 6, 7, 2, 3). Note that the sort order works correctly when I do not serialize/deserialize.

The very first task I attempt with the roadmap is to iterate through the milestones collection in a foreach loop, processing some data there and iterating through each of that milestone's steps as well.

Thanks, Paul

Ok I'll check it out. It's very strange.

So you fetch the data from the db, sorted in some order. You have, say, 5 items, they're then in 1, 2, 3, 4, 5 order. You serialize them to xml, transport that, read it back and the order is 1,4,3,2,5 (or whatever) ?

Thanks Otis. Yes, that's what I'm seeing.

As a side note, for each entity in the sorted collection, there is another child collection of varying length that is also sorted on itself. Perhaps there is some a relationship between the child collection sort and the final order...

Paul

Hi, I have a similar problem. I've been using the 2005 adapter scenario with the new web service addition. Everything worked perfectly until I came up to the following situation:

Two tables:

City Id StateId Name ...

State Id Name ...

I want to fecth the paged City EntityCollection with prefatch states via web service into a winforms datagrid on a client. Everything works great, sorting by the CityEntity columns produces the correct ordering. However, when I want to sort by the State name:

bucket.Relations.Add(CityEntity.Relations.StateEntityUsingStateId);
sortExpression = new SortExpression();
sortExpression.Add(SortClauseFactory.Create(StateFieldIndex.Name, SortOperator.Ascending));

I go to debug and I see the correct ordering after the adapter.FetchEntityCollection line:

...
using (DataAccessAdapter adapter = new DataAccessAdapter())
{
    totalRecords = (int)adapter.GetDbCount(entities, bucket);
    adapter.FetchEntityCollection(entities, bucket, 0, sortExpression, prefetchPath, pageNo, pageSize);
}
return entities;
} //here the web service method ends

In the client app's debug, immediately after the web method call, the ordering is messed up.

If it helps - I do have a prefetch path to State and it works fine, I don't think it has anything to do with this. I also have additional filtering conditions in the bucket, but they are only related to the CityEntity fields. The StateEntityUsingStateId is the only relation I add to the bucket. All Id fields are of type Guid (uniqueidentifier).

Can you please try to reproduce this? Am I doing something wrong here?

Additionaly, I saw the threads about paging & sorting problems and I tracked the generated queries in the profiler. Can you please confirm if I understood this right - if there is a relation in the bucket, so two tables are joined, the sql paging with temp table won't work (regardless of sorting column) - in that case you grab all data and do the paging & removing eventual duplicates inside your object model. Is this always the case or I missed something? In other words - is there a way for me to use the scenario above with the sql temp table paging enabled.

Thanks.

Actually, I was wrong in my assumptions. After further thorough debugging, I discovered that when I disable prefetchpath, all sorting works fine (even with additional bucket relation and the sorting column from related table). With prefetchpath included sorting behaves strange regarding of the sorting column. Mind that this is sorting on the main table (the one the entity collection is fetched for), I have no additional sorting on the prefetch path.

(in addition - all other behavior is the same as described above, with the prefetch path the collection is correctly sorted on the web service side, it is messed up on the client side).

Your descriptions are a little confusing. Please correct me if I'm wrong: - your query WITH prefetch path and sorting results in the CORRECT order of City, when you sort on state.name, for example if you query for page 3 the city names are properly sorted. - when you send that query to the client the order is wrong compared to the order the collection of cities had when it was send.

is this correct?

Please keep in mind that: if you order on state, you'll likely have multiple cities per state, so the order in which the cities are stored WITHIN a state in the resultset is undefined, UNLESS you also sort on city name. This is a normal SQL definition.

If I'm not correct here, please let me know what to check exactly so I don't spend hours on checking the wrong things

• your query WITH prefetch path and sorting results in the CORRECT order of City, when you sort on state.name, for example if you query for page 3 the city names are properly sorted.
• when you send that query to the client the order is wrong compared to the order the collection of cities had when it was send.

is this correct?

Yes.

if you order on state, you'll likely have multiple cities per state, so the order in which the cities are stored WITHIN a state in the resultset is undefined, UNLESS you also sort on city name. This is a normal SQL definition.

I know this, it is ok, I'm not saying that the city order within a state is wrong, it is undefined and I'm fine with that. I'm saying that the whole ordering by a state is wrong. And not just that, in my previous post I told you that the ordering by any city column without using state relation in the bucket and with a state prefetch path is wrong. Again, it is only wrong on the client, on the web service side everything is fine.

I hope you'll be able to reproduce this. Just a small note - when I used PageSize = 10 for paging, everything seemed fine at first. The thing is - if I sort by a column (for example - a state name) and request a page where all the 10 records have the same value for the sorted column (for example all 10 on the page 1 have state AL - Alabama), then everything's fine. If I come to the page where 10 records have different states, than the ordering can become something like AL, AL, IL, AL, ... instead of AL, AL, AL, IL... I think that you can reproduce the whole thing without paging at all, with it it's just more confusing.

Please help, I already did spent hours and hours checking this, I'm going slightly mad here . Just when I thought that nothing can stop me after finaly resolving that web service ShemaImporterExtension mess, here comes this ...

(By the way, after dealing quite a lot with CodeSmith, and similar tools, LLBL Gen is something I dreamed of for a long time. Heck, it's one of the best development tools ever. Thank you for it and keep up the good work.)

I'll look into it. I said that earlier in this thread and I appologize for not reporting back, but I'll re-check what's the cause of this. It should be a simple thing to reproduce. I do think it's something to do with the fact that multiple cities are in the same state. So what you'll get is this: - city A is deserialized. - its State object is deserialized. - the State object refers to other cities which are then deserialized as well. - AFTER these cities, the next city is processed.

This can lead to a wrong order, I think. But I've to run it through the debugger to be for sure. I can reproduce this with a set of orders and the related customers via prefetch path and sort on customer.country and the list of orders should be the same on the client as on the service.

Stay tuned.

(By the way, after dealing quite a lot with CodeSmith, and similar tools, LLBL Gen is something I dreamed of for a long time. Heck, it's one of the best development tools ever. Thank you for it and keep up the good work.)

Ok, I made a test and can reproduce it. Will look into it.

Ok this is the 'cause'. A City entity has a reference to a State entity. A State entity contains a collection of City objects. When XML is produced, it can be a City object in the State entity's City collection is already processed. The XML will then contain a <ProcessedObjectReference> tag. This tag is then used to create an object which is used to fill in the reference later on, after all xml nodes have been processed.

Now, when the deserialization code runs into a <ProcessedObjectReference> tag, it creates that object to later on be able to re-create the object, but it doesn't store the place where this object was located. This means that the entity which is re-created using the <ProcessedObjectReference > tag handler after all other nodes have been re-created, is added at the back of the list, not at the position it was found.

If the code should keep track of the order in which the entities are encountered, INCLUDING the <ProcessedObjectReference > tags, it would be able to insert the entities later on, instead of adding them.

Example: City C1, belongs to State S1 S1 has also C1 and C2. C3 belongs to State S2 C4 also belongs to S2.

Ordered, this means that in the collection, the order is C1, C2, C3, C4 and states are ordered: S1, S2.

Deserializing, It first sees C1. So that one is added. Then it sees S1, as a reference from C1 and starts to deserialize that one. The 'Cities' collection in S1 contains C1 and C2. C1 is already processed, C2 is new, so C2 is then processed. After that, the main collection is further processed as C1 and S1 are done. It then runs into C2, but that one is already processed (another <ProcessedObjectReference > tag) and it then runs into C3, finds S2, deserializes C4, and continues after that with the main collection, runs into C4's <ProcessedObjectReference > tag and is done.

At that moment, the collection order is: C1, C3 and 2 <ProcessedObjectReference> objects have to be processed, for C2 and C4. As said, these are added at the back, so teh final order is: C1, C3, C2, C4.

I can fix this, though that will be tomorrow (saturday).

Fixed in next build (uploaded later today, or tomorrow (sunday))

Hi, unfortunately I couldn't get it working. I downloaded your new runtime libraries and copied them according to the readme instructions. After not getting it working, I looked at the file version of the SD.LLBLGen.Pro.ORMSupportClasses.NET20.dll and it is 1.0.20051.060216 - that seems like February 16th, right? Both of the SD.LLBLGen.Pro.ORMSupportClasses.NET1X.dll files have the file version 1.0.20051.60224. Did you forget to include the 2.0 updated version? Can you check this please.

Oh, and thanks for the quick fix, I'm sure it works, I just need the right dll.

The version indeed is an old one. The build is automatic (build/packaging etc.) so it should have worked. I'll re-run the build process.

(edit) I forgot a using directive on the .net 2.0 assemblyinfo .cs, and the compile failed, though my nmake script didnt' report the error to finalbuilder.

Please re-download the runtime libraries archive.

It works now, thank you.

Can I just ask you - I already mentioned this in a previous post - "if there is a relation in the bucket, so two tables are joined, the sql paging with temp table won't work (regardless of the sorting column) - in that case you grab all data and do the paging & removing eventual duplicates inside your object model." Did I understand this right? Can I still somehow enable temp table paging in my city - state scenario described above. I use the FetchEntityCollection method to grab the paged cities, but I need the relation to states if I want to sort the paged collection by a state name.

Regards

Jaz wrote:

It works now, thank you.

Can I just ask you - I already mentioned this in a previous post - "if there is a relation in the bucket, so two tables are joined, the sql paging with temp table won't work (regardless of the sorting column) - in that case you grab all data and do the paging & removing eventual duplicates inside your object model." Did I understand this right? Can I still somehow enable temp table paging in my city - state scenario described above. I use the FetchEntityCollection method to grab the paged cities, but I need the relation to states if I want to sort the paged collection by a state name. Regards

It will work, but only if DISTINCT can be applied. If that's not the case, the JOIN can produce duplicates (the code doesn't check if this isn't possible, it always assumes the join will produce dupes) and paging is then not correct. So if DISTINCT can't be applied (e.g. when a blob/text field is in the select list) it falls back to client-side paging.

Is this the case?

It's an issue with low priority which is filed to be fixed in v2.