Collection copy... save... and byte[] property issue...

Posts   
 
    
zebranky
User
Posts: 24
Joined: 14-Mar-2006
# Posted on: 20-Jul-2006 17:49:33   

Hi again,

Seems it's my day for these odd issues simple_smile

I have a collection of entities called SupportingDocumentCollection. One of the properties of the entity is called DocumentData, which is a byte array (it's a BLOB data type in Oracle).

I am iterating like this:


SelectedSupportingDocumentCollection docToSave = new SelectedSupportingDocumentCollection();
foreach (SelectedSupportingDocumentEntity document in this.SelectedSupportingDocumentCollection)
{
    SelectedSupportingDocumentEntity newDocument = new SelectedSupportingDocumentEntity();
    newDocument.AppId = newEntity.AppId;
    newDocument.SupportingDocumentId = document.SupportingDocumentId;
    newDocument.DocumentData = document.DocumentData;
    newDocument.ContentType = document.ContentType;
    newDocument.DocumentTitle = document.DocumentTitle;
    newDocument.HaveHardCopy = document.HaveHardCopy;
    docToSave.Add(newDocument);
}

Then saving the docToSave with a call to SaveMulti(). I then get an exception "An exception was caught during the execution of an action query: Index was outside the bounds of the array.. Check InnerException, QueryExecuted and Parameters of this exception to examine the cause of this exception.". Here's its stack trace:


"   at SD.LLBLGen.Pro.ORMSupportClasses.ActionQuery.Execute()\r\n   at SD.LLBLGen.Pro.ORMSupportClasses.BatchActionQuery.Execute()\r\n   at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.ExecuteActionQuery(IActionQuery queryToExecute, ITransaction containingTransaction)\r\n   at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.AddNew(IEntityFields fields, ITransaction containingTransaction)\r\n   at Vcms.Data.EntityClasses.SelectedSupportingDocumentEntityBase.InsertEntity() in C:\\projects\\vcms\\Projects\\Vcms\\Vcms.Data\\EntityBaseClasses\\SelectedSupportingDocumentEntityBase.cs:line 690"

... and the trace output ...


Method Enter: CreateInsertDQ
Method Enter: CreateSingleTargetInsertDQ
Generated Sql query: 
    Query: INSERT INTO "VCMS"."SELECTEDSUPPORTINGDOCUMENT" ("SUPPORTINGDOCUMENTID", "VARAPPLICATIONID", "HARDCOPY", "DOCUMENTTITLE", "DOCUMENTDATA") VALUES (:SupportingDocumentId1, :AppId2, :HaveHardCopy3, :DocumentTitle4, :DocumentData5)
    Parameter: :SupportingDocumentId1 : Binary. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: binary lob.
    Parameter: :AppId2 : Binary. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: binary lob.
    Parameter: :HaveHardCopy3 : Int16. Length: 0. Precision: 1. Scale: 0. Direction: Input. Value: 1.
    Parameter: :DocumentTitle4 : String. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: Sworn Statement.
    Parameter: :DocumentData5 : Object. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: binary lob.

Method Exit: CreateSingleTargetInsertDQ
Method Exit: CreateInsertDQ

Any suggestions?

I have also tried copying the byte data across to the newDocument instance by doing this, with the same exception:


newDocument.DocumentData = new byte[document.DocumentData.Length];
document.DocumentData.CopyTo(newDocument.DocumentData, 0);

Although I would prefer to simply assign the old bytes to the new entity instance to reduce memory overhead (the DocumentData could be a few meg's per entity, of which there could be 6 or more, and do not really want to add undue strain on the server wink ), any suggestions are welcome!

Here's the table script:


CREATE TABLE "VCMS"."SELECTEDSUPPORTINGDOCUMENT" 
(
    "SUPPORTINGDOCUMENTID" RAW(16) NOT NULL ENABLE, 
    "VARAPPLICATIONID" RAW(16) DEFAULT NULL NOT NULL ENABLE, 
    "HARDCOPY" NUMBER(1,0) NOT NULL ENABLE, 
    "DOCUMENTTITLE" VARCHAR2(255 BYTE), 
    "DOCUMENTDATA" BLOB, 
    "CONTENTTYPE" VARCHAR2(100 BYTE), 
    CONSTRAINT "PK_SELSUPPDOC" PRIMARY KEY ("SUPPORTINGDOCUMENTID", "VARAPPLICATIONID") ENABLE, 
    FOREIGN KEY ("VARAPPLICATIONID")
    REFERENCES "VCMS"."VARAPPLICATION" ("VARAPPLICATIONID") ENABLE, 
    FOREIGN KEY ("SUPPORTINGDOCUMENTID")
    REFERENCES "VCMS"."SUPPORTINGDOCUMENT" ("SUPPORTINGDOCUMENTID") ENABLE
) ;

I will note that if the DocumentData property is an empty array then the SaveMulti() goes through fine.

Using LLBLGEN Pro v1.0.2005.1 Final, June 19th 2006. Using Self-Servicing. Using VS 2005, ASP.NET 2. Using Oracle 10g Data Provider.

Thanks, Brian Johnson

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39614
Joined: 17-Aug-2003
# Posted on: 21-Jul-2006 11:52:50   

I tried to insert a blob in my testtable:


[Test]
public void BlobInsertTest()
{
    EmployeesEntity toInsert = new EmployeesEntity();
    toInsert.LastName = "Test";
    toInsert.Email = "Foo@foo.com";
    toInsert.HireDate = DateTime.Now;
    toInsert.JobId = "IT_PROG";
    byte[] pictureData = new byte[5];
    for (int i = 0; i < 5; i++)
    {
        pictureData[i] = (byte)i;
    }
    toInsert.Picture = pictureData;
    Assert.IsTrue(toInsert.Save());
    Assert.IsTrue((toInsert.EmployeeId > 0));

    EmployeesEntity toFetch = new EmployeesEntity(toInsert.EmployeeId);
    for (int i = 0; i < 5; i++)
    {
        Assert.AreEqual(toFetch.Picture[i], toInsert.Picture[i]);
    }

    Assert.IsTrue(toInsert.Delete());
}

which succeeded with the following query:


    Query: INSERT INTO "HR"."EMPLOYEES" ("EMPLOYEE_ID", "LAST_NAME", "EMAIL", "HIRE_DATE", "JOB_ID", "PICTURE") VALUES (:EmployeeId1, :LastName2, :Email3, :HireDate4, :JobId5, :Picture6)
    Parameter: :EmployeeId1 : Int32. Length: 0. Precision: 6. Scale: 0. Direction: Input. Value: 0.
    Parameter: :LastName2 : String. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: Test.
    Parameter: :Email3 : String. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: Foo@foo.com.
    Parameter: :HireDate4 : Date. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: 21-7-2006 11:45:55.
    Parameter: :JobId5 : String. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: IT_PROG.
    Parameter: :Picture6 : Object. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: binary lob.

My guess is that the issue is with the RAW typed columns. As you've changed these to char(16) bytes, it might be this issue is now no longer present. Could you confirm, please?

Frans Bouma | Lead developer LLBLGen Pro
zebranky
User
Posts: 24
Joined: 14-Mar-2006
# Posted on: 21-Jul-2006 13:16:51   

The change from RAW to CHAR(32 byte) made no difference here cry .

I can also save these entities just like you in your test case. What I am doing though is copying the data across to a new entity which is a 'copy' of the old one.

As a test, insert a couple of these items in your database, then load them all into a collection. Once loaded, iterate them and create new entities with the same details as the original (except PK of course). Insert these new entities into a new collection that will be used to save all the entities in one call to SaveMulti().

Something along these lines:


SelectedSupportingDocumentCollection docToSave = new SelectedSupportingDocumentCollection();
foreach (SelectedSupportingDocumentEntity document in this.SelectedSupportingDocumentCollection)
{
    SelectedSupportingDocumentEntity newDocument = new SelectedSupportingDocumentEntity();
    newDocument.AppId = newEntity.AppId;
    newDocument.SupportingDocumentId = document.SupportingDocumentId;
    newDocument.DocumentData = document.DocumentData;
    newDocument.ContentType = document.ContentType;
    newDocument.DocumentTitle = document.DocumentTitle;
    newDocument.HaveHardCopy = document.HaveHardCopy;
    docToSave.Add(newDocument);
}
docToSave.SaveMulti(); // Error here!

Try to assign the new entity's data with the old entity's data (e.g. toInsert.Picture = oldInsert.Picture), and call a SaveMulti() on the new collection you created.

This is where I get the error.

Walaa avatar
Walaa
Support Team
Posts: 14950
Joined: 21-Aug-2005
# Posted on: 21-Jul-2006 15:30:09   

newDocument.DocumentData = new byte[document.DocumentData.Length]; document.DocumentData.CopyTo(newDocument.DocumentData, 0);

Would you try the following:

newDocument.DocumentData = new byte[0];
newDocument.DocumentData = document.DocumentData;
zebranky
User
Posts: 24
Joined: 14-Mar-2006
# Posted on: 21-Jul-2006 16:38:39   

Walaa wrote:

Would you try the following:

newDocument.DocumentData = new byte[0];
newDocument.DocumentData = document.DocumentData;

Same error still...

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39614
Joined: 17-Aug-2003
# Posted on: 24-Jul-2006 09:35:11   

What I wonder: do you assign the same byte[] array to multiple entities?

The stack trace you have posted starts with the Execute method, however are there also ODP.NET method calls above it in the original stacktrace? It seems the error happens inside the oracle client or inside oracle itself, and my guess is that it's something with the byte streams send to the server: if the same object is passed, the stream is already sent... (wild guess, I have no clue what's causing this)

Frans Bouma | Lead developer LLBLGen Pro
zebranky
User
Posts: 24
Joined: 14-Mar-2006
# Posted on: 25-Jul-2006 09:52:11   

Otis wrote:

What I wonder: do you assign the same byte[] array to multiple entities?

No, I simply assign entity A's byte array to Entity B's byte array. Entity A already exists in the database, and Entity B is new. When I save Entity B it fails with the errors noted.

Having said the above, the error still happens even if I create a new byte array for Entity B and copy the bytes across from Entity A individually. When doing this, the source byte array should not be affecting the new byte array, so that should eliminate that possibility.

Here's the new code snippet that still does not work:


SelectedSupportingDocumentCollection docToSave = new SelectedSupportingDocumentCollection();
foreach (SelectedSupportingDocumentEntity document in this.SelectedSupportingDocumentCollection)
{
    SelectedSupportingDocumentEntity newDocument = new SelectedSupportingDocumentEntity();
    newDocument.AppId = newEntity.AppId;
    newDocument.SupportingDocumentId = document.SupportingDocumentId;
    newDocument.DocumentData = new byte[document.DocumentData.LongLength];
    for (long index = 0; index < document.DocumentData.LongLength; index++)
    {
        newDocument.DocumentData[index] = document.DocumentData[index];
    }
    newDocument.ContentType = document.ContentType;
    newDocument.DocumentTitle = document.DocumentTitle;
    newDocument.HaveHardCopy = document.HaveHardCopy;
    docToSave.Add(newDocument);
}

The stack trace:


at SD.LLBLGen.Pro.ORMSupportClasses.ActionQuery.Execute()
at SD.LLBLGen.Pro.ORMSupportClasses.BatchActionQuery.Execute()
at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.ExecuteActionQuery(IActionQuery queryToExecute, ITransaction containingTransaction)
at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.AddNew(IEntityFields fields, ITransaction containingTransaction)
at Vcms.Data.EntityClasses.SelectedSupportingDocumentEntityBase.InsertEntity() 
in C:\projects\vcms\Projects\Vcms\Vcms.Data\EntityBaseClasses\SelectedSupportingDocumentEntityBase.cs:line 690

This is the line I get the first exception on in the debugger:


/// <summary> Performs the insert action of a new Entity to the persistent storage.</summary>
/// <returns>true if succeeded, false otherwise</returns>
protected override bool InsertEntity()
{
    SelectedSupportingDocumentDAO dao = (SelectedSupportingDocumentDAO)CreateDAOInstance();
    // This next line throws the exception
    return dao.AddNew(base.Fields, base.Transaction);
}

Edit: The SQL is not generated before this error, so this error seems to happen before (or during) the generation of the SQL for the insert.

I will try see if perhaps saving the entities without the data, then assigning the data and saving again will make a difference.

zebranky
User
Posts: 24
Joined: 14-Mar-2006
# Posted on: 25-Jul-2006 10:23:01   

Otis wrote:

The stack trace you have posted starts with the Execute method, however are there also ODP.NET method calls above it in the original stacktrace?

Apologies, I did not answer this in the original message. But no, that is the only stack trace output I get for that exception.

I think the original Generated SQL output was confusing, as the test case in that scenario may have had no data assigned for the first document saved (the SaveMulti() calls each entity's Save() iteratively it appears). The document saves fine if there is no data assigned to it, only when I try to save data too do I get the problem.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39614
Joined: 17-Aug-2003
# Posted on: 25-Jul-2006 11:20:52   

Ok, so the error occurs inside Oracle or the CLI client, not inside ODP.NET.

I'll try with your table layout and routine to repro this, although it looks like it would do normal blob inserts like I tested before...

The oracle version you're using is 10g?

Frans Bouma | Lead developer LLBLGen Pro
zebranky
User
Posts: 24
Joined: 14-Mar-2006
# Posted on: 25-Jul-2006 11:56:16   

Otis wrote:

Ok, so the error occurs inside Oracle or the CLI client, not inside ODP.NET.

I'll try with your table layout and routine to repro this, although it looks like it would do normal blob inserts like I tested before...

The oracle version you're using is 10g?

I am using 10g client with 9i server.

Just keep in mind that the only way this seems to break is when I try to use a loaded entity to assign data to a new entity. I have been able to test if this breaks if I try to save the new entity without data, THEN set its data and save it again within the same transaction. I have not been able to test a workaround yet, which is to save the entity without data, then reload it entirely (within the transaction, see http://www.llblgen.com/TinyForum/Messages.aspx?ThreadID=7042) and then set its data and save again.

I know I have no problem saving these entities with data from an uploaded file in ASP.NET. So it's only happening on assigning data from an existing entity. (EDIT: Although setting the data by copying rather than assigning also seems to break).

Does all this make sense? I just went cross-eyed wink

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39614
Joined: 17-Aug-2003
# Posted on: 25-Jul-2006 13:33:31   

It still succeeds:


[Test]
public void BlobInsertTest()
{
    EmployeesEntity toInsert = new EmployeesEntity();
    toInsert.LastName = "Test";
    toInsert.Email = "Foo@foo.com";
    toInsert.HireDate = DateTime.Now;
    toInsert.JobId = "IT_PROG";
    byte[] pictureData = new byte[5];
    for (int i = 0; i < 5; i++)
    {
        pictureData[i] = (byte)i;
    }
    toInsert.Picture = pictureData;
    Assert.IsTrue(toInsert.Save());
    Assert.IsTrue((toInsert.EmployeeId > 0));

    EmployeesEntity toFetch = new EmployeesEntity(toInsert.EmployeeId);
    for (int i = 0; i < 5; i++)
    {
        Assert.AreEqual(toFetch.Picture[i], toInsert.Picture[i]);
    }

    EmployeesCollection copies = new EmployeesCollection();
    for (int i = 0; i < 5; i++)
    {
        EmployeesEntity newEntity = new EmployeesEntity();
        newEntity.LastName = toFetch.LastName;
        newEntity.Email = toFetch.Email + i.ToString();     // UC 
        newEntity.HireDate = toFetch.HireDate;
        newEntity.JobId = toFetch.JobId;
        newEntity.Picture = toFetch.Picture;
        copies.Add(newEntity);
    }

    Assert.AreEqual(5, copies.SaveMulti(false));
    Assert.IsTrue(toInsert.Delete());
    copies.DeleteMulti();
}

I create several copies from an entity I fetched, then saved then in a collection like you did. Also 9i with 10g client (10.1.0.401)

Frans Bouma | Lead developer LLBLGen Pro
zebranky
User
Posts: 24
Joined: 14-Mar-2006
# Posted on: 28-Jul-2006 14:47:16   

Otis wrote:

It still succeeds:



    Assert.AreEqual(5, copies.SaveMulti(false));

I create several copies from an entity I fetched, then saved then in a collection like you did. Also 9i with 10g client (10.1.0.401)

Hmm, strange... I see you call SaveMulti(false), you think that will have anything to do with it? I just call SaveMulti()...

Will see what else I can produce to see if there's anything else happening...

Walaa avatar
Walaa
Support Team
Posts: 14950
Joined: 21-Aug-2005
# Posted on: 28-Jul-2006 15:06:10   

Hmm, strange... I see you call SaveMulti(false), you think that will have anything to do with it? I just call SaveMulti()...

There is no difference here, as the default value is false.

public int SaveMulti()
        {
            return SaveMulti(false);
        }