Purge trash script not handling large amounts of files


(Yogi Bear50) #1

Hi All

 

 

Currently our main trash contains about 6000 records; :wacko: which needs to be removed instead, we tried to run the purge_trash.php , but it crashes with oracle errors :o .

 

 

Is there a way of removing these records from the trash via an oracle database command or how can we improve the following purge_trash.php to handle more records etc.

 

 

Please note we are using version 4.0.3 .

 

 

Cheers  


(Nic Hubbard) #2

Have you looked in the error log to see why it crashed? I have a feeling it is a memory issue. 

 

Did purging it the normal way not work either?


(Yogi Bear50) #3

Hi Nic

 

Thanks for getting back to me, the automated purge_trash.php script is generating the following errors “Unable to Commit, transaction has already been aborted" {SYS0220}

 

Purging the normal way with that amount of assets just freezes the application completely, therefore I wondering whether there is a better automated scripts via the ORACLE database command which can be used or whether the purge_trash.php script can be used to handle large amounts of records to be deleted without the above mentioned error occurring.


(Marcus Fong) #4

As far as I’m aware you can’t purge Matrix’s trash via a database query, and I strongly suspect you’d be very likely to make a mess of your Matrix system if you tried.

In my experience, “unable to commit, transaction has already been aborted” is a symptom rather than a cause. It means that Matrix tried to perform a number of database queries in a single transaction, but one or more of them threw an error and caused the transaction as a whole to abort. The “unable to commit” message is displayed when Matrix then tries to end the transaction by committing its changes.


The key question is what database query failed and caused the transaction to abort? If you were running PostgreSQL I’d suggest checking the PostgreSQL database logs, but since you’re using Oracle I’d probably check the Matrix error logs first. If there’s nothing relevant in there, then you might need to ask your friendly Oracle DBAs if they can see anything.


Failing that, since your forum account indicates you’re a Squiz client, you could lodge a support ticket and ask Squiz Support to investigate.


(Peter Sheppard) #5

Wasn't there an update to that script at some point in the last year to solve an issue with Oracle's limit of not being able to specify more than 1000 parameters in an "IN (x, y, z)" query?


(Marcus Fong) #6

I’ve checked the CVS logs for the purge_trash.php script and the purge trash HIPO, and I don’t see anything like that:

http://public-cvs.squiz.net/cgi-bin/viewvc.cgi/mysource_matrix/scripts/purge_trash.php?view=log

http://public-cvs.squiz.net/cgi-bin/viewvc.cgi/mysource_matrix/core/hipo/jobs/hipo_job_purge_trash.inc?view=log


A search of the Matrix bug tracker didn’t show anything recent referring to Oracle and purging trash either.


#7

It’s been advised that once the trash gets to large amounts of asset 700 assets plus the system will chug! We have a trash purge set for weekly trashes. I’m pritty sure the transaction has to happen through the hipo - not straight on the database or your left with all matter of weirdness. We also run oracle with matrix.