Our system is getting pretty large these days. Our backups are about 3.7 GB, and the _admin interface is getting to be slow. The frontend is nice and fast, which is great. But, I am wondering if there is anything I can do, besides upgrading hardware, to speed up the admin a bit?
Is there any scripts that I can run to do some more cleanup? Or, and other suggestions?
[quote]Our system is getting pretty large these days. Our backups are about 3.7 GB, and the _admin interface is getting to be slow. The frontend is nice and fast, which is great. But, I am wondering if there is anything I can do, besides upgrading hardware, to speed up the admin a bit?
Is there any scripts that I can run to do some more cleanup? Or, and other suggestions?[/quote]
Well there are other ways to get more performance from the same hardware. What we did was:
- Install a dynamic content cache like the zend accelerator or eAccelerator (http://eaccelerator.net/) - this speeds thing up alot.
- Looking that the PUC http headers i don't see a squid or any other cache/load balancer in front of MSM? If you take the strain of serving the pages direclty away from MSM then it speeds things up as well… or did for us.
HTH
K
Did this help with the speed of your Admin interface, such as saving page and creating assets? What about loading pages when a user is logged in?
This is what I am most concerned about.
[quote]Did this help with the speed of your Admin interface, such as saving page and creating assets? What about loading pages when a user is logged in?
This is what I am most concerned about.[/quote]
It does help with the admin interface speed. if MSM doesn't have to serve the content directly for normal visitors then its only really dealing with logged in users. I'm not sure I have it setup perfectly (sometimes have to do a force refresh to see the edit buttons etc) but I'm sure the squiz guys could tell you exactly how to do it.
K
Hey Nic, I know a lot of the regular maintenance stuff helps to keep Matrix and therefore the admin interface running as best you can on your hardware. Particularly cleaning up sessions and if you are on PostgreS a regular vacuum. Caching is almost essential and if you have a lot of content changes happening all the time you can dull down the logs that are recorded in the sys config so that there is less work to be done, however this does mean that there will be less evidence to go on when things go wrong. If you are running roll-back or roles they will suck a lot of juice and will slow things down a little.
[quote]Our system is getting pretty large these days. Our backups are about 3.7 GB, and the _admin interface is getting to be slow. The frontend is nice and fast, which is great. But, I am wondering if there is anything I can do, besides upgrading hardware, to speed up the admin a bit?
Is there any scripts that I can run to do some more cleanup? Or, and other suggestions?[/quote]
Hi Nick
one this that may make a small difference is cleaning out your trash bin regularly. Our trash is automatically cleaned out every night in the early hours of the morning
hope this helps?
regards
Tim
Yes, I know that we have regular vacuum running on Postgres. Checked our crontab and we didn't have session cleanup. Strange. So I added that. Any other suggestions?
[quote]Hi Nick
one this that may make a small difference is cleaning out your trash bin regularly. Our trash is automatically cleaned out every night in the early hours of the morning
hope this helps?
regards
Tim[/quote]
Yeah, our trash always gets cleaned out each morning. Thanks for the idea though.
Keep on top of remaps and linking, especially within design files. If you have a number of domains and a bunch of different design files move them out and put them under different site assets. That way you can reduce the url's and links associated with designs. If you have 10 domains and 20 design files in the design folder each design will be maintaining links for all 10 domains, even if they don't need the other domains.
eaccelerator or apc for caching your php would help in most instances. are you able to determine where the slowness exists? is it IO, slow queries, CPU load or something else?
If you haven't been cleaning your sessions, it might be worth deleting the entire cache directory because all of the old inodes will still be allocated to that directory, making reads slow.
Cleaning SQ_INTERNAL_MSG, SQ_CACHE, and you might have other big tables with rubbish in them as well - worth checking out.
We only manage one URL on our site, as we have apache rewrite any non www. links to add www. so that Matrix won't have to manage both http:// and http://www. links. In addition to this, I clean out our remaps about once a week. we only have 3 pages of remaps, which I keep in an excel file to keep it clean, then reupload the list when I need to clear out any that are not being used.
[quote]eaccelerator or apc for caching your php would help in most instances. are you able to determine where the slowness exists? is it IO, slow queries, CPU load or something else?
If you haven't been cleaning your sessions, it might be worth deleting the entire cache directory because all of the old inodes will still be allocated to that directory, making reads slow.
Cleaning SQ_INTERNAL_MSG, SQ_CACHE, and you might have other big tables with rubbish in them as well - worth checking out.[/quote]
Yes, I think I will look into adding eaccelerator.
As for knowing where the slowness is. What is the best method for finding this out? I have munin installed, but I am not sure how to check things like slow queries.
Thanks guys, I can kick butt at Matrix implementation, but sometimes when it comes to server stuff I get a little confused. 
[quote]Yes, I think I will look into adding eaccelerator.
As for knowing where the slowness is. What is the best method for finding this out? I have munin installed, but I am not sure how to check things like slow queries.
Thanks guys, I can kick butt at Matrix implementation, but sometimes when it comes to server stuff I get a little confused. :)[/quote]
The other things that can cause slowness is a bloated sq_message table and sq_cache table.
We clear the sq_cache table each day as old entries are not purged, and a very large number of entries can slow down the system when the Matrix cache is updated.
We do not log messages (sq_message) to the database at all. If we need to find something, greping the files is easy.
Richard
Richard,
What is the best method for clearing an entire table in the DB?
[quote]We do not log messages (sq_message) to the database at all. If we need to find something, greping the files is easy.[/quote]
To do this, did you just remove all the entries from the Messaging Service Configuration?
Thanks.
How many items do you have in the root of the asset map?
What is the hardware spec of your database server? Has there been any optimisation done to the config of postgres?
If backend actions are slow, a common culprit is a massive internal message table, as both Shane and Richard have mentioned.
[quote]To do this, did you just remove all the entries from the Messaging Service Configuration?[/quote]
Yes. Remove anything you don't want to see on the Logs screen of an asset. If you have access to your server and good log rotation and retention, you don't need anything in the boxes, although I would just move the whitelist content to the blacklist so always remember what those values were.
To clean out your message table, you can truncate it in the DB: TRUNCATE sq_internal_msg
Your other option is to keep the logs you want and use a script to clean them out. The script you use for this is: scripts/remove_internal_message.php
[quote]How many items do you have in the root of the asset map?
What is the hardware spec of your database server? Has there been any optimisation done to the config of postgres?[/quote]
We only have about 15 items at the root.
Our servers are:
Webserver -
2.8GHz Pentium D with 800MHz Front Side Bus
3 GB Memory
two 250GB SATA 7200 RPM harddrives (mirrored)
SAS5iR RAID Controller
Database -
Dual 2.0GHz Xeon with 1333MHz Front Side Bus
5 GB RAM
six 146GB SCSI harddrives in a RAID 5 array
PERC 5/i RAID Controller
You think those servers are adequate?
And yes, postgres has been tuned by Squiz when they installed our system.
Seems a little faster. When I make a change in the admin and commit it still takes a bit.
One things to watch with php opcode caches: almost all of them cause segfaults, and if you really really have to use one you will probably need some way to detect the apache segfaults and restart apache. Checking the log once per minute seems to the be the norm, so your site could be down for one minute.
We could never get any accelerator to work without segfaulting.
And to be honest, they did not make any difference to the load on our servers. (Perhaps the pages got built slightly faster though).
Xcache allegedly does not segfault, but I have not tested it.
cheers
Richard