Caching on Asset Listings to make like Static Files?

(Nic Hubbard) #1

I have a few asset listings that are hardly ever going to change. They are also complex listings that take about 10 seconds to generate. So, I would like these asset listings to almost be like static files, and only manually refresh the cache on them rather than ever having expired cache.


What is the best way to do this? Should I customize the root node, and set a super long cache expiry? I just want them to act like __data files (obviously they can't be) and only clear their cache when I manually do so. I don't ever want users to see the lag of the file getting generated after cache is expired.



Ideas?

(Duncan Robertson) #2

Sometimes you can cut that down to 1/10 by using the 'Whats New' asset than an asset listing. This depends on a lot, including if you require a bit more dynamic ability. Give it a go?

(Nic Hubbard) #3

[quote]
Sometimes you can cut that down to 1/10 by using the 'Whats New' asset than an asset listing. This depends on a lot, including if you require a bit more dynamic ability. Give it a go?

[/quote]



This could be an option. But rarely will anything in most of these lists be "new". Do you have a way that you can still use the what's new asset with old assets?

(Anton Babushkin) #4

Are you running Squid in front of your site?


If so, the best way would definitely be to do it via the Matrix Cache Manager + the Squid config.



One thing to remember is the __data directory functions differently to assets that get cached in Matrix. Assets in this directory don't go through any code execution, they're retrieved instantly. If you wanted to replicate this, you could always write a script that would convert a given page into a HTML file that is stored in this directory. But the maintenance of this wouldn't be as easy as Matrix + Squid cache configs.

(Nic Hubbard) #5

[quote]
Are you running Squid in front of your site?



If so, the best way would definitely be to do it via the Matrix Cache Manager + the Squid config.



One thing to remember is the __data directory functions differently to assets that get cached in Matrix. Assets in this directory don't go through any code execution, they're retrieved instantly. If you wanted to replicate this, you could always write a script that would convert a given page into a HTML file that is stored in this directory. But the maintenance of this wouldn't be as easy as Matrix + Squid cache configs.

[/quote]



No, we are not using Squid, just normal Matrix cache.



I realize they are different than __data files. But since I assume that a cached version of an asset listing is pretty fast. And since I am not updating these listings much, I wanted to continually use that cached file until I choose to refresh it.

(Rhulse) #6

[quote]
Sometimes you can cut that down to 1/10 by using the 'Whats New' asset than an asset listing. This depends on a lot, including if you require a bit more dynamic ability. Give it a go?

[/quote]



This is true, but only when the date constraint reduces the size of the data set fetched from the DB.



If there is not a reasonable date constraint, then you are hosed.



Squid is a very good option for this scenario on a non-replicated system. You can set a long cache time for these assets in Matrix, but set the cache headers to a shorter time to fool squid. You then add a trigger to clear the matrix cache and squid when any content is updated.



Richard

(Aleks Bochniak) #7

Are you using any pagination on the asset listing?

(Anton Babushkin) #8

Root Node Specific cache will definitely be your best friend then Nic =)

(Nic Hubbard) #9

[quote]
Are you using any pagination on the asset listing?

[/quote]



No, it is just one large listing.

(Chris Smith) #10

[quote]
I have a few asset listings that are hardly ever going to change. They are also complex listings that take about 10 seconds to generate.

[/quote]



I'm interested in why they take so long to generate. Any long queries showing up in postgres logs that need attention?

(Nic Hubbard) #11

[quote]
I'm interested in why they take so long to generate. Any long queries showing up in postgres logs that need attention?

[/quote]



I just did a test and the slowness is due to using list_current_asset_id. While it sucks that they are slow, I really need this feature. I wonder if in the new Squizlabs performance tests they could speed up that function? :slight_smile:

(Aleks Bochniak) #12

How many assets are you listing, generally?

(Nic Hubbard) #13

[quote]
How many assets are you listing, generally?

[/quote]



Probably 50-70.



I just did a little work so that all the listings and nested list_current_asset_id listings would be using direct children only. It cut the time in half. Not sure why I was not doing that before…

(Robin Shi) #14

[quote]
Root Node Specific cache will definitely be your best friend then Nic =)

[/quote]

Yeah very true. I have an asset listing list 2000+ assets, I have customised the cache to 30days, which is the max. expiry, it works good. It still takes a while for the first public user to load the page after the cache expires though.