I have a quick question to generally throw out there. I'm busy building a system that requires information to be updated on page in 'real time'.
Context: As an asset is published public users can instantly see the results. It's like a traditional push service I guess.
I've happily written a bit a Jquery which used a timed event refresh on an id:
var refreshId = setInterval(function()
{
$('#push').load('http://mydomain.com/asset-listing/_nocache');
}, 15000);
This results in the id push (DIV) reloading the _nocache results of an asset lister every 15 seconds.
The question I have is really about server load. Would Matrix be able to hand this number of requests? If 1000 users all have a 15 minute session on the push service I'm looking at 60000 page requests over a quarter hour. Too much?
240k uncached page views an hour is an very highly loaded site. Even 240k cached page views is very high.
I doubt your system would be able to cope. I'm sure the load time on the uncached listing would get up to > 15 seconds quickly, causing the system to get into some real trouble.
Realistically for this sort of thing you are going to need to have a static page server (or a darn fast db based system) to churn out requests this fast. You might be able to have a trigger that fires an external script that updates your static page directly from the Matrix db. (There is not a trigger that supports this at present). You may have to use something faster than PHP to do so for performance reasons.
Then you'd set the static page server to ensure you get a 304 from the ajax call if there was no change. jQuery can be set up to to only update the page onSuccess.
[quote]I have a quick question to generally throw out there. I'm busy building a system that requires information to be updated on page in 'real time'.
Context: As an asset is published public users can instantly see the results. It's like a traditional push service I guess.
I've happily written a bit a Jquery which used a timed event refresh on an id:
var refreshId = setInterval(function()
{
$('#push').load('http://mydomain.com/asset-listing/_nocache');
}, 15000);
This results in the id push (DIV) reloading the _nocache results of an asset lister every 15 seconds.
The question I have is really about server load. Would Matrix be able to hand this number of requests? If 1000 users all have a 15 minute session on the push service I'm looking at 60000 page requests over a quarter hour. Too much?[/quote]
Polling is not an efficient method of doing updates, you really want to do push not pull.
There is some interesting work going on now using long lasting http connections to do push, largely using XMPP (the jabber IM protocol), and there are javascript XMPP clients too. This way really scales, and is built on tested components (instant messaging). Went to a few interesting talks about this at FOSDEM last month.
Pushing out an XMPP update message from a trigger shouldnt involve too much development work, then you just need to set up the server and build the client side javascript to receive the messages. Give me a call if you want to discuss more.
There are a few articles around on this, look for xmpp push...
The idea is that clients make a HTTP request; the server accepts the connection but doesn’t reply until the data is ready; and there is a high timeout on the connection. When the server then has the data ready, it sends it to the client down the HTTP connection. The result is: low-latency updates that don’t require polling and are more bandwidth efficient.
Exciting stuff - but whether you use BOSH in this situation doesn’t make a huge difference - the main problem is finding out when the output of the asset listing is going to change (without regenerating it). This is quite difficult and we bump into the same problem when trying to do caching in Matrix.
The best solution I can think of is to use a trigger which can detect when the asset listing is most likely to change. For example, when an article is added to a news section. This trigger could then write some data (like a timestamp) to a file or HTTP resource somewhere, which acts as an identifier to tell your AJAX client when the listing has changed. Your AJAX client can poll this resource and when it detects an update it then re-requests the asset listing separately (using /_nocache or /_recache).
If you wanted to make those updates completely instantaneous then you could use BOSH - but it would require writing some kind of separate BOSH server which gets rather complicated for one small task (I also did a quick Google search and I can’t find any implementation of a BOSH server that isn’t tied directly to XMPP).
Note that the idea above is the same one that we employ when creating triggers that manually clear the cache on certain asset listings when we know for sure that what they are listing has changed (ie. assets have been created or updated under a listing’s root node).
