Caching 301 in remap manager


(Dw Andrew) #1

Hi,
I found we had a bit of a performance issue with some highly trafficed matrix redirects, each one is served directly by apache, and naturally has db connection overheads etc. Looking into the code, is there any real reason cache headers are not attached to the 301?



I added some cache headers to core/assets/system/remap_manager/remap_manager.inc so browsers and proxies will cache the 301, based on the system cache time. Theoretically I suppose the remap expiry date could be used.







Headers before:

Request:

http://test.edu.au/testurl


    
    HTTP/1.0 301 Moved Permanently
    Date: Wed, 18 Aug 2010 01:01:59 GMT
    Server: Apache/2.2.3 (Red Hat)
    X-Powered-By: PHP/5.1.6
    Expires: Thu, 19 Nov 1981 08:52:00 GMT
    Cache-Control: max-age=3600, public
    Pragma: no-cache
    Set-Cookie: SQ_SYSTEM_SESSION=15go3hb6j2gt7rkbs6jv86pu27; expires=Wed, 18 Aug 2010 02:01:59 GMT; domain=.test.edu.au; path=/;
    Location: http://test.edu.au/
    Content-Length: 0
    Content-Type: text/html; charset=UTF-8
    X-Cache: MISS from prx1.test.edu.au
    X-Cache-Lookup: MISS from prx1.test.edu.au:80
    Via: 1.0 prx1.test.edu.au:80 (squid/2.6.STABLE21)
    Connection: keep-alive



AFTER:
    
    HTTP/1.0 301 Moved Permanently
    Server: Apache/2.2.3 (Red Hat)
    X-Powered-By: PHP/5.1.6
    Expires: Wed, 18 Aug 2010 02:56:04 GMT
    Cache-Control: max-age=3600, public
    Pragma: cache
    Location: http://test.edu.au/
    Content-Length: 0
    Content-Type: text/html; charset=UTF-8
    Age: 597
    X-Cache: HIT from prx1.test.edu.au
    X-Cache-Lookup: HIT from prx1.test.edu.au:80
    Via: 1.0 prx1.test.edu.au:80 (squid/2.6.STABLE21)
    Connection: keep-alive



I got a boost once squid was able to cache the 301.
Some initial load tests show some improvement, mileage will vary because I don't think curl is using browser side cache:

Before:
time for i in `jot 250`; do curl -i http://test.edu.au/testurl; done

real 0m41.044s
user 0m0.826s
sys 0m1.112s


After:
time for i in `jot 250`; do curl -i http://test.edu.au/testurl; done

real 0m17.944s
user 0m0.837s
sys 0m1.159s




I took the code from mysource.inc (I think) so it's almost all dupe code:
    
    	public function loadRemapFromURL($protocol=NULL, $url=NULL)
    	{
    		require_once SQ_FUDGE_PATH.'/db_extras/db_extras.inc';
		if (is_null($protocol)) $protocol = current_protocol();
		if (is_null($url)) {
			$url = strip_url(current_url(FALSE, TRUE));
		}

		$url = $protocol.'://'.$url;

		try {
			$bind_vars = Array(
								'date_value' => ts_iso8601(time()),
							  );
			$time = MatrixDAL::executeOne('core', 'toDate', $bind_vars);
			$bind_vars = Array(
							'url'  => $url,
							'time' => $time,
						 );
			$remap_url = MatrixDAL::executeOne('remap_manager', 'getCurrentRemapURL', $bind_vars);
		} catch (DALException $e) {
			throw new Exception('Unable to get current remap URL for "'.$url.'" due to database error: '.$e->getMessage());
		}

		// if we have a remap url, display it
		if (!empty($remap_url)) {
			header('HTTP/1.1 301 Moved Permanently');
                    // Send Cacheable Header based on cache manager default setting
                    if (SQ_CONF_SEND_CACHEABLE_HEADER && SQ_CONF_SEND_404_CACHEABLE_HEADER && current_protocol() != 'https') {
                    	$cm = $GLOBALS['SQ_SYSTEM']->am->getSystemAsset('cache_manager');
                            header('Pragma: cache');

                            $browser_cache_expiry = $cm->attr('browser_cache_expiry');
                           	if (empty($browser_cache_expiry)) {
                            	$browser_cache_expiry = $cm->attr('expiry');
                            }
			//more cache headers
                    header('Cache-Control: max-age='.$browser_cache_expiry.', public');
			header('Expires: '.gmdate('D, d M Y H:i:s', time() + $browser_cache_expiry).' GMT');
                    }

			header('Location: '.$remap_url);

			return TRUE;
		}
		return FALSE;

	}//end loadRemapFromURL()

(Edison Wang) #2

Thank you very much for your detailed post.


I have verified the code and will include this enhancement in next release.

Here is the feature request: #4626


(Dave Letorey) #3

This is Awesome Ndrw


Brilliant post and well done for getting a new feature added to Matrix



Dave


(Dw Andrew) #4

Great, thanks for checking it out :slight_smile: