High CPU usage - packages/matrix_feed/scripts/run_feed.php

(Gavin Boyd) #1

When we run this script “packages/matrix_feed/scripts/run_feed.php” - We use this script for Matrix Imports.

When the script is used our server CPU goes through the roof, 98-100%

Has anyone else experienced this?

Version: Squiz Matrix v5.4.0.2

(Marcus Fong) #2

Hi, Gavin - while that particular package would have been installed for you by Squiz, it isn’t one of the baseline Matrix packages.

To obtain support for it you should either lodge a support ticket or contact your Squiz account manager.

(Gavin Boyd) #3

Ok, that’s off to support. Iit is a very useful script, as it uses xslt to translate xml which is usually generated from an external data source. Is there an alternate baseline Matrix package or approach for importing data and creating assets in Matrix via a cron?

(Marcus Fong) #4

Well, there is an XML import script, but it wasn’t really intended for importing XML from external data sources so it doesn’t have the XSLT translation layer you describe.

From what I’ve seen it’s more common these days to retrieve data and display it dynamically using Matrix’s REST capabilities, rather than actually import the data into Matrix assets.

(Gavin Boyd) #5

We find it very powerful when we can import and build assets using data from external sources, you can then take full advantage of Matrix’s capabilities. We have used REST, integrating with various API’s, but only for in page components. For large areas of a site, for example a course finder or people finder, have found it better to have the assets built in Matrix, so we can for example bolt on extra metadata for additional information, for example a marketing strapline. However using a full REST approach for a large site is interesting. Any good Squiz examples in the wild out there?

(Marcus Fong) #6

I see what you mean. My own experience with projects like that here at Squiz has been that course finders are typically centred around Funnelback rather than Matrix.

Funnelback either crawls the necessary data or the data is pushed into Funnelback, then Matrix uses REST to query Funnelback as necessary and displays the output (usually after some SSJS processing).

(Gavin Boyd) #7

Marcus, thanks for the info, that’s really helpful hearing about various implementations.

“Matrix uses REST to query Funnelback as necessary and displays the output” - we actually do our course finder this way, however there is usually a need to serve a URL for each course. Which we do using Matrix, course pages are standard page assets in Matrix (imported from an external database by a cronned import)

I think you are saying storing external data as assets in matrix is taken out of the equation in most implementations, how do you handle routing in this case for each course detail page? Is Angular or SSJS purely used for this implemented through Matrix?

I have looked at a few squiz universities, and they all serve their course pages as assets from Matrix, with a single Matrix page using REST to display funnelback search results.

(Marcus Fong) #8

You’re right, the course pages themselves typically are Matrix assets - it’s the search component that’s Funnelback REST. But that usually doesn’t involve the kind of regular cronned import you describe.

As regards routing a URL to a specific object (in this case a course) when using dynamic REST for the object’s details, the implementations I can recall have usually relied on a predictable URL structure and mapping that to a specific Matrix REST asset with query parameters via transparent rewrites at the webserver level (Apache / nginx / Openresty).