A new Matrix API is coming...

system · 2019-08-19 10:54:29 UTC

A brand new Content Delivery API feature for Squiz Matrix is coming and we are looking for early user feedback.

This is a companion discussion topic for the original entry at https://matrix.squiz.net/a?id=25160

aleks · 2019-08-20 08:17:38 UTC

Anyone playing around with this yet?

JohnGill · 2019-08-20 10:33:08 UTC

I’ve had a look, but I don’t really get what it’s for.

Simple example

GET http://matrix5532.local/__api/assets/78?contents=true

{
   "updated" : {
      "date" : "2019-08-20T19:34:22+10:00",
      "user_id" : "10"
   },
   "type" : "news_item",
   "published" : {
      "date" : "2019-08-20T19:34:22+10:00",
      "user_id" : "10"
   },
   "created" : {
      "date" : "2019-08-20T19:33:50+10:00",
      "user_id" : "10"
   },
   "version" : "0.1.0",
   "contents" : "\t\t\t<h1>Clickbait headline</h1>\n\t\t\t\t<div id=\"sq_news_summary\">\n\t\t\t<p>Article summary</p>\n\t\t</div>\n\t\t\t\t<div id=\"sq_news_body\">\n\t\t\t<p>Article body</p>\n\t\t</div>\n\t\t\t\t<div id=\"sq_news_contact\">\n\t\t\t\t\t<p>Contact: A Reporter</p>\n\t\t\t\t\t<p>Phone: 8675309</p>\n\t\t\t\t\t<p>Email: example@example.com</p>\n\t\t\t\t</div>\n\t\t",
   "name" : "Clickbait headline",
   "short_name" : "Clickbait headline",
   "type_name" : "News Item",
   "status_changed" : {
      "date" : "2019-08-20T19:34:22+10:00",
      "user_id" : "10"
   },
   "status" : {
      "code" : "LIVE",
      "name" : "Live",
      "id" : 16
   },
   "id" : "78"
}

Given that it’s read-only, it seems to cover territory that was already achievable with (admittedly manual) Paint Layouts and Asset LIstings.

I knocked together a quick Paint Layout to produce the same output when called as http://matrix5532.local/fakeapi/_nocache?param=78

{
   "status_changed" : {
      "user_id" : "%globals_get_param^as_asset:asset_status_changed_by_assetid%",
      "date" : "%globals_get_param^as_asset:asset_status_changed_iso8601%"
   },
   "short_name" : "%globals_get_param^as_asset:asset_short_name%",
   "created" : {
      "user_id" : "%globals_get_param^as_asset:asset_created_by_assetid%",
      "date" : "%globals_get_param^as_asset:asset_created_iso8601%"
   },
   "type_name" : "%globals_get_param^as_asset:asset_type%",
   "updated" : {
      "user_id" : "%globals_get_param^as_asset:asset_updated_by_assetid%",
      "date" : "%globals_get_param^as_asset:asset_updated_iso8601%"
   },
   "status" : {
      "name" : "%globals_get_param^as_asset:asset_status_description%",
      "id" : %globals_get_param^as_asset:asset_status_code%
   },
   "type" : "%globals_get_param^as_asset:asset_type_code%",
   "contents" : "%globals_get_param^as_asset:asset_contents_raw^json_encode%",
   "published" : {
      "user_id" : "%globals_get_param^as_asset:asset_published_by_assetid%",
      "date" : "%globals_get_param^as_asset:asset_published_iso8601%"
   },
   "name" : "%globals_get_param^as_asset:asset_name%",
   "version" : "%globals_get_param^as_asset:asset_version%",
   "id" : "%globals_get_param^as_asset:asset_assetid%"
}

The API takes about 40ms, and the fakeAPI takes about 90ms, so it seems faster and less set up, but it doesn’t seem to enable anything that was totally out of reach. Depending on whether or not the speed advantage holds up across the other API calls (/assets/78/children), there might be some kinds of custom reporting that will be faster using this API than any of the alternatives (Asset Listings, JS API, SOAP API, asset map calls). It also cleans up the situation of having to manually build JSON and be careful with all the escaping.

I’d love to see some example use cases for this, it would help explain why it’s a priority. Without that context, it’s disappointing to see a new API being added when the existing ones are in serious need of attention.

JohnGill · 2019-08-21 08:05:55 UTC

Some more concrete feedback.

Content negotiation

The __api endpoint should be returning json to requests that either have no Accept: header or allow for */*, but in both cases it incorrectly returns a 406 response instead. This is contrary to RFC 7231, and it’s needlessly inconvenient to make __api URLs not work in the browser.

Implications of all public attributes and metadata fields being accessible

Currently, I suspect that many Matrix sites are in the situation where there are assets which are Live, with public read permissions, but due to the nature of the available paint layouts, not all of the attributes/fields are generally accessible to the public. e.g Comments and notes stored in unlisted MD fields.

(obviously this may not be advisable, and there may be other ways of reaching this data such that it’s not actually secure. It would be interesting to get a sense of how safe Squiz considers such “unlisted” fields on public assets to be).

__api seems to provide raw access to every aspect of every public asset in the system, which would do away with the ability to have this kind of unlisted field. Has there been any discussion around allowing __api only for assets within certain locations? It seems like a significant change to make the raw data of every public asset trivially extricable.

tbaatar · 2019-08-23 15:25:23 UTC

Great news and 1 step in the right direction.

I’d be keen to understand if Squiz is aiming to make it full blown headless CMS with the capabilities of integrating with other services such as Netlify, Contentful and Github etc with write/management API endpoints?

BenFreke · 2019-08-29 06:41:19 UTC

Hi John

Content Negotiation: This has been changed for v1, where it assumes json is acceptable if no Accepts is provided.

Attributes and Metadata fields publicly accessible: This is great feedback.

Allow API only for Assets within certain locations: Yes, we’ve discussed this but it’s not available in the beta due to the performance and complexity this adds. It’s on the internal roadmap but we haven’t yet committed to it.

Thanks for taking the time to look at the Beta, it’s greatly appreciated.

tbaatar · 2019-08-29 11:02:45 UTC

Hi Ben,

Can you share the development roadmap if it is not a huge secret?
Reading along the lines of Bart’s update seems to suggest further endpoints will be made available or am I lettting my imagination go wild here.

This new API is getting built to empower developers to use Squiz Matrix as a Headless CMS and to take the integration capabilities of the platform to a whole new level.

BenFreke · 2019-08-29 23:20:33 UTC

Hi

Unfortunately we’re not yet ready to share this roadmap publicly. Obviously Summit is fast approaching so we will look to share it before then if at all possible.

Separately, part of the reason of releasing the Beta is to get feedback to help us build the roadmap. So if there is a particular use case or feature you would like it to have, if you can let us know we can add that to our list of things to consider when we finalise the next step of the roadmap (even if we can’t publish it).

It is safe to say that more endpoints are coming. What they are and what they do is TBD.

Bart · 2019-09-02 10:56:17 UTC

Further to Ben’s comment, we are definitely working to make sure that the API doesn’t unexpectedly open up read access points to assets that you don’t want it to read, especially for the public user.

We are looking to have similar controls as the JS API has, where you can control root node restrictions and endpoints per API.

It’s worth mentioning that this is just the first step to improve the headless / API capabilities in Matrix. Once we have the content delivery API done, we’ll be working on creating a new Content Management API so you can more easily create your own admin interfaces and integrations for pushing data to Matrix. Ontop of that, we’ll also look at adding a GraphQL layer so that you can have complete control over the data structures that the delivery API returns to you. Our goal is to make the API of Matrix as developer friendly as possible, which includes making it fast to enable and configure, but also work in a way that supports modern web development practices. We want developers to work less and less in the Admin interface configuring assets and jumping between screens, and more in their local dev environments (Node, Ruby, PHP, whatever). This is a smaller piece in a much larger developer experience enablement strategy within the whole Squiz product suite (DXP).

tbaatar · 2019-09-02 13:27:46 UTC

Amazing news Bart.

When can we expect to see Content Management API?

Many thanks.
Tuguldur

Adam-DPTI · 2019-09-03 00:33:23 UTC

Will this API open the possibility of design assets? We have a third party software that mimics the design of our Squiz Matrix site that currently requires ‘copy and paste’ of the rendered header and footer being delivered by Squiz into the 3rd party software to replicate the look.

Can this feature do an API call to an ESI design asset that is used in our design or something similar to ensure a “single source of truth”?

Bart · 2019-09-08 10:50:41 UTC

Not until next year some time. We’ll most likely first focus on adding the GraphQL layer, but we’ll evaluate the priorities early next year.

It depends on how your design is set up. The API will simply just give you data about an asset in JSON format, so if your 3rd party software supports that now, it should be possible. You can achieve this type of functionality with the existing feature set of Matrix though.

JohnGill · 2019-10-10 05:47:31 UTC

Now that 5.5.4.0 is out (but not yet at https://matrix.squiz.net/releases/vm) is there any word on what has changed in the Content API since the beta?

Was the ability to only expose parts of the tree added?

BenFreke · 2019-10-10 06:02:18 UTC

Hi John

The Content API is now a separate package, so it is not part of the 5.5.4.0 release.

We expect to tag the v1.0.0 release either late this week or early next week, along with release notes and documentation on how to use install and use it.

And yes, it is possible to restrict the portions of the tree available to the API

tbaatar · 2019-12-23 16:56:23 UTC

Hi Ben,

Any news when the Conetnt API package will be released? and how it can be installed?

Thanks.

BenFreke · 2020-01-06 01:28:52 UTC

Hi

I’ll get that information added to the official documentation.

That is listed here: https://matrix.squiz.net/manuals/content-api

tbaatar · 2020-01-07 17:07:19 UTC

Hi Ben,

Thanks for getting back.
So this Content API is only for Squiz clients? or anyone with administrator knowledge can install it by running the packages?

Little bit confusing as the packages manual does not state anything about being a client of Squiz.

Thanks.

BenFreke · 2020-01-14 04:49:39 UTC

Sorry for the delay getting back to you.

It’s downloadable from the DXP Marketplace. https://marketplace.squiz.net/extensions/content-api

Next time I’ll just reference that, as that does link to the official documentation as well.

thompsonmax · 2020-01-14 14:48:36 UTC

Hello everybody. I am new to this forum. Thanks for the information.