Keyword modifiers


(Justin Cormack) #1

We have this week added keyword modifiers to the Matrix core as you might have seen in the dev newsletter. I thought I would put this here as there is no other documentation yet. This will be available in the 3.24.0RC1 release in early August. The drive for this was for escaping of data for web services, but it is of general use for all sorts of things.


The syntax is: %keyword~modifier:arg~modifier:arg% ie use ~ to specify the modifiers and : if they have any arguments, and you can use multiple modifiers in a chain.



eg if you want to get the first 10 words out of the body of an asset as a summary you might use %asset_contents~striphtml~maxwords:10% in your paint layout.



The currently available modifiers are listed below. They are mainly aimed at data passing and security, but a few extra things added by popular demand…

urlencode

urldecode

base64encode

base64decode

lowercase

uppercase

nl2br - new lines to <br />

escapequotes - add slashes to quotes

escapehtml - convert & < > to html entities

striphtml - remove all html tags

maxchars - limit to x characters

maxwords - limit to x words

wordcount - number of words

charcount - number of bytes

capitalize - first letter capital

titleize - all first letters capitals

trim - remove whitespace at start and end

underscore - replace white space with underscores

xpath - Args are delimeter, xpath expression, extracts the parts of XML using xpath and then puts the delimiter between if there are multiple ones.

empty - One argument; text to display if the keyword result is empty.

tag - One argument; the tag name to wrap around the text.

tagif - Same as above, except do nothing if the text is empty.

cdata - Escape CDATA closing markup (this correctly works even with nested CDATA etc)



The most complex one is probably the xpath one. The following extracts all the titles from an RSS feed and wraps them in ol and li tags:

%asset_contents~xpath:</li><li>://item/title~tagif:li~tagif:ol%



We are using this to extract single fields from web services that return XML eg SOAP (more on that shortly when the web services work goes into core).



Note that with these changes, if you use a modifier on eg %globals_get% that currently has a whitelist of allowed characters, the whitelist is ignored as you can safely escape these values yourself (eg using escapehtml).


(Nic Hubbard) #2

Very cool! I can't wait to try this out in 3.24!


(Scott Hall) #3

That's 0rsm!


(Robin Shi) #4

From Matrix to Transformers! :D’ /> :D <img src=‘http://forums.matrix.squiz.net/public/style_emoticons/<#EMO_DIR#>/biggrin.gif’ class=‘bbc_emoticon’ alt=’:smiley:


(Justin Cormack) #5

Just adding the rest of the docs here. If you have to do xpath namespaces you need to do:

    %keyword~xpath:://y:Longitude xmlns:y=urn:yahoo:maps%


to process a Yahoo maps API call that returns

    51.559509-0.142860London, NW5 1SNUnited KingdomGB

(Duncan Robertson) #6

Justin showed me a demo of this yesterday and I can confirm that it is 0rsm.


(James Hurst) #7

Howdy all.


Just thought I'd update this thread with the complete list of modifiers. As Justin explained, you can chain as any modifiers as you like, and some of them accept args separated by :



Most of them are self explanatory, but I've added notes to the complicated ones…



  • urlencode
    [*]urldecode
    [*]base64encode
    [*]base64decode
    [*]lowercase
    [*]uppercase
    [*]nl2br
    [*]escapequotes
    [*]escapehtml
    [*]striphtml
    [*]maxchars:n
    [*]maxwords:n
    [*]wordcount
    [*]charcount
    [*]capitalize
    [*]titleize
    [*]trim
    [*]underscore
    [*]xpath
    // Args are delimeter, xpath
    //
    // Example (ordered list of RSS item titles, using tagif modifier to ol list):
    //
    // %asset_contents~xpath:</li><li>://item/title~tagif:li~tagif:ol%
    //
    // The path lists all titles in the feed, delimited by </li><li>, and then conditionally wrapped in <li>,
    // which in turn is conditionally wrapped in <ol>.

    [*]empty:replacement
    // Will echo text specified in replacement if the keyword is empty.

    [*]notempty:replacement
    // Same as above with reverse logic. Allows you to display something only if the keyword has something in it.

    [*]tag:tagname
    // Wrap the keyword value in an xml tag, prints a single closing tag if the keyword is empty.

    [*]tagif
    // Wrap the keyword value in an xml tag, prints nothing if the keyword is empty.

    [*]cdata
    // Wrap the keyword in a CDATA block. Handles escaping CDATA within CDATA.

    [*]increment

    [*]decrement

    [*]negate

    [*]abs

    [*]sign:plus:minus:zero
    // Outputs +, -, or 0 depending on the sign of an number. Args allow you to override what is displayed.

    [*]ceil

    [*]floor

    [*]round:precision
    // Default precision is 0.

    [*]add:n

    [*]subtract:n

    [*]modulo:n

    [*]divide:n

    [*]multiply:n

    [*]eq:x:true:false
    // Prints 1 if keyword equals x, nothing if not. Args allow you to override what is displayed.

    [*]neq:x:true:false

    [*]gt:x:true:false

    [*]lt:x:true:false

    [*]gte:x:true:false

    [*]lte:x:true:false


This will all be in the 3.24.0RC1 release. Here's some examples from a paint layout which I used during testing, just to help get people started when the RC is out.

    
Metadata
%asset_metadata_Test%
globals_asset_metadata_Test:64~uppercase
%globals_asset_metadata_Test:64~uppercase%
globals_user_name
%globals_user_name%
globals_user_name~xmltag:em
%globals_user_name~xmltag:em%
globals_user_namse~empty:replace
%globals_user_namse~empty:replace%
globals_user_name~cdata
%globals_user_name~cdata%
globals_user_namse~empty:replacewith]]>asd~cdata
%globals_user_namse~empty:replacewith]]>asd~cdata%
globals_user_name~urlencode
%globals_user_name~urlencode%
globals_user_name~lowercase
%globals_user_name~lowercase%
globals_user_name~uppercase
%globals_user_name~uppercase%
globals_user_name~lowercase~titleize
%globals_user_name~lowercase~titleize%
globals_user_name~lowercase~capitalize
%globals_user_name~lowercase~capitalize%
asset_contents
%asset_contents%
asset_contents~escapehtml
%asset_contents~escapehtml%
asset_contents~striphtml
%asset_contents~striphtml%
asset_contents~striphtml~nl2br
%asset_contents~striphtml~nl2br%
asset_contents~striphtml~maxchars:5
%asset_contents~striphtml~maxchars:3%
asset_contents~striphtml~maxwords:5
%asset_contents~striphtml~maxwords:5%
asset_contents~striphtml~wordcount
%asset_contents~striphtml~wordcount%
asset_contents~striphtml~charcount
%asset_contents~striphtml~charcount%
asset_contents~divide:2
%asset_contents~divide:2%
asset_contents~divide:2~round:2
%asset_contents~divide:2~round:2%
asset_contents~multiply:2
%asset_contents~multiply:2%
asset_contents~negate
%asset_contents~negate%
asset_contents~increment
%asset_contents~increment%
asset_contents~neq:23
%asset_contents~neq:23%
asset_contents~sign
%asset_contents~sign%
asset_contents~negate~sign
%asset_contents~negate~sign%
asset_contents~notempty:asdasd:/asdasf/asfas//asda
%asset_contents~notempty:asdasd:/asdasf/asfas//asda%


Oh yeah, and stand by for the REST package, where using many of these modifiers starts to make more sense :)

James.

(Duncan Robertson) #8

I'm hearing good things about 3.24 generally.


(Nic Hubbard) #9

I heard that 3.24 would solve Global Warming. At least that is what I heard.

(Duncan Robertson) #10

3.24 feeds starving children. At least that's what I heard.


(Nic Hubbard) #11

Maybe Greg can confirm this.

(Greg Sherwood) #12

I heard 3.24 will be the first CMS on Mars, but I'm sure it's doing all that other stuff too.


(Keith Brown) #13

Did the keyword modifiers go into the 3.24.0 final? Installed that today and wanted use them. Putting it into an asset listing just outputs the keyword 'unprocessed'. i.e.


[html]%asset_metadata_category~escapehtml%

%asset_name~escapehtml%[/html]



Unless I've misunderstood how they work which is always a possibility.



cheers



K


(Greg Sherwood) #14

They will be in 3.24.1 but will use a ^ instead of a ~ as that character was reserved by a metadata keyword and failed testing.


(Shane Weddell) #15

Hey Greg, is it true that functionality is being released into the stable branch after the dev lock down a month before release? Seems there is a lot of functionality released in the dot 1,2 and 3 versions in the Matrix 3.2#. + versions.


Would be nice if feeding starving children could be backwards compatible with earlier versions.


(Greg Sherwood) #16

Functionality that is unlikely to cause core instability is allowed in the latest stable branch for a few point releases. Anything really core has to wait in dev.


Once a new stable branch is released, the previous stable branch becomes classed as a "hardened" branch and does not get any new features or even medium-risk performance improvements. Users can rely on that branch for stability. At that time, the previous hardened branch is discontinued.



So for example, 3.22 wont be getting any new features now. 3.24 will still get low-risk changes until the end of the year. 3.20 wont see any more releases, but Squiz will continue to support these systems under SLAs and back-port bug fixes as required.



So this keyword modifier change wont get into 3.22 and it is really only going into 3.24 because it's been in testing there for a couple of months.


(Shane Weddell) #17

Cool, I bed the BDM's are happy about that:)


(Tomlinson) #18

That looks brilliant, but I'm stuck on 3.22 for the moment. Is there an built-in way to get a summary of an asset contents field, or do I have to create one as metadata. Sorry if It's a dumb question. Just can't seem to find the answer anywhere.


(Nic Hubbard) #19

[quote]
That looks brilliant, but I'm stuck on 3.22 for the moment. Is there an built-in way to get a summary of an asset contents field, or do I have to create one as metadata. Sorry if It's a dumb question. Just can't seem to find the answer anywhere.

[/quote]



What do you mean by an assets contents field? You mean an asset attribute?


(Tomlinson) #20

[quote]
What do you mean by an assets contents field? You mean an asset attribute?

[/quote]



Sorry for not being more specific, I mean the first 100 characters of %asset_contents_raw% in order to display a summary of page contents in an asset list.