Data migration and integration with other systems

Hello everyone,


Background: We are currently setting up MySource Matrix RC1 on a Suse Linux 9.0 server for the purpose of evaluating the product. We are a small (around 30 employees) Norwegian government agency administering international education programmes.



Our current publishing system is custom-developed and Lotus Notes/Domino-based. The main web database needs to be moved/migrated to a CMS/CMF such as MySource, but we need to retain the possibility of including queries from other systems (usually via ODBC) in pages on the new system.



Without yet having much experience with MySource Matrix, does anyone know to what degree it lends itself to the following (or to what extent it can be modified to support it):



1: The possibility to include ODBC/JDBC queries on other databases in MySource pages and other assets, perhaps as assets themselves, and



2: Import/migration of pages and other assets from other databases into the MySource database.



I am grateful for any information, and I will try to post my experiences on our set-up and evaluation of MySource Matrix on this board. I should mention that the initial impression (from reading the documentation) is very good, and that the other products we are evaluating include Plone, WebGUI, and EzPublish.



Daniel Arnrup

Executive Officer IT

SIU, Norway

There are a couple of things you can do to query other systems. Both involve creating new asset types.


MySource Matrix supports Shadow Assets - assets that do not exist within the Matrix system itself, but can be browsed using the asset map. LDAP users and groups are an example of this. Essentially, you can create an asset called a Bridge and the bridge can query an external database and provide asset like information to the rest of the system. The system thinks of these Shadow Assets as real pages, files etc but the Bridge is the asset that is really doing all the work. This is the most complex way of doing things but gives the best result if it can be done.



The other way is to simply create a custom asset type that does the query and the printing of information for you. A custom asset type can execute whatever PHP code you require. If PHP can connect to your database, you can get and present the information in whatever format you require.



There are no tools built into MySource Matrix to set these sorts of asset types up. They require custom development as the differences between external systems are too great to produce a really good generic module.



If you would like any more specific information I'd be happy to help.

Thanks, I will experiment with this as soon as we have everything up and running.

For us to use Matrix requires a good deal of data migration, and I do believe it should be possible to create a generic import asset for pages following a very basic format - title/subject, ingress and body (plus perhaps metadata as one or
more chunks).



Could you provide a very general instruction for importing existing pages into Matrix? I here mean the minimum of information required - which tables and fields must be updated, and what are the required values? I have begun going through the database created by Matrix but perhaps you could save us some work by giving some hints on the database structure.



Since you have worked closely with government agencies, do you have any examples of how they have handled migration of data from previous systems into Mysource/Matrix?

Its almost impossible to import data directly into the Matrix database tables because of the way asset attributes are stored. The preferred method is to use the Matrix API (available soon!) to create and modify the assets instead.


We have a few clients that have written/are writing migration/import scripts from their existing infrastructure to Matrix using this method.

Thanks - is there any possibility of making sample scripts available when the API is released?

We certainly hope to do exactly that. At the moment, most of the scripts are not ready for public consumption, as they're written specfically for a particular environment.


We plan to have the API documentation released at the same time as the 3.1.0 full release of Matrix.