My Sysadmin has just done some monitoring of our AD server, and reported that in a 2 hour window there are approx 550,000 events in the security log - 99% of which are from our Matrix server.
This doesn't right to me - as my understanding was that AD was only contacted during the user's login process.
Even if Matrix was polling AD for every permission of every asset - this still seems too big a number as usage of our matrix system is less than 5000 hits a day.
We are using Matrix 3.10.3. On searching through the forums yesterday, I saw some advice from Avi that LDAP V3 and 'disallow referrals' should be ticked - they weren't, so I selected them. There doesn't appear to be any change as a result of this.
Do we have something configured incorrectly in our Matrix system? Any advice on what might be causing this and how to fix it?
Can I recommend you log a support request for this so that we can investigate your system specifically?
If you are using the admin interface and loading information about LDAP users or groups, LDAP will also be contacted. Matrix does not keep a cache of any LDAP requests, except for the groups that a user is in.
So if you open the LDAP tree, or view a page edited/created/published by an LDAP user, or view the permissions screen where LDAP users or groups are listed, Matrix will need to requests information about that LDAP resource.
Maybe your requests are coming from these type of events. If so, there are no options to enable that can turn that off. It is just the way LDAP integration work. Of course, we could build an LDAP cache, but that would take some time.
I'm pretty sure that I am the only user that would know where the LDAP items are located in the backend - and I don't go there very often. We don't have any other users/administrators working with permissions.
I've logged a support ticket.
This is not correct. Every time an LDAP user visits the page, we need to load their details to find their name, email etc, just like a regular Matrix user. If all your users view the site as LDAP users, you'd have a lot of requests.
I've also posted this comment in your support request.
So your Matrix system is not doing anything unusual. If you need the LDAP traffic reduced, you can either create a mirror of the directory to let Matrix use, or get us to build LDAP caching into the system for you. If you are really concerned, the mirror would be my suggestion.
Resonse from support :
[quote]Our developers have also said that every time an LDAP user visits a page, Matrix needs to load their details to find their name, email etc, just like a regular Matrix user. This will require an LDAP query every single time, since Matrix doesn't store a cache of any LDAP queries. As a result, there will be a great deal of LDAP requests going on. It is probably possible for our development team to build LDAP caching into Matrix, but it would take significant amounts of development time.[/quote]
This seems like an inefficient way of doing things - were the particular reasons why it was constructed this way? Surely it would have been obvious from early on that caching LDAP user/group info would significantly improve the LDAP functions.
How are your larger clients (eg govt. departments) finding speed as a result of this?
[quote]Resonse from support :
This seems like an inefficient way of doing things - were the particular reasons why it was constructed this way? Surely it would have been obvious from early on that caching LDAP user/group info would significantly improve the LDAP functions.[/quote]
The goal of LDAP integration is allow LDAP users and groups to work with Matrix without having to store their information locally. If we cached every request we did, we'd be creating a copy of the directory locally, which is not acceptable to many of our clients and is against the original goal of the project.
[quote]How are your larger clients (eg govt. departments) finding speed as a result of this?[/quote]
I haven't heard of any problems at all. LDAP performance is good because LDAP is designed for these type of queries and the directory is always on the same network as the Matrix server. The number of LDAP queries has never been identified as a bottleneck in any Matrix system.
As for clients with very large directories, they run perfectly fine. Those are also the sort of directories we don't want to cache requests for, as we would end up with tens of thousands of unique cache entries, which would (potentially) decrease Matrix's own caching performance.
Greg, would there be a way to create a middle-of-the-road solution?
What I mean is: cache the user's LDAP details on login, so that each page load of the current session doesn't need to go back to the LDAP server, but don't actually store that information anywhere in Matrix (beyond the session itself).
[quote]Greg, would there be a way to create a middle-of-the-road solution?
What I mean is: cache the user's LDAP details on login, so that each page load of the current session doesn't need to go back to the LDAP server, but don't actually store that information anywhere in Matrix (beyond the session itself).[/quote]
However we treat LDAP users, we have to treat normal users as well. So we cant cache their details in the session because the bridge doesn't know about logins, but it does the requests, and I don't want all user data cached in Matrix.
The solution to this is caching of the LDAP requests, which is no harder than caching the user on login anyway. As a user of Matrix, you'd need to decide which is more important: fewer requests to the LDAP directory or more up-to-date LDAP information. If you have no current LDAP problems, then you'd be silly not to want the up-to-date info.
And like I said before, you can always mirror the directory, which clients would typically do if they don't have the directory local to the Matrix box.
[quote]However we treat LDAP users, we have to treat normal users as well. So we cant cache their details in the session because the bridge doesn’t know about logins, but it does the requests, and I don’t want all user data cached in Matrix.
The solution to this is caching of the LDAP requests, which is no harder than caching the user on login anyway. As a user of Matrix, you’d need to decide which is more important: fewer requests to the LDAP directory or more up-to-date LDAP information. If you have no current LDAP problems, then you’d be silly not to want the up-to-date info.
And like I said before, you can always mirror the directory, which clients would typically do if they don’t have the directory local to the Matrix box.
[right][post=“15324”]<{POST_SNAPBACK}>[/post][/right][/quote]
Just to be sure on something. If a user logs into a site (ie an intranet site) and begins surfing under https each page request requires an ldap lookup to see what that person can do (ie: group associations)?
Is this correct?
A :blink:
[quote]Just to be sure on something. If a user logs into a site (ie an intranet site) and begins surfing under https each page request requires an ldap lookup to see what that person can do (ie: group associations)?
[right][post=“15427”]<{POST_SNAPBACK}>[/post][/right][/quote]
Correct, but its not just used for permissions, its also used to create the User asset itself.
[quote]Correct, but its not just used for permissions, its also used to create the User asset itself.
[right][post=“15428”]<{POST_SNAPBACK}>[/post][/right][/quote]
So the short story is that if you have an authenticated site then your LDAP Server needs to be able to handle large numbers of small requests.
What would happen if an LDAP tree had a large number of groups, would this in turn have the potential to cause spped issues when accessing the LDAP Service?
A
The size of the LDAP directory wont matter. We are only checking to see which groups the user has listed in their memberof attribute. We then match those IDs to the IDs stored in the Matrix DB.
Our LDAP queries are small and very targeted.
[quote]The size of the LDAP directory wont matter. We are only checking to see which groups the user has listed in their memberof attribute. We then match those IDs to the IDs stored in the Matrix DB.
Our LDAP queries are small and very targeted.
[right][post=“15431”]<{POST_SNAPBACK}>[/post][/right][/quote]
Sounds great. We are just finalising the LDAP Service Matrix shall authenticate against. One of the questions was if Matrix did alot of requests per login or if it only did one…Thanks for letting me know…
Cheers,
A