Customising Documentum’s Netegrity Siteminder SSO plugin pt 2

July 1, 2013 at 9:51 am | Posted in Performance | Leave a comment
Tags: , ,

The 1st part of this article introduced the motivation and architecture behind web-based Single Signon systems and Documentum’s SSO plugin. This 2nd part of the article discusses limitations in the out of the box plugin and a customisation approach to deal with the issue.

Sometimes you don’t want SSO

Whilst SSO is a great boon when you just want to login and get on with some work there are situations when it is positively unwanted. A case in point is electronic sign off of documents in systems like Documentum Compliance Manager (DCM). The document signoff screens in DCM require entry of a username and password (a GxP requirement) yet the out-of-the-box netegrity plugin only understands SSO cookies, it doesn’t know what to do with passwords.

Inside the plugin

Before looking at the solution let’s look in detail as how the out-of-the-box plugin works. When the dm_netegrity plugin receives an authentication request it contacts the SiteMinder application via the SiteMinder Agent API (SiteMinder libraries are included with the Content Server installation). The following API calls are made to the SiteMinder server:

  1. Sm_AgentApi_Init(). Sets up the connection to the SiteMinder server.

  2. Sm_AgentApi_DoManagement(). “Best practice” call to the SiteMinder server passing an authentication agent identifications string: Product=DocumentumAgent,Platform=All,Version=5.2,Label=None.

  3. Sm_AgentApi_DecodeSSOToken(). Passes the SSO token to SiteMinder to confirm that the token is valid i.e. that it has been produced by that SiteMinder infrastructure. If the call returns a success code then the token is valid. A session specification is also returned to the calling program – this is the identifier that connects the SSO token to the session originally created on the SiteMinder infrastructure.

  4. Sm_AgentApi_IsProtected(). Checks whether SiteMinder regards the web application context as a protected resource. This call is probably needed to fill in a data structure that is used in the in the next call.

  5. Sm_AgentApi_Login(). One of the input parameters to this call is the session specification (from step 3). If the session specification is passed then SiteMinder will will do some verification checks on the session (has it expired? is the user active?) and then return the user LDAP identifier. The plugin uses this information to check that the token is for the correct user.


The out of the box (OOB) dm_netegrity plugin provided by EMC is setup to authenticate users who have previously authenticated against SiteMinder and received an SSO token in their browser session. In our case, where authentication with a username and password is required, there is no support in the DCM application for re-authenticating against the SiteMinder SSO solution. Where such authentication is attempted the OOB plugin will return an authentication failure as it is not designed to authenticate usernames and passwords against SiteMinder.

One way to solve this problem is to add support in the authentication plugin for authenticating against a username and password as well as a SSO token. Since SSO tokens are very large (several hundreds of characters) whilst passwords are generally significantly smaller, we can use the length of the authentication token to decide whether the token is an SSO credential or a password. In practice something like 20 characters is a good cutoff point. If the length is greater than this limit it is treated as an SSO credential and processed as described above. If the length is 20 characters or less it is treated as a password and processed using the following API calls.

  1. Sm_AgentApi_Init(). Sets up the connection to the SiteMinder server.

  2. Sm_AgentApi_DoManagement(). “Best practice” call to the SiteMinder server passing an authentication agent identifications string: Product=DocumentumAgent,Platform=All,Version=5.2,Label=None.

  3. Sm_AgentApi_IsProtected(). Checks whether SiteMinder regards the web application context as a protected resource.

  4. Sm_AgentApi_Login(). Since Sm_AgentApi_DecodeSSOToken() has not been called no session specification is available and is not passed into the Login call (compare the out-of-the-box logic). However if the username and password are passed to the Login function  SiteMinder will validate the credentials. If a success return code is received the user is authenticated, otherwise the user is not authenticated.

Implementation and Deployment

Source code for the out of the box plugin is provided in the Content Server installation. It is written in C++ and has a makefile that covers a number of operating systems. To get this to work for 64-bit Linux took a little manipulation of the compiler and linker options.

The customisation should be deployed as a single *nix shared library. When the file is deployed to $DOCUMENTUM/dba/auth on the Content Server it is available as a dm_netegrity plugin (after a Content Server restart).

Note: the out-of-the-box library must not be present in the auth directory as this will cause a conflict when the plugins are loaded by Content Server and both try to register themselves as ‘dm_netegrity’.


The solution is fairly simple in concept, the devil is in the details of compile/link, deployment and testing. If you think you need to implement customised SSO for your project and want some help designing and implementing your solution please contact me for consulting work – initial advice is not charged.

Troubleshooting weird DCM messages

July 24, 2012 at 5:34 pm | Posted in Performance | Leave a comment

This came up on the ECN forum today and the message is so obscure (but quite common) that I thought it worth writing up the troubleshooting notes.

The original post is here. The poster was trying to create a Change Notice or Change Request in Documentum Compliance Manager (DCM) and got the following error message in a dialog box:

The System can not complete your request. The action you have chosen is no longer valid because of a change in repository

This seems to be a generic message that DCM pops up whenever an ‘onexecutiononly’ pre-condition check fails.

What’s a pre-condition?

A pre-condition is a framework built into Documentum WDK (the framework that DCM, Webtop, Taskspace, WebPublisher, etc are built on) that allows menu options to be programatically turned-on/turned-off/greyed-out/hidden in the browser interface. To give an example a developer may have created a component to display the contents of a folder and for each entry there can be different menu options available such as View, Edit, Check-out, checkin, Create PDF rendition and so on. Now if a document is not checked out it doesn’t make sense for the checkin option to be available. In fact it would be just confusing if that selection was left available (WDK applications tend to be confusing enough as it is). So a pre-condition is a piece of code that can be run for each item which will return either true or false to decide whether a menu option is available.

What’s an ‘onexecutiononly’ pre-condition?

With great power comes great responsibility! Imagine you have 100 objects in a folder and you have 40 or 50 menu options for each one (not untypical). That’s 4,000 – 5,000 pre-condition checks. If the pre-condition code just does calculations and checks based on information available or cached on the application server then generally this is not a problem and your UI should remain pretty responsive. However if your pre-condition runs a query against against the content server, however ‘fast’, or does an object fetch (e.g. using IdfSession.getObjectBy…()) then you are going to suffer some pretty sluggish UI performance.

The WDK references do warn about this in the section on pre-conditions however it seems that this warning was not heeded in DCM 5.3 (naughty EMC). Generally navigating around DCM5.3 is pretty miserable for most productions users and the best that can be suggested is to upgrade to DCM 6.x (by the way if you absolutely have to stay on DCM5.x but can bear some development and testing effort to alleviate the pain then there are some code-based possibilities).WDK 6 introduced a new pre-condition setting – onexecutiononly – which was taken up by the DCM developers to ‘fix’ the performance problems they had introduced.

‘onexecutiononly’ means that the pre-condition is not evaluated when the list of objects is rendered onto the screen but only when the user selects the menu option in the user interface. As a result you no longer have 1000s of pre-conditions running when rendering the interface. Of course in a way this rather ‘neuters’ the power of the pre-condition because now we could have, for instance, check-in available for documents that aren’t checked out. If we try to checkin the document the pre-condition will return false and we will get a warning message on the screen. Typically like the one the poster saw when trying to create a change notice or change request. In that particular case there are likely to be some checks in the pre-condition code for a newchangerequest or newchangenotice action and they have ‘failed’. At the time of writing the problem hadn’t been fully resolved so I’ll update this entry if any new information comes to light.

Blog at
Entries and comments feeds.