Can’t start the docbase after installing new Oracle home

June 11, 2009 at 4:54 pm | In Performance | Leave a Comment
Tags:

This is a very windows specific solution however similar things could occur on *nix
This is a problem I ran into while investigating an oracle problem. I had a VMWare image that I used to run Oracle and Content Server. Content Server had always run fine on it whenever I switched the VM on. Then this morning it didn’t. Coincidentally I had just installed a new Oracle home (ie installation) to test out some issues to do with a particular Oracle patch version. Too much of a coincidence? Of course.

The problem actually arose when I subsequently tried to create a new docbase for an entirely different investigation. I started the docbase config program entered all the information until I got to the bit where it takes the database password and tests the database connection – FAILURE.

I did a couple of tests, first of all start up SQL Plus and try and connect to the database, in this case the database connect identifier is DB1:

SQL> connect system/@DB1
ERROR:
ORA-12154: TNS: could not resolve the connect identifier specified

OK so SQL Plus seems to have a similar problem to the Content Server config program. So What happens when you type connect …/…@DB1? Well assuming you are using the standard TNSNAMES name resolution it looks for the file tnsnames.ora in the %ORACLE_HOME%\network\admin folder and uses that to convert DB1 into a request to the TNSListener.

The problem was I hadn’t defined ORACLE_HOME – never needed to – so presumably it was using a default value that had now changed after the new oracle home installation.

So I set an explicit oracle home environment variable:

set ORACLE_HOME=d:\oracle\product\10.2.0\db_1

and that fixed SQL Plus. By opening up the control panel System app I set the environment variable for the dmadmin user so now the configuration program , and all the docbases could start.

Update: every Documentum on Oracle installation I have seen has used TNSNAMES, using something like Oracle Internet Directory (OID) is not a good idea according to support note esg97155.

Jython on Documentum part 3

March 12, 2009 at 5:44 pm | In Continous Integration, Jython | Leave a Comment
Tags:

It’s been a while since I’ve blogged and even longer since I blogged about Jython on Documentum. Sadly excessive work schedules have prevented me from blogging as much as I would like and it’s unlikely to change for the forseeable future. Anyway I’m waiting for a build to run so I’ve got 5 minutes for a coffee and a swift blog.

Since my last Jython on Documentum post I’ve been using jython in a real-live project as part of a build system. The next couple of posts will record some things that I found out.

First up is organising script files. In my previous posts I had just been using jython by changing to the installation directory, saving a .py file and running it from the installation directory. This is not really a great way to organise your script files. I’ve found the following organisation to work well though no doubt there are other ways to do the same thing.

First I have my jython installation in say c:\jython2.2.1. So anytime I want to kick off a jython script I do the following:

  • Start a cmd prompt
  • set path=%path%;c:\jython2.2.1
  • jython

Next I organise my jython scripts in the source code repository in their own folder structure. Something like /scripts/jython/dctm or something similar. The scripts can then be included in builds or deployments as necessary.

A useful object to know about is the sys.path object. This is represents the classpath that jython uses to pickup python libraries, this includes libraries you write yourself (just standard jython code files). You can easily import an external library using the following code snippet (which I have seared into my brain!):


import sys
sys.path.append('scripts/jython/dctm')
import dctm

sys is a standard python library that contains a lot of useful system functions/features. One of them is the aforementioned sys.path. The append method allows you to add extra paths that jython can use to look for libraries. In this case the relative path script/jython/dctm is where I keep my standard dctm.py library. After these lines I can use the dctm object to access any of the methods in dctm.py.

So my standard dctm.py is stored scripts/jython/dctm, my project specific jython file (e.g. myfile.py) is also stored in this folder structure. By navigating to the top-level folder (the one containing the scripts folder) I can run jython like this:

jython myfile.py

WebPublisher Apply Presentation from the command line

February 3, 2009 at 9:25 am | In Continous Integration | 3 Comments
Tags:

Ever thought it would be useful to be able to utilise the WebPublisher Apply Presentation functionality from the command line but didn’t want to have a whole application server install to make it work? Perhaps you need to put together installation scripts or an automated testing environment. Here is the ‘under-the-covers’ method that is behind the Apply Presentation logic in WebPublisher.

Much of the back-end logic that is exposed by the WebPublisher front end goes through a java server method called wcm_utils. Suppose you have:

  • a docbase DB1
  • install owner of dmadmin
  • xsl template with object id 090ff0ff80003852
  • xml object to be transformed 090ff0ff8000f921

then you need to call the wcm_utils method with the following arguments:

-docbase_name DB1 -user dmadmin -action 12
    -object_id 090ff0ff80003852 -object_ids 090ff0ff8000f921

for example in IAPI this would be:

apply,c,NULL,DO_METHOD,METHOD,S,wcm_utils,
     ARGS,S,-docbase_name DB1 -user dmadmin -action 12
     -object_id 090ff0ff80003852 -object_ids 090ff0ff8000f921

You can also use this interface to transform more than 1 xml object, just separate the -object_ids parameter with colons:

apply,c,NULL,DO_METHOD,METHOD,S,wcm_utils,
     ARGS,S,-docbase_name DB1 -user dmadmin -action 12
     -object_id 090ff0ff80003852 -object_ids
    090ff0ff8000f921:090ff0ff8000f922:090ff0ff8000f924

Why DocApps are bad

January 27, 2009 at 5:23 pm | In Continous Integration, Development | 4 Comments

A recent comment on one of my posts pointed out that you could script DocApp installs using an unsupported ant task. This is certainly a useful facility however it doesn’t overcome the basic problem with Documentum Application Builder and DocApps – they don’t integrate well with source code control systems that are used to maintain all the other code and artefacts needed to develop a working system.

Think how you probably develop code for your system. You probably store your source code in a repository (Visual SourceSafe, Subversion, CVS, etc.). You probably have integrations in your IDE to allow you to automatically checkout and checkin code. You probably build your code automatically from the source code system. This is is all fairly standard development practice (hopefully!).

If you are a little more advanced then maybe you also package and deploy your application automatically and you store automated tests in your source code control system. The tests would be automatically executed whenever you build and deploy from your souce code control system.

Basically the source code control system is the lynchpin of your whole development effort. Using version control and labelling you can see how different releases have progressed and maybe track where and when bugs were introduced.

DocApps sit completely outside of this setup, you are effectively using a development docbase as an alternative source code control system – without all the benefits of a properly featured SCC! In an ideal world all your Documentum configuration assets would live in the source code control system – that includes things like type definitions. Now all the source artefacts for your development live in one place. Hopefully this is exactly what Composer will allow you to do.

Why do I get DM_DOCBROKER_E_NO_SERVERS_FOR_DOCBASE

January 22, 2009 at 2:51 pm | In Performance | 2 Comments
Tags:

This question comes up so many times on the forums it’s time to add it to the blog. It probably helps to understand the role of the Documentum Docbroker (or Connection Broker isn’t it amusing how the marketing people insist on changing the names for these things but the old names live on in the code!).

Definitions

Most of this information is in the Content Server manuals but a few definitions:

Docbase/Repository. A collection of content files and metadata that is managed as a single entity. Usually consists of one or more files systems and a RDBMS. Can also include an XML Store (from D6.5) and other more esoteric options like Centera. Some people include the Full-text index as part of the docbase.

Content Server. A server-based process (or set of processes is you are on *nix) that provides an API for clients to access and control the underlying docbase. Each server will access exactly one docbase, but each docbase could have multiple servers possibly distributed to different machines and maybe other parts of the world.

Docbroker/Connection Broker. Each content server will be configured to project connection information (what IP address and port the server is on, which docbase it serves). When client applications want to connect to a server to access a docbase they first ask a docbroker for the connection details. If they receive connections details they then connect to the correct server. The details of this are all contained within the DMCL connect or DFC IDfSession. Note that clients are configured with which docbroker(s) to connect to not which servers to connect to.

Troubleshooting

In a simple out-of-the-box implementation you usually have 1 docbase, served by 1 content server projecting to 1 docbroker all on the same server, so I’ll continue this discussion on that basis although it is easily extensible to more complicated deployments.

If you get the above error message then you can conclude that the client application sucessfully connected to a docbroker and asked for the connection details for a docbase. The docbroker however did not know about any servers that served the docbase.

A couple of questions to ask yourself are as follows. First, is the content server for the docbase running? Use Task Manager or ps to check that there is a process running. Usually the process is called documentum.exe (or just documentum) but earlier versions of the windows software used dm_serverv4.exe. If you have a machine with multiple content servers possibly serving different docbases then you have a slight problem as you want to check if the content server for one particular docbase is running. In Unix this is easy as the full command line is available in ps, but in Windows it’s more difficult. Process Explorer, formerly from SysInternals but now owned by Microsoft, gives this information.

If you know the server is running then you need to find out why the docbroker doesn’t know about it. First of all check the server log especially the startup section – there is a DM_DOCBROKER_I_PROJECTING message that tells you which docbroker(s) the server is projecting to. Is it the docbroker your client is configured for?

It maybe that the server has tried to connect but has been unsuccessful e.g. due to firewall problems. There will be an error message in the log stating the problem. You then need to work out what configuration must be changed to allow the projection.

The content server is unnecessarily picky about which IP addresses it binds to and I have seen issues with docbroker and content server on a machine with multiple network adapters where the content server is binding to one adapter and the docbroker is binding to another. Even though they are on the same machine they are trying to communicate on different IP addresses.

By working your way through this troubleshooting process you should be able to resolve 9 out of 10 (or maybe 99 out of 100) examples of this problem

Momentum 2008 – Composer

December 3, 2008 at 10:26 pm | In Architecture, D6, Momentum | 7 Comments
Tags: ,

Ever since I got back from Momentum it’s been work, work, work. That’s what happens when you take 4 days off to look around at what’s going on. I recall that I was going to post some more thoughts on some of the other products that I saw.

I went to David Louie’s presentation on Composer. Have to say I was impressed with what I saw. This maybe because I’ve been developing with Eclipse for a while now, so having something that integrates natively with this environment is a big plus. Whilst there are many interesting functional features of Composer I was most interested in a single slide that compared Composer with Application Builder.

First Composer doesn’t require a connection to the docbase to get your work done. You can of course import objects from a docbase, but you can also import from a docapp archive.

Secondly Composer can install your application (a DAR, similar to a DocApp in concept) into a docbase via a GUI installer but you can also use something called Headless Composer which is a GUI-less installer that runs from the command line. Not absolutely sure on the specifics at this point but possibly uses ant. David said that there are details in the documentation – I will be sure to try it out and post my findings at a later date.

This last point was of great interest to me as I’m currently investigating how to run Documentum development using a continuous integration approach. Being able to deploy your artifacts from the command line, and therefore from some overall automated controlling process is essential to making continuous integration a reality. On this note I also spoke to Erin Samuels (Sharepoint Product Manager) and Jenny Dornoy (Director, Customer Deployments). I hope that the sharepoint web parts SDK that is likely to integrate into MS Visual Studio will also have support for a headless installer, and also that Documentum/EMC products generally support the continuous integration approach.

Momentum 2008 – Centrestage

November 13, 2008 at 11:00 pm | In D6, Momentum | 1 Comment
Tags: , ,

Of course the star of the show was Centrestage. If you don’t know what Centrestage is (where have you been?), in a single sentence, it’ s the next generation of Documentum client providing Web 2.0 features, a significantly different customisation model (compared with WDK) and no-cost/low-cost licencing model.

I won’t go into too much detail about the features except to say they include basic content services, personal spaces, team spaces, blogs, wikis, rss, tagging and faceted search. The time line was set as 1.0 to be released April 2009 (the beta version is available on the download site), 1.5 to be released after that and then a D7 version released by the end of 2009.

What did interest me was some of the details of the architecture and development environment. This is a web client that implements rich client functionality using Javascript. Centrestage uses a library ExtJS v2.2, that has powerful DHTML manipulation facilities. All the back-end logic is provided via DFS which is accessed via a technology called DWR v2.0. DFS provides a SOAP/WS-* interface which is difficult to call via Ajax. DWR (Direct Web Remoting) solves this problem – take a look at the wikipedia link, it’s a fascinating idea.

The UI is composed from numerous separate components which, in concept at least, are like Sharepoint WebParts. Since each component needs to be rendered on the page separately I wondered whether this would mean that a page with, say, 20 components would need 20 separate network calls to display the page. In a high-latency network environment this could be a performance nightmare. Apparently the DWR library allows for batching of requests – it means that having numerous components on the page could be displayed using a smaller number of network requests.

Momentum 2008 – XML Store

November 13, 2008 at 8:26 am | In Architecture, D6, Momentum, Performance | 2 Comments
Tags: , ,

On Tuesday and Wednesday I attended a load more sessions covering XML Store, Centrestage, Composer, Sharepoint and Web Content Management. In the next few posts I’ll share some of my thoughts and impressions, starting with XML Store.

For those that don’t know, EMC purchased a company called X-hive a while back. X-hive have an XML database product and that has now been integrated into the full Content Server stack. The easiest way to picture this is to take the old picture of the repository as consisting of a relational database and a file system and add in a third element, the XML Store.

From 6.5 (possibly sp1, I don’t remember) all XML is stored in the XML store. The XML Store is built around the many XML standards that are in existence such as XQuery, XSL and the XML full-text query standard.

The XML is not stored in the usual textual XML format but in a DOM format. This presumably is to allow them to implement various types of index and to optimise the query access patterns. The performance claims for the database are impressive although they need to be taken with a pinch of salt. As with all benchmarking, vendors will target specific goals in the benchmark. However your real-life workloads could be very different. If you are expecting high-throughput for an application using the XML store I suggest you put some work into designing and executing your own benchmarks.

In addition to indexes there is also a caching facility. This was only talked about at a high-level, however just as relational database performance experts made a career in 1990s out of sizing the buffer cache properly so we may see something similar with XML database installations. We may see them suffering poor performance as a result of under-sized hardware and mis-configuration. As always don’t expect this to just work without a little effort and research.

One other point I should make is that the XML Store is not limited to the integrated Content Server implementation. You can also install instances of XML Store separately. For example the forthcoming Advanced Site Caching Servicees product provides for a WebXML target. This is essentially an XML Store database installed alongside the traditional file system target that you currently get with SCS. You can then use the published XML to drive all sorts of clever dynamic and interactive web sites.

Momentum 2008 part 1

November 11, 2008 at 4:22 pm | In Architecture, D6, Momentum | 2 Comments
Tags: , ,

All this week I am at Momentum in Prague. It’s a great opportunity to catch up with Documentum employees, partners and users, and also to see what is going in the Documentum world.

I arrived yesterday morning, and attended the Sharepoint Integration product advisory forum. The forum was run by Erin Samuels and Andrew Chapman. The session centred around a number of topics relating to Sharepoint-Documentum integration.

First of all there was a round-table on the kind of integration scenarios people were facing. Interestingly, and reassuringly, there seem to be far fewer ‘maverick’ implementations as Andrew called them. Maverick implementations are where sharepoint is installed as a generic application that can be just configured and used by any department and team without any kind of guidance or direction from IT. This leads to silos of information and lack of control of any kind over the information. Whilst departments like this quick and easy delivery of applications it stores up problems for the organisation as it is no longer able to utilise or manage enterprise-wide data.

Andrew then talked about a new product that is due to come out called Journalling. Whilst I don’t think the naming is great (maybe that’s not how it is going to be sold but it was certainly the name used for the technology) the principle was very powerful. It uses the Microsoft-provided Sharepoint EBS interface to allow you to redirect where sharepoint stores its data. By default sharepoint will store content and metadata in a SQL server database. Each sharepoint instance will require a sql server instance (apparently) and this can easily become a big data management problem. Furthermore as sql server stores all content as BLOBs (Binary Large OBjects) there can be scalability issues.

With Documentum EBS implementation, content is (transparently to the user) stored in a Documentum repository rather than SQL server (there is just a ‘journal’ entry in sharepoint representing the object). This provides all kinds of useful benefits such as being able to leverage Documentum’s data storage scalability, EMC hierachical storage management, de-deduplication and so on.

At this point there was a big discussion around a point introduced by Andrew. Since the data is now stored in Documentum we can access it via Documentum clients; for example you average user might be creating content in sharepoint across the organisation, but you have power users who need the full power of Documentum interfaces to work with the data. But what operations should documentum clients be allowed on sharepoint originated data? Read or other types of operation that don’t modify the content/metadata are fine, but should we allow update or delete access? If yes then there is additional work required as right now an update outside of sharepoint would cause sharepoint to throw an error the next time a user accesses the object. Predictably there was an almost equal 3-way split over who wanted no Documentum access, read-only/no-modify access and total control.

Later on I got to meet up with some people that I only know from the Documentum forums and blogs: Johnny Gee, Erin Riley and Jorg Kraus. It was great to finally get to speak to these guys after years of interacting over the web.

Query Formatting

September 19, 2008 at 2:12 pm | In Performance, Troubleshooting | 4 Comments
Tags:

What does this query do?

SELECT ALL 'y' AS notnavigatable,o.r_object_type AS r_object_type,
o.r_lock_owner AS r_lock_owner,o.r_object_id AS r_object_id,'y' AS
selectable,'0' AS isfolder,upper(o.object_name) AS objname,
o.owner_name AS owner_name,o.r_version_label AS r_version_label,
o.i_is_reference AS i_is_reference,o.a_content_type AS a_content_type,
o.object_name AS object_name,100 AS idunion,o.r_is_virtual_doc AS
r_is_virtual_doc,'' AS navigatable,o.r_link_cnt AS r_link_cnt,
o.r_full_content_size AS r_content_size FROM dm_process o
WHERE ( r_definition_state = 2) AND NOT
((FOLDER('/Resources', DESCEND) OR FOLDER('/System')) OR
FOLDER('/System/DistributionList Templates', DESCEND) AND
o.object_name LIKE 'dmSendTo%') ORDER BY 13 ASC,7 ASC,
4 ASC,9 ASC

I don’t really have an idea either until I format it into something readable:

SELECT ALL 	'y' AS notnavigatable,
		o.r_object_type AS r_object_type,
		o.r_lock_owner AS r_lock_owner,
		o.r_object_id AS r_object_id,
		'y' AS selectable,'0' AS isfolder,
		upper(o.object_name) AS objname,
		o.owner_name AS owner_name,
		o.r_version_label AS r_version_label,
		o.i_is_reference AS i_is_reference,
		o.a_content_type AS a_content_type,
		o.object_name AS object_name,
		100 AS idunion,
		o.r_is_virtual_doc AS r_is_virtual_doc,
		'' AS navigatable,
		o.r_link_cnt AS r_link_cnt,
		o.r_full_content_size AS r_content_size
FROM 		dm_process o
WHERE 		( r_definition_state = 2)
AND 		NOT (
			(   FOLDER('/Resources', DESCEND)
			 OR FOLDER('/System')
			)
		     OR FOLDER('/System/DistributionList Templates', DESCEND)
		     AND o.object_name LIKE 'dmSendTo%'
		)
ORDER BY 13 ASC,7 ASC,4 ASC,9 ASC

It is now immediately apparent which object types or registered tables are being queried (dm_process in this case), which attributes are in the select list and what the query predicates are. I generally use tabs to space out each block (e.g. the attributes in the select list are all aligned to a tab) and to show nesting of predicates and sub-queries.

Another benefit of formatting the query is that, generally, you are not that interested in the attribute select list when tuning queries. As a rule you look first at the object types/tables and predicates involved, along with any additional clauses that affect how queries will be processed like ORDER BY and GROUP BY. By formatting the query you can immediately spot the important FROM, WHERE, ORDER BY and GROUP BY clauses.

Here’s another example of a properly formatted query with multiple object types to illustrate another point:

select 	p.retainer_root_id as retainer_root_id,
	t.r_object_id as retained_object_id,
	p.event_date as event_date,
	p.r_object_id,
	p.object_name as retention_name,
	p.r_policy_id,
	p.retention_policy_id,
	p.r_retention_status,
	p.entry_date,
	p.qualification_date,
	p.r_object_type,
	p.phase_name,
	q.object_name as policy_name,
	s.object_name as retention_policy_name
from 	dm_sysobject (all) t,
	dmc_rps_retainer p,
	dm_policy q,
	dmc_rps_retention_policy s
where 	q.r_object_id=p.r_policy_id
and 	s.r_object_id=p.retention_policy_id 
and 	p.r_object_id in (
		select 	i_retainer_id
		from 	dm_sysobject (all)
		where 	(r_object_id = '0900379780155e1b')
	)
and 	t.r_object_id  ='0900379780155e1b'
and	p.retainer_root_id is not null

Where there are multiple types or tables in the FROM clause I put them, one line for each, one after the other. Then in the predicate (where) clause I ensure that the join conditions come first followed by the other, filtering, clauses.

You can immediately see in this example that dmc_rps_retainer, dm_policy and dmc_rps_retention_policy all have join conditions but dm_sysobject (all) does not. This should raise alarm bells as if there is no join condition for a table then all the rows of the table will be joined to the rowset formed by the other table joins – this could potentially result in massive amounts of work for the database engine.

In this case there is no need to worry as we can see further down that there is a filtering condition t.r_object_id =’0900379780155e1b’ which will ensure that only a single row will be output for joining to the other tables.

It is impossible to effectively tune queries without understanding what they do and it is very difficult to understand what a query does without formatting it. With only a little practice it is easy to format even very large queries in a few minutes.

Next Page »

Blog at WordPress.com. | Theme: Pool by Borja Fernandez.
Entries and comments feeds.