Xense Profiler for DFC now open source

May 4, 2011 at 11:44 am | Posted in D6, Performance, Xense Profiler | 4 Comments
Tags: , ,

I’m really excited to announce that from today Xense Profiler for DFC – the Documentum 6.x performance profiling tool – is now an open source project (DFCprof).

The DFCprof project is hosted on Sourceforge where you can download fully-functional binaries or build your own copy from the source code. The software is totally free – that’s free as in ‘free beer’ as well as ‘free speech’!

DFCprof can be used in a number of different ways. Most people will be interested in using it as a standalone application to process a DFC trace file and create a performance analysis report.

DFCprof basic architecture

Just download the application from sourceforge, extract the files and you are ready to go.

Alternatively you can embed the dfcprof-x.x.x.jar library into your java project and use the trace parsing facility from there. I’ll be posting more details on the DFCprof parser API in due course. I’ll also be talking about the roadmap for future DFCprof features. Feel free to drop me a line in the comments if there are particular things you would like the project to do.

Changes in Xense Profiler for DFC version 1.1

August 22, 2008 at 12:04 pm | Posted in D6, Performance, Xense Profiler | Leave a comment
Tags: , , , ,

An updated set of release notes have now been released to the website. In amongst a number of smaller cosmetic changes are some work I have been doing on making the profiler less sensitive to the specific tracing properties used to generate the trace.

One of the great virtues of the DMCL trace was the simplicity of invocation. There are only 2 different properties trace_level and trace_name with the latter being optional and defaulting to api.log. These could be set specifically in the dmcl.ini file or specified using and api call (e.g. trace,c,10,mytrace.log would start a level 10 trace to mytrace.log).

However the DFC trace has a lot of different options many of which have very significant implications for the formatting and content of the trace file. Suffice to say this makes reliable parsing of the dfc trace a challenge. To manage this complexity we took the decision to mandate the setting of certain parameters when generating the trace. Whilst this makes the development task easier it is not very user-centric. There is a risk that users will generate traces ‘incorrectly’ and then find that the profiler either doesn’t produce any results or even worse produces erroneous results.

The latest version, Xense Profiler for DFC v1.1, removes the requirement to have the dfc.tracing.include_rpcs flag set correctly; the profiler will correctly process files whatever the setting of include_rpcs used to generate the trace. There still some restrictions on the trace properties that can be used to generate the trace file – the details are in the release notes.

Future versions will aim to further remove these restrictions, in addition we will probably include a catalogue of trace formats that we can test against and at least warn the user if an incorrect trace format is being used.

Xense Profiler for DFC beta 2

August 7, 2008 at 8:56 pm | Posted in D6, Performance, Xense Profiler | 1 Comment
Tags: , , , ,

The first beta for Xense Profiler for DFC finished 31st July but we have decided to extend the program for another 3 months. If you are an existing beta group member you should have received an email containing a new licence key and a link to the updated beta (if you didn’t get the email let us know). If you haven’t already signed up for the beta then you can do so here.

When we sent out the renewal emails to the existing beta group members we also asked for some feedback. I’ll repeat the questions here as we would really like to here from anyone who has used (or tried to use) the software; any feedback helps us improve the product:

Beta Questionnaire

  1. Was it easy to download and install the software?
  2. Was the license process easy to follow and complete? How could it be improved?
  3. Did the software work first time or was there configuration that you had to perform?
  4. Was the invocation of the software and the command line syntax intuitive and easy to understand?
  5. DFC tracing has a confusing array of options and the intial beta required certain tracing parameters to be set. Was this clear or did this cause you problems? Did you resolve any problems or did you leave it as ‘requiring too much of my time’?
  6. Did the HTML reports display adequately on your browser? If not what browser and os were you using?
  7. Was the meaning and interpretation of the various reports clear? Would it be useful if the documentation could cover the theory and approach to tuning? Would motivational examples help your tuning work and the use of the software?
  8. The Xense Profiler for Documentum 5 systems includes a CSV output facility that allows the raw trace data to be converted to CSV for import into (for example) Excel. Would such a feature be useful in the Xense Profiler for DFC?
  9. If we could change or improve one thing about Xense Profiler for DFC what would it be?

Documentum 5 Profiler

May 8, 2008 at 10:06 am | Posted in Performance, Xense Profiler | Leave a comment

Xense Profiler v1.3, the Documentum 5 performance profiler, has now been officially released after an extended and successful beta period. During that time there have been no bugs or issues reported.

If you need to quickly analyse systems for performance problems Xense Profiler is the fastest and most convenient tool for the job. No need to import files into Excel for analysis; the Xense Profiler, dmclprof, analyses DMCL trace files and creates HTML-based reports that provide the information you need to diagnose system performance problems.

One of the new features of v1.3 is a Top Queries report. In previous versions you had to rely on scanning through the Query Summary report to find long-running queries; for large traces with lots of queries this could be inconvenient.

Top Queries reports the Top 20 longest-running queries. In this case longest-running means the queries taking the most time to complete. Since its inception Xense Profiler has calculated the true cost of a query. Most other approaches to performance analysis simply record the time taken to complete a DfQuery.execute() call, Xense Profiler aggregates the duration of the execute() call and all the corresponding next() calls as well. Only in this way can you be sure that you really have identified the long-running queries in your system. The documentation is now on-line with examples of the reports.

If you are interested in using Xense Profiler for your systems you should check out the information on the Xense website.

Performance Profiling for D6

April 28, 2008 at 4:36 pm | Posted in D6, Performance, Xense Profiler | Leave a comment

As many of you will be aware, whilst the traditional approach to performance tuning Documentum prior to D6 was to generate and analyse a DMCL trace this option is no longer available in D6. As part of the effort to remove reliance on native code libraries (ie dmcl40.dll or equivalent Unix shared libraries) EMC has rewritten the DMCL layer in Java.

Iin effect the API no longer exists., there are no create,c,…/get,c,…/getfile,c,…/etc calls anymore. DFC code now calls directly into the RPC layer (now rewritten in Java as opposed to the native code Netwise implementation). API-aware utilities such as iapi and dmbasic still have support for API calls but in effect this is a layer on top of the DFC code (rather than underneath it as it used to be).

The standard way to analyse performance problems in D6 is to create a DFC trace and analyse that. Not only is this format very different from the DMCL trace format but there is also substantial change and improvement over the Documentum 5 DFC trace format. There are now around 34 different tracing parameters that can be set in the dfc.properties which give you a lot of control over what is output and in what format.

This week we have uploaded the beta version of the Xense Profiler for DFC. Xense Profiler for DFC is a port of the Xense Profiler, a Documentum 5 DMCL trace analyser. Xense Profiler for DFC is designed to work with D6 DFC trace files and produces performance profiling reports that allow you to identify the causes of D6 performance problems. Xense Profiler for DFC beta is available for download and is free for the duration of the beta period (Until 31 July 2008). If you are already working on D6 products I would urge you to try it out; as an incentive all users who register for the download will receive a substantial discount off the list price when the production version is released.

Update 7 August 2008: We have extended the beta period for another 3 months

Xense Profiler v1.3 beta

December 19, 2007 at 8:52 pm | Posted in Performance, Xense Profiler | Leave a comment

Xense Profiler v1.3 beta is now on the Xense website for download by existing customers. Prospective customers can purchase Xense Profiler from the Xense website or via our reseller Dell ASAP.

There are a number of changes in 1.3 but the key ones are:

  • A new Top Queries section on the summary report. This lists the 10 ten longest running queries (using the true query duration).
  • Performance improvements especially for parsing dmcl 5.3 traces

The Top Queries report is a great addition to the summary report. The Query Summary is excellent for getting an understanding of all the queries being issued by a Documentum application; it allows you to spot patterns in the queries being run. However when your application is suffering from 1 or 2 slow-running queries the Top Queries report allows you to quickly home in on the culprit.

The performance improvements are much needed when processing 5.3 traces. If you examine 5.3 dmcl trace files you will find there is a lot of extra debug type information whenever a collection object is transferred from the Content Server to the client. A bug in Xense Profiler v1.2 caused excessive CPU usage whenever the new 5.3 trace lines were being processed.

Momentum Monaco

November 8, 2007 at 3:58 pm | Posted in D6, Momentum, Performance, Troubleshooting, Xense Profiler | Leave a comment

I have been at the Momentum Conference in Monaco all week seeing all the exciting new stuff in D6 (and beyond) and I also presented on Performance Tuning in D6. I have uploaded the presentation here: Momentum 2007 Monaco – Inside EMC Documentum.

The presentation talks about the implications of removing the native code DMCL from DFC. There is no longer a dmcl trace facility which has been the cornerstone of performance tuning Documentum in the past. I suggest that the DFC can be used in a similar way. I then go on to discuss how a software tool could analyse the trace and I suggest there are 3 useful views of the performance data:

  • Top long running DFC calls
  • Top long running queries
  • Call profile

Update: Xense now has a beta version of a tool that does this, Xense Profiler for DFC.

I demonstrate 4 performance tuning example scenarios and show how the different views can be used to interpret the trace. Each of these example scenarios illustrate a performance problem pattern.

I should point out something: One of the things I say in the presentation is that there are not yet any performance tuning analysis scripts for D6 available from EMC. A subsequent discussion with Chase Harris revealed that there are some new scripts and they should be generally available. I’ll post some information about how to get hold of them when I find out.

Update: The scripts are here

Performance Tuning D6

October 17, 2007 at 7:56 am | Posted in D6, Momentum, Performance, Xense Profiler | Leave a comment

One of the big changes in Documentum 6 is that DFC is no longer reliant on DMCL to talk to the Content Server. In versions of DFC prior to D6, native code DMCL (a Windows DLL or a Unix shared library) was bound into the DFC process space and DFC would call DMCL functions via JNI. DMCL in turn would execute RPC calls to the Content Server functions. In D6 this whole function chain is now implemented in Java effectively replacing native-code DMCL with Java equivalents.

There are some significant consequences of this change. DMCL.ini is no longer used to set set client parameters such as docbroker host and port, etc. There are now equivalent dfc.properties keys. In fact EMC has taken the opportunity to rationalise the naming of DFC keys thoughout. This is documented in a number of places but the most useful is the Documentum 6 System Migration Guide.

The other big change is that there is no DMCL trace facility. This has a big effect on the performance tuning methodology, which for a decade has usually started with running a DMCL trace. The clear replacement for this is the DFC trace which has undergone some changes since version 5.3. I will be presenting at Momentum on the topic of DFC trace and its use in performance tuning D6. If you are currently implementing D6 or will be soon then you will want to get to grips with the significant change in approach. The talk will include some of the reports generated from the Xense Profiler for D6 which is currently under development.

Update 26 April 2006: The beta for Xense Profiler for DFC is now available for download free of charge.

Xense Profiler Offer

October 10, 2007 at 6:48 am | Posted in D6, Performance, Xense Profiler | Leave a comment

If you haven’t tried out the Xense Profiler, the Documentum Performance Profiling tool developed by my company, you should have a look at our new evaluation offer.

Anyone supplying a trace for evaluation before the end of October will qualify for a 50% discount. This prices the Xense Profiler v1.2 at just $250. You will get a free upgrade to v1.3 when it is released (beginning of November) and also a free version of the Xense Profiler for D6, the Documentum 6 DFC trace profiler when released.

When you send us a trace we put it through the Xense Profiler and send you back a copy of the report. I’ll also put in a short analysis note outlining the key points.

Content Server 6 Server Tracing

September 5, 2007 at 10:37 am | Posted in D6, Troubleshooting, Xense Profiler | 3 Comments

Loads of D6 documentation is appearing on Powerlink. I’m going through the Content Server Admin Guide over the next few days and I’ll pick out some of the stuff that looks interesting. The first thing that caught my eye was that Tracing and Logging has its own chapter. Previously this was all lumped into the Tools chapter. A very timely revision.

2 important changes here, first sqltrace is now on by default. This means that all the SQL being generated from a DQL command (very often there is a 1-to-many relationship between DQL and SQL) is output to the content server session log. I think this is a great improvement from the troubleshooting and performance tuning aspect.

Secondly there is a new server tracing option called rpctrace. The documentation states ‘Turns on tracing of RPC calls’. I will be having a look at the tracing format and the information being dumped by this trace option as again it is potentially very useful.

Next Page »

Blog at WordPress.com.
Entries and comments feeds.