Document and Records Management

SynopsisButler Group does not believe that organisations in general have a clear understanding of their obligations, under current and pending regulations, to keep electronic copies of information, such as e-mails, as well as hard copy, such as letters and faxes. This is leading to confusion as to whether information should be kept at all, and if so for how long it needs to be retained. In our opinion, the only effective way to safely retain information, in a format where it cannot be altered, is to implement a Document and Records Management (DRM) solution. Unfortunately few organisations outside of the public sector have even a basic understanding of Records Management (RM), and how this differs from Document Management (DM) - the Report explains the important differences between these apparently similar disciplines.

It is the view of Butler Group that the general ignorance surrounding DRM will cost organisations dearly as regulators get tougher on companies that fail to discover and retrieve information within the demanded timeframe.

This Report will be of interest to IT Directors and CIOs, who will be likely candidates to hold the responsibility for managing DRM strategy in an organisation.

Key FindingsKey Findings

An ignorance of the obligations under current and pending regulations to retain information will cost organisations dearly as regulators get tougher on companies that fail to discover and retrieve information.

Document and Records Management requires a change in culture from secretive to public, individual to corporate, and paper to electronic.

20% of the knowledge capital of any organisation is still in a paper format.

Paper-based records have no built-in provision for disaster recovery.

More requests for information are made for litigation than compliance.

Far too much information is retained by organisations; records should be retained that are required to run the business, or to meet statutory requirements.

Organisations need to balance the risk of disposing of information over that posed by retaining it.

In the public sector information is more likely to become a permanent record than in the private sector.

The selection of a DRM solution should be driven by the ability to address the organisation's requirements and not based on whether it conforms to TNA 2002 or other standards.

Without a DRM system knowledge workers spend up to 80% of their time looking for information.

DRM should be available to all employees and not just a few, and must be transparent to the end-user.

Management Summary

Introduction

Butler Group does not believe that organisations in general have a clear understanding of their obligations, under current and pending regulations, to retain information. This is leading to confusion as to whether information should be retained, and if so for how long it needs to be retained. In our opinion, the only effective way to safely retain information, in a format where it cannot be altered, is to implement a Document and Records Management (DRM) solution. Unfortunately few organisations outside of the public sector have even a basic understanding of Records Management (RM), and how this differs to Document Management (DM). It is the view of Butler Group that this ignorance will cost organisations dearly as regulators get tougher on companies that fail to discover and retrieve information within the requested timeframe

Business Issues

In Butler Group's opinion, much of the confusion over the difference between DM and RM arises because of the integrated DRM functions offered by vendors, which in turn means that the boundaries between the two are often blurred. Because of the ability to embed DRM into applications that users are familiar with, such as Microsoft Word, and drive many of the DRM features through automated workflow, much of the functionality is hidden from the end-user and even the organisation's management. Furthermore, because records can be automatically declared using the workflow capability, the user may not even be aware that the status of a document has changed to that of a record. We therefore believe that every organisation should have a professional Records Manager to provide oversight and control of all aspects of DRM.

The majority of private sector organisations have not implemented RM. Many will already have some form of DM, often within an Enterprise Content Management (ECM) solution, but there are still companies that have no system in place to effectively manage information. In these organisations up to 80% of the organisation's knowledge capital is stored on local hard disks, never backed-up, and unavailable to other users who could derive value from it. An absence of centralised management of information also results in duplicated information, multiple versions of the same document, and even inaccurate information, which may be sent to partners, or customers. Butler Group believes that the absence of an effective information management system can directly affect the bottom line of the organisation through lost business.

Numerous studies have shown that it can take information workers up to 80% of the time spent on a piece of information actually locating it, with the other 20% spent working on it. If the information being searched for is a record, and its retrieval is required for reference only, then this percentage can be even higher. Whilst not every person in the organisation, can be considered an information worker, this undoubtedly is a major area of inefficiency in organisations and one where demonstrable benefits can easily be seen.

For many organisations the RM policy is to simply finalise a document or other piece of information, archive it off to off-line storage (typically tape) and store the tape or other medium off-site. Retained e-mails are also archived to back-up tapes and again stored off-site. It is not until the organisation is asked to discover and retrieve information, typically for litigation, that it realises that this approach is naive and actually places the organisation at risk of a large fine for a failure to disclose information.

It is in this type of situation, as well as for compliance, that we feel RM has such a vital role to play. Many organisations fail to realise that unless information is stored in a RM system where it cannot be altered, it is difficult to prove to a regulator or a court that a piece of information has not been amended in any way. Therefore organisations must implement RM to provide this proof.

Many organisations have been put off implementing effective archiving or RM systems because they feel that the managerial overheads are too high. In Butler Group's opinion, this is an area where organisations can learn a great deal from the experience of the public sector. In many parts of the public sector there has been a position of Records Manager for many years. Traditionally, the role of the Records Manager was to file and manage the distribution of paper-based records. With the development of Electronic DRM (EDRM) applications, this role has evolved into managing electronic as well as paper documents, with the Records Manager becoming a corporate Information Manager. We believe that private sector organisations must take a leaf out of the public sector's book and appoint a dedicated person to manage information, as information is the greatest asset of an organisation and should be maintained as such.

Another prohibiting factor to the implementation of DRM is reluctance from users. Change Management is one of the trickiest areas that organisations have to overcome in order to achieve a successful implementation. DRM involves huge cultural change from secretive to public, as users need to become used to sharing information; individual-to-corporate, as information is no longer stored on local drives; and paper-to-electronic, as the organisation stores more information electronically. This is a problem that is experienced by both the private and public sectors.

Butler Group believes that many organisations considering implementing DRM are misguided in the belief that The National Archives (TNA) 2002, or another standards-approved RM solution will provide them with a superior product. The requirements of the organisation and how well the solution fits these needs should be the overriding drivers, unless an approved system is mandated as in the case of public sector bodies. The approved systems have been developed to address the requirements of specific sectors, and can be overly restrictive for general use, not because they lack functionality (which is extensive and manages all aspects of RM) but because some of the features required for general use are disabled.

Much of the information within an organisation, in both the private and public sectors, is still in a paper-format, despite the fact that we are supposedly in the era of the 'paperless office'. Storing records in a paper format carries inherent risks. Because there are normally no copies, there is no disaster recovery provision. We therefore urge organisations to implement some form of EDRM for its key information, and wherever possible to maintain all records electronically.

Technology Issues

It is the belief of Butler Group that organisations have failed to fully exploit technology to help in the management of documents and records. One of the major issues is the ability to discover and retrieve information. This may be for internal purposes, for regulators, the courts, or in the UK public sector for Subject Access Requests (SARs) under the Freedom Of Information (FOI) Act 2000. This requires the use of search and retrieval technology. For organisations implementing DRM, search techniques are normally included out-of-the-box, but for organisations that have not yet deployed DRM, searching the indexes of back-up tapes, or searching the hard disks of storage devices are not cost-effective or time-efficient search methods. We believe that it is inevitable that organisations will face increasingly large fines for a failure to disclose information when requested, through an inability to locate it. Organisations must therefore ensure that they have effective search technologies in place to provide them with the ability to discover information.

However, search techniques alone are not sufficient to discover specific documents and records from what in the future may be many petabytes of information. Effective indexing and classification of information is also required. Again the public sector can teach the private sector a great deal. Its Records Managers have been creating fileplans or classification systems for records and documents for many years. An effective classification system will make searching for information much speedier and more effective. This is one reason why Butler Group believes that organisations implementing DRM must have a dedicated Records or Information Manager, who can develop and maintain a classification system or fileplan.

To date, many DRM implementations have been departmental, or delivered only to select users. We believe that DRM must be implemented enterprise-wide, with access to specific information and information-types via access rights and permissions. This is the only effective way that knowledge capital can be shared across the organisation. It also addresses the major issue faced by a large number of organisations, which is that of information being stored on local devices, where it is never backed-up or protected.

DRM runs much wider than simply creating and editing, and retrieving documents and records. It manages the entire lifecycle of information, including its storage. Information Lifecycle Management (ILM) is therefore an important element of DRM, and we believe that organisations must implement an ILM strategy for cost-effective DRM. Although only around 10% of ILM is related to technology, this does include the media that the information is stored on at any point in its lifecycle. It is our opinion that information must be stored on the media that best reflects the age and value of an item of information, and that the ability must exist to be able to move the information - even if it is a record - back up the storage hierarchy to higher-performance disk should it, once more, need to be accessed regularly. Butler Group believes that in the future DRM will support the ability to automatically move information up and down the storage hierarchy, based on user demand, but this level of sophistication is currently not generally available.

Market Issues

There are two types of products that provide DRM functionality, ECM solutions where DRM is a sub-set of much wider information management capabilities, and niche EDRM products. Butler Group believes that there is validity in both approaches and has reflected this by reviewing both ECM vendors with DRM functionality and EDRM vendors for this Report.

As the ECM market place continues to consolidate, we believe that the larger ECM vendors will acquire most of the niche EDRM players, particularly those that are TNA 2002 and other standards approved. We have already seen the beginnings of this through the acquisition of Tower Technology by Vignette, and Valid Information Systems by Hummingbird. One of the major benefits of this approach for the ECM vendors is that they gain immediate TNA 2002 approval, which enables them to bid for UK public sector contracts. The reason that many of the major ECM vendors are only now seeking TNA 2002 approval, is because they are US-headquartered, and therefore have chosen DoD 5015.2 approval first, which is the US standard, as a priority over international standards.

It is not only the EDRM vendors that will be subject to acquisition. Many ECM vendors will also either be acquired, or will merge with other vendors to consolidate customer-bases and become more prominent players. Butler Group believes that eventually there will be around six major ECM vendors, and some of these will comprise vendors that do not currently play or predominate in this space. We expect Microsoft and Oracle to enter this arena through the acquisition of ECM or EDRM vendors. Furthermore, following the example of EMC, it is more than likely that other storage vendors will undertake vertical integration by entering this market, again through the acquisition of existing players.

We do not, however, see major risks for customers in this shakeout. Experience has shown, for example with Vignette and IBM, that the existing customers of the acquired vendors are offered security, long-term commitment, and additional functionality/options for the future.

Report Structure

Report Structure

A brief summary of each section of the Report is included below in order to help direct the reader to particular areas of interest.

Section 2 - Business Issues

This section provides the introduction to Document and Records Management (DRM) and puts the subject into context. It includes Butler Group's definitions of documents, records, Document Management (DM) and Records Management (RM), and sets the scene for the Report, examining the various business issues and drivers.

Section 3 - Technology Features

Section three of this Report looks at the various technology areas that Butler Group expects to see in DRM solutions and includes the core functionality of DRM; the lifecycle steps in a successful DRM strategy through from the creation of a document to the disposition of a record.

Section 4 - Architectures and Models

This section examines the various architecture approaches offered by vendors such as the combination of servers and repositories, the various client options, and the interface options. Also covered in this section are the different approaches to DRM from the specialist Electronic DRM (EDRM) vendors and the functionally larger offerings from the Enterprise Content Management (ECM) vendors.

Section 5 - Market Issues

Market Issues examines the various DRM implementation options, and makes recommendations as to the best way to address this daunting task. There are a number of case studies of organisations that have implemented DRM to provide guidance to organisations considering an implementation.

Section 6 - Tables

This section first presents Butler Group's Features Matrix, which allows the selected DRM solutions to be seen side-by-side in terms of features and capabilities. A number of supplementary tables are also included in this section.

Section 7 - Comparisons

This section includes comparisons of the vendors featured in this Report.

Section 8 - Technology Audits

This section contains in-depth Technology Audits for the vendors reviewed in this Report. These are as follows:

Diagonal Solutions - Wisdom

EMC Documentum - Documentum 5

Fabasoft - eGov-Suite Version 6

FileNet - FileNet Records Manager

Hummingbird - Hummingbird Enterprise Version 5.1.05

Hummingbird (Valid Information Systems) - R/KYV Version v9.1

Hyperwave - eKnowledge Suite and eRecords Suite

IBM - DB2 Document Manager, DB2 Records Manager

Interwoven - EDMS Suite

Meridio - Meridio 4.2

Open Text - LiveLink Enterprise 9.5

Stellent - Stellent Content Management Version 7.2

Tower Software - TRIM Context

Vignette - Vignette Records & Documents Release 4.4

Section 9 - Vendor Profiles

This section contains brief profiles of a number of relevant vendors and technologies not covered by a Technology Audit.