Most complexity does not become apparent until after a system has been deployed. Initially documentation may seem to require little more than a word processor but as product variants emerge, traceability requirements increase and repeated use of warning messages across dozens of topics becomes apparent, the simple writer-centric tool often employed for initial documentation creation is no longer adequate.
These criteria hold up under load and make trade-offs that most vendors will demonstrate against.
Structure has to be enforced, not encouraged
Many tools allow you to add structure to your content. Few tools prevent you from removing it. Unconstrained structure is prone to collapse under the pressure of a hard deadline. Thousands of topics, almost all formatted correctly, is a state that few authors even realize has been reached.
Validation at save time – not just an external process run at some later time – against a fully defined schema for complex work. If authors are working in DITA, S1000D or even a constrained subset of DocBook then this validation should be native to the platform.
Specialization without forking
Standard document types hardly ever exactly fit the requirements for a special product. You should test whether a system for managing documents also supports a constrained version of a document type (a “specialized document type”) that remains compatible with the standard version. Content model elements, even a custom set of them, should still be processed by standard transforms (e.g. In XSLT). In particular, they should still be exportable to other systems, etc. A custom element set that only your instance of the system understands is a liability.
Scalability is about relationships, not file count
There’s a big difference between storing a hundred thousand topics and managing the dependencies of a huge graph of linked topics. In this scenario, it’s essential to test how a system handles a topic referenced by 9 publications across 3 different products. Can the system tell you what will break if you make changes to a given topic?
Ask for a demonstration of where a topic is used, impact analysis before a topic is to be deleted, and link validation for the entire document set, not only for a single map. In addition, test how long a full build would take for the amount of content you plan to create, and how your team of writers would perform under the following scenario, six writers are creating content and trigger full builds at the same time. The response times that seemed OK with a small sample library will likely deteriorate dramatically with your production volume of content.
- Search that indexes element attributes and metadata, not just body text
- Branching and merging that operates on topics, not whole repositories
- Baselines that capture the exact state of a publication for audit or recall
- Bulk operations that can retag or revalidate thousands of topics without manual passes
Reuse mechanisms and their maintenance cost
The most common mistake when it comes to reuse is to pick a reuse strategy and fail to recognize the ongoing cost of that strategy. That means that typically the maximum number of reuse strategies that can be supported is three.
| Mechanism | Best suited to | Maintenance burden |
| Topic reuse via maps | Whole procedures shared across publications | Low, provided context assumptions are documented |
| Content references at element level | Warnings, prerequisites, repeated table rows | Moderate, breaks silently if source is restructured |
| Variables and keys | Product names, part numbers, version strings | Low, but requires disciplined key scope governance |
| Conditional profiling | Model variants, audience tiers, regional differences | High, grows combinatorially without attribute limits |
Governing conditions before they multiply
Conditional attributes are generally a difficult feature to get right. Therefore, restrict the set of profiling values to a fixed and small set of values. Ensure this vocabulary is reviewed on a regular basis and that any additions require sufficient justification. In addition, report which combinations of conditions are published and which result in empty output for any given scenario.
Publishing flexibility and the cost of each output
As output requirements change more frequently than your content models, a powerful documentation platform is one where adding a new deliverable is a matter of configuration as opposed to development. Teams that need this kind of configuration-driven output will find the MadCap Flare authoring platform a sensible starting point for evaluation, since target definitions and skins are treated as settings rather than code.
How do you manage stylesheets versus content? Can you regenerate a publication that was published two years ago, and re-create it in the exact form it was released in, using the exact transforms that were in place at that time? In highly regulated industries, this is not a ‘nice to have’ but evidence of a system.
Collaboration that reflects how the work is really done
Most importantly, a documentation system needs to support other roles than that of writers. We need to be able to manage source information provided by engineers, review comments by reviewers, and translated content by translators, and have legal sign off against published content in the system.
- Review in a rendered view, with comments anchored to specific elements
- Role-based permissions at branch, map, and topic level
- Locking or merge handling that prevents concurrent edits from overwriting each other
- Translation packages that export only changed segments and reimport cleanly
Judging fit before commitment
Use the toughest publication you have, not a “sample” publication that looks clean. Migrate your messiest legacy documentation and then try to publish the resulting topics in two different formats. Have a real reviewer try to work with the resulting content. The problems you find will be the ones you live with.
