I previously blogged about a very comprehensive survey-paper on semantic annotation. Besides the survey the paper also lists seven requirements for semantic annotation systems in the context of a document centred knowledge management approach and reviews them in the light of the annotation tools surveyed. Here are the seven requirements from Uren et al. (2006):

• Requirement 1—standard formats: … Using standard formats is preferred, wherever possible, because the investment in marking up resources is considerable and standardization builds in future proofing because new tools, services, etc., which were not envisaged when the original semantic annotation was performed may be developed. Compliance with standards also frees companies from the constraints of proprietary formats when choosing knowledge management software. It is the activity of the W3C in developing and promoting international standards for the SemanticWeb that has convinced us that this route is worth following in knowledge management. Two types of standard are required, standards for describing ontologies such as the Web Ontology Language OWL and standards for annotations such as the W3C’s RDF annotation schema.
  Many of the reviewed tools already use W3C standards.
• Requirement 2—user centered/collaborative design: … Annotation can potentially become a bottleneck if it is done by knowledge workers with many demands on their time. Since few organizations have the capacity to employ professional annotators, it is crucial to provide knowledge workers with easy to use interfaces that simplify the annotation process and place it in the context of their everyday work. A good approach would be a single point of entry interface, so that the environment in which users annotate documents is integrated with the one in which they create, read, share and edit them. System design also needs to facilitate collaboration between users, which is a key facet of knowledge work with experts from different fields contributing to and reusing intelligent documents. …
  … More attention needs to be paid to build in or plug-in semantic annotation facilities in commonly used packages to encourage knowledge workers to view annotation as part of the authoring process not as an afterthought, and also to supporting annotation in collaborative environments, …
• Requirement 3—ontology support (multiple ontologies and evolution): … annotation tools need to be able to support multiple ontologies. For example, in a medical context, there may be one ontology for general metadata about a patient and other technical ontologies that deal with diagnosis and treatment. … In addition, systems will have to cope with changes made to ontologies over time, such as incorporating new classes or modifying existing ones. In this case, the problem is ensuring consistency between ontologies and annotations with respect to ontology changes. …
  … Ontology maintenance, which directly affects the maintenance of annotations, is poorly supported, or not supported at all, by the current generation of tools. This perhaps reflects the intended user groups; with the assumption being that knowledge workers will use existing ontologies rather than editing or creating them. … A genuinely integrated semantic annotation environment should give the user automatic support for ontology maintenance, for example, using text mining methods to suggest new classes as they emerge in documents and spotting inconsistencies between new and existing annotations. …
• Requirement 4—support of heterogeneous document formats: … Documents will be in many different formats including word processor files, spreadsheets, graphics files and complex mixtures of different formats. This presents a technical challenge rather than a research challenge, but dealing with multiple document formats is a prerequisite for integrating annotation into existing work practices.
• Requirement 5—document evolution (document and annotation consistency): Ontologies change sometimes but some documents change many times. … What should happen to the annotations on a document when it is revised, poses both technical and application specific questions. … Annotation environments need to help knowledge workers maintain appropriate annotations as documents change.
  The survey did not discover any concerted work on these lines.
• Requirement 6—annotation storage options: The Semantic Web model assumes that annotations will be stored separately from the original document, whereas the “word processor” model assumes that comments are stored as an integral part of the document, which can be viewed or not as the reader prefers. …
  … However, separate storage of annotations has advantages for KM. … It also makes it easy to produce different views of a document for users with different roles in an organization or different access rights, thus facilitating knowledge sharing and collaboration. We therefore argue that separate storage is the better model, even when extra overheads are required to maintain links between a document and its annotations.
• Requirement 7—automation: Another aspect of easing the knowledge acquisition bottleneck is the provision of facilities for automatic mark-up of document collections to facilitate the economical annotation of large document collections. To achieve this, the integration of knowledge extraction technologies into the annotation environment is vital. …
  Language technologies present usability challenges when deployed for knowledgeworkers since most are research tools or designed for use by specialists. … In addition to the usability challenges there are also research challenges, among which we have highlighted the extraction of relations as important for semantic annotation.