Unstructured Information Management Architecture SDK

UIMA processing occurs through a series of modules called analysis engines. The result of analysis is an assignment of semantics to the elements of unstructured data, for example, the indication that the phrase "Washington" refers to a person's name or that it refers to a place. UIMA supports the rendering of these results in conventional structures (for example, relational databases or search engine indices), where the content of the original unstructured information may efficiently be accessed according to its inferred semantics.

UIMA is specifically designed to support the developer in the creation, integration, deployment, and sharing of components across platforms and among disperse teams with different skills working to develop advanced analytics.

The UIMA Software Development Kit (SDK) is a software system that includes a run-time framework, APIs, and tools for implementing, composing, packaging, and deploying UIMA components. It comes with a semantic search engine for indexing and querying over the results of analysis.

The UIMA run-time framework allows developers to plug in their components and applications and run them on different platforms and according to different deployment options that range from tightly-coupled (running in the same process space) to loosely-coupled (distributed across different processes or machines for greater scale, flexibility, and recoverability).

Annotations are not limited to text. A label may annotate a region of an image or a segment of audio. The same concepts apply.

• the subject of analysis (the artifact being analyzed, such as the document)
• the analysis results or metadata (such as annotations, parse trees, relations, entities, etc.)
• indices to the analysis results
• the type system (a schema for the analysis results).

UIMA defines another type of component, the CAS Consumer, which is intended to maintain state across documents and is typically associated with some resource such as a database or search engine that aggregates analysis results across an entire collection.

A Collection Processing Engine (CPE) is a component that the UIMA developer creates by specifying a CPE descriptor. A CPE descriptor points to a series of UIMA components, including a Collection Reader, CAS Initializer, Analysis Engine(s), and CAS Consumers. These components organized in a particular flow define a collection analysis job that acquires documents from a source collection, initializes CASs with document content, performs document analysis, and then produces collection level results (for example, search engine index, database, etc). The CPM is the execution engine for a CPE.

UIMA has advanced that state of the art along a number of dimensions including support for distributed deployments in different middleware environments; easy framework embedding in different software product platforms (key for commercial applications); broader architectural converge with its collection processing architecture; support for multiple-modalities; support for efficient integration across programming languages; support for a modern software engineering discipline calling out different roles in the use of UIMA to develop applications; the extensive use of descriptive component metadata to support development tools; and component discovery and composition. (Please note that not all these features are available in this release of the SDK.)

<casProcessor deployment="remote" name="MyCustomAnnotator">
	<descriptor>
		<include href="/home/esadmin/config/col1.parserdriver/specifiers/
		EsSocketService.xml"/>
	</descriptor>
	<filter/>
	<errorHandling>
		<errorRateThreshold action="continue" value="0/100"/>
		<maxConsecutiveRestarts action="terminate" value="3"/>
		<timeout max="30000"/>
	</errorHandling>
	<checkpoint batch="1"/>
	<deploymentParameters>
		<parameter name="transport" type="string"
			value="com.ibm.es.control.casprocessor.server.CasProcessorSocketTransport"/>
	</deploymentParameters>
</casProcessor>

The time-out value is specified in milliseconds in the error handling section of the casProcessor. If the annotator does not return earlier, increase this time-out value in order to trigger a time-out event. After increasing the time-out value for the custom annotator, it is also necessary to increase the time-out value for the CPM output queue. The necessary setting is also in the EsCpeDescriptor.xml at the end of the file. The tag is called

<outputQueue dequeueTimeout="100000" .../>

Increase this time-out value by the same factor used for the custom annotator.

	<casProcessor deployment="remote" name="MyCustomAnnotator">
		<descriptor>
			<include href="/home/esadmin/config/col1.parserdriver/specifiers/
			EsSocketService.xml"/>
		</descriptor>
		<filter/>
		<errorHandling>
			<errorRateThreshold action="continue" value="0/100"/>
			<maxConsecutiveRestarts action="terminate" value="3"/>
			<timeout max="30000"/>
		</errorHandling>
		<checkpoint batch="1"/>
		<deploymentParameters>
			<parameter name="transport" type="string"
				value="com.ibm.es.control.casprocessor.server.CasProcessorSocketTransport"/>
		</deploymentParameters>
	</casProcessor>

The timeout value is specified in milliseconds in the error handling section of the casProcessor. If the annotator does not return earlier, increase this timeout value in order to trigger a timeout event. After increasing the timeout value for the custom annotator, it is also necessary to increase the timeout value for the CPM output queue. The necessary setting is also in the EsCpeDescriptor.xml at the end of the file. The tag is called

<outputQueue dequeueTimeout="100000" .../>

Increase this timeout value by the same factor used for the custom annotator.

NodeRoot/master_config/colID_config.ini

Within the file, search for an expression such as

sessionN.type=casprocessor

sessionN.max_heap=size in MB

OmniFind must be restarted so the changes become effective.

The default heap size is set to 200 MB. Be careful with increasing that heap size. For additional help, see the memory recommendations in the OmniFind installation guide.

OmniFind log level: UIMA log level
Error: Error
Warning: Warning
Informational: Info
not mapped: Config , Fine, Finer, Finest

Note that the mapping for Error and Warning messages can not be changed. By default, only the custom annotator log message with the levels Info, Warning, and Error are written to the log file. This default behavior can be replaced with a special log-level mapping for log levels below Info, as follows:

1. Modify the tokenizer.properties config file in the following directory:
   EsNodeRoot/master_config/parserservice/
2. Inside this file is a level configuration setting, such as
   trevi.tokenizer.jedii.InformationalLevelMapping=Info
3. In order to see more than UIMA annotator Info messages in the log file, replace this log level value with the desired UIMA log level. For example, use the following in order to see all UIMA annotator log messages in the OmniFind audit log:
   trevi.tokenizer.jedii.InformationalLevelMapping=Finest

1. If you enable native XML search on the parse panel of your collection, OmniFind will create an Annotation of type com.ibm.es.tt.MarkupTag for each XML tag found in a document. This annotation contains all the information of the original XML tag, namely, its attributes and their content. Moreover, OmniFind will automatically index these XML tags under the name in which they appear in the XML file, so you can use them for semantic searching right away. Your annotators could access these annotations in their processing instead of relying on the XML tags. They would need to iterate over com.ibm.es.tt.MarkupTag and look at the name feature of the annotation in order to find out which XML tag it represented originally.
2. You can specify a so-called "XML to CAS" mapping file. In this file, you specify which XML tags should be mapped to which CAS types. OmniFind will automatically create annotations for these XML tags. This would make it even easier for your annotators to access certain XML tags than in Option 1. For example, if one annotator is interested only in content within <technicianComments>, you could specify a mapping from this tag to a type com.yourco.TechnicianComment. Then your annotator need iterate only over annotations of this type. In the case of "XML to CAS" mapping, OmniFind doesn't index the XML tags automatically. If you still want to search for, say, <technicianComments>, you have two options:
   • Additionally enable native XML search.
   • In your CAS2Index mapping, add a rule that maps com.yourco.TechnicianComment to the span technicianComment.

• Changes to the JAR file structure (some classes have been repackaged).
• Removal of implementation for methods that were deprecated in UIMA 1.0.
• Removal of Eclipse 2 version of Component Descriptor Editor (please use Eclipse 3.0 or above for running this tool).

Release 1.2

• CAS Consumers can now be included within Aggregate Analysis Engines.
• Collection Processing Engines can now specify overriding parameter values when operating in Integrated mode.
• New metadata elements in the component descriptors specify whether the components can be multiply deployed and whether they modify the CAS.
• The alphaWorks version includes an initial version of the C++ enablement layer, permitting existing analytic components written in C++ to be integrated into the UIMA SDK. It supplies JNI (Java Native Interface) code to invoke and pass arguments to UIMA components written in C++, and it provides a C++ library and framework that gives these components access to the CAS. This layer is available for Windows® and Linux®. Some documentation is provided through a QuickStart introduction, and there is detailed documentation for the required C++ APIs.

Release 1.1

• This version is incorporated into IBM's enterprise search solution, WebSphere Information Integrator OmniFind Edition, allowing search to be augmented using UIMA analytics.
• Support for multiple Subjects of Analysis has been added - this is documented in a new chapter in the UIMA SDK User's Guide and Reference.
• The Component Descriptor Editor has been greatly enhanced for Eclipse 3, allowing you to edit most UIMA descriptor files (the main exception being the Collection Processing Engine descriptors).
• A new GUI-based tool allows interactive semantic search querying.
• The Collection Processing Manager has been enhanced, and the SDK now includes examples and documentation describing how to use it.

It will be some time before the Apache UIMA Version 2 code is ready for distribution. Earlier versions and the current Version 2 beta will remain available here on alphaWorks. Some user adaptation is expected to be needed when switching to the Apache UIMA project -- for example, the name spaces are updated with org.apache prefixes (in place of com.ibm prefixes).

OASIS has established a technical committee to work on the standardization of UIMA; participation is open to interested parties who are members of OASIS.