What is Document Intelligence?

By George Harpur, Co-founder and CEO at Aluma

In an era where digital transformation has evolved from an aspiration to a requirement for most organizations to remain competitive, companies everywhere are on a quest to maximize automation of business-centric processes. But despite billions invested in digital working technologies, the mission to improve automation continues to be challenged by unstructured content.

While many organizations have integrated some degree of automation across basic business processes, traditional data capture technologies are limited in scope and scale. In an unlimited, high-volume document universe, businesses need to adopt intelligent, mission-critical automation strategies to see significant ROI with the fastest time-to-value. This means investing in high-performing, intelligent document processing that works in combination with AI, the modern technology that Forrester refers to as “Document Intelligence.”

The Need to Rethink Traditional Document Processing

The majority of business processes today–from customer-facing processes to back-office operations, or compliance workflows, are hampered by high volumes of information in unrigid formats. These can be formatted documents, PDFs, images, or emails–otherwise known as unstructured data. A recent AIIM survey showed that nearly 40% of companies felt that getting control of unstructured information in their customer-facing processes was their number one priority to enrich the overall experience of their customers, while over 50% want to gain control of unstructured information internally to improve productivity and drive automation throughout the organization.

To automate manual workflows, organizations must ingest, classify, and extract unstructured data from the source–financial documents, contracts, digital assets, images, etc.–and turn that information into actionable data insights for further processing.

While viable document processing technology has been available for decades, organizations still struggle to conquer their unstructured data. So what’s getting in the way?

The Old Way

Traditional data capture technologies like optical character recognition (OCR) operate off-line in isolation. These applications tend to be standalone, and data capture occurs asynchronously before uploading into the business process. While these technologies can be highly performant once configured, deploying these systems is no easy feat, and the slightest modification requires specialist intervention. Not to mention, integration with other business systems and applications can be highly limited.

Though it may be tempting to deploy a pre-packaged document processing tool, the often monolithic, heavyweight capture platforms available not only break the bank but are highly inflexible in terms of practical application. To the other extreme, organizations may be inclined to source different automation components individually as needed, but as the tech stack starts to pile up, this can become technically and commercially hard to pull off.

The New Way: Intelligent Document Processing

By moving data capture from off-line to in-line, the automation technology becomes an integral part of the business flow rather than an isolated flow of its own, blurring the boundary between the technology and the complete business process. Intelligent Document Processing (IDP) seeks to do just this. Defined as a set of technologies that can be used to understand and turn unstructured and semi-structured data into a structured format, when enhanced with AI capabilities, IDP works to capture, classify, and extract the most difficult-to-automate, unstructured data.

The technology scans the unstructured document for any relevant embedded information before appropriately processing or enriching it in the form of highlights or bookmarks.

Ideally, the technology needs to successfully coordinate and manage the control flow, queuing, scaling, monitoring, logging, and tracking. With so many moving parts, we’re left with a broad scope for AI to automate what are otherwise tedious manual tasks.

IDP + AI = Document Intelligence

This is where document intelligence capabilities come into play. Document intelligence within an intelligent automation platform is essentially “prebuilt intelligent document processing capabilities that make AI accessible to non-technical business users” (Forrester).

Document intelligence combines intelligent document processing and AI to convert unstructured data locked in documents into data insights. Using a powerful combination of data management, cognitive capture, artificial intelligence, and machine learning (ML), document intelligence helps organizations classify documents and extract information to automate actions and improve productivity.

So, what exactly is needed to enable this in an efficient and scalable way?

For IDP to deliver on its promise, three key components are required:

Fast Configuration

It’s essential to have a multi-tiered approach where everything is pre-configured, and the system can learn on its own where possible. If customization is needed, it should only be simple tailoring of existing components.

Ease of Deployment and Integration

An IDP system should be easy to deploy and seamlessly integrate with existing business systems to enhance existing processes, rather than become a process in and of itself.

Adaptable ML

Adaptability is critical to allow for new document types or variants. Machine learning does this best, but not all ML can learn on the fly. It’s important to leave space for human ingenuity if the technology can’t deliver on a particular scenario. Hybrid systems that utilize ML but combine this with manual configuration as needed will deliver the greatest results.

In sum, document intelligence requires a set of technologies that work together, can be individually selected and scaled, and are easily integrated into any existing business system while flexible enough to work in tandem with other technologies when necessary.

Bringing the Best of Both Worlds

The best-of-both-worlds approach uses a flexible combination of these different components from a single source, providing a set of containerized microservices under a unified REST API. At Aluma, our technology works at the interception of these core components to deliver IDP as a set of cloud-native microservices wrapped into one unified platform and accessible through a common interface.

Aluma can be used to intercept and enhance documents in various ways throughout their processing journey. For example, the technology can enhance standard office scanning devices by intercepting the scanned image and responding with suggestions for filing based on a combination of OCR and machine learning.

For business processes like accounts payable or invoice processing, Aluma works by embedding intelligence into existing applications such as Gmail to analyze the contents and attachments and provide the service with a suitable reply for processing or filing. The technology can also enhance an existing repository or document archive to improve search and retrieval, improve compliance, simplify document retention, or streamline records navigation.

The Bigger Picture

Today, organizations face an increasingly pressing need to rethink the traditional approach to document processing and automation. Modern automation technologies can be effective on their own, but using any technology in isolation has its limitations in time. By leveraging these technologies in combination with AI and machine learning, we can revisit existing systems with new eyes and explore uncharted areas of the document universe.

With a high-performing intelligent document processing platform that works in combination with AI, organizations can begin to improve process automation for a wide range of use cases, streamlining operations and boosting workforce productivity while minimizing cost and business disruption.