The Back and Front of Application Integration

From an IT perspective, adding value under this model requires the ability to expose data, business rules and other information assets residing in disparate systems, and to connect them, rapidly and cost-effectively, into ensembles worth more than the sum of their parts. This article surveys back-end, middleware and front-end techniques for getting code and data trapped in discrete networked systems to work together.

Building on a common foundation

A basic approach to integration is to maintain data independently of business logic, in repositories where multiple applications can access it. For instance, an employee database could serve as the common link between several Human Resources applications, such as benefits sign-up, training compliance and performance tracking. Integrating applications under this scheme amounts to writing queries and updates against a shared, hopefully well documented data model. Due in large part to the success of relational database products and their associated standards (e.g., SQL, normal forms, ODBC), data integration continues to predominate as a means for leveraging assets between systems.

As a foundation layer, back-end data has some problems, however. For one thing, it presupposes a common data model across a set of applications, which can hamper developers and limit flexibility. Even within a well-defined domain such as HR, for example, a self-service application targeted at employees and a decision-support tool aimed at managers would likely demand different schemas. In addition, data-level integration offers little help connecting to existing legacy or packaged applications. Sometimes the data layers of such systems are exposed and well documented; more often talking to them requires custom connectors, with attendant cost, change management and vendor lock-in concerns.

A range of solutions exists to mitigate these and other problems with data integration. Metadata repositories, for instance, can reduce the need for a common schema by adding a layer of abstraction over individual databases. So-called ETL (Extract, Transform, Load) tools can help replicate data from one system to another, re-mapping as needed. And standards for wrapping legacy data continue to evolve, making connectors ever more productive and less brittle. As an enterprise-class solution, data integration isn't going away anytime soon.

Starting in the middle

Another approach to application integration focuses on sharing executable code rather than back-end data. This includes both traditional function libraries and APIs [Application Programming Interfaces], created to bridge software hosted on a single platform, as well as cross-platform tools such as Remote Procedure Calls (RPCs), Object Request Brokers (ORBs), and Message-Oriented Middleware (MOM). The essence of all these techniques, which vary widely in their particulars, is to expose functionality resident in one application for use by others, regardless of location or host platform.

Several factors - the ubiquity of Internet standards, the emergence in J2EE of an industry-standard interoperablity platform, and rapid acceptance of XML as a platform-neutral message format, to name three - are driving investment in this kind of integration. Solutions appear under such umbrella terms as 'middleware,' 'enterprise application integration' (EAI), infrastructure software, B2Bi and related categories. Keeping in mind that business logic tends to reside on the middle tiers of a distributed architecture, the term middleware nicely underscores the category's process orientation.

Middleware offers some important benefits relative to data-level integration. It lends itself better to object-oriented programming, exposing data and code together at a level of abstraction sufficient to potentiate reuse by a range of applications. Java programming in particular facilitates process-level integration, which is directly supported by numerous J2EE services, including remote method invocation (RMI), Java messaging service (JMS) and Java Connector Architecture (JCA).

The downside of this approach is complexity. Real-world middleware projects tend to be costly, long-lead affairs requiring significant expertise. Web Services, an emerging application assembly framework that enables XML-based messaging between remote objects, promises to streamline integration, at least for applications that (natively or via translation) speak XML.

Taking the front door

Having toured the back-end and middle-tier approaches to integration, it remains to consider whether applications might be connected the way humans use them: through their user interfaces. In fact, a simple form of front-end integration called screen scraping has been used for over a decade to extend mainframe applications in the enterprise. Leveraging the character-based displays associated with many legacy applications, screen scraping tools provide a rapid, if fragile, interface to otherwise closed systems.

As a rule, front-end integration techniques exploit some dependable constraint in the presentation layer as a bridge between systems. In the case of screen scrapers, the constraint is the character-based display or "green screen." Another type of front-end integration, HTML scraping, relies on the browser-based output of Web-based applications to open them up. Tools in this class, such as CrossWeave's Composite Application Platform, identify components in the HTML delivered by Web applications and make these components available for repurposing and integration.

For non-Web applications, emulating the user as an integration paradigm is more complex. It requires software capable of reading graphic user interface (GUI) calls, correlating them to application objects, and exposing these as programmble controls, in something approaching real-time. An innovative start-up, Anysoft, has invested significant research and development in solving this problem. Their solution allows nearly any application to be integrated with others via the presentation layer, non-intrusively and without complex infrastructure. While not as scalable as a middleware solution, this approach promises to add a new cost-benefit profile to the integration landscape in the coming months - a timely addition, given the mandate for companies to leverage existing assets without busting budgets.

Gordon Benett is a technology strategist with over 16 years experience analyzing, architecting and developing information systems. He is currently with Aberdeen Group (Boston, MA), where as a Senior Research Analyst he follows the Enterprise Java and Middleware markets. Gordon founded Intranet Journal in 1996 and remains a reader and contributing author. He welcomes your comments at gbenett@mediaone.net.

Printer Friendly Version