Under Pressure: Why Many Businesses Will Need To Master Extreme Transaction Processing

By Hugo Moreno

December 18, 2016

Until recently, only certain companies, such as high-frequency trading firms or digitally native companies like Facebook and Google, built their business models around extreme transaction processing – the ability to process at least two million transactions per hour. Today, however, enterprises across industries must figure out how to ingest and process extraordinary numbers of transactions from an expanding variety of sources concurrently, quickly and reliably, with no tolerance for latency, loss, inconsistency or failure.

A recent Forbes Insights briefing report, “Doing Business In-the-Moment: Transforming Transaction Processing for the Digital Economy,” examines this new competitive necessity.

In many ways, digital business innovation depends on transactions that are measured in milliseconds or microseconds – millions of them per hour. Companies ranging from industrial giants like GE to startups like Ripple Labs – which sells services based on blockchain technology, the distributed ledger system that underpins Bitcoin transactions – require near-real-time processing of billions of transactions to deliver their digital products and services.

And today’s transactions go beyond traditional orders and payments. Even payments have expanded: They now include machine-to-machine transactions, micropayments, peer-to-peer money transfers, pay-per-second products and services, mobile apps and purchases through TV set-top boxes.

To compete in the digital economy, most enterprises will need to transform their transaction processing systems, rethinking their transaction processing architecture and databases for extreme speed and extremely diverse workloads. No single technology solution will fit every organization or business case, and the evolution to extreme transaction processing is as much a change management challenge as a technology one. Companies will have to establish the value of extreme transaction processing for their businesses to develop the right processes and systems to meet their particular high-volume processing requirements.

The digital economy is already generating data at an exponentially increasing rate. The amount of information that organizations must process will grow dramatically, driven in part by the increased volume of transactions that will be performed not just by humans but by machines, as well. Leaving aside the Internet of Things, each of the world’s smartphones has multiple sensors, including a magnetometer, barometer, thermometer, gyroscope, proximity sensor, accelerometer and light sensor, all generating data and, potentially, transactions.

But capturing enormous amounts of concurrent transactions in large and rapidly growing databases with full data consistency, high availability, instant failover and zero data loss takes new technology, people and processes that most large enterprises have not employed before. Doing so presents significant challenges for the average large enterprise, whose existing IT infrastructure is built to process traditional transactions such as orders, payments, enrollment, account creation or customer cases from centralized applications in batches.

A successful transformation depends on understanding the business problems extreme transaction processing would solve, and then putting the appropriate technology in place. Here are five steps:

1. Define which transactions have value. A sanitation company, for example, may save half a million dollars annually by installing $150,000 worth of IoT devices in garbage bins and dumpsters to monitor when they need to be emptied. But in many cases, the business outcomes from extreme transaction processing – and therefore its value – may not be instantly clear.

2. Start small, and experiment. Once the potential business case for extreme transaction processing is clear, companies should begin to experiment with new transaction processing technologies.

3. Consider the cloud. Because companies can pay only for the computing they use, the public cloud offers flexibility that supports the shifting workload levels and volumes of data inherent in extreme transaction processing.

4. Examine your business processes, as well as the IT infrastructure, databases and applications that support them. Pilot projects may point to ways to simplify and streamline.

5. Choose your solutions. Ultimately, different businesses will approach extreme transaction processing in unique ways. But the agenda is clear: It’s time to figure out what business value new data-producing transactions can deliver, and then put in place the systems and processes to capture it.


This article was written by Hugo Moreno from Forbes and was legally licensed through the NewsCred publisher network.