Why Data-First Applications Will Come To Rule Enterprise Software

  • Peter Wagner
  • Software

This is the original version of a blog post that recently appeared on VentureBeat. A deeper dive into the subject can be found on our "Perspectives" section here.

We are on the cusp of an upheaval in software applications for businesses that will put billions of dollars of IT spending up for grabs. In the old world, applications concentrated on driving corporate efficiency through workflow, while data and analytics took a back seat. In the emerging one, data’s in the driver’s seat and a new generation of “data-first” applications will give companies that use them a distinct advantage over competitors still using the prior generation of services.

This revolution is in its early days, but data-first services are starting to emerge in many of the major enterprise-application categories. Companies such as Vlocity in customer relationship management (CRM), Moogsoft in IT operations and Kanjoya in human resources are amongst startups driving the new, data-first approach.

The original enterprise software paradigm spawned huge businesses such as SAP and large parts of Oracle, and their legacy services remain a potent force. By mapping out key business workflows, writing software to codify them, and then repeating this play across a wide range of processes, they established the pursuit of efficiency as the main driver of value creation. The subsequent SaaS revolution improved the software delivery and distribution model massively, but it also deflected attention from functional innovation that would have delivered even more value. As one top SaaS CEO explained to me recently: “The cloud idea turned out to be so big that we never got to the other ideas in our plan”.

Many of those neglected ideas had to do with data. Data-first applications differ from workflow-first ones in several respects. Architecturally speaking, they are built around a scalable data-centric core that is highly flexible with regard to data type, structure and the nature of the processing to be done on the data. This represents an inversion compared to prior architectures which led with business logic.

Unlike their predecessors, the new services also rely heavily on embedded algorithms. High-frequency trading, consumer fraud detection and ad targeting were early examples of this tectonic shift. These involved an incredibly high scale and velocity of interactions, which meant no human could be in the loop. At the same time, a glitch wouldn’t bankrupt a company, kill a patient or cause any number of other catastrophic outcomes. In processes where the stakes are much higher, data-first applications are likely to be deployed in support of skilled operators and analysts wherever good data sets can be put to use (see chart).

What really distinguishes data-first applications, however, is the virtuous data cycle they make possible. The data they generate are used to power additional, domain-specific applications, driving additional insights. This raises the hugely exciting prospect of a virtual breeder reactor of business-process optimization and insight generation. The cycle isn’t new: consumer web companies such as Google and Facebook have been running this play for years. What is new is that this phenomenon is now invading numerous business categories.

Many early examples have emerged in sales and marketing, where new incremental revenue generated makes it relatively easy to demonstrate a swift return on investment. The data-first model will have most value in industry-specific applications. Veeva is the canonical example of this. The company built its footprint—and its data set—with a standard CRM application for life sciences, subsequently rolling out other data-first services such as Veeva Network and Veeva OpenData in the healthcare arena. Veterans from Veeva and CRM pioneer Siebel Systems have now teamed up at Vlocity to target other verticals.

With lots of data, complex operations and highly technical users, IT is a natural place for data-first applications. The incident-management application of Moogsoft, a Wing portfolio company, consumes data from various IT systems, applications and even external sources; analyzes the data using algorithms; and delivers an intelligent view of service-affecting situations in real time. It also creates a virtuous data cycle by capturing information about how incidents are resolved to build an historical data set of key people, symptoms and cures that is then used to derive recommendations for action in future situations.

Cyber security is another area where the new generation of applications is taking off. Today’s Security Information and Event Management products are easily overwhelmed by the volume and diversity of data streaming towards them. To tackle this challenge, Securonix, Fortscale and Exabeam have developed services that ingest numerous data streams and employ big data analytics to identify anomalous behaviors and create measures of potential risk.

Even in HR, which has far less data and far fewer technical users, new entrants are championing a data-first approach. They include HiQ, which uses data science and public information to identify employee flight risks, and Kanjoya, which uses the data generated by a social network that it created, called the “Experience Project”, to train algorithms for emotional analysis of free-text employee survey responses.

These and other data-first attackers will initially appear with focused use cases, and will integrate with incumbent systems for legacy data access, as new data collectors, and as analytical coprocessors. The first impression is all very complementary. However, this is likely to be just the first step in a wholesale transformation that will be every bit as big as the SaaS wave before it. The smarter incumbents are already trying to respond, but this could well be an even more difficult transition for them than the leap to the cloud. Data-first alternatives are already capturing beachheads that will allow them some day to topple empires.