Are you caught up with the real-time (r)evolution?

This post originally appeared on InfoWorld on December 1, 2017.

Are you caught up with the real-time (r)evolution?

Real-time analytics have been around since well before computers. Here's where the evolution of real time is headed in 2018

People often think of “real time” as a new concept, but the need for real-time notification and decisioning has been around forever. Gartner defines real-time analytics as “the discipline that applies logic and mathematics to data to provide insights for making better decisions quickly.” This definition has always held true, it is just the meaning and measurement of real time that has evolved. As we approach the end of 2017, it’s a good time to look at the evolution of real-time decision making and ask yourself—are you caught up?

Real time B.C. (before computers)

Before there were computers, people were finding ways to use real-time information. For instance, in 500 BC, Philippides ran 25 miles from Marathon to Athens, Greece, to alert the magistrates of the victory at the Battle at Marathon, and in 200 BC, smoke signals were used on the Great Wall of China to warn those further down the wall of enemy size and attack direction.

Fast-forward to 1775 when Paul Revere developed a real-time alerting system using lanterns. We all know the famous phrase “one if by land, two if by sea,” and after initiating this real-time alert, he rode on horseback to deliver the information to towns around Boston along with 40 other horseback riders to spread the word throughout eastern Massachusetts.

Real-time A.D. (after digital)

With the invention of computers, we upgraded from “human speed” to “machine speed.” For example, telephone call routing was initially executed by human switchboard operators, but by 1983, computers automated this process, enabling real-time voice communication for almost everyone.

People also began interacting with real-time systems, typically through an intermediary. The stock market is a good example—if you wanted to buy or sell securities, you had to make a phone call to your stockbroker who would execute the trade on your behalf using the real-time stock exchange trading software.

From an organizational standpoint, computers enabled companies to gather transactions into a database and produce monthly, weekly or even daily reports. At this early stage, nightly batch processing of data was generally viewed as “real-time data.”

Today in real time

Every day most of us interact with real-time systems and execute real-time decisions, possibly without even realizing it. For instance, going back to our stock example, trades can be executed in seconds by the consumer without a broker.

Take credit card fraud as another example. What was previously fraud detection has evolved into fraud prevention thanks to advancements in real-time technology, and preventing the unauthorized transaction from being processed can save corporations millions of dollars.

Smartphones have helped advance the capabilities of real-time processing thanks to the personalized data they can ingest. Tracking the location of consumers in real time allows companies to deliver personalized offers based on what they are close to. Mobile traffic applications like Waze provide real-time incident alerts and traffic updates to redirect drivers to the fastest route.

Tomorrow in real time

We are witnessing the commoditization of real-time information, alerts, and decisions, and these capabilities will proliferate across all application domains, including online gaming, casino gaming, financial compliance and risk, fraud prevention, digital ad technology, telecom policy and billing, and more.

As we look ahead to 2018, we’ll see real-time advancements in the following areas:

• Self-driving cars will process data from countless sensors and ultimately get passengers to their destinations safely and efficiently, saving thousands of lives a year in the process.
• Real-time threat detection and remedy for data security breaches and DDoS attacks will alert consumers as they occur.
• Wearable health sensors (they already exist in your smartwatch) will not only monitor health statistics, but will also combine that telemetry with personal medical history to know when and how to notify doctors if necessary.
• Augmented reality will become part of our everyday lives. When traveling in a foreign country, AR on cell phones and tablets will provide “live” translations of street signs and shops and integrate reviews, recommendations and advertisements on whatever signposts or buildings you scan.

The value of real time is the ability to make decisions as soon as possible based on the most readily available information. Information that used to take days to receive can now be ingested and analyzed in fractions of a second, and 2018 will only cut that time down even more, empowering companies to leverage real-time information to increase corporate profit, protect resources and even save lives.

 

What real-time application pattern works for you?

This post originally appeared on InfoWorld on September 27, 2017.

What real-time application pattern works for you?

3 common real-time application patterns that require a real-time decision

 

At first glance, building a real-time application may sound like a daunting proposition, one that involves technical challenges as well as a significant financial investment, especially when you have an application goal of responding within a fraction of a second. But advances in hardware, networking, and software—both commercial as well as open source—make building real-time applications today very achievable. So what do these real-time applications look like?

This article presents three common real-time application patterns that require a real-time decision, meaning a response returned or transaction executed based on real-time input. To determine which pattern to apply to your application, you must first define your real-time objective. Ask yourself: How fast does the application need to respond?

Each application pattern addresses a particular level of real-time response: sub-millisecond, milliseconds, or 100 milliseconds and greater.

Pattern 1: Embedded applications—delivering responses in sub-milliseconds

To achieve sub-millisecond response, you need to eliminate any server-side networking and embed your application onto a computer or hardware appliance. This is the bleeding edge of real-time processing for more specialized applications that are not very common. This pattern is relevant for areas such as high frequency trading applications, nuclear power plant systems and signal processing and sensor applications.

Delivering sub-millisecond responses involves low-level programming, often at the kernel level. Standard kernels, operating systems, and device drivers can add unwanted processing overhead resulting in extra latency. Applications that care about every microsecond or nanosecond, every clock cycle, should seek to eliminate this overhead and code directly on the hardware. Alternatively, if you can withstand some additional latency, you can forgo writing low-level code and build and run your application directly on the operating system, embedding a data store such as SQLite, if needed.

Pattern 2: High speed OLTP—delivering responses in milliseconds

This is the classic client-server OLTP application architecture where a client application talks to a server-side application and database. These applications are very common —  you have likely interacted with them several times today without even realizing it. These applications detect credit card fraud, compute personalized webpages, and deliver optimized digital ads. For instance, when you use your iPhone or Android phone to make a call, run an app, or access the internet, several decisions (transactions) must be made by the telco provider before your action is allowed to occur: Is your account valid? Do you have enough quota (voice or data)? What policy should apply to the action (throttling etc.)? And each transaction must respond in milliseconds.

Optimizing network performance between the client application and the server allows for low-latency responses for high-speed OLTP application patterns. Low-cost gigabit ethernet (GigE) and relatively low-cost 10GigE networking is readily available to most application developers. Network performance can be further optimized by keeping the application on the same network switch or rack as the server or minimally on the same LAN. In other words, keep the client and server in close proximity. Within the server, the application and database usually minimize blocking disk I/O, either by avoiding it completely, by applying sequential I/O, or by using advanced storage such as SSDs or the newly emerging non-volatile RAM.

One additional point worth noting is that with next generation in-memory data stores and caches, it is even possible to achieve low single-digit millisecond latency with highly available clustered data stores; that is, databases and systems spanning multiple nodes or processes.  Today, many shared-nothing, in-memory databases, data grids, and NoSQL stores offer highly available data stores with predictable low latency (often single-digit millisecond) response times.

Pattern 3: Streaming fast data pipelines—delivering responses in seconds

A fast data pipeline, historically rooted in complex event processing (CEP) applications, is becoming a more broadly deployed real-time application pattern today. In this application pattern, a never-ending stream of immutable events is being ingested with real-time analytics applied.

Typical applications have a queuing or streaming system that delivers events, ultimately feeding the data lake, managed by Hadoop, Spark, or a data warehouse. Before arriving at the historical archive, the event stream is processed by a fast data store or computational engine. It is the role of this engine to aggregate, dedupe, and compute real-time analytics on incoming events and generate real-time alerts or decisions as required. The analytics are often displayed on a dashboard, and alerts or decisions are generated. A person or business process reacts to the alert, in human speed. A few seconds is often enough time to ensure any late data has arrived to inform the decision.

In this pattern, data flows in one direction. This real-time engine often holds a predetermined amount of “hot data,” either in the form of continuously computed analytics or a database of the last hour, day, or week’s worth of data. Older data is delivered to the historic data lake or data warehouse.

Advances in queuing systems like Kafa, in-memory databases, data grids, and NoSQL data stores make implementing this pattern possible. This pattern has broad usage across the internet of things (IoT), electric smart grids, log file management, and mobile in-game analytic processing, among others. We’ll be seeing more of this pattern in future applications.

The age of real time is now

If you are just starting out with your real-time application, first consider what response rate your problem domain requires. If it requires sub-millisecond response, consider an embedded application. If your application is high-velocity OLTP, explore high-performance network configurations and new offerings in low-latency data store and in-memory database technology. If you need to handle relentless streams of data, consider a fast data-pipeline architecture.

Low-cost computing, readily accessible high-speed networking, and numerous open source and commercial data storage software offerings capable of low-latency data processing means that real-time applications are no longer out of reach.

 

This post originally appeared on InfoWorld on August 28, 2017. 

 Measuring real-time

I explore the different types of real-time applications and how real-time is measured based on the application need

The term “real-time” is thrown around a lot these days, but it’s a buzzword that is often surrounded by ambiguity. Every day, it seems a new product is announcing its real-time capability. But how is real-time measured? It certainly isn’t measured in days (or even hours)—so is it measured in:

• Nanoseconds?
• Microseconds?
• Milliseconds?
• Seconds?
• Minutes?
• All of the above?

Everyone, from developers to software corporate marketing departments to even consumers, seems to have a slightly different answer. So let’s explore the question “What does ‘real-time’ really mean?”

Let’s begin with the dictionary definition:

Real-time—“of or relating to applications in which the computer must respond as rapidly as required by the user or necessitated by the process being controlled.”

While this definition continues with the subjective theme, it does confirm that the correct answer to how to measure real-time is “All of the above.” The meaning of the term real-time varies based on application need—the amount of time a computer (the application) takes to respond and the acceptable latency is as fast as required by the problem domain.

Rather than look at applications and determine if they are real-time or not, let’s examine various time units and understand the types of real-time applications that require those response rates:

Nanoseconds: A nanosecond (ns) is one billionth of a second. Admiral Grace Hopper famously explained a nanosecond using an 11.8-inch wire, as that is the maximum distance electricity can travel in one nanosecond. This quick video of Hopper is worth watching if you haven’t yet seen it.

With this in mind, it is easy to see why nanoseconds are the unit used to measure the speed of hardware, such as the time it takes to access computer memory. Worrying about nanosecond latency is at the bleeding edge of real-time computing and is primarily driven by innovation with hardware and networking technology.

Microseconds: A microsecond (µs) is one millionth of a second. Real-time applications that worry about microsecond latency are high-frequency trading (HFT) applications. Financial trading firms spend large sums of money investing in the latest networking and computer hardware to eliminate microseconds of latency within their trading platforms. A trading decision has to be made in as few microseconds as possible in order to execute ahead of competition and thus maximize profit. 

Milliseconds: A millisecond (ms) is one one-thousandth of a second. To put this in context, the speed of a human eye blink is 100 to 400 milliseconds, or between a 10th and half of a second. Network performance is often measured in milliseconds. Real-time applications that worry about latency in milliseconds include telecom applications, digital ad networks, and self-driving cars. The decision on what optimal ad to display or whether there is enough balance to let a cellphone call proceed must be made on the order of 100 milliseconds.

Seconds: We’re starting slow down here. We’re still in the realm of real-time, but we are now venturing into near real-time. Sub-minute processing time is often more than good enough for applications that process log files, computing analytics on event streams, as well as alerting applications. These real-time applications drive actions and decisions that are made in human-reaction time rather than machine-time. Reducing the response time by one tenth of a second (100ms), which may be costly to implement, has no change in value for the application.

Minutes: Waiting minutes may seem like an eternity to a high-frequency trading application. However, consider package shipment and delivery alerts or ecommerce stock availability notifications. Those applications certainly feel real-time to me—the fact that I receive a “delivery notification” text message within 10 minutes of a delivery made to my home is very satisfying.

Finally, though I discounted it up front, let’s briefly consider hours and days. While this time range is generally not regarded as true real-time, if you’ve been getting finance or sales reports on a monthly, weekly, or daily basis, and now you can get up-to-date reports every hour, that may be as real-time as you need. The modernization of these applications is often termed as upgrading from “batch” to “real-time.”

The old proverb is correct: Time is money. Throughout history, the ability to make real-time decisions has meant the difference between life and death, between profit and loss. The value of time has never been higher and therefore speed has never been more critical to business applications of all kinds.

Luckily, we live in an age where fast computing is very affordable and making decisions in real-time is economically achievable for most applications. The first step is determining the appropriate definition of real-time that aligns with the needs of your business applications.

In-Memory Database Sizing

 This post originally appeared on VoltDB.com in June, 2014.  

In-Memory Database Sizing

Sizing an in-memory database does not follow conventional database sizing rules.

For traditional databases, you buy a decent server machine, likely one with many CPU cores and reasonable memory, and then focus on application IOPS (I/O Operations per Second).  If you are really going to stress the database, you must choose disks that can support the I/O needs of your application, today and in the future. Because these systems often use many disks to achieve high I/O performance, capacity is usually an afterthought.

With in-memory databases, throw out everything you know about sizing databases. Performance is now the afterthought, but capacity must be carefully considered.  

To size your in-memory clustered database application you must consider the following factors:

●      First, compute the size of the data your application expects to host.  This means computing the size of all rows in all tables.  And also computing the size of all table indexes. VoltDB provides an easy-to-use interactive sizing tool in our application catalog report, described later in this post.

●      If you want a highly available database cluster, all data must be stored redundantly. You’ll need to multiply your logical data size by the replication factor to understand your memory requirements. For example, if you have 100GB of table and index data, and you want your database to be able to withstand losing one of the nodes in the cluster, you will want each datum to be stored twice, and on two different physical machines. This means that you need to size the data requirement of your database to be 200GB (100GB of data X 2 copies of the data).

●      Allow for some extra memory, perhaps an additional 25%, for database and operating system overhead. VoltDB uses extra memory for storing intermediate results, maintaining an undo log, caching SQL plans and buffering network data. Note that on a machine with 4GB, you’ll want to leave a larger fraction of memory free than on a machine with 256GB.

This tallied memory total can now help you decide how many machines your system requires.  Note at no time did we need to determine how much CPU we required, or how many IOPS were needed.  Because you are sizing your database based on data, memory size is the most important calculation you will likely use. Generally speaking, in-memory databases like VoltDB operate so fast that CPU utilization is not a problem - there is plenty of headroom for growth. As for disk I/O, most in-memory systems use sequential logs on disk, eschewing costly random I/O. With this reduced dependency on disk performance, VoltDB can achieve tremendous throughput, even on commodity spinning disks.

Often, you’ll be faced with a choice between many machines with less memory per machine or a small number of high-memory machines. There are lots of reasons to lean one way or another. For one, you should try to run with at least three nodes if availability or redundancy is a concern. Additionally, some administrator operations run faster with less per-machine data. At some point however, too many nodes can be a management hassle. Most VoltDB users run with 3 to 30 nodes per cluster for these reasons.

Planning for Data Capacity and Growth

Sizing VoltDB databases is fairly straight-forward.  Once you have settled on your application’s database schema, point your web browser to the database Catalog Report and choose the “Size Worksheet”, found at this URL: http://localhost:8080/#z (note the database must be running).

 

This sizing worksheet is automatically built from your application’s schema. It allows you to enter your expected number of rows for each table and then computes the amount of memory required for the database cluster as a whole.

Note that the footprint of your database is the size of the number of rows as well as the size of all indexes and materialized views. You must also leave additional memory for the operating system.  Note that this assumes that VoltDB is the only active application on these machines.  If you plan on sharing the machines with other applications, something we do not recommend, your memory requirements will naturally be larger.

Expanding your Database Capacity

A well behaved IT environment will monitor the performance and capacity of the applications running the business.

Because VoltDB processes transactions incredibly fast, it is often the case that there is plenty of transaction throughput headroom and there are lots of extra CPU cycles available. Monitoring the memory usage of the database cluster becomes much more important. VoltDB provides monitoring interfaces, as well as Nagios and New Relic plug-ins that make monitoring memory usage easy. These tools allow you to define capacity alerts, much the way you would define disk space usage alerts for legacy database systems. Should your in-memory database memory usage grow to a high level, signaling a capacity limit alert, it may be time to add more storage capacity to your database.

With VoltDB, adding capacity is easy: simply add more nodes.  VoltDB supports adding nodes to a running cluster without interrupting ongoing operations or sacrificing ACID transactional guarantees.

If you do not wish to add more nodes to your database cluster, doing a rolling memory upgrade of machines in a redundant VoltDB cluster is an alternative. Since VoltDB uses active replication within a cluster, removing and replacing a node can be done transparently to a user’s application with no data loss and often unnoticeable performance impact.

In-Memory Database provisioning… A New Way of Thinking

If you are provisioning and rolling out a new application using an in-memory database you will have to shift your thinking about how you size your deployment. With all data residing in memory you will need to provision enough memory in your database cluster to hold all of your data, and a bit more memory for regular operating system operations.

Disks are only used for durability and most in-memory systems like VoltDB work great with commodity hardware, disks included. Sizing for IOPS is a thing of the past!