A Well-Designed Intraday Data Research Environment Can Help Traders Outsmart the Competition

By Scott Mayster, Global Product Manager, Tick Data LLC, a subsidiary of OneMarketData LLC

Today’s financial markets have been roiled by volatility, global economic turmoil, and the pressure of continued regulatory obligations. To navigate these dynamic conditions, remain compliant, and drive competitive advantage, traders need fast and efficient access to quality market data and the ability to effectively analyze it. Only then can today’s market participants interpret trading patterns and develop new strategies while saving time and money on data management and ensuring ongoing compliance. What firms and their traders really need is a robust, versatile intraday data research environment.

A well-designed data research environment helps firms perform pre-trade research, including building algorithmic trading models and performing critical strategy back testing, and post-trade analytics, such as BestEx and Transaction Cost Analysis (TCA). Unfortunately, many firms don’t know where to start; they find the prospect of building this type of environment themselves daunting, especially when it comes to understanding how to acquire and manage the data component of the system.

This is understandable. It is easy to become overwhelmed with the cost, complexity, performance, and scalability requirements associated with creating such a system. As a result, small and large firms alike increasingly look to leverage a pre-built or hosted historical data service to help them create the optimal environment that will enable them to perform research on a tick-by-tick basis. But data is only one aspect. Here are the critical factors firms need to consider before moving forward, along with some recommendations:

Don’t reinvent the wheel if you can avoid it.

Open source analysis and database platforms are appealing due to their lack of upfront costs. But whether you use an open source solution or pay for the software, the license fee is small compared to the cost of implementation. It’s worth keeping in mind that analysis platforms that lack business intelligence (BI) specific to the financial markets require significant build-out and optimization. You need to be aware of the hidden costs of implementing more generic solutions, including the expenditure of time to go-live. Consider looking at the platforms available that offer research and/or analysis environments with pre-built indicators and tools, data loaders, feed handles, trading functions (i.e. different order types, for back-testing and trading in real-time), and other functions specific to financial time series data and trading.

Budget for costs related to acquiring quality real-time and historical data – they can be significant.

A key concept to understand is that the accuracy and reliability of the output of any analytics platform – such as a trading model, execution quality monitoring, or TCA – depends on the accuracy and reliability of the underlying data. But quality tick-by-tick market data alone is insufficient for building the data component of a research environment. The platform must be able to link disparate data sets, including proprietary trading information, normalized tick-by-tick market data, and quality reference data, under a common thread to maximize efficiency and create smarter output. You will save substantial time and money by acquiring the data from a reputable vendor or opting for a managed service running on shared or dedicated infrastructure. When it comes to market data, the build-vs-buy decision is an easy one.

Ensure compatibility.

Unless you are completely separating your research and trading functions into two distinct platforms, you will want your real-time data to be compatible with your historical data format. As conventions tend to vary from vendor to vendor, you may need to build internal maps for formatting variations, such as ticker symbol conventions, displayed decimal places, delimiters, and time zones. You may need to build custom feed handlers to unify formats and should also consider if you need to be FIX-compliant so you can execute trades directly from the platform.

Embrace the cloud.

Cloud adoption has come a long way in a few short years, and you would be wise to take full advantage of the efficiencies, power, and cost savings of running your research environment in a cloud infrastructure. Machines that are used for analysis and nightly optimizations spend most of their time idle, but when it’s time to run those overnight processes, they generally require a massive amount of processing power. That’s why elastic computing is ideal for this type of application. If you can quickly, easily, and cost-effectively scale up and down, you can save significant time and money.

Don’t be afraid to leverage AaaS.

An Analytics as a Service (AaaS) solution can offer access to analysis tools, programming environments and data on a remotely hosted server, generally in the cloud these days. This type of offering is fast to implement and offers a subscription-based service, which means the upfront costs can be much lower than building traditional analysis environments. A managed solution also allows firms to focus on their core business without having to think about maintenance. Keep in mind that you will need a service that can process enormous amounts of stored data. You may save money with a managed solution running in a cloud environment by paying a bundled fee for the database license, infrastructure, and data.

Driving your program forward

Once you have clearly assessed your firm’s use cases for a data research environment, taking into consideration your specific needs and budget constraints for market data, infrastructure, and analytics, it is time to determine which components you want to build, buy, lease, or subscribe to. Then you can best evaluate and select the right solution or solutions. Most firms do not have the time or money to build a data research environment from scratch, but with the existing options available today, no firm should have to. Whether choosing an end-to-end solution that offers everything you need or staking out the middle ground by creating an environment comprised of multiple solutions, a well-designed intraday data research environment is within reach of trading firms of any size.