In 2017, I received an email asking for a meeting to talk about market data. I initially thought it would be a standard sales meeting, but soon it became clear that it was anything but.
When I met my contact in a nondescript coffee shop in downtown New York City, he told me a story about market data that has stuck with me ever since.
Prior to 9/11, a well-known large financial firm was located in a building adjacent to the World Trade Center. While their building was not attacked during the events that transpired on 9/11, the collapse of the WTC blew out the buildings of their nearby office, destroying nearly everything in their path and scattering rubble everywhere.
The namesake family who still ran this particular financial firm was looking to salvage what they could amongst the debris. During the course of this recovery, they came across old market data tape drives from the 1960s onward.
The patriarch (still head of the financial firm) was planning to throw out those tapes until the youngest and “IV” in their family line told him to wait. “IV” understood the power of data and insisted that rather than throw away the tapes, they should convert them to a more modern, digital format.
My contact then shared with me a small sample of the data, and I was awestruck. I was looking at 1969 tick data for an American icon: Ford. This was data that almost nobody alive today had seen before.
I then realized the gravity of the story I had just heard: had “IV” not convinced his father to save the tapes, this piece of financial history would have been lost forever, disappearing into the ethers of knowledge that once was.
Each day in the course of running Tiingo.com, a platform built on providing clean, reliable financial data, I am continually reminded of the fragility of financial data. And I don’t just mean single points of failure from non-redundant data storage, but also conflicting information about what has happened on any given day.
Our success stems from people testing our data and discovering its cleanliness for themselves. What they don’t see is the massive behind-the-scenes cleaning effort demanded by data sources that continuously conflict.
For example, as recently as last year, there was a period of two to three months where 6% to 8% of data points among centralized U.S. equity exchange data providers disagreed on open, high, low, and closing prices as well as volumes.
Since 2014 (when we launched our firm), tracking these discrepancies and their causes has been relatively easy compared to tracking data from years ago.
Why does this matter? Because, each day, we gatekeep the history of data as we go through painstaking lengths to verify older data. If a customer raises a questionable data point regarding information that appeared as recently as 2005, for example, we then proceed to comb over financial filings as well as our own news database of over 30 million articles in search of any reason that the price discrepancy may exist.
This process quickly becomes incredibly labor intensive, sometimes taking between two to six hours to trace down a single price discrepancy. In some cases, nothing we find can explain the price discrepancy, so we are forced to accept that the price movement may be real based on circumstantial evidence and our evolving understanding of market dynamics at the time.
These are the cases that keep us up at night. And then I am forced to ask the question:
“What if there is a better way to record financial history?”
Blockchain solves this not-so-rhetorical question by allowing data to exist openly for everyone in a redundant way that also immutably records history. One location being attacked will not destroy data, and those in charge of data cannot accidentally (or purposefully) change historical data points.
Blockchain and oracle networks like Chainlink create a real-time mapping of our history. Moreover, Chainlink Price Feeds don’t ignore price variations, but rather map them with full transparency.
Chainlink ultimately allows us to write this data in immutable storage, further cementing our human and financial history. It is for this reason that Tiingo has made its data available on-chain via our oracle and integration with Chainlink Price Feeds.
In the infancy of the stock market, we could not fathom what data would be used for. In fact, I asked “IV” of the aforementioned financial firm: “Why didn’t the exchanges have this old tick data?”
His response was that stock exchanges did not anticipate that the data would be useful, and in the 1960s-80s, they contracted data storage to an outside firm. Both the exchange and the firm did not think data that old would be useful, so they decided to throw it out.
If there’s anything to be learned from this story and applied to crypto markets in their infancy, it’s that it is our responsibility to future generations to ensure that this financial history is recorded in both its glory and volatility. This financial price data may one day be used for financial modeling and gaining insight into what makes a successful crypto project stand out.
This data will also be used to better understand our society and how to build projects with longevity. Yet we probably cannot even fathom all the ways this data will be used 100 years from now – that’s why it is our job to ensure the data remains clean, accurate, and available for future generations to build a more stable, prosperous, and equitable world.
About Tiingo
Tiingo is an award-winning financial analytics and data platform that powers thousands of asset managers, technology firms, and individuals across the globe. Unique in its position as both a consumer of its data and distributor, Tiingo is trusted by firms across the industry for its clean and unique datasets, which are engineered for the discovery of new alpha sources. With a mission deeply rooted in democratizing access to financial analytics and data, Tiingo is also a leader in financial education for future generations, having partnered with research departments at universities across the world to advocate for financial markets literacy.
Formed in 2014, Tiingo holds the belief that love is the ideal way to conduct business. We are a team made up of artists, engineers, and algorithmic hedge fund traders. Some of us have been professional photographers, and others have created trading algos managing hundreds of millions of dollars. We are united with the same goal: to make everyone’s life easier in the ways we know how.
To learn more about Tiingo’s APIs, visit https://api.tiingo.com. To learn more about the Tiingo analytics engine, visit https://www.tiingo.com.