Hey Gokhanuck,
That’s a really good question. Finding trustworthy market data is such a vital part of making any algorithmic trading work smoothly. Here’s what I’ve learned from experience and the industry over the years:
First off, many traders get their live data directly from their broker’s API. It’s pretty handy because it’s integrated and quick, but the quality can vary quite a bit depending on the broker. So, it’s a good idea to double-check that the data you’re getting is accurate and consistent.
For more serious research and backtesting, many people turn to premium data providers like Bloomberg, Thomson Reuters, Quandl, or TickData. They tend to have cleaner, more reliable historical data, though they can be costly. Still, they’re worth it if you need precision and depth.
There are also more affordable APIs from platforms like Alpha Vantage, IEX Cloud, or Polygon.io. These are good for real-time and historical data, but they often have limits on how much you can request. If you’re doing high-frequency trading, you might need to combine these sources or consider paid plans to get faster, more reliable data.
When it comes to data quality and latency, a few best practices help:
- Use multiple sources and cross-reference data to catch discrepancies.
- Implement validation checks to flag anomalies or missing data.
- Save raw data for audits or reprocessing if needed.
- For strategies sensitive to delays, consider colocating your servers near the exchanges or using direct exchange feeds if possible.
Dealing with bad or noisy tick data is another challenge. Techniques like filtering out outliers, smoothing data, or applying real-time corrections can really improve the quality. Some traders also subscribe to exchange correction feeds or apply their own adjustments.
Lastly, it’s important to find a balance between cost and reliability based on your trading style. High-frequency strategies often justify investing in direct exchange feeds, while less frequent trading might do just fine with premium APIs or broker data.
In summary, combining different sources, validating data thoroughly, and understanding each source's limitations are key to running a robust trading system.
Hope that helps, and happy trading!
Best,
