Securities data cleaning

[MoneyWeaver]

Do you take time on validating and cleaning large scale historical data sets. If yes what software do you use?

 

[johnkm]

I use openrefine.org from Google which is good, although I still need to run some custom routines for timeseries data (such as gap tests etc)

 

[MoneyWeaver]

Thanks - I'll take a look. I don't want to reinvent the wheel. Are there other specific timeseries functions that would be usefull for price series eg multiple source price comparison, gap tests, outlier testing, winsorizing, multiple price blending, price syncing, fair value recalculation, scaling & ranking, random data gen .... ?