Situation:
The client was building an AI-driven brokerage platform from the ground up. The data science and engineering teams were ingesting massive amounts of 3rd party data and rapidly building ML models to power its core platform. Potential new data sources needed to be prequalified to inform data procurement decisions, and once procured, data needed to be cleansed and ingested into the pipeline. The client’s Chief Data Scientist recognized that off-the-shelf data profiling and management tools would not be useful given the nuanced judgements required to determine whether a particular data value was “good”, and determined that typical outsourced consultants could not handle the task given the domain complexity, the volume of data being ingested, and the speed at which the work needed to be executed.
Results:
The data science team has been able to deliver their models 2X faster
