Garbage In,
Gold Out?

How AI addresses messy input data

By Nick Green

It’s fair to be sceptical of hype-driven claims around AI.

One area of scepticism I’m hearing repeated, however, is GIGO: Garbage In, Garbage Out; the idea that AI requires data cleansing and structured data before it can be useful. While this is true of classic machine learning, it seems to miss a crucial, later development in AI.

Lessons From Chess

In the 2010s, chess engines had far surpassed human champions, and the highest level competition was among chess engine developers. Stockfish 8, generally the strongest of these ‘brute force’ models, could calculate 70,000,000 board positions per second, covering every conceivable search tree of reasonable moves.

In 2017, out of the blue, Google DeepMind announced a new kind of chess engine, AlphaZero. A closed-source, complete outsider with no game history (chess engine enthusiasts were as surprised as anyone else), it only calculated a mere 80,000 positions per second. To the chess world’s surprise, DeepMind was confident enough to release AlphaZero through a 100-game competition against Stockfish 8.

AlphaZero vs. Stockfish 8 · 100 Games, 2017:

  • Draws: 72
  • AlphaZero Wins: 28
  • AlphaZero Losses: 0

After dominating the series, to satisfy doubters, the engines played again in 2018 with new conditions, such as longer time controls, fairer to Stockfish, but AlphaZero dominated once again* before disappearing from the chess world.

Classical engines depended on heavy human preparation. Stockfish 8 relied on Hand-Crafted Evaluation (HCE), human experts embedding their knowledge to influence long term strategy with age-old ideas from controlling the centre, to keeping knights off the edge of the board (“a knight on the rim is dim” goes the expression from chess teachers).

AlphaZero vs. Stockfish 8
Game 3 · 2017
♛︎♜︎♝︎♚︎♝︎♟︎♟︎♟︎♟︎♟︎♟︎♟︎♞︎♟︎♟︎♟︎♛︎♟︎♚︎♜︎♟︎♝︎a8b7c6d5e4f3g2h1
30.Bxg6: AlphaZero takes the pawn on g6, giving up a bishop for no obvious gain. Viewers initially assumed it’s a mistake. By move 34, Black has the extra piece but no safe place for his king.

In the case of AlphaZero, its lead developer, David Silver, wasn’t much of a chess player at all. He never needed to be. As a Deep Learning model, AlphaZero was given a very simple reward function: Using these legal chess moves, win.

With zero human prejudice about what was good or bad board position, AlphaZero generated synthetic training data by playing itself millions of times over.

Grandmaster observers of the games were amazed by AlphaZero’s counter-intuitive strategies — in particular, its willingness to sacrifice pieces early, and play moves that every chess student is taught not to.

I always wondered how it would be if a superior species landed on earth and showed us how they played chess. Now I know.
Peter Heine Nielsen · Magnus Carlsen’s Second

AlphaZero shocked the world of chess engine developers, but didn’t render them obsolete. In the years that followed, grandmasters learned many lessons, and challenged many age-old assumptions about chess strategy. Today’s Stockfish 19, back on top of the chess engine podium, combines Deep Learning with human-engineered search.

So what does all this mean for your own business’s data models, and your messy data?

AI as the Data Cleanser

Garbage In, Garbage Out remains true for late-stage data processing — a classical forecast built on distorted data will only extrapolate the distortions.

Unsupervised Deep Learning, however, is often most useful in early-stage processing. Rather than humans dictating narrow boundaries around say, inventory categories or outlier thresholds, Deep Learning can discover patterns among several — or even thousands of dimensions across numbers and text (and if required, images or sound).

This creates a new mid-stage of processing; Deep Learning presents a series of matching clusters to the human domain expert, who labels, approves or rejects for thousands of records at a time.

Illustrative: spare part descriptions embedded with a sentence-transformer model, reduced to 3D with UMAP and clustered with HDBSCAN. Lines link records that are likely the same part.

A good example of this is in your Android or iPhone photo app.

In the ‘People’ section, you might see an image of a friend or family member, nudging you to give this person a name. Rather than you having to label every photo with a name, deep learning found a cluster (distinctive lines and shades, representing the same person), reducing perhaps hundreds of cleansing decisions to just one.

iOS Photos · People
Apple’s deep learning model identifies hundreds of photos of the same person, but asks you to provide their name.

After you label her ‘Mum’, not only will the model label all her photos in your history, it will correctly label future photos you take of your mother.

We use the same approach for business data when we flag, label and fix via deep learning. And the point of exploring data is often to find the surprises, yet a classical approach tends to treat unusual signals as outliers and filter them out.

Classical vs. Deep Learning First · ETL’s Transform Stage
StageClassicalDeep Learning First
ExploreHumanQueries, pivots, spot checksAIEmbeddings, clustering, anomaly flags
CleanseHumanRow by row, rule by ruleAIHumanHuman reviews and corrects AI’s clusters
FeaturesHumanHand-picks and engineers inputsAILearns its own representations

Unlike chess, business has no fixed rulebook and no single definition of “winning” – which is why the real world still needs human judgement. But applied well, AI lets analysts see more, and cleanse more, than ever before. Rather than a blocker, ‘garbage’ data is often a supporting reason to deploy AI.

* In the 2018 series; time-controls were extended from 1 minute to 3 hours per move, and Stockfish used 44 CPU Cores. After 1,000 games: AlphaZero achieved 155 wins, with 6 losses and 839 draws.

Subscribed. The next piece will arrive in your inbox.

That didn’t send. Please try again.

Get each new piece by email when it’s published. Nothing else.

Thanks. Your feedback has been sent.