Amazon’s iPhone App Uses Image Recognition to “See” Real-World Products You Want to Buy

  • Share
  • Read Later
Amazon Flow

Among the many things that make Amazon one of the most successful merchants in the history of merchandising is the zeal with which it removes every possible obstacle that might stand in the way of you buying something. Today, it’s adding a feature to its Amazon Mobile app for iPhone designed to let you find and purchase products — especially food and other household consumables — without so much as typing in their names.

The feature is called Flow, and it’s a┬áspruced-up take on a search tool the iPhone app previously had, Snap It. That version was designed only for books, CDs, DVDs and video games, and it made you press an on-screen shutter button and then a “Use Photo” button. Flow recognizes far more stuff, and it does it on the fly: The moment it thinks it’s figured out what something is, it shows it to you, and you can go on pointing it at additional products if you choose.

In both its old form and its new one, the feature uses image-recognition technology that’s similar to the Goggles search option available in Google’s mobile search apps.

How well does Flow work? Pretty well, when I tried it at home and in my office. (Amazon provided me with early access to the feature.) It recognized most packaged items available from Amazon in anywhere from a second or so to a few seconds. In fact, it does so with such eagerness that when I accidentally swept it across my messy desk, it spotted an old issue of TIME in all the clutter, correctly identified it and gave me the opportunity to buy a copy from a used-magazine seller through Amazon. It also noticed my MacBook Pro sitting there, though it thought it was a MacBook Air.

Among other things, Flow’s technology is hunting for recognizable logos, and when it got confused, it usually got the company right, but not the product. For instance, it thought that a Scotch tape dispenser was a roll of Scotch tape, and that a package of Kleenex paper towels was Kleenex toilet tissue.

The bigger issue is that even when Flow guesses correctly, as it usually does, the products it identifies usually come in a lot of variants — different sizes and different colors, for example. (As seen in the screen shot above, it was smart enough to identify a pack of Post-it Notes — but not to detect that they were blue Post-it Notes.) In such cases, it shows you one pick, and lets you browse through others to find the one you want. Which means that Amazon can’t tie it into its 1-Click feature for utterly frictionless buying.

The scenario in which the feature feels like it has the most potential to be a serious time-saver is creating a shopping list. If you’re the type who buys groceries and other household necessities from Amazon, you could stand in your kitchen and whip through the items you need more of at a fast clip, maybe without even bothering to remove them from the refrigerator or cupboard. (As products get identified, they go into your history, so you could place the order at your convenience — although finding your history is trickier than it should be, and it doesn’t seem to sync across devices.)

One application Amazon isn’t playing up is the controversial practice of showrooming — going to a retail store, checking out a product in person and then sneakily buying it from Amazon, presumably at a lower price. I aimed the app at a bunch of boxed gadgets I happened to have handy, and it identified many of them — although here again, it sometimes jumped to conclusions based on a logo. For example, it thought that my Belkin home charger was a car charger. I don’t think the app is a game-changer in this context; if you’re only trying to pull up one item at a time, it just won’t save you that much time.

The new version of the iPhone app with Flow should be available by the time you read this. No word on whether the feature will show up in Amazon’s Android app or other venues.