Google Wants to Use Your Instagram Brunch Pics to Count Calories

image via (cc) flickr user bittermelon

We may be on the cusp of a culinary revolution. No, not in terms of how we prepare, cook, and eat our meals. Not even in terms of local sourcing, or ethical farming conditions.

I’m talking about food photography.

During a session titled “Semantic Image Segmentation Using Deep Learning & Graphical Models” at RE.WORK’s recent Deep Learning Summit in Boston, Google researcher Kevin Murphy presented a tantalizing glimpse at what might be the future of Instagrammed brunches: Artificial intelligence calorie-counting.

The experimental system, dubbed “Im2Calories,” relies on a combination of sophisticated pixel analysis, software, and the search giant’s deep library of data. The result is an app that can, ostensibly, scan a photo of food and determine not only what it is, but how much of it there is. Using that data, Im2Calories can then extrapolate a rough calorie estimate of everything you’re about to eat. The more the system is used, the more accurate it gets as its database fills with newer, more varied food imagery. Practically speaking, Im2Calories’s technology will remove manual input and guesswork from health-tracking software, instead allowing users to simply snap a picture of their meal, adjust as necessary (the program is still very much experimental), and let that image fill in the requisite data itself.

Google doesn’t necessarily have the best track record when it comes to lifestyle products and services (think: Google Glass, Google Plus, Google Wave, etc), but having recently submitted a patent for the technology behind Im2Calories, the company could theoretically find themselves the only game in town when it comes to the commercial application of this type of auto-analysis tech. Medical Daily points out that, in its current stage, Im2Calories works best with low-res images (which are easier to analyze, pixel-wise) although Google plans to expand the technology to accommodate higher resolution photographs.

And food is just the beginning. As Murphy explained to Popular Science:

“If we can do this for food, that's just the killer app. Suppose we did street scene analysis. We don't want to just say there are cars in this intersection. That's boring. We want to do things like localize cars, count the cars, get attributes of the cars, which way are they facing. Then we can do things like traffic scene analysis, predict where the most likely parking spot is. And since this is all learned from data, the technology is the same, you just change the data.”

For the time being, Im2Calories seems to still be in the experimental “we may never actually see it in the wild” stage. Still, if it—or something like it—makes it to market, high-res food porn could one day become an indispensable tool for healthy eating.

[via medical daily]