Leafsnap, a new mobile app that identifies plants by leaf shape, is launched by Smithsonian and collaborators

The Smithsonian Institution, Columbia University and the University of Maryland have pooled their expertise to create the world’s first plant identification mobile app using visual search—Leafsnap. This electronic field guide allows users to identify tree species simply by taking a photograph of the tree’s leaves. In addition to the species name, Leafsnap provides high-resolution photographs and information about the tree’s flowers, fruit, seeds and bark—giving the user a comprehensive understanding of the species.

Smithsonian botanist John Kress uses the new mobile app to correctly identify a katsura tree (Cercidiphyllum japonicum) growing in the Smithsonian’s Enid A. Haupt Garden on the National Mall in Washington, D.C.

“We wanted to use mathematical techniques we were developing for face recognition and apply them to species identification,” said Peter Belhumeur, professor of computer science at Columbia and leader of the Columbia team working on Leafsnap. “Traditional field guides can be frustrating—you often do not find what you are looking for. We thought we could redesign them using today’s smartphones and visual recognition technology.”

David Jacobs of the University of Maryland and Belhumeur approached John Kress, research botanist at the Smithsonian’s National Museum of Natural History, to collaborate on remaking the traditional field guide for the 21st century.

“Leafsnap was originally designed as a specialized aid for scientists and plant explorers to discover new species in poorly known habitats,” said John Kress, leader of the Smithsonian team working on Leafsnap. Kress was digitizing the botanical specimens at the Smithsonian when first contacted by Jacobs and Belhumeur, so the match between a botanist and computer scientists came at a perfect time. “Now Smithsonian research is available as an app for the public to get to know the plant diversity in their own backyards, in parks and in natural areas. This tool is especially important for the environment, because learning about nature is the first step in conserving it.”

In addition to identifying and providing information about plants, Leafsnap also can map a specific plant’s location and save the location for future reference. (Photos by John Barrat)

Users of Leafsnap will not only be learning about the trees in their communities and on their hikes—they will also be contributing to science. As people use Leafsnap, the free mobile app automatically shares their images, species identifications and the tree’s location with a community of scientists. These scientists will use the information to map and monitor population growth and decline of trees nationwide. Currently, Leafsnap’s database includes the trees of the Northeast, but it will soon expand to cover the trees of the entire continental United States.

The visual recognition algorithms developed by Columbia University and the University of Maryland are key to Leafsnap. Each leaf photograph is matched against a leaf-image library using numerous shape measurements computed at points along the leaf’s outline. The best matches are then ranked and returned to the user for final verification.

“Within a single species leaves can have quite diverse shapes, while leaves from different species are sometimes quite similar,” said Jacobs, a professor of computer science at the University of Maryland. “So one of the main technical challenges in using leaves to identify plant species has been to find effective representations of their shape, which capture their most important characteristics.”

The algorithms and software were developed by Columbia and the University of Maryland, and the Smithsonian supervised the identification and collection of leaves needed to create the image library used for the visual recognition in Leafsnap. In addition, the not-for-profit organization Finding Species was hired and supervised by the Smithsonian to acquire the detailed species images seen in the Leafsnap app and on the Leafsnap.com website.

The app is available for the iPhone, with iPad and Android versions to be released later this summer.

You might also like:

  1. Smithsonian botanist writes book on his discoveries in the secret land of Myanmar
  2. “Billy club” leaf beetle has been hiding in Smithsonian collections since 1959
  3. Scientists find excess nitrogen favors plants that respond poorly to rising CO2
  4. A well-defended territory is what some female hummingbirds find most attractive in a mate
 

 
  • Kristi

    HOORAY! I have been waiting for such an app! How exciting!

  • Rob Savio

    How do you log in to Leafsnap as a new user?

  • robbie

    can we buy this app now are is it in the work now. im looking for plant and tree id app thanks robbie

  • Chuck

    OK, now we need an Android version ASAP.

  • William

    Sounds like a great app… but I have a blackberry. Is there an app for us?

  • Kim

    Hello,
    Will this work for identifying all plants?

    • Antonella Francesca

      No, it has not clue about my house plants!

  • Ken Baumgardner

    Please hurry up and get the Android version done. I can’t wait! Sooo cool.

  • Terry

    Please, Please get an Android version! I was so disappointed when I got my smartphone that I couldn’t use this app.

  • http://www.johnfields.com John Fields

    Cant Wait for the Windows Version!! Really!

  • samantha

    I can’t find leafsnap on the app store on apple. where can i get this app?

  • JR

    Is there a Droid version available?

  • http://www.camaxo.com Layton Patrick

    Is there a Symbian version available? We were thinking to do a trip with the kids and show them different species of trees. We have got to an agreement with a phone seller and they will provide us with a phone for each child. Those phones work with Symbian OS.

  • larry

    will leafsnap be available for laptops. i have a web cam and would like to bring the leaves to the laptop. if not how long. thanks i am a botinist at heart. not the brightest crayon in the box though. thanks larry

  • http://www.bobdpt.it creazione siti web

    nice app thank for post

  • http://www.squidoo.com/geodesic-dome-kit Green living

    This is what it’s all about. We have so many useless apps, but one that educates people and grows their knowledge is something that we need more of. This could be so useful to children especially. Thank you, thank you, thank you!

  • http://www.active-net.it/ Creazione Siti

    Great!  I was looking many months for a plant identification mobile app .

  • misterbill

    What happened to the Android version? It’s a year later and I don’t see it anywhere.

  • Marianne

    Really want this app for Android, and want it now. Both for trees, herbs, wild plants, insects… In a perfect world, I would just snap a picture and get up a list of “definitely” or “most likely” hits based on my position, and with additional filters if necesary (such as “sump”, “heather”, “farmed field” or whatever is relevant).

  • Otterbound

    Let me know when this comes out for Icecream.. Android Please…..

  • Guest

    Android please!

  • Kristina

    Have the iPhone and can’t wait to get it

  • http://www.facebook.com/profile.php?id=1365873945 Mamie Waites

    very disappointed! I live in the south and was hoping that this app would help to identify different species of helpful and harmful plants—especially
    poison ivy. Out of the ten photos I snapped of a variety of plant life, not
    rare plants mind you, NONE of them were identifiable. Many were identified
    wrong. This is a great idea, I just wish it worked!

  • H.

    it doesn’t work.

  • http://www.facebook.com/ryanpugh10 Ryan Pugh

    This app sucks.