Skip to content
Fork of the NY Times tagger with improved testing, bugfixes
Branch: master
Clone or download
Pull request Compare This branch is 50 commits ahead of nytimes:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bin
hooks
ingredient_phrase_tagger
tests
.coveragerc
.dockerignore
.gitignore
.python-version
.travis.yml
Dockerfile
LICENSE.md
README.md
build.sh
dev_requirements.txt
docker_build
docker_train_prod_model
nyt-ingredients-snapshot-2015.csv
requirements.txt
setup.py
template_file
test_e2e

README.md

CRF Ingredient Phrase Tagger

Build Status Docker Pulls License

Fork Notes

This is a fork of the original NY Times ingredient-phrase-tagger. This fork is maintained by Michael Lynch

This fork maintains the design of the original ingredient-phrase-tagger, but adds bugfixes and additional features to aid in future development:

These improvements were described in a series of blog posts on mtlynch.io:

Zestful

Zestful is a managed ingredient-parsing service based on this library. It has higher accuracy and more frequent updates:

Overview

This repo contains scripts to extract the Quantity, Unit, Name, and Comments from unstructured ingredient phrases. We use it on Cooking to format incoming recipes. Given the following input:

1 pound carrots, young ones if possible
Kosher salt, to taste
2 tablespoons sherry vinegar
2 tablespoons honey
2 tablespoons extra-virgin olive oil
1 medium-size shallot, peeled and finely diced
1/2 teaspoon fresh thyme leaves, finely chopped
Black pepper, to taste

Our tool produces something like:

{
    "qty":     "1",
    "unit":    "pound"
    "name":    "carrots",
    "other":   ",",
    "comment": "young ones if possible",
    "input":   "1 pound carrots, young ones if possible",
    "display": "<span class='qty'>1</span><span class='unit'>pound</span><span class='name'>carrots</span><span class='other'>,</span><span class='comment'>young ones if possible</span>",
}

We use a conditional random field model (CRF) to extract tags from labelled training data, which was tagged by human news assistants. We wrote about our approach on the New York Times Open blog. More information about CRFs can be found here.

On a 2012 Macbook Pro, training the model takes roughly 30 minutes for 130k examples using the CRF++ library.

Development

On OSX:

brew install crf++
python setup.py install

Docker:

docker pull mtlynch/ingredient-phrase-tagger

Quick Start

To begin, you must train a model:

MODEL_DIR=$(mktemp -d)
./docker_train_prod_model $MODEL_DIR
MODEL_FILE=$(find $MODEL_DIR -name '*.crfmodel')

From there, you can convert ingredients by piping them into stdin:

echo '
2 tablespoons honey
1/2 cup flour
Black pepper, to taste' | bin/parse-ingredients.py --model-file $MODEL_FILE
[
  {
    "display": "<span class='qty'>2</span><span class='unit'>tablespoons</span><span class='name'>honey</span>",
    "input": "2 tablespoons honey",
    "name": "honey",
    "qty": "2",
    "unit": "tablespoon"
  },
  {
    "display": "<span class='qty'>1/2</span><span class='unit'>cup</span><span class='name'>flour</span>",
    "input": "1/2 cup flour",
    "name": "flour",
    "qty": "1/2",
    "unit": "cup"
  },
  {
    "comment": "to taste",
    "display": "<span class='name'>Black pepper</span><span class='other'>,</span><span class='comment'>to taste</span>",
    "input": "Black pepper, to taste",
    "name": "Black pepper",
    "other": ","
  }
]

Authors

License

Apache 2.0.

You can’t perform that action at this time.