Notebook 09: TensorFlow and spaCy API updates

[mrdbourke]

A list of updates for different TensorFlow API changes as well as spaCy API changes to Notebook 09: Milestone Project 2: SkimLit 📄🔥

TextVectorization API update

As of TensorFlow 2.6+, TextVectorization is out of experimental and available via tf.keras.layers.TextVectorization:

New (TensorFlow 2.6+):

import tensorflow as tf

text_vectorizer = tf.keras.layers.TextVectorization(...)

Old:

import tensorflow as tf

text_vectorizer = tf.keras.layers.experimental.preprocessing.TextVectorization(...)

Functionality remains the same.

---

spaCy sentencizer API update

In the "Make example predictions" section of Notebook 09, the spaCy API has changed slightly:

from spacy.lang.en import English
nlp = English() # setup English sentence parser

# New (.add_pipe() only instead of .create_pipe()) 
sentencizer = nlp.add_pipe("sentencizer") # create sentence splitting pipeline object

Old:

from spacy.lang.en import English
nlp = English() # setup English sentence parser

# Old version of spaCy
sentencizer = nlp.create_pipe("sentencizer") # create sentence splitting pipeline object 
nlp.add_pipe(sentencizer) # add sentence splitting pipeline object to sentence parser

Full code (as of May 2023):

# Create sentencizer - Source: https://spacy.io/usage/linguistic-features#sbd 
from spacy.lang.en import English
nlp = English() # setup English sentence parser

# New version of spaCy
sentencizer = nlp.add_pipe("sentencizer") # create sentence splitting pipeline object

# Create "doc" of parsed sequences, change index for a different abstract
doc = nlp(example_abstracts[0]["abstract"]) 
abstract_lines = [str(sent) for sent in list(doc.sents)] # return detected sentences from doc in string type (not spaCy token type)

# View the abstract lines
abstract_lines