Chapter 15 worked answers: Preparing Text and Multimodal Data
On this page
Compare your reasoning after an attempt. Different valid implementations are welcome when they preserve the observation set and the stated meaning.
B2-15-E01 · Compare Unicode spellings
Normalize and casefold so the two entries compare equal while retaining raw s.
Normalization resolves the composed/decomposed representation; case folding handles case. Neither removes the original evidence.
s=pd.Series(["Café","CAFÉ"],dtype="string")
normalized=s.str.normalize("NFKC").str.casefold()
assert normalized.iloc[0]==normalized.iloc[1] and s.iloc[0]!=s.iloc[1]
print(normalized.tolist())['café', 'café']B2-15-E02 · Count documents and occurrences
Compute the occurrence count of red and its document frequency; verify 2 and 1.
Repeated use in one document increases occurrences without increasing the number of documents containing the term.
docs=pd.Series(["red red house","blue house"],dtype="string")
tokens=docs.str.split();occurrences=tokens.explode().value_counts()
document_frequency=sum("red" in set(t) for t in tokens)
assert occurrences["red"]==2 and document_frequency==1
print(int(occurrences["red"]),document_frequency)2 1B2-15-E03 · Verify representation dimensions
Interpret shape as samples, time steps and features; retrieve sample 1, step 0, feature 2.
The second sample starts at flat offset 12; its first step has values 12–15.
batch=np.arange(24,dtype="float32").reshape(2,3,4)
value=batch[1,0,2]
assert batch.shape==(2,3,4) and value==14
print(batch.shape,value)(2, 3, 4) 14.0Return to practice
Change one supplied input, predict the effect and rerun your own implementation. Return to the exercise prompts to identify a task you can now explain independently.