libraries I used:
Session Security: Flask uses sessions to store user-specific information across multiple requests. The secret key is used to securely sign the session cookie, ensuring that it cannot be tampered with by the client. This prevents potential security vulnerabilities such as session tampering or session hijacking.
CSRF Protection: Cross-Site Request Forgery (CSRF) is a type of attack where unauthorized commands are transmitted from a user that the web application trusts. The secret key is used in Flask to generate tokens for preventing CSRF attacks.
#TF-IDF(term frequency-inverse document frequency) IDF = log(total_documents / (frequency + 1)) TF-IDF = count / (len(words))
In the given implementation of TfidfVectorizer, the feature extraction process involves the following major steps:
-
Building Vocabulary: The
_compute_idfmethod iterates over the input data and builds a vocabulary by counting the frequency of each word across all documents. It updates thevocabdictionary, where the keys are unique words and the values are their corresponding frequencies. -
Calculating IDF: The
idfvalues are computed based on the frequencies stored in thevocabdictionary. The IDF value for each word is calculated using the formulaIDF = log(total_documents / (frequency + 1)), wheretotal_documentsis the total number of documents in the input data. The IDF values are stored in theidfdictionary. -
Transforming into TF-IDF Vectors: The
transformmethod processes each document in the input data. For each document, it creates a dictionaryword_countsto count the frequency of each word. It then calculates the term frequency (TF) for each word by dividing its count by the total number of words in the document. -
Assigning TF-IDF Values: For each word in
word_counts, thetransformmethod checks if the word exists in the vocabulary (self.vocab). If it does, it retrieves the corresponding IDF value from theidfdictionary. The TF-IDF value is then computed by multiplying the TF and IDF values. These TF-IDF values are assigned to the corresponding positions in thefeaturesarray, which represents the TF-IDF feature vectors for the documents. -
Applying IDF Weighting: After all the TF-IDF values are assigned to the
featuresarray, the method applies IDF weighting by element-wise multiplication with the IDF values stored in theidfdictionary.
The output of the transform method is the features array, which contains the TF-IDF feature vectors for the input documents. Each row in the array corresponds to a document, and each column corresponds to a unique word in the vocabulary. These feature vectors capture the importance of each word in each document, considering both term frequency and inverse document frequency.
here is how the accuracy score works from sklearn.metrics import accuracy_score
y_true = [0, 1, 1, 0, 1]
y_pred = [0, 1, 0, 0, 1]
accuracy = accuracy_score(y_true, y_pred) print(f"Accuracy: {accuracy}")