Skip to content

Repository files navigation

libraries I used:

Flask, pandas, scikit-learn's TfidfVectorizer, LogisticRegression, and accuracy_score, secret key

Session Security: Flask uses sessions to store user-specific information across multiple requests. The secret key is used to securely sign the session cookie, ensuring that it cannot be tampered with by the client. This prevents potential security vulnerabilities such as session tampering or session hijacking.

CSRF Protection: Cross-Site Request Forgery (CSRF) is a type of attack where unauthorized commands are transmitted from a user that the web application trusts. The secret key is used in Flask to generate tokens for preventing CSRF attacks.

in the featue extraction method

#TF-IDF(term frequency-inverse document frequency) IDF = log(total_documents / (frequency + 1)) TF-IDF = count / (len(words))

In the given implementation of TfidfVectorizer, the feature extraction process involves the following major steps:

  1. Building Vocabulary: The _compute_idf method iterates over the input data and builds a vocabulary by counting the frequency of each word across all documents. It updates the vocab dictionary, where the keys are unique words and the values are their corresponding frequencies.

  2. Calculating IDF: The idf values are computed based on the frequencies stored in the vocab dictionary. The IDF value for each word is calculated using the formula IDF = log(total_documents / (frequency + 1)), where total_documents is the total number of documents in the input data. The IDF values are stored in the idf dictionary.

  3. Transforming into TF-IDF Vectors: The transform method processes each document in the input data. For each document, it creates a dictionary word_counts to count the frequency of each word. It then calculates the term frequency (TF) for each word by dividing its count by the total number of words in the document.

  4. Assigning TF-IDF Values: For each word in word_counts, the transform method checks if the word exists in the vocabulary (self.vocab). If it does, it retrieves the corresponding IDF value from the idf dictionary. The TF-IDF value is then computed by multiplying the TF and IDF values. These TF-IDF values are assigned to the corresponding positions in the features array, which represents the TF-IDF feature vectors for the documents.

  5. Applying IDF Weighting: After all the TF-IDF values are assigned to the features array, the method applies IDF weighting by element-wise multiplication with the IDF values stored in the idf dictionary.

The output of the transform method is the features array, which contains the TF-IDF feature vectors for the input documents. Each row in the array corresponds to a document, and each column corresponds to a unique word in the vocabulary. These feature vectors capture the importance of each word in each document, considering both term frequency and inverse document frequency.


here is how the accuracy score works from sklearn.metrics import accuracy_score

True labels

y_true = [0, 1, 1, 0, 1]

Predicted labels

y_pred = [0, 1, 0, 0, 1]

accuracy = accuracy_score(y_true, y_pred) print(f"Accuracy: {accuracy}")

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages