Skip to content

Latest commit

 

History

History
35 lines (26 loc) · 1.6 KB

OBJECTIVES.md

File metadata and controls

35 lines (26 loc) · 1.6 KB
Raw

Acres99

Features

  • Add logging in every steps.
  • Solve drop_duplicates=True issue.
  • Understand why drop_duplicates=True does not remove the newer duplicate rows from the DataFrames. For reference see logs.
  • If expand=False check that CSV file is not exists. If exists raise FileExistsError.
  • Also, add a parameter to export_dfs method which rewrite the facets DataFrames.
  • Create a function which fetches the data from website and store them in CSV formate periodically.
  • Handle exceptions for validation of Pydantic.
  • Make the code more comprehensive and modular. Now it is more coupled and not easy to use individually and understand.
  • If any error occurs after fetching the data show a button to download the un-cleaned data and then create a page to upload those data to clean.
  • Add a button to download the raw response (without validating through Pydantic).
  • Add new features to SRP class to get a better data for model building and data analysis.
  • Check how to identify the rental apartments/properties.
  • Show the logging into the app.

New objectives

Now I want to convert this project into Streamlit app.

  • Find all the cities id to create URLs.
  • Create a separate module which handles the DataFrames operations.
  • Create class to deal with API requests.
  • Add a parameter to pass the city ID.

TODO

  • Improve logging.

    • Only one logging file will be there.
    • More comprehensive logging message.

  • Tackle with first data saving error. (No column to parse in the csv file.)