Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scrape Vendor APIs for Available Numbers #8

Closed
uncheckederror opened this issue Nov 29, 2019 · 3 comments
Closed

Scrape Vendor APIs for Available Numbers #8

uncheckederror opened this issue Nov 29, 2019 · 3 comments
Labels
enhancement New feature or request

Comments

@uncheckederror
Copy link
Collaborator

@danry25 Right now the limitation on speed is due to the latency of the TeleAPI that this application is dependent on.

Ideally we would scrape all of the data required for this app once a day from the TeleAPI and then cache it in a database like SQLite or PostgreSQL so that we can control performance.

Originally posted by @uncheckederror in #7 (comment)

@uncheckederror uncheckederror added the enhancement New feature or request label Nov 29, 2019
@uncheckederror
Copy link
Collaborator Author

Per our internal discussion the plan is to spin up a copy of PostgreSQL and then write a console application that bulk scrapes these APIs and loads the data into a table. At which point NumberSearch will be refactored to reduce the scope of it's responsibilities. Rather than directly querying these vendor APIs it will instead query against this ingest table to look up available phone numbers. When a phone number is selected NumberSearch will directly query the API that provided that phone number to capture its current state and verify its continued availability, then it will bump to a form to collect the customer's information, and finally wrap that all up into an email that is sent by the application to the customer's email and the order fulfillment email.

@uncheckederror
Copy link
Collaborator Author

To support this goal we should do this work items:

  • Break the data models out into a dotnet standard 2.1 library called NumberSearch.DataAccess to be shared between multiple projects.
  • Create a console app called NumberSearch.Ingest that can be run daily to ingest data into PostgreSQL.
  • Integrate a mail client into NumberSearch.Mvc that can handle the order creation process.

@uncheckederror
Copy link
Collaborator Author

This is complete as of commit c421efc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
No open projects
Development

No branches or pull requests

1 participant