Improve performance by updating Python client code

[leovegoda]

Is your feature request related to a problem? Please describe.
Refactor of the complex sync, update, tasks and fetch components.

Who is affected by the problem?
PeeringDB users get slower performance.

What is the impact?
Improve performance, simplify code, and upgrade to Python 3.

Are there security concerns?
No.

Are there privacy concerns?
No.

Describe the solution you'd like
Proposal from 20C:

These components handle the synchronization of data from the PeeringDB API.

They currently use an antiquated python2 task concurrency implementation that was originally meant to improve performance of initial synchronization.

We think the code for this can be vastly simplified with python3 asyncio and make it much easier to update the client in the future.

• remove all the old task concurrency code
  • determine if we gain anything from async; if we do, use python's asyncio module instead
• refactor the fetch and update logic to make it easier to understand and expand upon in the future
  • vastly simplified approach
  • make use of batch inserts

Do you think this feature will require a formal design?
Yes.

Describe alternatives you've considered
Do nothing.

Could this feature request need support from the Admin Committee?
No.

What is the proposed priority?
ASAP

Provide a rationale for any/all of the above
The old code uses Python 2, which is no longer actively supported.

[grizz]

+1

[martinhannigan]

+1 From: Leo Vegoda ***@***.***> Date: Monday, July 17, 2023 at 12:55 PM To: peeringdb/peeringdb ***@***.***> Cc: Subscribed ***@***.***> Subject: [peeringdb/peeringdb] Improve performance by updating Python client code (Issue #1412) Is your feature request related to a problem? Please describe. Refactor of the complex sync, update, tasks and fetch components<https://github.com/peeringdb/peeringdb-py/tree/master/src/peeringdb>. Who is affected by the problem? PeeringDB users get slower performance. What is the impact? Improve performance, simplify code, and upgrade to Python 3. Are there security concerns? No. Are there privacy concerns? No. Describe the solution you'd like Proposal from 20C: These components handle the synchronization of data from the PeeringDB API. They currently use an antiquated python2 task concurrency implementation that was originally meant to improve performance of initial synchronization. We think the code for this can be vastly simplified with python3 asyncio and make it much easier to update the client in the future. · remove all the old task concurrency code o determine if we gain anything from async; if we do, use python's asyncio module instead · refactor the fetch and update logic to make it easier to understand and expand upon in the future o vastly simplified approach o make use of batch inserts Do you think this feature will require a formal design? Yes. Describe alternatives you've considered Do nothing. Could this feature request need support from the Admin Committee? No. What is the proposed priority? ASAP Provide a rationale for any/all of the above The old code uses Python 2, which is no longer actively supported. — Reply to this email directly, view it on GitHub<#1412>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFA2YQXMUHAF54CDXULBSTDXQVU7JANCNFSM6AAAAAA2NG367Y>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[arnoldnipper]

+1