Remove manys calls to cursor_to_dict #16431
Conversation
|
(Also I secretly hate |
| @@ -101,7 +101,7 @@ | |||
| class PusherConfig: | |||
| """Parameters necessary to configure a pusher.""" | |||
|
|
|||
| id: Optional[str] | |||
| id: Optional[int] | |||
There was a problem hiding this comment.
This has had the wrong type forever, but was consistent with itself. Now that we're pulling the proper type from the database layer we had to update it here too.
| return ( | ||
| TaskSchedulerWorkerStore._convert_row_to_task( | ||
| ( | ||
| row["id"], | ||
| row["action"], | ||
| row["status"], | ||
| row["timestamp"], | ||
| row["resouce_id"], | ||
| row["params"], | ||
| row["result"], | ||
| row["error"], | ||
| ) | ||
| ) | ||
| if row | ||
| else None | ||
| ) |
There was a problem hiding this comment.
I'm hoping to convert simple_select_* methods too, but that's a bigger job. So for now we have this ugliness.
| row["result"] = db_to_json(row["result"]) | ||
| return ScheduledTask(**row) | ||
| def _convert_row_to_task(row: ScheduledTaskRow) -> ScheduledTask: | ||
| task_id, action, status, timestamp, resource_id, params, result, error = row |
There was a problem hiding this comment.
Alternately we could just do row[0], etc.
This comment applies in a bunch of spots, let me know which style you prefer.
There was a problem hiding this comment.
does task_id, *rest = row work? or even task_id, *_ = row?
There was a problem hiding this comment.
Those help readability, but from a quick test it would allocate another list:
>>> first, *rest = (1, 2, 3, 4)
>>> first
1
>>> rest
[2, 3, 4]
>>> first, *_ = (1, 2, 3, 4)
>>> _
[2, 3, 4]
There was a problem hiding this comment.
Sorry if I'm unclear, I mean below this line of code we could do
ScheduledTaskRow(
task_id=row[0],
action=row[1],
...
)There was a problem hiding this comment.
(If there's no particular opinion I'll just merge this.)
There was a problem hiding this comment.
Oh, I see. No strong opinions here.
|
Err I'm a bit scared about #16431 (comment) actually |
Yes, I plan to poke at more. |
Instead of calling
cursor_to_dictand then immediately unpacking thedict/ changing the form to something else we should just use the raw tuples returned from the cursor.A smoke test of this reduced my sync (for a rather small account on my test server) from 2757 allocations to 1861. Reducing the total allocated memory by
_wait_for_sync_from_userfrom 217.2 KiB to 146.7 KiB.