Generator as Coroutines
Generator as Coroutines
- cooperative multitasking (cooperative routines)
- concurrent not parallel (python program execute on a single thread)
The way to create coroutines:
- generators (asyncio)
- native coroutines (using async /await)
Concepts
- concurrency: tasks start, run and complete in overlapping time periods
- parallelism: tasks run simultaneousely
- cooperative: control relinquished to other task voluntarily, control by application(developer)
- preemptive: control relinquished to other task involuntarily, control by the OS.
some sort of scheduler involved
- Global Interpreter Lock(GIL)
Only one native thread excutes at a time.
Use Process based parallelism to avoid GIL. Not Thread based.
The Python threading module uses threads instead of processes. Threads uniquely run in the same unique memory heap. Whereas Processes run in separate memory heaps. This makes sharing information harder with processes and object instances. One problem arises because threads use the same memory heap, multiple threads can write to the same location in the memory heap which is why the global interpreter lock(GIL) in CPython was created as a mutex to prevent it from happening.
Make the right choice
- CPU Bound => Multi processing
- I/O Bound, Fast I/O, Limit Connections => Muilti Threading
- I/O Bound, Slow I/O, Many Connections => Concurrency
Use deque
Much more efficient way to implement the stack and queue.
Operate 10,000 items take 1,000 times average:
| (times in seconds) | list | deque |
|---|---|---|
| append(right) | 0.87 | 0.87 |
| pop(right) | 0.002 | 0.0005 |
| insert(left) | 20.8 | 0.84 |
| pop(left) | 0.012 | 0.0005 |
Use unlimited deque with deque() or deque(iterable)
Use limited deque with deque(maxlen=n). If full, a corresponding number of items are discarded from the opposite end.
Implement producer / consumer coroutine using deque
Implement simple event loop