Generator as Coroutines

Generator as Coroutines

  • cooperative multitasking (cooperative routines)
  • concurrent not parallel (python program execute on a single thread)

The way to create coroutines:

  • generators (asyncio)
  • native coroutines (using async /await)

Concepts

  • concurrency: tasks start, run and complete in overlapping time periods
  • parallelism: tasks run simultaneousely

image

  • cooperative: control relinquished to other task voluntarily, control by application(developer)
  • preemptive: control relinquished to other task involuntarily, control by the OS.

    some sort of scheduler involved

image

  • Global Interpreter Lock(GIL)

    Only one native thread excutes at a time.

    Use Process based parallelism to avoid GIL. Not Thread based.

    The Python threading module uses threads instead of processes. Threads uniquely run in the same unique memory heap. Whereas Processes run in separate memory heaps. This makes sharing information harder with processes and object instances. One problem arises because threads use the same memory heap, multiple threads can write to the same location in the memory heap which is why the global interpreter lock(GIL) in CPython was created as a mutex to prevent it from happening.

Make the right choice

  • CPU Bound => Multi processing
  • I/O Bound, Fast I/O, Limit Connections => Muilti Threading
  • I/O Bound, Slow I/O, Many Connections => Concurrency

Use deque

Much more efficient way to implement the stack and queue.

Operate 10,000 items take 1,000 times average:

(times in seconds) list deque
append(right) 0.87 0.87
pop(right) 0.002 0.0005
insert(left) 20.8 0.84
pop(left) 0.012 0.0005

Use unlimited deque with deque() or deque(iterable)
Use limited deque with deque(maxlen=n). If full, a corresponding number of items are discarded from the opposite end.

Implement producer / consumer coroutine using deque

Implement simple event loop

Read more “Generator as Coroutines”

發表留言

Generator

Generator

  • A type of iterator
  • generator function: function that uses yield statement
  • implement the iterator protocal, call next
  • raise StopIteration exhausted

Less code

Implement an iterator

Implement a generator

More efficient

Generator Comprehensions

  • local scope
  • lazy evaluation
  • is an iterator, can be exhausted

Delegating Generator

Use the syntax yield from to yield items in a generator

發表留言