Browse Source

Coroutine section now covers asyncio instead of generator based coroutines

pull/52/head
Jure Šorn 5 years ago
parent
commit
d58dce0135
3 changed files with 129 additions and 87 deletions
  1. 106
      README.md
  2. 108
      index.html
  3. 2
      parse.js

106
README.md

@ -13,7 +13,7 @@ Contents
**   ** **3. Syntax:** **         ** **[`Args`](#arguments)**__,__ **[`Inline`](#inline)**__,__ **[`Closure`](#closure)**__,__ **[`Decorator`](#decorator)**__,__ **[`Class`](#class)**__,__ **[`Duck_Types`](#duck-types)**__,__ **[`Enum`](#enum)**__,__ **[`Exceptions`](#exceptions)**__.__
**   ** **4. System:** **        ** **[`Print`](#print)**__,__ **[`Input`](#input)**__,__ **[`Command_Line_Arguments`](#command-line-arguments)**__,__ **[`Open`](#open)**__,__ **[`Path`](#path)**__,__ **[`Command_Execution`](#oscommands)**__.__
**   ** **5. Data:** **             ** **[`JSON`](#json)**__,__ **[`Pickle`](#pickle)**__,__ **[`CSV`](#csv)**__,__ **[`SQLite`](#sqlite)**__,__ **[`Bytes`](#bytes)**__,__ **[`Struct`](#struct)**__,__ **[`Array`](#array)**__,__ **[`MemoryView`](#memory-view)**__,__ **[`Deque`](#deque)**__.__
**   ** **6. Advanced:** **   ** **[`Threading`](#threading)**__,__ **[`Operator`](#operator)**__,__ **[`Introspection`](#introspection)**__,__ **[`Metaprograming`](#metaprograming)**__,__ **[`Eval`](#eval)**__,__ **[`Coroutine`](#coroutine)**__.__
**   ** **6. Advanced:** **   ** **[`Threading`](#threading)**__,__ **[`Operator`](#operator)**__,__ **[`Introspection`](#introspection)**__,__ **[`Metaprograming`](#metaprograming)**__,__ **[`Eval`](#eval)**__,__ **[`Coroutine`](#coroutines)**__.__
**   ** **7. Libraries:** **      ** **[`Progress_Bar`](#progress-bar)**__,__ **[`Plot`](#plot)**__,__ **[`Table`](#table)**__,__ **[`Curses`](#curses)**__,__ **[`Logging`](#logging)**__,__ **[`Scraping`](#scraping)**__,__ **[`Web`](#web)**__,__ **[`Profile`](#profiling)**__,__
**                                 ** **[`NumPy`](#numpy)**__,__ **[`Image`](#image)**__,__ **[`Animation`](#animation)**__,__ **[`Audio`](#audio)**__.__
@ -1209,10 +1209,15 @@ class MyIterable:
```
```python
>>> z = MyIterable([1, 2, 3])
>>> iter(z)
<generator object MyIterable.__iter__>
>>> 1 in z
>>> obj = MyIterable([1, 2, 3])
>>> itr = iter(obj)
>>> [next(itr), next(itr), next(itr)]
[1, 2, 3]
>>> [el for el in iter(obj)]
[1, 2, 3]
>>> [el for el in obj]
[1, 2, 3]
>>> 1 in obj
True
```
@ -2095,10 +2100,10 @@ from operator import itemgetter, attrgetter, methodcaller
```python
import operator as op
elementwise_sum = map(op.add, list_a, list_b)
product_of_elems = functools.reduce(op.mul, <collection>)
sorted_by_second = sorted(<collection>, key=op.itemgetter(1))
sorted_by_both = sorted(<collection>, key=op.itemgetter(1, 0))
product_of_elems = functools.reduce(op.mul, <collection>)
elementwise_sum = map(op.add, list_a, list_b)
LogicOp = enum.Enum('LogicOp', {'AND': op.and_, 'OR' : op.or_})
last_el = op.methodcaller('pop')(<list>)
```
@ -2238,49 +2243,64 @@ ValueError: malformed node or string
```
Coroutine
---------
* **Any function that contains a `'(yield)'` expression returns a coroutine.**
* **Coroutines are similar to iterators, but data needs to be pulled out of an iterator by calling `'next(<iter>)'`, while we push data into the coroutine by calling `'<coroutine>.send(<el>)'`.**
* **Coroutines provide more powerful data routing possibilities than iterators.**
Coroutines
----------
* **Coroutines have a lot in common with threads, but unlike threads, they only give up control when they call another coroutine and they don’t use as much memory.**
* **Coroutine definition starts with `'async'` and its call with `'await'`.**
* **`'asyncio.run(<coroutine>)'` is the main entry point for asynchronous programs.**
* **Fucntions wait(), gather() and as_completed() can be used when multiple coroutines need to be started at the same time.**
### Helper Decorator
* **All coroutines must first be "primed" by calling `'next(<coroutine>)'`.**
* **Remembering to call next() is easy to forget.**
* **Solved by wrapping coroutine functions with the following decorator:**
#### Starts a terminal game where you control an asterisk that must avoid numbers:
```python
def coroutine(func):
def out(*args, **kwargs):
cr = func(*args, **kwargs)
next(cr)
return cr
return out
```
import asyncio, collections, curses, enum, random
### Pipeline Example
```python
def reader(target):
for i in range(10):
target.send(i)
target.close()
P = collections.namedtuple('P', 'x y') # Position
D = enum.Enum('D', 'n e s w') # Direction
@coroutine
def adder(target):
def main(screen):
curses.curs_set(0) # Makes cursor invisible.
screen.nodelay(True) # Makes getch() non-blocking.
asyncio.run(main_coroutine(screen)) # Starts running asyncio code.
async def main_coroutine(screen):
state = {'*': P(0, 0), **{id_: P(30, 10) for id_ in range(10)}}
moves = asyncio.Queue()
coros = (*(random_controller(id_, moves) for id_ in range(10)),
human_controller(screen, moves),
model(moves, state, *screen.getmaxyx()),
view(state, screen))
await asyncio.wait(coros, return_when=asyncio.FIRST_COMPLETED)
async def random_controller(id_, moves):
while True:
value = (yield)
target.send(value + 100)
@coroutine
def printer():
moves.put_nowait((id_, random.choice(list(D))))
await asyncio.sleep(random.random() / 2)
async def human_controller(screen, moves):
while (ch := screen.getch()) != 27:
key_mappings = {259: D.n, 261: D.e, 258: D.s, 260: D.w}
if ch in key_mappings:
moves.put_nowait(('*', key_mappings[ch]))
await asyncio.sleep(0.01)
async def model(moves, state, height, width):
while state['*'] not in {p for id_, p in state.items() if id_ != '*'}:
id_, d = await moves.get()
p = state[id_]
deltas = {D.n: P(0, -1), D.e: P(1, 0), D.s: P(0, 1), D.w: P(-1, 0)}
new_p = P(*[sum(a) for a in zip(p, deltas[d])])
if 0 <= new_p.x < width-1 and 0 <= new_p.y < height:
state[id_] = new_p
async def view(state, screen):
while True:
value = (yield)
print(value, end=' ')
```
screen.clear()
for id_, p in state.items():
screen.addstr(p.y, p.x, str(id_))
await asyncio.sleep(0.01)
```python
>>> reader(adder(printer()))
100 101 102 103 104 105 106 107 108 109
curses.wrapper(main)
```
<br>
@ -2329,7 +2349,7 @@ with open('test.csv', encoding='utf-8', newline='') as file:
Curses
------
#### Clears the terminal, prints a message and waits for an ESC key press:
#### Clears the terminal, prints a message and waits for the ESC key press:
```python
from curses import wrapper, curs_set, ascii
from curses import KEY_UP, KEY_RIGHT, KEY_DOWN, KEY_LEFT

108
index.html

@ -1147,10 +1147,15 @@ Hello World!
<pre><code class="python language-python hljs"><span class="hljs-meta">&gt;&gt;&gt; </span>z = MyIterable([<span class="hljs-number">1</span>, <span class="hljs-number">2</span>, <span class="hljs-number">3</span>])
<span class="hljs-meta">&gt;&gt;&gt; </span>iter(z)
&lt;generator object MyIterable.__iter__&gt;
<span class="hljs-meta">&gt;&gt;&gt; </span><span class="hljs-number">1</span> <span class="hljs-keyword">in</span> z
<pre><code class="python language-python hljs"><span class="hljs-meta">&gt;&gt;&gt; </span>obj = MyIterable([<span class="hljs-number">1</span>, <span class="hljs-number">2</span>, <span class="hljs-number">3</span>])
<span class="hljs-meta">&gt;&gt;&gt; </span>itr = iter(obj)
<span class="hljs-meta">&gt;&gt;&gt; </span>[next(itr), next(itr), next(itr)]
[<span class="hljs-number">1</span>, <span class="hljs-number">2</span>, <span class="hljs-number">3</span>]
<span class="hljs-meta">&gt;&gt;&gt; </span>[el <span class="hljs-keyword">for</span> el <span class="hljs-keyword">in</span> iter(obj)]
[<span class="hljs-number">1</span>, <span class="hljs-number">2</span>, <span class="hljs-number">3</span>]
<span class="hljs-meta">&gt;&gt;&gt; </span>[el <span class="hljs-keyword">for</span> el <span class="hljs-keyword">in</span> obj]
[<span class="hljs-number">1</span>, <span class="hljs-number">2</span>, <span class="hljs-number">3</span>]
<span class="hljs-meta">&gt;&gt;&gt; </span><span class="hljs-number">1</span> <span class="hljs-keyword">in</span> obj
<span class="hljs-keyword">True</span>
</code></pre>
<div><h3 id="collection">Collection</h3><ul>
@ -1727,14 +1732,14 @@ db = connector.connect(host=&lt;str&gt;, user=&lt;str&gt;, password=&lt;str&gt;,
<li><strong><code class="python hljs"><span class="hljs-string">'='</span></code> - native byte order</strong></li>
<li><strong><code class="python hljs"><span class="hljs-string">'&lt;'</span></code> - little-endian</strong></li>
<li><strong><code class="python hljs"><span class="hljs-string">'&gt;'</span></code> - big-endian (also <code class="python hljs"><span class="hljs-string">'!'</span></code>)</strong></li>
</ul><div><h4 id="integertypesuseacapitalletterforunsignedtypestandardsizesareinbrackets">Integer types. Use a capital letter for unsigned type. Standard sizes are in brackets:</h4><ul>
</ul></div><div><h4 id="integertypesuseacapitalletterforunsignedtypestandardsizesareinbrackets">Integer types. Use a capital letter for unsigned type. Standard sizes are in brackets:</h4><ul>
<li><strong><code class="python hljs"><span class="hljs-string">'x'</span></code> - pad byte</strong></li>
<li><strong><code class="python hljs"><span class="hljs-string">'b'</span></code> - char (1)</strong></li>
<li><strong><code class="python hljs"><span class="hljs-string">'h'</span></code> - short (2)</strong></li>
<li><strong><code class="python hljs"><span class="hljs-string">'i'</span></code> - int (4)</strong></li>
<li><strong><code class="python hljs"><span class="hljs-string">'l'</span></code> - long (4)</strong></li>
<li><strong><code class="python hljs"><span class="hljs-string">'q'</span></code> - long long (8)</strong></li>
</ul></div></div><div><h4 id="floatingpointtypes">Floating point types:</h4><ul>
</ul></div><div><h4 id="floatingpointtypes">Floating point types:</h4><ul>
<li><strong><code class="python hljs"><span class="hljs-string">'f'</span></code> - float (4)</strong></li>
<li><strong><code class="python hljs"><span class="hljs-string">'d'</span></code> - double (8)</strong></li>
</ul></div>
@ -1840,10 +1845,10 @@ lock.release()
<pre><code class="python language-python hljs"><span class="hljs-keyword">import</span> operator <span class="hljs-keyword">as</span> op
elementwise_sum = map(op.add, list_a, list_b)
product_of_elems = functools.reduce(op.mul, &lt;collection&gt;)
sorted_by_second = sorted(&lt;collection&gt;, key=op.itemgetter(<span class="hljs-number">1</span>))
sorted_by_both = sorted(&lt;collection&gt;, key=op.itemgetter(<span class="hljs-number">1</span>, <span class="hljs-number">0</span>))
product_of_elems = functools.reduce(op.mul, &lt;collection&gt;)
elementwise_sum = map(op.add, list_a, list_b)
LogicOp = enum.Enum(<span class="hljs-string">'LogicOp'</span>, {<span class="hljs-string">'AND'</span>: op.and_, <span class="hljs-string">'OR'</span> : op.or_})
last_el = op.methodcaller(<span class="hljs-string">'pop'</span>)(&lt;list&gt;)
</code></pre>
@ -1941,46 +1946,63 @@ MyMetaClass.__base__ == type <span class="hljs-comment"># MyMetaClass is
ValueError: malformed node or string
</code></pre></div>
<div><h2 id="coroutine"><a href="#coroutine" name="coroutine">#</a>Coroutine</h2><ul>
<li><strong>Any function that contains a <code class="python hljs"><span class="hljs-string">'(yield)'</span></code> expression returns a coroutine.</strong></li>
<li><strong>Coroutines are similar to iterators, but data needs to be pulled out of an iterator by calling <code class="python hljs"><span class="hljs-string">'next(&lt;iter&gt;)'</span></code>, while we push data into the coroutine by calling <code class="python hljs"><span class="hljs-string">'&lt;coroutine&gt;.send(&lt;el&gt;)'</span></code>.</strong></li>
<li><strong>Coroutines provide more powerful data routing possibilities than iterators.</strong></li>
</ul><div><h3 id="helperdecorator">Helper Decorator</h3><ul>
<li><strong>All coroutines must first be "primed" by calling <code class="python hljs"><span class="hljs-string">'next(&lt;coroutine&gt;)'</span></code>.</strong></li>
<li><strong>Remembering to call next() is easy to forget.</strong></li>
<li><strong>Solved by wrapping coroutine functions with the following decorator:</strong></li>
</ul><pre><code class="python language-python hljs"><span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">coroutine</span><span class="hljs-params">(func)</span>:</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">out</span><span class="hljs-params">(*args, **kwargs)</span>:</span>
cr = func(*args, **kwargs)
next(cr)
<span class="hljs-keyword">return</span> cr
<span class="hljs-keyword">return</span> out
</code></pre></div></div>
<div><h2 id="coroutines"><a href="#coroutines" name="coroutines">#</a>Coroutines</h2><ul>
<li><strong>Coroutines have a lot in common with threads, but unlike threads, they only give up control when they call another coroutine and they don’t use as much memory.</strong></li>
<li><strong>Coroutine definition starts with <code class="python hljs"><span class="hljs-string">'async'</span></code> and its call with <code class="python hljs"><span class="hljs-string">'await'</span></code>.</strong></li>
<li><strong><code class="python hljs"><span class="hljs-string">'asyncio.run(&lt;coroutine&gt;)'</span></code> is the main entry point for asynchronous programs.</strong></li>
<li><strong>Fucntions wait(), gather() and as_completed() can be used when multiple coroutines need to be started at the same time.</strong></li>
</ul><div><h4 id="startsaterminalgamewhereyoucontrolanasteriskthatmustavoidnumbers">Starts a terminal game where you control an asterisk that must avoid numbers:</h4><pre><code class="python language-python hljs"><span class="hljs-keyword">import</span> asyncio, collections, curses, enum, random
P = collections.namedtuple(<span class="hljs-string">'P'</span>, <span class="hljs-string">'x y'</span>) <span class="hljs-comment"># Position</span>
D = enum.Enum(<span class="hljs-string">'D'</span>, <span class="hljs-string">'n e s w'</span>) <span class="hljs-comment"># Direction</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">main</span><span class="hljs-params">(screen)</span>:</span>
curses.curs_set(<span class="hljs-number">0</span>) <span class="hljs-comment"># Makes cursor invisible.</span>
screen.nodelay(<span class="hljs-keyword">True</span>) <span class="hljs-comment"># Makes getch() non-blocking.</span>
asyncio.run(main_coroutine(screen)) <span class="hljs-comment"># Starts running asyncio code.</span>
<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">main_coroutine</span><span class="hljs-params">(screen)</span>:</span>
state = {<span class="hljs-string">'*'</span>: P(<span class="hljs-number">0</span>, <span class="hljs-number">0</span>), **{id_: P(<span class="hljs-number">30</span>, <span class="hljs-number">10</span>) <span class="hljs-keyword">for</span> id_ <span class="hljs-keyword">in</span> range(<span class="hljs-number">10</span>)}}
moves = asyncio.Queue()
coros = (*(random_controller(id_, moves) <span class="hljs-keyword">for</span> id_ <span class="hljs-keyword">in</span> range(<span class="hljs-number">10</span>)),
human_controller(screen, moves),
model(moves, state, *screen.getmaxyx()),
view(state, screen))
<span class="hljs-keyword">await</span> asyncio.wait(coros, return_when=asyncio.FIRST_COMPLETED)
<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">random_controller</span><span class="hljs-params">(id_, moves)</span>:</span>
<span class="hljs-keyword">while</span> <span class="hljs-keyword">True</span>:
moves.put_nowait((id_, random.choice(list(D))))
<span class="hljs-keyword">await</span> asyncio.sleep(random.random() / <span class="hljs-number">2</span>)
<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">human_controller</span><span class="hljs-params">(screen, moves)</span>:</span>
<span class="hljs-keyword">while</span> (ch := screen.getch()) != <span class="hljs-number">27</span>:
key_mappings = {<span class="hljs-number">259</span>: D.n, <span class="hljs-number">261</span>: D.e, <span class="hljs-number">258</span>: D.s, <span class="hljs-number">260</span>: D.w}
<span class="hljs-keyword">if</span> ch <span class="hljs-keyword">in</span> key_mappings:
moves.put_nowait((<span class="hljs-string">'*'</span>, key_mappings[ch]))
<span class="hljs-keyword">await</span> asyncio.sleep(<span class="hljs-number">0.01</span>)
<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">model</span><span class="hljs-params">(moves, state, height, width)</span>:</span>
<span class="hljs-keyword">while</span> state[<span class="hljs-string">'*'</span>] <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> {p <span class="hljs-keyword">for</span> id_, p <span class="hljs-keyword">in</span> state.items() <span class="hljs-keyword">if</span> id_ != <span class="hljs-string">'*'</span>}:
id_, d = <span class="hljs-keyword">await</span> moves.get()
p = state[id_]
deltas = {D.n: P(<span class="hljs-number">0</span>, <span class="hljs-number">-1</span>), D.e: P(<span class="hljs-number">1</span>, <span class="hljs-number">0</span>), D.s: P(<span class="hljs-number">0</span>, <span class="hljs-number">1</span>), D.w: P(<span class="hljs-number">-1</span>, <span class="hljs-number">0</span>)}
new_p = P(*[sum(a) <span class="hljs-keyword">for</span> a <span class="hljs-keyword">in</span> zip(p, deltas[d])])
<span class="hljs-keyword">if</span> <span class="hljs-number">0</span> &lt;= new_p.x &lt; width<span class="hljs-number">-1</span> <span class="hljs-keyword">and</span> <span class="hljs-number">0</span> &lt;= new_p.y &lt; height:
state[id_] = new_p
<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">view</span><span class="hljs-params">(state, screen)</span>:</span>
<span class="hljs-keyword">while</span> <span class="hljs-keyword">True</span>:
screen.clear()
<span class="hljs-keyword">for</span> id_, p <span class="hljs-keyword">in</span> state.items():
screen.addstr(p.y, p.x, str(id_))
<span class="hljs-keyword">await</span> asyncio.sleep(<span class="hljs-number">0.01</span>)
curses.wrapper(main)
</code></pre></div></div>
<div><h3 id="pipelineexample">Pipeline Example</h3><pre><code class="python language-python hljs"><span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">reader</span><span class="hljs-params">(target)</span>:</span>
<span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(<span class="hljs-number">10</span>):
target.send(i)
target.close()
<span class="hljs-meta">@coroutine</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">adder</span><span class="hljs-params">(target)</span>:</span>
<span class="hljs-keyword">while</span> <span class="hljs-keyword">True</span>:
value = (<span class="hljs-keyword">yield</span>)
target.send(value + <span class="hljs-number">100</span>)
<span class="hljs-meta">@coroutine</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">printer</span><span class="hljs-params">()</span>:</span>
<span class="hljs-keyword">while</span> <span class="hljs-keyword">True</span>:
value = (<span class="hljs-keyword">yield</span>)
print(value, end=<span class="hljs-string">' '</span>)
</code></pre></div>
<pre><code class="python language-python hljs"><span class="hljs-meta">&gt;&gt;&gt; </span>reader(adder(printer()))
<span class="hljs-number">100</span> <span class="hljs-number">101</span> <span class="hljs-number">102</span> <span class="hljs-number">103</span> <span class="hljs-number">104</span> <span class="hljs-number">105</span> <span class="hljs-number">106</span> <span class="hljs-number">107</span> <span class="hljs-number">108</span> <span class="hljs-number">109</span>
</code></pre>
<p><br></p>
<div><h1 id="libraries">Libraries</h1><div><h2 id="progressbar"><a href="#progressbar" name="progressbar">#</a>Progress Bar</h2><pre><code class="python language-python hljs"><span class="hljs-comment"># $ pip3 install tqdm</span>
<span class="hljs-keyword">from</span> tqdm <span class="hljs-keyword">import</span> tqdm
@ -2010,7 +2032,7 @@ pyplot.clf() <span class="hljs-comment"># Clea
</code></pre></div></div>
<div><h2 id="curses"><a href="#curses" name="curses">#</a>Curses</h2><div><h4 id="clearstheterminalprintsamessageandwaitsforanesckeypress">Clears the terminal, prints a message and waits for an ESC key press:</h4><pre><code class="python language-python hljs"><span class="hljs-keyword">from</span> curses <span class="hljs-keyword">import</span> wrapper, curs_set, ascii
<div><h2 id="curses"><a href="#curses" name="curses">#</a>Curses</h2><div><h4 id="clearstheterminalprintsamessageandwaitsfortheesckeypress">Clears the terminal, prints a message and waits for the ESC key press:</h4><pre><code class="python language-python hljs"><span class="hljs-keyword">from</span> curses <span class="hljs-keyword">import</span> wrapper, curs_set, ascii
<span class="hljs-keyword">from</span> curses <span class="hljs-keyword">import</span> KEY_UP, KEY_RIGHT, KEY_DOWN, KEY_LEFT
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">main</span><span class="hljs-params">()</span>:</span>

2
parse.js

@ -193,7 +193,7 @@ function fixPageBreaksFile() {
function fixPageBreaksStruct() {
const formatDiv = $('#floatingpointtypes').parent().parent().parent().parent()
move(formatDiv, 'floatingpointtypes')
move(formatDiv, 'integertypesusecapitalletterforunsignedtypestandardsizesareinbrackets')
move(formatDiv, 'integertypesuseacapitalletterforunsignedtypestandardsizesareinbrackets')
move(formatDiv, 'forstandardsizesstartformatstringwith')
}

|||||||
100:0
Loading…
Cancel
Save