Comprehensive Python Cheatsheet =============================== [Download text file](https://raw.githubusercontent.com/gto76/python-cheatsheet/master/README.md), [Buy PDF](https://transactions.sendowl.com/products/78175486/4422834F/view), [Fork me on GitHub](https://github.com/gto76/python-cheatsheet) or [Check out FAQ](https://github.com/gto76/python-cheatsheet/wiki/Frequently-Asked-Questions). ![Monty Python](web/image_888.jpeg) Contents -------- **   ** **1. Collections:** ** ** **[`List`](#list)**__,__ **[`Dictionary`](#dictionary)**__,__ **[`Set`](#set)**__,__ **[`Tuple`](#tuple)**__,__ **[`Range`](#range)**__,__ **[`Enumerate`](#enumerate)**__,__ **[`Iterator`](#iterator)**__,__ **[`Generator`](#generator)**__.__ **   ** **2. Types:** **          ** **[`Type`](#type)**__,__ **[`String`](#string)**__,__ **[`Regular_Exp`](#regex)**__,__ **[`Format`](#format)**__,__ **[`Numbers`](#numbers)**__,__ **[`Combinatorics`](#combinatorics)**__,__ **[`Datetime`](#datetime)**__.__ **   ** **3. Syntax:** **         ** **[`Args`](#arguments)**__,__ **[`Inline`](#inline)**__,__ **[`Closure`](#closure)**__,__ **[`Decorator`](#decorator)**__,__ **[`Class`](#class)**__,__ **[`Duck_Type`](#duck-types)**__,__ **[`Enum`](#enum)**__,__ **[`Exception`](#exceptions)**__.__ **   ** **4. System:** **        ** **[`Exit`](#exit)**__,__ **[`Print`](#print)**__,__ **[`Input`](#input)**__,__ **[`Command_Line_Arguments`](#command-line-arguments)**__,__ **[`Open`](#open)**__,__ **[`Path`](#path)**__,__ **[`OS_Commands`](#oscommands)**__.__ **   ** **5. Data:** **             ** **[`JSON`](#json)**__,__ **[`Pickle`](#pickle)**__,__ **[`CSV`](#csv)**__,__ **[`SQLite`](#sqlite)**__,__ **[`Bytes`](#bytes)**__,__ **[`Struct`](#struct)**__,__ **[`Array`](#array)**__,__ **[`Memory_View`](#memory-view)**__,__ **[`Deque`](#deque)**__.__ **   ** **6. Advanced:** **   ** **[`Threading`](#threading)**__,__ **[`Operator`](#operator)**__,__ **[`Introspection`](#introspection)**__,__ **[`Metaprograming`](#metaprograming)**__,__ **[`Eval`](#eval)**__,__ **[`Coroutines`](#coroutines)**__.__ **   ** **7. Libraries:** **      ** **[`Progress_Bar`](#progress-bar)**__,__ **[`Plot`](#plot)**__,__ **[`Table`](#table)**__,__ **[`Curses`](#curses)**__,__ **[`Logging`](#logging)**__,__ **[`Scraping`](#scraping)**__,__ **[`Web`](#web)**__,__ **[`Profile`](#profiling)**__,__ **                                 ** **[`NumPy`](#numpy)**__,__ **[`Image`](#image)**__,__ **[`Audio`](#audio)**__,__ **[`Games`](#pygame)**__,__ **[`Data`](#pandas)**__,__ **[`Cython`](#cython)**__.__ Main ---- ```python if __name__ == '__main__': # Runs main() if file wasn't imported. main() ``` List ---- ```python = [from_inclusive : to_exclusive : ±step_size] ``` ```python .append() # Or: += [] .extend() # Or: += ``` ```python .sort() .reverse() = sorted() = reversed() ``` ```python sum_of_elements = sum() elementwise_sum = [sum(pair) for pair in zip(list_a, list_b)] sorted_by_second = sorted(, key=lambda el: el[1]) sorted_by_both = sorted(, key=lambda el: (el[1], el[0])) flatter_list = list(itertools.chain.from_iterable()) product_of_elems = functools.reduce(lambda out, el: out * el, ) list_of_chars = list() ``` * **Module [operator](#operator) provides functions itemgetter() and mul() that offer the same functionality as [lambda](#lambda) expressions above.** ```python = .count() # Returns number of occurrences. Also works on strings. index = .index() # Returns index of first occurrence or raises ValueError. .insert(index, ) # Inserts item at index and moves the rest to the right. = .pop([index]) # Removes and returns item at index or from the end. .remove() # Removes first occurrence of item or raises ValueError. .clear() # Removes all items. Also works on dictionary and set. ``` Dictionary ---------- ```python = .keys() # Coll. of keys that reflects changes. = .values() # Coll. of values that reflects changes. = .items() # Coll. of key-value tuples that reflects chgs. ``` ```python value = .get(key, default=None) # Returns default if key is missing. value = .setdefault(key, default=None) # Returns and writes default if key is missing. = collections.defaultdict() # Creates a dict with default value of type. = collections.defaultdict(lambda: 1) # Creates a dict with default value 1. ``` ```python = dict() # Creates a dict from coll. of key-value pairs. = dict(zip(keys, values)) # Creates a dict from two collections. = dict.fromkeys(keys [, value]) # Creates a dict from collection of keys. ``` ```python .update() # Adds items. Replaces ones with matching keys. value = .pop(key) # Removes item or raises KeyError. {k for k, v in .items() if v == value} # Returns set of keys that point to the value. {k: v for k, v in .items() if k in keys} # Returns a dictionary, filtered by keys. ``` ### Counter ```python >>> from collections import Counter >>> colors = ['blue', 'blue', 'blue', 'red', 'red'] >>> counter = Counter(colors) >>> counter['yellow'] += 1 Counter({'blue': 3, 'red': 2, 'yellow': 1}) >>> counter.most_common()[0] ('blue', 3) ``` Set --- ```python = set() ``` ```python .add() # Or: |= {} .update() # Or: |= ``` ```python = .union() # Or: | = .intersection() # Or: & = .difference() # Or: - = .symmetric_difference() # Or: ^ = .issubset() # Or: <= = .issuperset() # Or: >= ``` ```python = .pop() # Raises KeyError if empty. .remove() # Raises KeyError if missing. .discard() # Doesn't raise an error. ``` ### Frozen Set * **Is immutable and hashable.** * **That means it can be used as a key in a dictionary or as an element in a set.** ```python = frozenset() ``` Tuple ----- **Tuple is an immutable and hashable list.** ```python = () = (, ) = (, [, ...]) ``` ### Named Tuple **Tuple's subclass with named elements.** ```python >>> from collections import namedtuple >>> Point = namedtuple('Point', 'x y') >>> p = Point(1, y=2) Point(x=1, y=2) >>> p[0] 1 >>> p.x 1 >>> getattr(p, 'y') 2 >>> p._fields # Or: Point._fields ('x', 'y') ``` Range ----- ```python = range(to_exclusive) = range(from_inclusive, to_exclusive) = range(from_inclusive, to_exclusive, ±step_size) ``` ```python from_inclusive = .start to_exclusive = .stop ``` Enumerate --------- ```python for i, el in enumerate( [, i_start]): ... ``` Iterator -------- ```python = iter() # `iter()` returns unmodified iterator. = iter(, to_exclusive) # A sequence of return values until 'to_exclusive'. = next( [, default]) # Raises StopIteration or returns 'default' on end. = list() # Returns a list of iterator's remaining elements. ``` ### Itertools ```python from itertools import count, repeat, cycle, chain, islice ``` ```python = count(start=0, step=1) # Returns updated value endlessly. Accepts floats. = repeat( [, times]) # Returns element endlessly or 'times' times. = cycle() # Repeats the sequence endlessly. ``` ```python = chain(, [, ...]) # Empties collections in order. = chain.from_iterable() # Empties collections inside a collection in order. ``` ```python = islice(, to_exclusive) = islice(, from_inclusive, to_exclusive [, +step_size]) ``` Generator --------- * **Any function that contains a yield statement returns a generator.** * **Generators and iterators are interchangeable.** ```python def count(start, step): while True: yield start start += step ``` ```python >>> counter = count(10, 2) >>> next(counter), next(counter), next(counter) (10, 12, 14) ``` Type ---- * **Everything is an object.** * **Every object has a type.** * **Type and class are synonymous.** ```python = type() # Or: .__class__ = isinstance(, ) # Or: issubclass(type(), ) ``` ```python >>> type('a'), 'a'.__class__, str (, , ) ``` #### Some types do not have built-in names, so they must be imported: ```python from types import FunctionType, MethodType, LambdaType, GeneratorType ``` ### Abstract Base Classes **Each abstract base class specifies a set of virtual subclasses. These classes are then recognized by isinstance() and issubclass() as subclasses of the ABC, although they are really not.** ```python >>> from collections.abc import Sequence, Collection, Iterable >>> isinstance([1, 2, 3], Iterable) True ``` ```text +------------------+------------+------------+------------+ | | Sequence | Collection | Iterable | +------------------+------------+------------+------------+ | list, range, str | yes | yes | yes | | dict, set | | yes | yes | | iter | | | yes | +------------------+------------+------------+------------+ ``` ```python >>> from numbers import Integral, Rational, Real, Complex, Number >>> isinstance(123, Number) True ``` ```text +--------------------+----------+----------+----------+----------+----------+ | | Integral | Rational | Real | Complex | Number | +--------------------+----------+----------+----------+----------+----------+ | int | yes | yes | yes | yes | yes | | fractions.Fraction | | yes | yes | yes | yes | | float | | | yes | yes | yes | | complex | | | | yes | yes | | decimal.Decimal | | | | | yes | +--------------------+----------+----------+----------+----------+----------+ ``` String ------ ```python = .strip() # Strips all whitespace characters from both ends. = .strip('') # Strips all passed characters from both ends. ``` ```python = .split() # Splits on one or more whitespace characters. = .split(sep=None, maxsplit=-1) # Splits on 'sep' str at most 'maxsplit' times. = .splitlines(keepends=False) # Splits on \n,\r,\r\n. Keeps them if 'keepends'. = .join() # Joins elements using string as separator. ``` ```python = in # Checks if string contains a substring. = .startswith() # Pass tuple of strings for multiple options. = .endswith() # Pass tuple of strings for multiple options. = .find() # Returns start index of first match or -1. = .index() # Same but raises ValueError if missing. ``` ```python = .replace(old, new [, count]) # Replaces 'old' with 'new' at most 'count' times. = .translate() # Use `str.maketrans()` to generate table. ``` ```python = chr() # Converts int to Unicode char. = ord() # Converts Unicode char to int. ``` * **Also: `'lstrip()'`, `'rstrip()'`.** * **Also: `'lower()'`, `'upper()'`, `'capitalize()'` and `'title()'`.** ### Property Methods ```text +---------------+----------+----------+----------+----------+----------+ | | [ !#$%…] | [a-zA-Z] | [¼½¾] | [²³¹] | [0-9] | +---------------+----------+----------+----------+----------+----------+ | isprintable() | yes | yes | yes | yes | yes | | isalnum() | | yes | yes | yes | yes | | isnumeric() | | | yes | yes | yes | | isdigit() | | | | yes | yes | | isdecimal() | | | | | yes | +---------------+----------+----------+----------+----------+----------+ ``` * **Also: `'isspace()'` checks for `'[ \t\n\r\f\v…]'`.** Regex ----- ```python import re = re.sub(, new, text, count=0) # Substitutes all occurrences with 'new'. = re.findall(, text) # Returns all occurrences as strings. = re.split(, text, maxsplit=0) # Use brackets in regex to include the matches. = re.search(, text) # Searches for first occurrence of the pattern. = re.match(, text) # Searches only at the beginning of the text. = re.finditer(, text) # Returns all occurrences as match objects. ``` * **Search() and match() return None if they can't find a match.** * **Argument `'flags=re.IGNORECASE'` can be used with all functions.** * **Argument `'flags=re.MULTILINE'` makes `'^'` and `'$'` match the start/end of each line.** * **Argument `'flags=re.DOTALL'` makes dot also accept the `'\n'`.** * **Use `r'\1'` or `'\\1'` for backreference.** * **Add `'?'` after an operator to make it non-greedy.** ### Match Object ```python = .group() # Returns the whole match. Also group(0). = .group(1) # Returns part in the first bracket. = .groups() # Returns all bracketed parts. = .start() # Returns start index of the match. = .end() # Returns exclusive end index of the match. ``` ### Special Sequences * **By default digits, alphanumerics and whitespaces from all alphabets are matched, unless `'flags=re.ASCII'` argument is used.** * **Use a capital letter for negation.** ```python '\d' == '[0-9]' # Matches any digit. '\w' == '[a-zA-Z0-9_]' # Matches any alphanumeric. '\s' == '[ \t\n\r\f\v]' # Matches any whitespace. ``` Format ------ ```python = f'{}, {}' = '{}, {}'.format(, ) ``` ### Attributes ```python >>> from collections import namedtuple >>> Person = namedtuple('Person', 'name height') >>> person = Person('Jean-Luc', 187) >>> f'{person.height}' '187' >>> '{p.height}'.format(p=person) '187' ``` ### General Options ```python {:<10} # ' ' {:^10} # ' ' {:>10} # ' ' {:.<10} # '......' {:<0} # '' ``` ### Strings **`'!r'` calls object's [repr()](#class) method, instead of [str()](#class), to get a string.** ```python {'abcde'!r:10} # "'abcde' " {'abcde':10.3} # 'abc ' {'abcde':.3} # 'abc' ``` ### Numbers ```python { 123456:10,} # ' 123,456' { 123456:10_} # ' 123_456' { 123456:+10} # ' +123456' {-123456:=10} # '- 123456' { 123456: } # ' 123456' {-123456: } # '-123456' ``` ### Floats ```python {1.23456:10.3} # ' 1.23' {1.23456:10.3f} # ' 1.235' {1.23456:10.3e} # ' 1.235e+00' {1.23456:10.3%} # ' 123.456%' ``` #### Comparison of presentation types: ```text +---------------+-----------------+-----------------+-----------------+-----------------+ | | {} | {:f} | {:e} | {:%} | +---------------+-----------------+-----------------+-----------------+-----------------+ | 0.000056789 | '5.6789e-05' | '0.000057' | '5.678900e-05' | '0.005679%' | | 0.00056789 | '0.00056789' | '0.000568' | '5.678900e-04' | '0.056789%' | | 0.0056789 | '0.0056789' | '0.005679' | '5.678900e-03' | '0.567890%' | | 0.056789 | '0.056789' | '0.056789' | '5.678900e-02' | '5.678900%' | | 0.56789 | '0.56789' | '0.567890' | '5.678900e-01' | '56.789000%' | | 5.6789 | '5.6789' | '5.678900' | '5.678900e+00' | '567.890000%' | | 56.789 | '56.789' | '56.789000' | '5.678900e+01' | '5678.900000%' | | 567.89 | '567.89' | '567.890000' | '5.678900e+02' | '56789.000000%' | +---------------+-----------------+-----------------+-----------------+-----------------+ ``` ```text +---------------+-----------------+-----------------+-----------------+-----------------+ | | {:.2} | {:.2f} | {:.2e} | {:.2%} | +---------------+-----------------+-----------------+-----------------+-----------------+ | 0.000056789 | '5.7e-05' | '0.00' | '5.68e-05' | '0.01%' | | 0.00056789 | '0.00057' | '0.00' | '5.68e-04' | '0.06%' | | 0.0056789 | '0.0057' | '0.01' | '5.68e-03' | '0.57%' | | 0.056789 | '0.057' | '0.06' | '5.68e-02' | '5.68%' | | 0.56789 | '0.57' | '0.57' | '5.68e-01' | '56.79%' | | 5.6789 | '5.7' | '5.68' | '5.68e+00' | '567.89%' | | 56.789 | '5.7e+01' | '56.79' | '5.68e+01' | '5678.90%' | | 567.89 | '5.7e+02' | '567.89' | '5.68e+02' | '56789.00%' | +---------------+-----------------+-----------------+-----------------+-----------------+ ``` ### Ints ```python {90:c} # 'Z' {90:b} # '1011010' {90:X} # '5A' ``` Numbers ------- ### Types ```python = int() # Or: math.floor() = float() # Or: = complex(real=0, imag=0) # Or: ± j = fractions.Fraction(0, 1) # Or: Fraction(numerator=0, denominator=1) = decimal.Decimal() # Or: Decimal((sign, digits, exponent)) ``` * **`'int()'` and `'float()'` raise ValueError on malformed strings.** * **Decimal numbers can be represented exactly, unlike floats where `'1.1 + 2.2 != 3.3'`.** * **Precision of decimal operations is set with: `'decimal.getcontext().prec = '`.** ### Basic Functions ```python = pow(, ) # Or: ** = abs() # = abs() = round( [, ±ndigits]) # `round(126, -1) == 130` ``` ### Math ```python from math import e, pi, inf, nan, isinf, isnan from math import cos, acos, sin, asin, tan, atan, degrees, radians from math import log, log10, log2 ``` ### Statistics ```python from statistics import mean, median, variance, stdev, pvariance, pstdev ``` ### Random ```python from random import random, randint, choice, shuffle = random() = randint(from_inclusive, to_inclusive) = choice() shuffle() ``` ### Bin, Hex ```python = ±0b # Or: ±0x = int('±', 2) # Or: int('±', 16) = int('±0b', 0) # Or: int('±0x', 0) '[-]0b' = bin() # Or: hex() ``` ### Bitwise Operators ```python = & # And = | # Or = ^ # Xor (0 if both bits equal) = << n_bits # Shift left (>> for right) = ~ # Not (also: - - 1) ``` Combinatorics ------------- * **Every function returns an iterator.** * **If you want to print the iterator, you need to pass it to the list() function first!** ```python from itertools import product, combinations, combinations_with_replacement, permutations ``` ```python >>> product([0, 1], repeat=3) [(0, 0, 0), (0, 0, 1), (0, 1, 0), (0, 1, 1), (1, 0, 0), (1, 0, 1), (1, 1, 0), (1, 1, 1)] ``` ```python >>> product('ab', '12') [('a', '1'), ('a', '2'), ('b', '1'), ('b', '2')] ``` ```python >>> combinations('abc', 2) [('a', 'b'), ('a', 'c'), ('b', 'c')] ``` ```python >>> combinations_with_replacement('abc', 2) [('a', 'a'), ('a', 'b'), ('a', 'c'), ('b', 'b'), ('b', 'c'), ('c', 'c')] ``` ```python >>> permutations('abc', 2) [('a', 'b'), ('a', 'c'), ('b', 'a'), ('b', 'c'), ('c', 'a'), ('c', 'b')] ``` Datetime -------- * **Module 'datetime' provides 'date' ``, 'time' ``, 'datetime' `
` and 'timedelta' `
` classes. All are immutable and hashable.** * **Time and datetime objects can be 'aware' ``, meaning they have defined timezone, or 'naive' ``, meaning they don't.** * **If object is naive, it is presumed to be in the system's timezone.** ```python from datetime import date, time, datetime, timedelta from dateutil.tz import UTC, tzlocal, gettz, resolve_imaginary ``` ### Constructors ```python = date(year, month, day) = time(hour=0, minute=0, second=0, microsecond=0, tzinfo=None, fold=0)
= datetime(year, month, day, hour=0, minute=0, second=0, ...)
= timedelta(days=0, seconds=0, microseconds=0, milliseconds=0, minutes=0, hours=0, weeks=0) ``` * **Use `'.weekday()'` to get the day of the week (Mon == 0).** * **`'fold=1'` means the second pass in case of time jumping back for one hour.** * **`' = resolve_imaginary()'` fixes DTs that fall into the missing hour.** ### Now ```python = D/DT.today() # Current local date or naive datetime. = DT.utcnow() # Naive datetime from current UTC time. = DT.now() # Aware datetime from current tz time. ``` * **To extract time use `'.time()'`, `'.time()'` or `'.timetz()'`.** ### Timezone ```python = UTC # UTC timezone. London without DST. = tzlocal() # Local timezone. Also gettz(). = gettz('/') # 'Continent/City_Name' timezone or None. =
.astimezone() # Datetime, converted to passed timezone. = .replace(tzinfo=) # Unconverted object with new timezone. ``` ### Encode ```python = D/T/DT.fromisoformat('') # Object from ISO string. Raises ValueError.
= DT.strptime(, '') # Datetime from str, according to format. = D/DT.fromordinal() # D/DTn from days since Christ, at midnight. = DT.fromtimestamp() # Local time DTn from seconds since Epoch. = DT.fromtimestamp(, ) # Aware datetime from seconds since Epoch. ``` * **ISO strings come in following forms: `'YYYY-MM-DD'`, `'HH:MM:SS.ffffff[±]'`, or both separated by an arbitrary character. Offset is formatted as: `'HH:MM'`.** * **Epoch on Unix systems is: `'1970-01-01 00:00 UTC'`, `'1970-01-01 01:00 CET'`, ...** ### Decode ```python = .isoformat(sep='T') # Also timespec='auto/hours/minutes/seconds'. = .strftime('') # Custom string representation. = .toordinal() # Days since Christ, ignoring time and tz. = .timestamp() # Seconds since Epoch, from DTn in local tz. = .timestamp() # Seconds since Epoch, from DTa. ``` ### Format ```python >>> from datetime import datetime >>> dt = datetime.strptime('2015-05-14 23:39:00.00 +0200', '%Y-%m-%d %H:%M:%S.%f %z') >>> dt.strftime("%A, %dth of %B '%y, %I:%M%p %Z") "Thursday, 14th of May '15, 11:39PM UTC+02:00" ``` * **When parsing, `'%z'` also accepts `'±HH:MM'`.** * **For abbreviated weekday and month use `'%a'` and `'%b'`.** ### Arithmetics ```python = ±
# Returned datetime can fall into missing hour. = - # Returns the difference, ignoring time jumps. = - # Ignores time jumps if they share tzinfo object. = - # Convert DTs to UTC to get the actual delta. ``` Arguments --------- ### Inside Function Call ```python () # f(0, 0) () # f(x=0, y=0) (, ) # f(0, y=0) ``` ### Inside Function Definition ```python def f(): # def f(x, y): def f(): # def f(x=0, y=0): def f(, ): # def f(x, y=0): ``` Splat Operator -------------- ### Inside Function Call **Splat expands a collection into positional arguments, while splatty-splat expands a dictionary into keyword arguments.** ```python args = (1, 2) kwargs = {'x': 3, 'y': 4, 'z': 5} func(*args, **kwargs) ``` #### Is the same as: ```python func(1, 2, x=3, y=4, z=5) ``` ### Inside Function Definition **Splat combines zero or more positional arguments into a tuple, while splatty-splat combines zero or more keyword arguments into a dictionary.** ```python def add(*a): return sum(a) ``` ```python >>> add(1, 2, 3) 6 ``` #### Legal argument combinations: ```python def f(x, y, z): # f(x=1, y=2, z=3) | f(1, y=2, z=3) | f(1, 2, z=3) | f(1, 2, 3) def f(*, x, y, z): # f(x=1, y=2, z=3) def f(x, *, y, z): # f(x=1, y=2, z=3) | f(1, y=2, z=3) def f(x, y, *, z): # f(x=1, y=2, z=3) | f(1, y=2, z=3) | f(1, 2, z=3) ``` ```python def f(*args): # f(1, 2, 3) def f(x, *args): # f(1, 2, 3) def f(*args, z): # f(1, 2, z=3) def f(x, *args, z): # f(1, 2, z=3) ``` ```python def f(**kwargs): # f(x=1, y=2, z=3) def f(x, **kwargs): # f(x=1, y=2, z=3) | f(1, y=2, z=3) def f(*, x, **kwargs): # f(x=1, y=2, z=3) ``` ```python def f(*args, **kwargs): # f(x=1, y=2, z=3) | f(1, y=2, z=3) | f(1, 2, z=3) | f(1, 2, 3) def f(x, *args, **kwargs): # f(x=1, y=2, z=3) | f(1, y=2, z=3) | f(1, 2, z=3) | f(1, 2, 3) def f(*args, y, **kwargs): # f(x=1, y=2, z=3) | f(1, y=2, z=3) def f(x, *args, z, **kwargs): # f(x=1, y=2, z=3) | f(1, y=2, z=3) | f(1, 2, z=3) ``` ### Other Uses ```python = [* [, ...]] = {* [, ...]} = (*, [...]) = {** [, ...]} ``` ```python head, *body, tail = ``` Inline ------ ### Lambda ```python = lambda: = lambda , : ``` ### Comprehension ```python = [i+1 for i in range(10)] # [1, 2, ..., 10] = {i for i in range(10) if i > 5} # {6, 7, 8, 9} = (i+5 for i in range(10)) # (5, 6, ..., 14) = {i: i*2 for i in range(10)} # {0: 0, 1: 2, ..., 9: 18} ``` ```python out = [i+j for i in range(10) for j in range(10)] ``` #### Is the same as: ```python out = [] for i in range(10): for j in range(10): out.append(i+j) ``` ### Map, Filter, Reduce ```python from functools import reduce = map(lambda x: x + 1, range(10)) # (1, 2, ..., 10) = filter(lambda x: x > 5, range(10)) # (6, 7, 8, 9) = reduce(lambda out, x: out + x, range(10)) # 45 ``` ### Any, All ```python = any() # False if empty. = all(el[1] for el in ) # True if empty. ``` ### If - Else ```python = if else ``` ```python >>> [a if a else 'zero' for a in (0, 1, 2, 3)] ['zero', 1, 2, 3] ``` ### Namedtuple, Enum, Dataclass ```python from collections import namedtuple Point = namedtuple('Point', 'x y') point = Point(0, 0) ``` ```python from enum import Enum Direction = Enum('Direction', 'n e s w') direction = Direction.n ``` ```python from dataclasses import make_dataclass Creature = make_dataclass('Creature', ['location', 'direction']) creature = Creature(Point(0, 0), Direction.n) ``` Closure ------- **We have a closure in Python when:** * **A nested function references a value of its enclosing function and then** * **the enclosing function returns the nested function.** ```python def get_multiplier(a): def out(b): return a * b return out ``` ```python >>> multiply_by_3 = get_multiplier(3) >>> multiply_by_3(10) 30 ``` * **If multiple nested functions within enclosing function reference the same value, that value gets shared.** * **To dynamically access function's first free variable use `'.__closure__[0].cell_contents'`.** ### Partial ```python from functools import partial = partial( [, , , ...]) ``` ```python >>> import operator as op >>> multiply_by_3 = partial(op.mul, 3) >>> multiply_by_3(10) 30 ``` * **Partial is also useful in cases when function needs to be passed as an argument, because it enables us to set its arguments beforehand.** * **A few examples being: `'defaultdict()'`, `'iter(, to_exclusive)'` and dataclass's `'field(default_factory=)'`.** ### Non-Local **If variable is being assigned to anywhere in the scope, it is regarded as a local variable, unless it is declared as a 'global' or a 'nonlocal'.** ```python def get_counter(): i = 0 def out(): nonlocal i i += 1 return i return out ``` ```python >>> counter = get_counter() >>> counter(), counter(), counter() (1, 2, 3) ``` Decorator --------- **A decorator takes a function, adds some functionality and returns it.** ```python @decorator_name def function_that_gets_passed_to_decorator(): ... ``` ### Debugger Example **Decorator that prints function's name every time it gets called.** ```python from functools import wraps def debug(func): @wraps(func) def out(*args, **kwargs): print(func.__name__) return func(*args, **kwargs) return out @debug def add(x, y): return x + y ``` * **Wraps is a helper decorator that copies the metadata of the passed function (func) to the function it is wrapping (out).** * **Without it `'add.__name__'` would return `'out'`.** ### LRU Cache **Decorator that caches function's return values. All function's arguments must be hashable.** ```python from functools import lru_cache @lru_cache(maxsize=None) def fib(n): return n if n < 2 else fib(n-2) + fib(n-1) ``` * **CPython interpreter limits recursion depth to 1000 by default. To increase it use `'sys.setrecursionlimit()'`.** ### Parametrized Decorator **A decorator that accepts arguments and returns a normal decorator that accepts a function.** ```python from functools import wraps def debug(print_result=False): def decorator(func): @wraps(func) def out(*args, **kwargs): result = func(*args, **kwargs) print(func.__name__, result if print_result else '') return result return out return decorator @debug(print_result=True) def add(x, y): return x + y ``` Class ----- ```python class : def __init__(self, a): self.a = a def __repr__(self): class_name = self.__class__.__name__ return f'{class_name}({self.a!r})' def __str__(self): return str(self.a) @classmethod def get_class_name(cls): return cls.__name__ ``` * **Return value of repr() should be unambiguous and of str() readable.** * **If only repr() is defined, it will also be used for str().** #### Str() use cases: ```python print() print(f'{}') raise Exception() loguru.logger.debug() csv.writer().writerow([]) ``` #### Repr() use cases: ```python print([]) print(f'{!r}') >>> loguru.logger.exception() Z = dataclasses.make_dataclass('Z', ['a']); print(Z()) ``` ### Constructor Overloading ```python class : def __init__(self, a=None): self.a = a ``` ### Inheritance ```python class Person: def __init__(self, name, age): self.name = name self.age = age class Employee(Person): def __init__(self, name, age, staff_num): super().__init__(name, age) self.staff_num = staff_num ``` ### Multiple Inheritance ```python class A: pass class B: pass class C(A, B): pass ``` **MRO determines the order in which parent classes are traversed when searching for a method:** ```python >>> C.mro() [, , , ] ``` ### Property **Pythonic way of implementing getters and setters.** ```python class MyClass: @property def a(self): return self._a @a.setter def a(self, value): self._a = value ``` ```python >>> el = MyClass() >>> el.a = 123 >>> el.a 123 ``` ### Dataclass **Decorator that automatically generates init(), repr() and eq() special methods.** ```python from dataclasses import dataclass, field @dataclass(order=False, frozen=False) class : : : = : list/dict/set = field(default_factory=list/dict/set) ``` * **Objects can be made sortable with `'order=True'` and/or immutable and hashable with `'frozen=True'`.** * **Function field() is needed because `': list = []'` would make a list that is shared among all instances.** * **Default_factory can be any [callable](#callable).** #### Inline: ```python from dataclasses import make_dataclass = make_dataclass('', ) = make_dataclass('', ) = ('', [, ]) ``` ### Slots **Mechanism that restricts objects to attributes listed in 'slots' and significantly reduces their memory footprint.** ```python class MyClassWithSlots: __slots__ = ['a'] def __init__(self): self.a = 1 ``` ### Copy ```python from copy import copy, deepcopy = copy() = deepcopy() ``` Duck Types ---------- **A duck type is an implicit type that prescribes a set of special methods. Any object that has those methods defined is considered a member of that duck type.** ### Comparable * **If eq() method is not overridden, it returns `'id(self) == id(other)'`, which is the same as `'self is other'`.** * **That means all objects compare not equal by default.** * **Only the left side object has eq() method called, unless it returns NotImplemented, in which case the right object is consulted.** ```python class MyComparable: def __init__(self, a): self.a = a def __eq__(self, other): if isinstance(other, type(self)): return self.a == other.a return NotImplemented ``` ### Hashable * **Hashable object needs both hash() and eq() methods and its hash value should never change.** * **Hashable objects that compare equal must have the same hash value, meaning default hash() that returns `'id(self)'` will not do.** * **That is why Python automatically makes classes unhashable if you only implement eq().** ```python class MyHashable: def __init__(self, a): self._a = a @property def a(self): return self._a def __eq__(self, other): if isinstance(other, type(self)): return self.a == other.a return NotImplemented def __hash__(self): return hash(self.a) ``` ### Sortable * **With total_ordering decorator, you only need to provide eq() and one of lt(), gt(), le() or ge() special methods.** ```python from functools import total_ordering @total_ordering class MySortable: def __init__(self, a): self.a = a def __eq__(self, other): if isinstance(other, type(self)): return self.a == other.a return NotImplemented def __lt__(self, other): if isinstance(other, type(self)): return self.a < other.a return NotImplemented ``` ### Iterator * **Any object that has methods next() and iter() is an iterator.** * **Next() should return next item or raise StopIteration.** * **Iter() should return 'self'.** ```python class Counter: def __init__(self): self.i = 0 def __next__(self): self.i += 1 return self.i def __iter__(self): return self ``` ```python >>> counter = Counter() >>> next(counter), next(counter), next(counter) (1, 2, 3) ``` #### Python has many different iterator objects: * **Iterators returned by the [iter()](#iterator) function, such as list\_iterator and set\_iterator.** * **Objects returned by the [itertools](#itertools) module, such as count, repeat and cycle.** * **Generators returned by the [generator functions](#generator) and [generator expressions](#comprehension).** * **File objects returned by the [open()](#open) function, etc.** ### Callable * **All functions and classes have a call() method, hence are callable.** * **When this cheatsheet uses `''` as an argument, it actually means `''`.** ```python class Counter: def __init__(self): self.i = 0 def __call__(self): self.i += 1 return self.i ``` ```python >>> counter = Counter() >>> counter(), counter(), counter() (1, 2, 3) ``` ### Context Manager * **Enter() should lock the resources and optionally return an object.** * **Exit() should release the resources.** * **Any exception that happens inside the with block is passed to the exit() method.** * **If it wishes to suppress the exception it must return a true value.** ```python class MyOpen: def __init__(self, filename): self.filename = filename def __enter__(self): self.file = open(self.filename) return self.file def __exit__(self, exc_type, exception, traceback): self.file.close() ``` ```python >>> with open('test.txt', 'w') as file: ... file.write('Hello World!') >>> with MyOpen('test.txt') as file: ... print(file.read()) Hello World! ``` Iterable Duck Types ------------------- ### Iterable * **Only required method is iter(). It should return an iterator of object's items.** * **Contains() automatically works on any object that has iter() defined.** ```python class MyIterable: def __init__(self, a): self.a = a def __iter__(self): return iter(self.a) def __contains__(self, el): return el in self.a ``` ```python >>> obj = MyIterable([1, 2, 3]) >>> [el for el in obj] [1, 2, 3] >>> 1 in obj True ``` ### Collection * **Only required methods are iter() and len().** * **This cheatsheet actually means `''` when it uses `''`.** * **I chose not to use the name 'iterable' because it sounds scarier and more vague than 'collection'.** ```python class MyCollection: def __init__(self, a): self.a = a def __iter__(self): return iter(self.a) def __contains__(self, el): return el in self.a def __len__(self): return len(self.a) ``` ### Sequence * **Only required methods are len() and getitem().** * **Getitem() should return an item at index or raise IndexError.** * **Iter() and contains() automatically work on any object that has getitem() defined.** * **Reversed() automatically works on any object that has getitem() and len() defined.** ```python class MySequence: def __init__(self, a): self.a = a def __iter__(self): return iter(self.a) def __contains__(self, el): return el in self.a def __len__(self): return len(self.a) def __getitem__(self, i): return self.a[i] def __reversed__(self): return reversed(self.a) ``` ### ABC Sequence * **It's a richer interface than the basic sequence.** * **Extending it generates iter(), contains(), reversed(), index() and count().** * **Unlike `'abc.Iterable'` and `'abc.Collection'`, it is not a duck type. That is why `'issubclass(MySequence, abc.Sequence)'` would return False even if MySequence had all the methods defined.** ```python from collections import abc class MyAbcSequence(abc.Sequence): def __init__(self, a): self.a = a def __len__(self): return len(self.a) def __getitem__(self, i): return self.a[i] ``` #### Table of required and automatically available special methods: ```text +------------+------------+------------+------------+--------------+ | | Iterable | Collection | Sequence | abc.Sequence | +------------+------------+------------+------------+--------------+ | iter() | REQ | REQ | Yes | Yes | | contains() | Yes | Yes | Yes | Yes | | len() | | REQ | REQ | REQ | | getitem() | | | REQ | REQ | | reversed() | | | Yes | Yes | | index() | | | | Yes | | count() | | | | Yes | +------------+------------+------------+------------+--------------+ ``` * **Other ABCs that generate missing methods are: MutableSequence, Set, MutableSet, Mapping and MutableMapping.** * **Names of their required methods are stored in `'.__abstractmethods__'`.** Enum ---- ```python from enum import Enum, auto class (Enum): = = , = auto() ``` * **If there are no numeric values before auto(), it returns 1.** * **Otherwise it returns an increment of the last numeric value.** ```python = . # Returns a member. = [''] # Returns a member or raises KeyError. = () # Returns a member or raises ValueError. = .name # Returns member's name. = .value # Returns member's value. ``` ```python list_of_members = list() member_names = [a.name for a in ] member_values = [a.value for a in ] random_member = random.choice(list()) ``` ```python def get_next_member(member): members = list(member.__class__) index = (members.index(member) + 1) % len(members) return members[index] ``` ### Inline ```python Cutlery = Enum('Cutlery', 'fork knife spoon') Cutlery = Enum('Cutlery', ['fork', 'knife', 'spoon']) Cutlery = Enum('Cutlery', {'fork': 1, 'knife': 2, 'spoon': 3}) ``` #### User-defined functions cannot be values, so they must be wrapped: ```python from functools import partial LogicOp = Enum('LogicOp', {'AND': partial(lambda l, r: l and r), 'OR' : partial(lambda l, r: l or r)}) ``` * **Another solution in this particular case is to use built-in functions and\_() and or\_() from the module [operator](#operator).** Exceptions ---------- ### Basic Example ```python try: except : ``` ### Complex Example ```python try: except : except : else: finally: ``` * **Code inside the `'else'` block will only be executed if `'try'` block had no exception.** * **Code inside the `'finally'` block will always be executed.** ### Catching Exceptions ```python except : except as : except (, ...): except (, ...) as : ``` * **Also catches subclasses of the exception.** * **Use `'traceback.print_exc()'` to print the error message to stderr.** ### Raising Exceptions ```python raise raise () raise ( [, ...]) ``` #### Re-raising caught exception: ```python except as : ... raise ``` ### Exception Object ```python arguments = .args exc_type = .__class__ filename = .__traceback__.tb_frame.f_code.co_filename func_name = .__traceback__.tb_frame.f_code.co_name line = linecache.getline(filename, .__traceback__.tb_lineno) error_msg = traceback.format_exception(exc_type, , .__traceback__) ``` ### Built-in Exceptions ```text BaseException +-- SystemExit # Raised by the sys.exit() function. +-- KeyboardInterrupt # Raised when the user hits the interrupt key (ctrl-c). +-- Exception # User-defined exceptions should be derived from this class. +-- ArithmeticError # Base class for arithmetic errors. | +-- ZeroDivisionError # Raised when dividing by zero. +-- AttributeError # Raised when an attribute is missing. +-- EOFError # Raised by input() when it hits end-of-file condition. +-- LookupError # Raised when a look-up on a collection fails. | +-- IndexError # Raised when a sequence index is out of range. | +-- KeyError # Raised when a dictionary key or set element is not found. +-- NameError # Raised when a variable name is not found. +-- OSError # Failures such as “file not found” or “disk full”. | +-- FileNotFoundError # When a file or directory is requested but doesn't exist. +-- RuntimeError # Raised by errors that don't fall into other categories. | +-- RecursionError # Raised when the maximum recursion depth is exceeded. +-- StopIteration # Raised by next() when run on an empty iterator. +-- TypeError # Raised when an argument is of wrong type. +-- ValueError # When an argument is of right type but inappropriate value. +-- UnicodeError # Raised when encoding/decoding strings to/from bytes fails. ``` #### Collections and their exceptions: ```text +-----------+------------+------------+------------+ | | list | dict | set | +-----------+------------+------------+------------+ | getitem() | IndexError | KeyError | | | pop() | IndexError | KeyError | KeyError | | remove() | ValueError | | KeyError | | index() | ValueError | | | +-----------+------------+------------+------------+ ``` #### Useful built-in exceptions: ```python raise TypeError('Argument is of wrong type!') raise ValueError('Argument is of right type but inappropriate value!') raise RuntimeError('None of above!') ``` ### User-defined Exceptions ```python class MyError(Exception): pass class MyInputError(MyError): pass ``` Exit ---- **Exits the interpreter by raising SystemExit exception.** ```python import sys sys.exit() # Exits with exit code 0 (success). sys.exit() # Prints to stderr and exits with 1. sys.exit() # Exits with passed exit code. ``` Print ----- ```python print(, ..., sep=' ', end='\n', file=sys.stdout, flush=False) ``` * **Use `'file=sys.stderr'` for messages about errors.** * **Use `'flush=True'` to forcibly flush the stream.** ### Pretty Print ```python from pprint import pprint pprint(, width=80, depth=None, compact=False, sort_dicts=True) ``` * **Levels deeper than 'depth' get replaced by '...'.** Input ----- **Reads a line from user input or pipe if present.** ```python = input(prompt=None) ``` * **Trailing newline gets stripped.** * **Prompt string is printed to the standard output before reading input.** * **Raises EOFError when user hits EOF (ctrl-d/z) or input stream gets exhausted.** Command Line Arguments ---------------------- ```python import sys script_name = sys.argv[0] arguments = sys.argv[1:] ``` ### Argument Parser ```python from argparse import ArgumentParser, FileType p = ArgumentParser(description=) p.add_argument('-', '--', action='store_true') # Flag p.add_argument('-', '--', type=) # Option p.add_argument('', type=, nargs=1) # First argument p.add_argument('', type=, nargs='+') # Remaining arguments p.add_argument('', type=, nargs='*') # Optional arguments args = p.parse_args() # Exits on error. value = args. ``` * **Use `'help='` to set argument description.** * **Use `'default='` to set the default value.** * **Use `'type=FileType()'` for files.** Open ---- **Opens the file and returns a corresponding file object.** ```python = open('', mode='r', encoding=None, newline=None) ``` * **`'encoding=None'` means that the default encoding is used, which is platform dependent. Best practice is to use `'encoding="utf-8"'` whenever possible.** * **`'newline=None'` means all different end of line combinations are converted to '\n' on read, while on write all '\n' characters are converted to system's default line separator.** * **`'newline=""'` means no conversions take place, but input is still broken into chunks by readline() and readlines() on either '\n', '\r' or '\r\n'.** ### Modes * **`'r'` - Read (default).** * **`'w'` - Write (truncate).** * **`'x'` - Write or fail if the file already exists.** * **`'a'` - Append.** * **`'w+'` - Read and write (truncate).** * **`'r+'` - Read and write from the start.** * **`'a+'` - Read and write from the end.** * **`'t'` - Text mode (default).** * **`'b'` - Binary mode.** ### Exceptions * **`'FileNotFoundError'` can be raised when reading with `'r'` or `'r+'`.** * **`'FileExistsError'` can be raised when writing with `'x'`.** * **`'IsADirectoryError'` and `'PermissionError'` can be raised by any.** * **`'OSError'` is the parent class of all listed exceptions.** ### File Object ```python .seek(0) # Moves to the start of the file. .seek(offset) # Moves 'offset' chars/bytes from the start. .seek(0, 2) # Moves to the end of the file. .seek(±offset, ) # Anchor: 0 start, 1 current position, 2 end. ``` ```python = .read(size=-1) # Reads 'size' chars/bytes or until EOF. = .readline() # Returns a line or empty string/bytes on EOF. = .readlines() # Returns a list of remaining lines. = next() # Returns a line using buffer. Do not mix. ``` ```python .write() # Writes a string or bytes object. .writelines() # Writes a coll. of strings or bytes objects. .flush() # Flushes write buffer. ``` * **Methods do not add or strip trailing newlines, even writelines().** ### Read Text from File ```python def read_file(filename): with open(filename, encoding='utf-8') as file: return file.readlines() ``` ### Write Text to File ```python def write_to_file(filename, text): with open(filename, 'w', encoding='utf-8') as file: file.write(text) ``` Path ---- ```python from os import getcwd, path, listdir from glob import glob ``` ```python = getcwd() # Returns the current working directory. = path.join(, ...) # Joins two or more pathname components. = path.abspath() # Returns absolute path. ``` ```python = path.basename() # Returns final component of the path. = path.dirname() # Returns path without the final component. = path.splitext() # Splits on last period of the final component. ``` ```python = listdir(path='.') # Returns filenames located at path. = glob('') # Returns paths matching the wildcard pattern. ``` ```python = path.exists() # Or: .exists() = path.isfile() # Or: .is_file() = path.isdir() # Or: .is_dir() ``` ### DirEntry **Using scandir() instead of listdir() can significantly increase the performance of code that also needs file type information.** ```python from os import scandir ``` ```python = scandir(path='.') # Returns DirEntry objects located at path. = .path # Returns path as a string. = .name # Returns final component as a string. = open() # Opens the file and returns file object. ``` ### Path Object ```python from pathlib import Path ``` ```python = Path( [, ...]) # Accepts strings, Paths and DirEntry objects. = / [/ ...] # One of the paths must be a Path object. ``` ```python = Path() # Returns relative cwd. Also Path('.'). = Path.cwd() # Returns absolute cwd. Also Path().resolve(). = .resolve() # Returns absolute Path without symlinks. ``` ```python = .parent # Returns Path without final component. = .name # Returns final component as a string. = .stem # Returns final component without extension. = .suffix # Returns final component's extension. = .parts # Returns all components as strings. ``` ```python = .iterdir() # Returns dir contents as Path objects. = .glob('') # Returns Paths matching the wildcard pattern. ``` ```python = str() # Returns path as a string. = open() # Opens the file and returns file object. ``` OS Commands ----------- ### Files and Directories * **Paths can be either strings, Paths or DirEntry objects.** * **Functions report OS related errors by raising either OSError or one of its [subclasses](#exceptions-1).** ```python import os, shutil ``` ```python os.chdir() # Changes the current working directory. os.mkdir(, mode=0o777) # Creates a directory. Mode is in octal. ``` ```python shutil.copy(from, to) # Copies the file. 'to' can exist or be a dir. shutil.copytree(from, to) # Copies the directory. 'to' must not exist. ``` ```python os.rename(from, to) # Renames/moves the file or directory. os.replace(from, to) # Same, but overwrites 'to' if it exists. ``` ```python os.remove() # Deletes the file. os.rmdir() # Deletes the empty directory. shutil.rmtree() # Deletes the directory. ``` ### Shell Commands ```python import os = os.popen('').read() ``` #### Sends '1 + 1' to the basic calculator and captures its output: ```python >>> from subprocess import run >>> run('bc', input='1 + 1\n', capture_output=True, encoding='utf-8') CompletedProcess(args='bc', returncode=0, stdout='2\n', stderr='') ``` #### Sends test.in to the basic calculator running in standard mode and saves its output to test.out: ```python >>> from shlex import split >>> os.popen('echo 1 + 1 > test.in') >>> run(split('bc -s'), stdin=open('test.in'), stdout=open('test.out', 'w')) CompletedProcess(args=['bc', '-s'], returncode=0) >>> open('test.out').read() '2\n' ``` JSON ---- **Text file format for storing collections of strings and numbers.** ```python import json = json.dumps(, ensure_ascii=True, indent=None) = json.loads() ``` ### Read Object from JSON File ```python def read_json_file(filename): with open(filename, encoding='utf-8') as file: return json.load(file) ``` ### Write Object to JSON File ```python def write_to_json_file(filename, an_object): with open(filename, 'w', encoding='utf-8') as file: json.dump(an_object, file, ensure_ascii=False, indent=2) ``` Pickle ------ **Binary file format for storing objects.** ```python import pickle = pickle.dumps() = pickle.loads() ``` ### Read Object from File ```python def read_pickle_file(filename): with open(filename, 'rb') as file: return pickle.load(file) ``` ### Write Object to File ```python def write_to_pickle_file(filename, an_object): with open(filename, 'wb') as file: pickle.dump(an_object, file) ``` CSV --- **Text file format for storing spreadsheets.** ```python import csv ``` ### Read ```python = csv.reader() # Also: `dialect='excel', delimiter=','`. = next() # Returns next row as a list of strings. = list() # Returns list of remaining rows. ``` * **File must be opened with `'newline=""'` argument, or newlines embedded inside quoted fields will not be interpreted correctly!** ### Write ```python = csv.writer() # Also: `dialect='excel', delimiter=','`. .writerow() # Encodes objects using `str()`. .writerows() # Appends multiple rows. ``` * **File must be opened with `'newline=""'` argument, or '\r' will be added in front of every '\n' on platforms that use '\r\n' line endings!** ### Parameters * **`'dialect'` - Master parameter that sets the default values.** * **`'delimiter'` - A one-character string used to separate fields.** * **`'quotechar'` - Character for quoting fields that contain special characters.** * **`'doublequote'` - Whether quotechars inside fields get doubled or escaped.** * **`'skipinitialspace'` - Whether whitespace after delimiter gets stripped.** * **`'lineterminator'` - Specifies how writer terminates rows.** * **`'quoting'` - Controls the amount of quoting: 0 - as necessary, 1 - all.** * **`'escapechar'` - Character for escaping 'quotechar' if 'doublequote' is False.** ### Dialects ```text +------------------+--------------+--------------+--------------+ | | excel | excel-tab | unix | +------------------+--------------+--------------+--------------+ | delimiter | ',' | '\t' | ',' | | quotechar | '"' | '"' | '"' | | doublequote | True | True | True | | skipinitialspace | False | False | False | | lineterminator | '\r\n' | '\r\n' | '\n' | | quoting | 0 | 0 | 1 | | escapechar | None | None | None | +------------------+--------------+--------------+--------------+ ``` ### Read Rows from CSV File ```python def read_csv_file(filename): with open(filename, encoding='utf-8', newline='') as file: return list(csv.reader(file)) ``` ### Write Rows to CSV File ```python def write_to_csv_file(filename, rows): with open(filename, 'w', encoding='utf-8', newline='') as file: writer = csv.writer(file) writer.writerows(rows) ``` SQLite ------ **Server-less database engine that stores each database into a separate file.** ### Connect **Opens a connection to the database file. Creates a new file if path doesn't exist.** ```python import sqlite3 = sqlite3.connect('') # Also ':memory:'. .close() ``` ### Read **Returned values can be of type str, int, float, bytes or None.** ```python = .execute('') # Can raise a subclass of sqlite3.Error. = .fetchone() # Returns next row. Also next(). = .fetchall() # Returns remaining rows. Also list(). ``` ### Write ```python .execute('') .commit() ``` #### Or: ```python with : .execute('') ``` ### Placeholders * **Passed values can be of type str, int, float, bytes, None, bool, datetime.date or datetime.datetme.** * **Bools will be stored and returned as ints and dates as [ISO formatted strings](#encode).** ```python .execute('', ) # Replaces '?'s in query with values. .execute('', ) # Replaces ':'s with values. .executemany('', ) # Runs execute() many times. ``` ### Example **In this example values are not actually saved because `'con.commit()'` is omitted!** ```python >>> con = sqlite3.connect('test.db') >>> con.execute('create table person (person_id integer primary key, name, height)') >>> con.execute('insert into person values (null, ?, ?)', ('Jean-Luc', 187)).lastrowid 1 >>> con.execute('select * from person').fetchall() [(1, 'Jean-Luc', 187)] ``` ### MySQL **Has a very similar interface, with differences listed below.** ```python # $ pip3 install mysql-connector from mysql import connector = connector.connect(host=, …) # `user=, password=, database=`. = .cursor() # Only cursor has execute method. .execute('') # Can raise a subclass of connector.Error. .execute('', ) # Replaces '%s's in query with values. .execute('', ) # Replaces '%()s's with values. ``` Bytes ----- **Bytes object is an immutable sequence of single bytes. Mutable version is called bytearray.** ```python = b'' # Only accepts ASCII characters and \x00 - \xff. = [] # Returns int in range from 0 to 255. = [] # Returns bytes even if it has only one element. = .join() # Joins elements using bytes object as separator. ``` ### Encode ```python = bytes() # Ints must be in range from 0 to 255. = bytes(, 'utf-8') # Or: .encode('utf-8') = .to_bytes(n_bytes, …) # `byteorder='big/little', signed=False`. = bytes.fromhex('') # Hex numbers can be separated by spaces. ``` ### Decode ```python = list() # Returns ints in range from 0 to 255. = str(, 'utf-8') # Or: .decode('utf-8') = int.from_bytes(, …) # `byteorder='big/little', signed=False`. '' = .hex() # Returns a string of hexadecimal numbers. ``` ### Read Bytes from File ```python def read_bytes(filename): with open(filename, 'rb') as file: return file.read() ``` ### Write Bytes to File ```python def write_bytes(filename, bytes_obj): with open(filename, 'wb') as file: file.write(bytes_obj) ``` Struct ------ * **Module that performs conversions between a sequence of numbers and a bytes object.** * **Machine’s native type sizes and byte order are used by default.** ```python from struct import pack, unpack, iter_unpack = pack('', [, , ...]) = unpack('', ) = iter_unpack('', ) ``` ### Example ```python >>> pack('>hhl', 1, 2, 3) b'\x00\x01\x00\x02\x00\x00\x00\x03' >>> unpack('>hhl', b'\x00\x01\x00\x02\x00\x00\x00\x03') (1, 2, 3) ``` ### Format #### For standard type sizes start format string with: * **`'='` - native byte order** * **`'<'` - little-endian** * **`'>'` - big-endian (also `'!'`)** #### Integer types. Use a capital letter for unsigned type. Standard sizes are in brackets: * **`'x'` - pad byte** * **`'b'` - char (1)** * **`'h'` - short (2)** * **`'i'` - int (4)** * **`'l'` - long (4)** * **`'q'` - long long (8)** #### Floating point types: * **`'f'` - float (4)** * **`'d'` - double (8)** Array ----- **List that can only hold numbers of a predefined type. Available types and their sizes in bytes are listed above.** ```python from array import array = array('', ) # Array from collection of numbers. = array('', ) # Array from bytes object. = array('', ) # Treats array as a sequence of numbers. = bytes() # Or: .tobytes() ``` Memory View ----------- * **A sequence object that points to the memory of another object.** * **Each element can reference a single or multiple consecutive bytes, depending on format.** * **Order and number of elements can be changed with slicing.** ```python = memoryview() # Immutable if bytes, else mutable. = [] # Returns an int or a float. = [] # Mview with rearranged elements. = .cast('') # Casts memoryview to the new format. .release() # Releases the object's memory buffer. ``` ### Decode ```python .write() # Writes mview to the binary file. = bytes() # Creates a new bytes object. = .join() # Joins mviews using bytes object as sep. = array('', ) # Treats mview as a sequence of numbers. ``` ```python = list() # Returns list of ints or floats. = str(, 'utf-8') # Treats mview as a bytes object. = int.from_bytes(, …) # `byteorder='big/little', signed=False`. '' = .hex() # Treats mview as a bytes object. ``` Deque ----- **A thread-safe list with efficient appends and pops from either side. Pronounced "deck".** ```python from collections import deque = deque(, maxlen=None) ``` ```python .appendleft() # Opposite element is dropped if full. .extendleft() # Collection gets reversed. = .popleft() # Raises IndexError if empty. .rotate(n=1) # Rotates elements to the right. ``` Threading --------- * **CPython interpreter can only run a single thread at a time.** * **That is why using multiple threads won't result in a faster execution, unless at least one of the threads contains an I/O operation.** ```python from threading import Thread, RLock, Semaphore, Event, Barrier ``` ### Thread ```python = Thread(target=) # Use `args=` to set arguments. .start() # Starts the thread. = .is_alive() # Checks if thread has finished executing. .join() # Waits for thread to finish. ``` * **Use `'kwargs='` to pass keyword arguments to the function.** * **Use `'daemon=True'`, or the program will not be able to exit while the thread is alive.** ### Lock ```python = RLock() .acquire() # Waits for lock to be available. .release() # Makes the lock available again. ``` #### Or: ```python lock = RLock() with lock: ... ``` ### Semaphore, Event, Barrier ```python = Semaphore(value=1) # Lock that can be acquired 'value' times. = Event() # Method wait() blocks until set() is called. = Barrier(n_times) # Method wait() blocks until it's called 'n_times'. ``` ### Thread Pool Executor ```python from concurrent.futures import ThreadPoolExecutor with ThreadPoolExecutor(max_workers=None) as executor: # Does not exit until done. = executor.map(lambda x: x + 1, range(3)) # (1, 2, 3) = executor.map(lambda x, y: x + y, 'abc', '123') # ('a1', 'b2', 'c3') = executor.submit( [, , ...]) # Also visible outside block. ``` #### Future: ```python = .done() # Checks if thread has finished executing. = .result() # Waits for thread to finish and returns result. ``` ### Queue **A thread-safe FIFO queue. For LIFO queue use LifoQueue.** ```python from queue import Queue = Queue(maxsize=0) ``` ```python .put() # Blocks until queue stops being full. .put_nowait() # Raises queue.Full exception if full. = .get() # Blocks until queue stops being empty. = .get_nowait() # Raises queue.Empty exception if empty. ``` Operator -------- **Module of functions that provide the functionality of operators.** ```python from operator import add, sub, mul, truediv, floordiv, mod, pow, neg, abs from operator import eq, ne, lt, le, gt, ge from operator import and_, or_, not_ from operator import itemgetter, attrgetter, methodcaller ``` ```python import operator as op elementwise_sum = map(op.add, list_a, list_b) sorted_by_second = sorted(, key=op.itemgetter(1)) sorted_by_both = sorted(, key=op.itemgetter(1, 0)) product_of_elems = functools.reduce(op.mul, ) LogicOp = enum.Enum('LogicOp', {'AND': op.and_, 'OR' : op.or_}) last_el = op.methodcaller('pop')() ``` Introspection ------------- **Inspecting code at runtime.** ### Variables ```python = dir() # Names of local variables (incl. functions). = vars() # Dict of local variables. Also locals(). = globals() # Dict of global variables. ``` ### Attributes ```python = dir() # Names of object's attributes (incl. methods). = vars() # Dict of object's fields. Also .__dict__. = hasattr(, '') # Checks if getattr() raises an error. value = getattr(, '') # Raises AttributeError if attribute is missing. setattr(, '', value) # Only works on objects with __dict__ attribute. delattr(, '') # Equivalent to `del .`. ``` ### Parameters ```python from inspect import signature = signature() no_of_params = len(.parameters) param_names = list(.parameters.keys()) param_kinds = [a.kind for a in .parameters.values()] ``` Metaprograming -------------- **Code that generates code.** ### Type **Type is the root class. If only passed an object it returns its type (class). Otherwise it creates a new class.** ```python = type('', , ) ``` ```python >>> Z = type('Z', (), {'a': 'abcde', 'b': 12345}) >>> z = Z() ``` ### Meta Class **A class that creates classes.** ```python def my_meta_class(name, parents, attrs): attrs['a'] = 'abcde' return type(name, parents, attrs) ``` #### Or: ```python class MyMetaClass(type): def __new__(cls, name, parents, attrs): attrs['a'] = 'abcde' return type.__new__(cls, name, parents, attrs) ``` * **New() is a class method that gets called before init(). If it returns an instance of its class, then that instance gets passed to init() as a 'self' argument.** * **It receives the same arguments as init(), except for the first one that specifies the desired type of the returned instance (MyMetaClass in our case).** * **Like in our case, new() can also be called directly, usually from a new() method of a child class (**`def __new__(cls): return super().__new__(cls)`**).** * **The only difference between the examples above is that my\_meta\_class() returns a class of type type, while MyMetaClass() returns a class of type MyMetaClass.** ### Metaclass Attribute **Right before a class is created it checks if it has the 'metaclass' attribute defined. If not, it recursively checks if any of his parents has it defined and eventually comes to type().** ```python class MyClass(metaclass=MyMetaClass): b = 12345 ``` ```python >>> MyClass.a, MyClass.b ('abcde', 12345) ``` ### Type Diagram ```python type(MyClass) == MyMetaClass # MyClass is an instance of MyMetaClass. type(MyMetaClass) == type # MyMetaClass is an instance of type. ``` ```text +-------------+-------------+ | Classes | Metaclasses | +-------------+-------------| | MyClass --> MyMetaClass | | | v | | object -----> type <+ | | | ^ +--+ | | str ----------+ | +-------------+-------------+ ``` ### Inheritance Diagram ```python MyClass.__base__ == object # MyClass is a subclass of object. MyMetaClass.__base__ == type # MyMetaClass is a subclass of type. ``` ```text +-------------+-------------+ | Classes | Metaclasses | +-------------+-------------| | MyClass | MyMetaClass | | v | v | | object <----- type | | ^ | | | str | | +-------------+-------------+ ``` Eval ---- ```python >>> from ast import literal_eval >>> literal_eval('1 + 2') 3 >>> literal_eval('[1, 2, 3]') [1, 2, 3] >>> literal_eval('abs(1)') ValueError: malformed node or string ``` Coroutines ---------- * **Coroutines have a lot in common with threads, but unlike threads, they only give up control when they call another coroutine and they don’t use as much memory.** * **Coroutine definition starts with `'async'` and its call with `'await'`.** * **`'asyncio.run()'` is the main entry point for asynchronous programs.** * **Functions wait(), gather() and as_completed() can be used when multiple coroutines need to be started at the same time.** * **Asyncio module also provides its own [Queue](#queue), [Event](#semaphore-event-barrier), [Lock](#lock) and [Semaphore](#semaphore-event-barrier) classes.** #### Runs a terminal game where you control an asterisk that must avoid numbers: ```python import asyncio, collections, curses, enum, random P = collections.namedtuple('P', 'x y') # Position D = enum.Enum('D', 'n e s w') # Direction def main(screen): curses.curs_set(0) # Makes cursor invisible. screen.nodelay(True) # Makes getch() non-blocking. asyncio.run(main_coroutine(screen)) # Starts running asyncio code. async def main_coroutine(screen): state = {'*': P(0, 0), **{id_: P(30, 10) for id_ in range(10)}} moves = asyncio.Queue() coros = (*(random_controller(id_, moves) for id_ in range(10)), human_controller(screen, moves), model(moves, state, *screen.getmaxyx()), view(state, screen)) await asyncio.wait(coros, return_when=asyncio.FIRST_COMPLETED) async def random_controller(id_, moves): while True: moves.put_nowait((id_, random.choice(list(D)))) await asyncio.sleep(random.random() / 2) async def human_controller(screen, moves): while True: ch = screen.getch() key_mappings = {259: D.n, 261: D.e, 258: D.s, 260: D.w} if ch in key_mappings: moves.put_nowait(('*', key_mappings[ch])) await asyncio.sleep(0.01) async def model(moves, state, height, width): while state['*'] not in {p for id_, p in state.items() if id_ != '*'}: id_, d = await moves.get() p = state[id_] deltas = {D.n: P(0, -1), D.e: P(1, 0), D.s: P(0, 1), D.w: P(-1, 0)} new_p = P(*[sum(a) for a in zip(p, deltas[d])]) if 0 <= new_p.x < width-1 and 0 <= new_p.y < height: state[id_] = new_p async def view(state, screen): while True: screen.clear() for id_, p in state.items(): screen.addstr(p.y, p.x, str(id_)) await asyncio.sleep(0.01) curses.wrapper(main) ```
Libraries ========= Progress Bar ------------ ```python # $ pip3 install tqdm from tqdm import tqdm from time import sleep for el in tqdm([1, 2, 3]): sleep(0.2) ``` Plot ---- ```python # $ pip3 install matplotlib from matplotlib import pyplot pyplot.plot( [, label=]) pyplot.plot(, ) pyplot.legend() # Adds a legend. pyplot.savefig('') # Saves the figure. pyplot.show() # Displays the figure. pyplot.clf() # Clears the figure. ``` Table ----- #### Prints a CSV file as an ASCII table: ```python # $ pip3 install tabulate import csv, tabulate with open('test.csv', encoding='utf-8', newline='') as file: rows = csv.reader(file) header = [a.title() for a in next(rows)] table = tabulate.tabulate(rows, header) print(table) ``` Curses ------ #### Clears the terminal, prints a message and waits for the ESC key press: ```python from curses import wrapper, curs_set, ascii from curses import KEY_UP, KEY_RIGHT, KEY_DOWN, KEY_LEFT def main(): wrapper(draw) def draw(screen): curs_set(0) # Makes cursor invisible. screen.nodelay(True) # Makes getch() non-blocking. screen.clear() screen.addstr(0, 0, 'Press ESC to quit.') # Coordinates are y, x. while screen.getch() != ascii.ESC: pass def get_border(screen): from collections import namedtuple P = namedtuple('P', 'x y') height, width = screen.getmaxyx() return P(width-1, height-1) if __name__ == '__main__': main() ``` Logging ------- ```python # $ pip3 install loguru from loguru import logger ``` ```python logger.add('debug_{time}.log', colorize=True) # Connects a log file. logger.add('error_{time}.log', level='ERROR') # Another file for errors or higher. logger.('A logging message.') ``` * **Levels: `'debug'`, `'info'`, `'success'`, `'warning'`, `'error'`, `'critical'`.** ### Exceptions **Exception description, stack trace and values of variables are appended automatically.** ```python try: ... except : logger.exception('An error happened.') ``` ### Rotation **Argument that sets a condition when a new log file is created.** ```python rotation=||| ``` * **`''` - Max file size in bytes.** * **`''` - Max age of a file.** * **`'