Context Manager

  • Python

Context Manager

what is context

the state surrounding a section of code

why we need a context manager

  • writing try/finally every time can get cumbersom
  • easy to forget closing the file

use cases

Useful for program that needs Enter / Exit handeling

  • create / releasing resources
  • database transaction
  • set and reset decimal context

Common patterns

  • open / close
  • lock / release
  • change / reset
  • start / stop
  • enter / exit

protocal

implement these two dunder methods:

  • __enter__

    perform the setup, optionally return an object

  • __exit__

    receives error (silence or propagate)

    • need arguments exc_type, exc_value, exc_trace to handle exception
    • return True to silence exception

    perform clean up

examples

contextlib

nested contexts

Redis

  • Devops

Redis

compare to memcached

  • support persistant volume
    • RDB
    • AOF
  • support multiple data types
  • pub/sub

commands

  • redis-cli: command line interface
  • redis-sentinel: cluster managing tool
  • redis-server: run server
  • redis-benchmark: stress testing
  • redis-check-aof: check AOF
  • redis-check-dump: check RDB

configuration

Use redis.conf. Docker official redis image not contain this file. Mount it yourself or through redis-server arguments.

types

  • String: get, set, mget, mset
  • Integer: incr, decr, setbit
  • List: lpush, lrange, lpop
  • Hash Map: hset, hget, hmset, hmget
  • Set: sadd, smember, sdiff, sinter, sunion

use docker

Before start

To connect a container, you need to know the name and the port, in the associated networks to be able to discover the service.

There is no DNS resolution in docker deault bridge network. In default network, you need to specify --link to connect the containers. The --link is a legacy feature.

Therefore, create a user-defined network is recommanded, it provide automatic DNS resolution.

Create a bridge newrok

Run a redis instance in user-defined network

Run a redis-cli connect to the redis instance

Transaction

all commands are executed as a single isolated operation, serialized and executed sequentially
atomic: all failed or all succeed

  • MULTI: open a transaction and always return OK
  • EXEC: execute commands in transaction
  • DISCARD: flush commands and exit transaction
  • WATCH: check and set, if watched key changes, not execute

Errors

  • before EXEC: e.g. syntax error
  • after EXEC: e.g. value error

The pipeline discarding the transaction automatically if there was an error during the command queueing

… To be continued

Generator

  • Python

Generator

  • A type of iterator
  • generator function: function that uses yield statement
  • implement the iterator protocal, call next
  • raise StopIteration exhausted

Less code

Implement an iterator

Implement a generator

More efficient

Generator Comprehensions

  • local scope
  • lazy evaluation
  • is an iterator, can be exhausted

Delegating Generator

Use the syntax yield from to yield items in a generator

Memcached

  • Devops

Memcache

Store and retrieve data in memory(not persistent) base on specific hash function.

concepts

  • Slab: allocate as many pages as the ones available

  • Page: a memory area of default 1MB which contains as many chunks

  • Chunk: minimum allocated space for a single item

  • LRU: least recently used list

ref: Journey to the centre of memcached

we could say that we would run out of memory when all the available pages are allocated to slabs

memcached is designed to evict old/unused items in order to store new ones

every item operation (get, set, update or remove) requires the item in question to be locked

memcached only tries to remove the first 5 items of the LRU — after that it simply gives up and answers with OOM (out of memory)

Read More »Memcached

Iterable and Iterator

  • Python

Iterator & Iterable

iterator

  • get next item (__next__)
  • no indexes needed (Don’t need to be Sequence type)
  • consumable

iterable

  • collections that implement iterator

Protocal

Python need to count on certain funcionality: __next____iter__StopIteration

compare to sequence type

iteration can be more general than sequential indexing, we only need:

  • a bucket of items: collection, container
  • a way to get the next item, no need to care about ordering
  • an exception to raise if there is no next item

try to custom an iterator ourselfs:

Why re-create?

Seperate the Collection from the iterator

Iterable object

  • Maintaining the data of the collection is one object
  • Created once
  • implements __iter__, return a new iterator instance

Iterator object

  • Iterating over that data should be another object
  • throw away the iterator but don’t throw away the collection
  • Created every time
  • implements __iter__, return itself
  • implements __next__, return next item

iterable can be lazy

Caculate the next itme in an iterable until it’s actually requested

lazy evaluation

  • often used in class properties
  • properties of classes may not always populated when the object is created
  • value of property only becomes known when the property is requested/deferred

infnite iterables

  • itertools.cycle

Python Built-ins

  • range: return iterable
  • zip: return iterator
  • enumerate: return iterator
  • open: return iterator
  • reversed: return iterator

The type is important. Iterator object can be only iter over once.

iter()

when iter is called:

  • Python first looks for __iter__, if not then:
  • look for __getitem__ and create an iterator, if not then:
  • raise TypeError

Test it:

The __iter__ must return an iterator!

Iterating callable

iterator delegation

Example 1

Example 2

IP Subnetting

  • Devops
tags: ccna

IP Subnetting

Something you need to know first: Binary Odometer

10.1.1.254 + 1 = 10.1.1.255
10.1.1.255 + 1 = 10.1.2.0
10.1.2.0 + 1 = 10.1.2.1

in reverse:

10.1.2.0 – 1 = 10.1.1.255

Example 1

172.16.35.123/20 or 172.16.35.123 with the mask 255.255.240.0

Binary Method

image alt

Quick Method

Figure out the subnets:

First subnet = 172.16.32.0

Next subnet = 172.16.48.0

Broadcast address = next subnet – 1

172.16.32.0 + 1 = 172.16.32.1`

Last host = Broadcast – 1
172.16.47.255 - 1 = 172.16.47.254

Subnetting

  • Class A subnetting (255.0.0.0) support 1677214 (2^24) host per network, that way too much
  • Class B subnetting (255.255.0.0) support 16382 (2^16) host per network, that way too much
  • Class C subnetting (255.255.255.0) support 254 (2^8) host, more likely we subnet down to at least 254 hosts or even further

If you subnetting a network only has 2 hosts, you can subnet with (255.255.255.254) or CIDR as /31

Network, host number

  • Networks: 2^(network bits)
    • one allocate for the subnet
    • one allocate for the broadcast
  • Hosts: 2^(host bits) – 2

Subnetting to be short

  1. “stealing” or “taking away” bits from the host portion of an address, and
  2. allocating those bits to network portion

Example 2

Origin network 10.128.192.0/18 need at least 30 subnets as many hosts as possible

image

  1. draw the line with /18 to split network and host
  2. 2^5 > 30, need 5 subnet bit, draw the line to split subnet and host
  3. network/subnet portion is 8+8+7=23 bits, host portion is 32-23=9 bits
  • First subnet: 10.128.192.0/23
  • Second subnet: 10.128.194.0/23
  • Last subnet: 10.128.254.0/23