Devops

Memcached

  • Devops

Memcache

Store and retrieve data in memory(not persistent) base on specific hash function.

concepts

  • Slab: allocate as many pages as the ones available

  • Page: a memory area of default 1MB which contains as many chunks

  • Chunk: minimum allocated space for a single item

  • LRU: least recently used list

ref: Journey to the centre of memcached

we could say that we would run out of memory when all the available pages are allocated to slabs

memcached is designed to evict old/unused items in order to store new ones

every item operation (get, set, update or remove) requires the item in question to be locked

memcached only tries to remove the first 5 items of the LRU — after that it simply gives up and answers with OOM (out of memory)

Read More »Memcached

IP Subnetting

  • Devops
tags: ccna

IP Subnetting

Something you need to know first: Binary Odometer

10.1.1.254 + 1 = 10.1.1.255
10.1.1.255 + 1 = 10.1.2.0
10.1.2.0 + 1 = 10.1.2.1

in reverse:

10.1.2.0 – 1 = 10.1.1.255

Example 1

172.16.35.123/20 or 172.16.35.123 with the mask 255.255.240.0

Binary Method

image alt

Quick Method

Figure out the subnets:

First subnet = 172.16.32.0

Next subnet = 172.16.48.0

Broadcast address = next subnet – 1

172.16.32.0 + 1 = 172.16.32.1`

Last host = Broadcast – 1
172.16.47.255 - 1 = 172.16.47.254

Subnetting

  • Class A subnetting (255.0.0.0) support 1677214 (2^24) host per network, that way too much
  • Class B subnetting (255.255.0.0) support 16382 (2^16) host per network, that way too much
  • Class C subnetting (255.255.255.0) support 254 (2^8) host, more likely we subnet down to at least 254 hosts or even further

If you subnetting a network only has 2 hosts, you can subnet with (255.255.255.254) or CIDR as /31

Network, host number

  • Networks: 2^(network bits)
    • one allocate for the subnet
    • one allocate for the broadcast
  • Hosts: 2^(host bits) – 2

Subnetting to be short

  1. “stealing” or “taking away” bits from the host portion of an address, and
  2. allocating those bits to network portion

Example 2

Origin network 10.128.192.0/18 need at least 30 subnets as many hosts as possible

image

  1. draw the line with /18 to split network and host
  2. 2^5 > 30, need 5 subnet bit, draw the line to split subnet and host
  3. network/subnet portion is 8+8+7=23 bits, host portion is 32-23=9 bits
  • First subnet: 10.128.192.0/23
  • Second subnet: 10.128.194.0/23
  • Last subnet: 10.128.254.0/23

使用 Cloud Build 搭配 Helm 改善雲端部署

  • Devops

管理 Kubernetes 的服務時常有一些困擾:

  1. 須根據環境 (staging, production…) 去套用不同的設定及環境變數,整合不易
  2. secret 常是手動新增,如 cloudsql-proxy 的憑證,時間久了常忘記該 secret 是幹麻用的,及整個服務重新部署也會卡在這個手動步驟

使用 Helm 可以幫助我們管理這些部署檔案:

  1. 一鍵部署、移除
  2. 可根據不同的環境採用不同變數,有幾種可行的作法
  3. 可根據彈性的判斷式生成設定檔
  4. chart 的版本控制 (Release)

準備事項:

  1. 有個叢集
  2. Client 端安裝 Helm , Server 端安裝 Tiller
    image
  3. 叢集有 RBAC ,可以關閉 RBAC ,或給 Tiller 權限:
    • helm init預設使用的服務帳戶是default
    • 叢集的default服務帳戶綁定cluster-admin叢集角色

Helm 有提供 dependency 的功能,可以透過以下指令來部署全部的 subchart:

但這裡會有一個問題,即部署時沒辦法指定 subchart 要吃哪個 value file ,因若你的 chart 參數是分成values.prod.yamlvalues.staging.yaml,一旦 chart 打包成 package ,package 在使用時只能吃 values.yaml

下個版本可能提供相對應的作法,請參考相關 issue

透過 Cloud Build 部署

透過helm create建立並設定完 chart 後,希望能在 Cloud Build 的流程透過 helm 部署,這邊選用 cloud-builer-community 提供的helm映像檔

  1. Build 該映像檔並推至專案的 Container Registry
  2. 參考 example 來新增流程,如helm installhelm upgrade
  3. 若 RBAC 是啟用的狀態,須要給 Cloud Build 操作叢集的權限
    • Cloud Build 的服務帳戶綁定roles/container.admin角色及cluster-admin叢集角色,請參考相關指令
      image

IP Addressing

  • Devops
tags: ccna

IP Addressing

  • layer 3 logical address assigned by an administrator(MAC built in NIC)
  • used to identify specific devices on a network
  • every device on the internet has a unique IP Address

Street Analogy

  • network address portion
    • identifies a specific network
    • routers route traffic via routing tables, is based on network address(Network ID), not ip address
  • host address portion
    • identifies a specific endpoint on a network
    • we can use a protocal such as ARP to find the host

Ipv4

  • connectionless protocal: no sessions formd when transmitted, no status info
  • packets treated independently
    • may take different paths: load balancing, bandwidth, hopcount
  • hierarchical addressing sturture
  • best effort delivery
  • format
    • 32 bit with 4 octets
    • like DHL or FedEx routing parcel based on an address

Classes

  • Unicast Traffic
    • A
      • start with binary 0
      • range from 0(00000000).0.0.0 to 127(01111111).255.255.255
      • exceptions:
        • 127 is reserved for loopback: 127.0.0.1
        • 0 network is reserved for default network: 0.1.1.1
      • actual range from 1.0.0.0 to 126.255.255.255
      • portions
        • first 1 octets: Networks
        • last 3 octests: Hosts
    • B
      • start with binary 10
      • range from 128(10000000).0.0.0 to 191(10111111).255.255.255
      • portions
        • first 2 octets: Networks
        • last 2 octests: Hosts
    • C
      • start with binary 110
      • range from 192(11000000).0.0.0 to 223(11011111).255.255.255
      • portions
        • first 3 octets: Networks
        • last 1 octests: Hosts
  • Multicast
    • D
      • start with binary 111
  • reserved for other purposes
    • E
      • start with binary 1111

These classes replaced by Classless Inter-Domain Routing(CIDR) in 1993

Read More »IP Addressing

OSI Model

  • Devops
tags: ccna

OSI Model

  • By International Organization of Standard

Benefits

  • Standard and INTEROPERABILITY
  • Split development/role: hide developer from lower layer
  • Quicker development

Layers

You need to remember both the name and the layer number

  • Layer7: Application
  • Layer6: Presentation
  • Layer5: Session
  • Layer4: Transport
  • Layer3: Network
  • Layer2: DataLink
  • Layer1: Physical

Trick: All People Sleeping Through Networking Don’t Pass

Network Engineer: Focus on 1, 2, 3, 4 Layers
Web Developer: Focus on 5, 6, 7 Layers

Read More »OSI Model

Docker Swarm 進階實戰 (2)

  • Devops
tags: docker, devops

Docker Swarm進階實戰(2)

Service Logs

  • 是container logs的聚集
  • 可以印出全部tasks或單一task
  • 如果你沒有log system,這可能是唯一troubleshooting的方式
  • 沒有storage,、不支援log搜尋或轉送至其他系統
  • 如果你有設定--log-driver,也就是其他logging方式,此指令就抓不到任何logs

除非你是小型swarm或測試階段,才使用一般的service logs,長期還是需要其他log storage的解決方案

範例指令

印出時不被截斷

只取最近50筆

Linux跟Mac可以透過grep做簡單的篩選

Windows透過findstr

Docker Events

  • Docker engine或Swarm產生的操作(Action)紀錄
  • 紀錄service/node/secret/config/network等的新增修改刪除動作
  • 支援搜尋及formatting
  • 分兩種scope,分別是swarm level跟local level(單一node)
  • event存在實體記憶體,最多只能存1000筆,所以當node數量很多時,event就不堪用了,還是需要額外的logging system

範例指令 顯示local events,如果node是manager,會顯示swarm events

--since

--filter

Swarm Configs

可以在Swarm的Raft Database儲存字串或檔案,並mapping容器中,適合的應用場景為:

  • nginx設定或更改proxy config
  • 設定或更改資料庫config等 這樣可以取代bind mount(將資料儲存在硬碟),存在Raft能提高可用性 swarm config跟secret有異曲同工之妙,可以透過指令來新增刪除config files,要注意的是:
  • config是immutable,不能修改只能replace
  • 若有任何service使用到該config,要先移除service才能移除config
  • 如果是隱私資料一樣用secret來存

範例指令

用特定config來建立一個service

新增新的config來取代舊的,並更新有用到該config的service

stack file,3.3之後才支援config