Web Develop

深入理解 Nginx 讀書筆記 (第二章)

進程間的關係

  • Nginx 支持僅單進程(master)提供服務
  • 常態的部署是使用一個 master 進程來管理多個 worker 進程
  • Worker 數量與 CPU 核心數相等,進程切換代價最小

使用多進程的好處

  1. master 進程僅專注於純管理工作,為管理員提供命令行服務(啟動、停止、重配置、升級)
  2. master 進程需要比較大的權限,通常會以 root 使用者啟動
  3. 一個 worker 進程出錯後,其他 worker 仍然可以正常服務
  4. 充分利用 SMP(Symmetric multiprocessing) 多核架構,實現微觀上真正的多核併發處理
  5. Worker 通常不會進入睡眠狀態:可以同時處理多個請求,不像 Apache 每個進程只能同時處理一個請求,以致進程切換代價大

配置語法

每個模組都有自己感興趣的配置項,大部分模組都必須在 nginx.conf 中讀取到某個配置後才會啟用,例如只有當配置 http {…} 時, ngx_http_module 模組才會啟用,其他依賴的模組也才能正常使用

區塊配置項

  • 由名稱及一對大括號組成,如 http, server, location 都屬於區塊配置項
  • 傳入的參數取決於解析這個區塊配置項的模組
  • 大括號表示包含其中的配置同時生效
  • 可以嵌套,內層配置直接繼承外層
  • 當內外層配置發生衝突,以哪層配置為準,取決於解析這個區塊配置項的模組,例如範例的 gzip 開關

配置項語法格式

  • 名稱必須合法的(是某個 Nginx 模組想要處理的)
  • 傳入的參數取決於解析這個區塊配置項的模組
  • 若任一參數包含空格符,須要用單引號或雙引號包住
  • 以分號結尾

Read More »深入理解 Nginx 讀書筆記 (第二章)

深入理解 Nginx 讀書筆記 (第一章)

為什麼選擇 Nginx

  1. 更快: 1)單次請求更快響應 2) 在高峰期比其他服務器更快響應
  2. 高擴展性: 1)由耦合度極低模塊組成 2)模塊皆嵌入到2進制文件中執行
  3. 高可靠性: 1)模塊穩定 2)進程相對獨立 3)worker出錯可快速輪替
  4. 低內存消耗: 1)10,000個非活躍 HTTP Keep-Alive 連接僅消耗 2.5 MB
  5. 高併發: 1)單機支援 100,000 以上連接
  6. 熱部署: 1)基於 master 與 worker 進程分離 2)服務不間斷下,進行升級可執行元件、配置及更換日誌
  7. BSD 許可協議

開發準備工作

必要

  1. Linux 內核版本 2.6 以上 (須靠 epoll 處理高併發)
  2. GCC 編譯器編譯 C 語言

非必要

  1. G++,用來編譯 C++ 以編寫 HTTP 模塊
  2. PCRE(Perl 兼容正則表達式),用來在配置文件中使用正則表達式,pcre-devel 是使用 PCRE 做二次開發所需
  3. zlib, 用來對 HTTP 內容做 gzip 壓縮,減少網路傳輸量
  4. OpenSSL,支持 SSL 協議,或想使用 MD5 或 SHA 雜湊

目錄結構

  1. 源代碼目錄
  2. 編譯中間文件(置於源碼目錄底下,命名為objs)
  3. 部署目錄(莫認為 /usr/local/nginx)
  4. 日誌目錄

Linux 內核參數優化

  1. 須要修改內核參數,使得 Nginx 可以擁有更高的性能
  2. 通常根據業務特性進行調整,作為內容服務器、反向代理,或是提供縮圖用的服務器,會做不同調整

Read More »深入理解 Nginx 讀書筆記 (第一章)

Rabbitmq Exchange and its Types

Definition

exchange

Message routing agents.

Messages are not published directly to a queue. Instead, the publisher sends messages to an exchange.

Defined by the virtual host within RabbitMQ.

Predefined default exchanges are created at server starts.

Clients can create their own exchanges.

Exchange parameters:

  • type
  • name
  • durability
  • auto-delete (once last bound object is unbound from the exchange)

binding

Link between an exchange and a queue.

Use message attributes: routing key, header to route to the messages to queue(s).

Exchange Types

default exchange

No name. (usually referred by an empty string)

Every queue is automatically bound to the default exchange.

Binding key will be the queue name.

direct exchange

Similar to default exchange, but got a name and the bindings are not created automatically.

A message goes to the queue(s) with the binding key that exactly matches the routing key of the message.

fanout exchange

Routing key doesn’t have any effect.

Route message to all queues.

topic exchange

Similar to direct exchange, but use binding pattern not binding key.

A message goes to the queue(s) with the binding pattern that matches the routing key of the message.

image

header exchange

Similar to direct exchange, but use message header not routing key; use binding header not binding key

If x-match=all, then all header attributes should match

If x-match=any, then any header attributes match counts

image

Google Ads API Beta(v2.1) Short Notes

Prerequisites

  • Developer token
    • generate from a Google Ads Manager Account
    • you can use the same token for not linked account
    • token state: pending, approved
    • only approved token can connect to the API for production
    • Use any developer token if you using Test Manager Account view doc
  • Custermer ID
    • the account number of an Google Ads account
    • can be set or replace programmatically in the SDK
  • OAuth2 client credential
    • generate from a GCP Project/API Console
    • use for managing the API users
  • Client Library

Basic first call (installed application flow)

Step 1

  1. register an Google Ads Manager Account for production use
  2. take the developer token from UI
    > TOOL > SETTINGS > API Center

step refs:

You must use a production (non-test) manager account’s developer token to make API calls against a test account. Even if the token is pending approval, you can still use it to make calls against test accounts.

Step 2

  1. create a Google Account for testing
  2. use the Google Account to register a Google Ads Manager Account
  3. login and create one customer account (the customer created by test manager account will be test only)
  4. create a campaign under the customer account

step refs:

  • https://developers.google.com/google-ads/api/docs/first-call/overview#test_account
  • https://developers.google.com/adwords/api/docs/guides/accounts-overview#test_accounts

Step 3

If already has an OAuth client:

  1. assume the client ID is ORIGIN.apps.googleusercontent.com
  2. go to API console, find the OAuth2.0 client ID then download the client secret json file
  3. use this secret to request a generate refresh token

else:

  1. create a GCP project
  2. enable the GoogleAds API in API console page
  3. create a OAuth client, assume the client ID is NEW.apps.googleusercontent.com
  4. download the client secret json file
  5. use this secret to request a generate refresh token

step refs:

  • https://developers.google.com/adwords/api/docs/guides/accounts-overview#usingoauth2withtestaccounts

When requesting an OAuth2 refresh token, make sure you’re logged in as the test manager account user

To access a test account using OAuth2, the test manager account user must grant permission to your client application. Therefore, when requesting a refresh token, ensure you’re logged in as the test manager account rather than the production manager account.

you want to switch from the test manager account to the production manager account, simply reconfigure the client library to use the production manager account’s refresh token.

Step 4

  1. install client library
  2. create an sdk config file google-ads.yml and insert the values:
    • developer_token
    • client_id
    • client_secret
    • refresh_token
    • logincustomerid
  3. initial a client object that loads the config file
  4. make the first call to search the campaign we just created

Read More »Google Ads API Beta(v2.1) Short Notes

Python WSGI 及其應用

tags: python, wsgi, gunicorn, uwsgi

Python WSGI及其應用

WSGI是什麼?

  • 全名Python Web Server Gateway Interface
  • 基於CGI
  • Python用來描述了伺服器和請求處理程式之間傳輸資料的一種標準(協議)

PEP3333

基本原理跟目標

server、app、framework共用的界面

…proposes a simple and universal interface between web servers and web applications or frameworks: the Python Web Server Gateway Interface (WSGI).

不再重造輪子

…the goal of WSGI is to facilitate easy interconnection of existing servers and applications or frameworks, not to create a new web framework

只會支援python既有的release

…precludes WSGI from requiring anything that is not already available in deployed versions of Python

未來會增加部署的標準

…current version of WSGI does not prescribe any particular mechanism for “deploying” an application for use with a web server or server gateway. At the present time, this is necessarily implementation-defined by the server or gateway

Read More »Python WSGI 及其應用

初始Django專案並客製化設定檔 – Django Setup With Split Settings

  • Web Develop

Django Setup 專案初始化

假設我們想要建立一個django專案(1.9版本,linux為例),命名為mysite,並且不想要將特定的設定上傳,可以如何做:

clone你的倉庫(myrepo)至本地

建立虛擬環境

可以透過指令或IDE,我習慣會將環境命名為venv,然後放在專案目錄下

安裝django並開始一個專案

初始化django專案

別忘了把你安裝的套件加到mysite/requirements.txt

這時候你的tree應該長這樣

Read More »初始Django專案並客製化設定檔 – Django Setup With Split Settings