Nov 21

需要编译安装 ngx_http_substitutions_filter_module 模块,试图屏蔽掉广告但是美元成功,什么时候有时间再尝试一下。

worker_processes auto;
events {
  worker_connections 1024;
}
http {
  include  mime.types;
  default_type application/octet-stream;
  sendfile on;
  gzip on;
  server {
    listen 80;
    server_name yourdomain.com;

    location / {
      proxy_pass https://thepiratebay.org/;
      proxy_set_header Accept-Encoding "";
      proxy_set_header Host thepiratebay.org;
      proxy_set_header CF-Connecting-IP "";
      proxy_set_header Via "$host";
      proxy_ssl_verify off;
      proxy_ssl_server_name on;

      subs_filter 'thepiratebay.org' $host;
    }
  }
}

参考资料:https://proxybay.one/setup.html

Nov 02

1.直接寻找可用的 IP.

通过网络上的 ping 工具,检测得到 Google GHS 的 IP,分别打开下面列表中的网址,查询的地方输入 ghs.google.com 或者 ghs.googlehosted.com,里面得到的每一个 A Records 或响应 IP 就对应一个可以尝试的 GHS IP 地址,然后在本地通过 ping 测试验证是否能用。

http://bgp.he.net/dns/ghs.google.com
https://cloudmonitor.ca.com/en/dnstool.php
http://ping.chinaz.com/

可以将需要绑定的域名直接绑定到 A 记录,如果你有多个域名需要绑定的话,当一个 IP 被屏蔽的话,就需要修改所有的 A 记录。这里有一个小技巧,你可以添加一个可用的 IP 到域名 ghs.domain.com,然后通过 cname 的形式绑定 GAE 域名,这样当需要修改时仅仅需要修改一条 A 记录就可以了。

或者你可以使用网上仍然可用的由 GDG 维护的 ghs 替代域名:

www.goofan.net.g.xgslb.net

当然,这个需要添加 cname 记录。

2.使用 Nginx 直接反向代理
如果有 VPS 的话,可以直接安装 Ngrok,利用 Ngrok 反向代理 GAE,这种方法简单粗暴,一定能成功。但是如果你有多个 GAE 应用的话,就需要添加多段类似的代码,如果嫌麻烦的话可以参考下面第 3 种方法。

server
{
        listen   80;
        server_name xxx.com;

        location /
        {
                proxy_pass http://yyy.appspot.com;
                proxy_set_header Host "yyy.appspot.com";
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
}

3.使用 Nginx 反向代理 Google GHS
最好使用单独的 VPS 来反向代理 Google GHS,代码如下所示,然后添加 A 记录 ghs 到 VPS 的 IP,添加 cname 需要绑定的域名 到 ghs.domain.com。

upstream ghs {
   ip_hash;
   server ghs.google.com;
   server 72.14.203.121;
   server 72.14.207.121;
   server 74.125.43.121;
   server 74.125.47.121;
   server 74.125.53.121;
   server 74.125.77.121;
   server 74.125.93.121;
   server 74.125.95.121;
   server 74.125.113.121;
   server 216.239.32.21;
   server 216.239.34.21;
   server 216.239.36.21;
   server 216.239.38.21;
}

server {
   listen       80;
   server_name  ghs.domain.com;
#下行中的ghs换成自己定义的名称。
    log_format  ghs  '$remote_addr - $remote_user [$time_local] "$request" '
               '$status $body_bytes_sent "$http_referer" '
               '"$http_user_agent" $http_x_forwarded_for';
#下行中两处ghs换成自己定义的名称。
   access_log  /home/log/ghs.log ghs;

   location / {
     proxy_redirect off;
     proxy_set_header Host $host;
     proxy_pass http://ghs;
     proxy_set_header  X-Real-IP  $remote_addr;
     proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_redirect false;
   }

}

server {
   listen       443;
   server_name  ghs.domain.com;
   log_format  mail  '$remote_addr - $remote_user [$time_local] "$request" '
               '$status $body_bytes_sent "$http_referer" '
               '"$http_user_agent" $http_x_forwarded_for';
   access_log  /home/log/mail.log mail;

   location / {
     proxy_redirect off;
     proxy_set_header Host $host;
     proxy_pass http://ghs;
     proxy_set_header  X-Real-IP  $remote_addr;
     proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_redirect false;
   }
}

参考资料:
1.http://steven-wang.appspot.com/nginx-reverse-proxy-122001.html
2.Install Nginx and config Google ghs proxy

Nov 01

前面也写了一篇关于 Ngrok 搭建的文章:《树莓派上 Ngrok 的编译与使用》,但是看到《搭建 ngrok 服务实现外网访问局域网内的网站》这篇文章使用纯手工编译的步骤很详细,尤其是 go 语言的配置那部分内容,所以将文中的主要内容复制下来,以备不时之需。

事先的准备工作:添加 ngrok 服务域名的 DNS 解析。

选择支持泛解析的 DNS 服务商,如 Cloudns、DNSpod 国际版、zoneedit 等,分别添加 A 记录:

ngrok.chun.pro 记录值 1.2.3.4
*.ngrok.chun.pro 记录值 1.2.3.4

1.安装必要的工具和语言环境

sudo apt-get install build-essential golang mercurial git

2.升级 go 语言环境

# 看看是不是小于等于 1.2.1
go version
# 卸载
sudo apt-get purge golang*
#下载最新版并解压 https://golang.org/dl/
wget https://storage.googleapis.com/golang/go1.7.3.linux-386.tar.gz
tar -C /usr/local -xzf go1.7.3.linux-386.tar.gz
#创建目录
mkdir ~/.go
# 设置环境变量
vi ~/.profile
export GOROOT=/usr/local/go
export GOPATH=~/.go
export PATH=$PATH:$GOROOT/bin:$GOPATH/bin
source .profile

# 升级
sudo update-alternatives --install "/usr/bin/go" "go" "/usr/local/go/bin/go" 0
sudo update-alternatives --set go /usr/local/go/bin/go
go version

3.下载 ngrok 源码并编译服务端

git clone https://github.com/tutumcloud/ngrok.git ngrok
cd ngrok

#生成并替换源码里默认的证书,注意域名要修改为你自己的,这里是一个虚拟的测试域名
NGROK_DOMAIN="ngrok.chun.pro"
openssl genrsa -out base.key 2048
openssl req -new -x509 -nodes -key base.key -days 10000 -subj "/CN=$NGROK_DOMAIN" -out base.pem
openssl genrsa -out server.key 2048
openssl req -new -key server.key -subj "/CN=$NGROK_DOMAIN" -out server.csr
openssl x509 -req -in server.csr -CA base.pem -CAkey base.key -CAcreateserial -days 10000 -out server.crt

cp base.pem assets/client/tls/ngrokroot.crt
cp server.crt assets/server/tls/snakeoil.crt
cp server.key assets/server/tls/snakeoil.key

#开始编译,服务端客户端会基于证书来加密通讯,保证了安全性
GOOS=linux GOARCH=amd64 make release-server release-client
GOOS=linux GOARCH=386 make release-server release-client
GOOS=linux GOARCH=arm make release-server release-client
GOOS=linux GOARCH=arm64 make release-server release-client
GOOS=linux GOARCH=ppc64 make release-server release-client
GOOS=linux GOARCH=ppc64le make release-server release-client
GOOS=linux GOARCH=mips64 make release-server release-client
GOOS=linux GOARCH=mips64le make release-server release-client

GOOS=windows GOARCH=amd64 make release-server release-client
GOOS=windows GOARCH=386 make release-server release-client

GOOS=darwin GOARCH=amd64 make release-server release-client
GOOS=darwin GOARCH=386 make release-server release-client
GOOS=darwin GOARCH=arm make release-server release-client
GOOS=darwin GOARCH=arm64 make release-server release-client

GOOS=android GOARCH=arm make release-server release-client

GOOS=dragonfly GOARCH=amd64 make release-server release-client

GOOS=freebsd GOARCH=amd64 make release-server release-client
GOOS=freebsd GOARCH=386 make release-server release-client
GOOS=freebsd GOARCH=arm make release-server release-client

GOOS=netbsd GOARCH=amd64 make release-server release-client
GOOS=netbsd GOARCH=386 make release-server release-client
GOOS=netbsd GOARCH=arm make release-server release-client

GOOS=openbsd GOARCH=amd64 make release-server release-client
GOOS=openbsd GOARCH=386 make release-server release-client
GOOS=openbsd GOARCH=arm make release-server release-client

GOOS=plan9 GOARCH=amd64 make release-server release-client
GOOS=plan9 GOARCH=386 make release-server release-client

GOOS=solaris GOARCH=amd64 make release-server release-client

4.启动服务端
在服务器上运行下面的命令启动ngrok服务端

./bin/ngrokd -domain="ngrok.chun.pro" -httpAddr=":8081" -httpsAddr=":8082"

注意,这里 httpAddr 和 httpsAddr 是 ngrok 服务转发 http 和 https 请求的端口,为了避免和 Nginx/Apache 等的 80 端口冲突,使用了 8081 和 8082。

默认还会启动一个 4443 端口,用于跟活动的客户端进行通讯,如果需要更换端口,使用 -tunnelAddr=”:xxx”参数

现在你可以在浏览器里访问 http://ngrok.chun.pro:8081了,如果有一行提示,表示 ngrok 的服务端已经运行起来了

Tunnel ngrok.yourdomain.com:8081 not found

然后再访问 http://pi.ngrok.chun.pro:8081,如果有下面的提示,表示 A 记录也已经生效了。

Tunnel pi.ngrok.chun.pro:8081 not found

5.配置客户端参数

vi ngrok.cfg
# 填写如下信息,server_addr 指定了服务端的域名和与客户端通信的端口
server_addr: ngrok.chun.pro:4443
trust_host_root_certs: false

6.启动客户端

./ngrok -config=./ngrok.cfg -subdomain pi 127.0.0.1:80

如果连接正常,会有提示:

ngrok                                                        (Ctrl+C to quit)
Tunnel Status                 online
Version                       1.7/1.7
Forwarding                    http://pi.ngrok.chun.pro:8081 -> 127.0.0.1:80
Web Interface                 127.0.0.1:4040
# Conn                        5
Avg Conn Time                 192.70ms

7.nginx 反向代理 8081 端口

server {
        listen 80;
        server_name ngrok.chun.pro;
        location / {
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host:8081;
                proxy_set_header X-Nginx-Proxy true;
                proxy_set_header Connection "";
                proxy_pass http://127.0.0.1:8081;
        }
}

现在可以直接在浏览器访问 pi.ngrok.chun.pro,而不需要加 :8081 端口号。

另外,使用 Docker 搭建 Ngrok 服务器可以参考这篇文章:https://hteen.cn/docker/docker-ngrok.html

Dec 18

1.安装编译环境

apt-get install build-essential git gcc g++ make

2.安装 lua

apt-get install liblua5.1-0-dev liblua50-dev liblualib50-dev

3.安装 luajit

wget http://luajit.org/download/LuaJIT-2.0.3.tar.gz
tar -zxvf LuaJIT-2.0.3.tar.gz
cd LuaJIT-2.0.3
make PREFIX=/usr/local
make install

4.下载 nginx
因为 lua-nginx-module 只支持 Lua 5.1 或者 LuaJIT 2.0/2.1 版本,nginx 目前最高支持 1.9.3版本,所以需要设置 http_spdy_module 模块,1.9.5版本以上就可以支持
http_v2_module 模块了。

wget "http://nginx.org/download/nginx-1.9.3.tar.gz"
wget "ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.38.tar.gz"
wget "https://www.openssl.org/source/openssl-1.0.1j.tar.gz"
wget "http://zlib.net/zlib-1.2.8.tar.gz"

git clone https://github.com/cuber/ngx_http_google_filter_module
git clone https://github.com/yaoweibin/ngx_http_substitutions_filter_module
git clone https://github.com/openresty/lua-nginx-module.git
git clone https://github.com/simpl/ngx_devel_kit.git

tar xzvf nginx-1.9.3.tar.gz
tar xzvf pcre-8.38.tar.gz
tar xzvf openssl-1.0.1j.tar.gz
tar xzvf zlib-1.2.8.tar.gz

5.编译安装 nginx

./configure \
  --user=root \
  --group=root \
  --prefix=/usr/local/nginx \
  --with-pcre=../pcre-8.38 \
  --with-openssl=../openssl-1.0.1j \
  --with-zlib=../zlib-1.2.8 \
  --with-http_ssl_module \
  --with-http_stub_status_module \
  --with-http_ssl_module \
  --with-http_spdy_module \
  --with-http_gzip_static_module \
  --with-ipv6 \
  --with-http_sub_module \
  --add-module=../ngx_http_google_filter_module \
  --add-module=../ngx_http_substitutions_filter_module \
  --add-module=../ngx_devel_kit \
  --add-module=../lua-nginx-module

参考资料:https://github.com/openresty/lua-nginx-module/tree/master

6.编辑 nginx.conf

#user  nobody;
worker_processes  auto;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

pid        logs/nginx.pid;


events {
    use epoll;
    worker_connections 51200;
    multi_accept on;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  logs/access.log  main;

    sendfile        on;
    tcp_nopush     on;
    tcp_nodelay on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    fastcgi_connect_timeout 300;
    fastcgi_send_timeout 300;
    fastcgi_read_timeout 300;
    fastcgi_buffer_size 64k;
    fastcgi_buffers 4 64k;
    fastcgi_busy_buffers_size 128k;
    fastcgi_temp_file_write_size 256k;

    gzip on;
    gzip_min_length  1k;
    gzip_buffers     4 16k;
    gzip_http_version 1.0;
    gzip_comp_level 2;
    gzip_types       text/plain application/x-javascript text/css application/xml;
    gzip_vary on;
    gzip_proxied        expired no-cache no-store private auth;
    gzip_disable        "MSIE [1-6]\.";

    server {
        listen       80;
        server_name  scholar.xxx.com;
        rewrite ^(.*) https://scholar.xxx.com$1 permanent;
           }

    server {
        listen 443 spdy;
        server_name scholar.xxx.com;

        ssl on;
        ssl_certificate /usr/local/nginx/conf/xxx.crt;
        ssl_certificate_key /usr/local/nginx/conf/xxx.key;
        ssl_dhparam /usr/local/nginx/conf/dhparam2048.pem;
        add_header Strict-Transport-Security 'max-age=31536000; includeSubDomains; preload';
        ssl_prefer_server_ciphers on;
        ssl_ciphers 'kEECDH+ECDSA+AES128 kEECDH+ECDSA+AES256 kEECDH+AES128 kEECDH+AES256 kEDH+AES128 kEDH+AES256 DES-CBC3-SHA +SHA !aNULL !eNULL !LOW !MD5 !EXP !DSS !PSK !SRP !kECDH !CAMELLIA !RC4 !SEED';
        ssl_protocols TLSv1.2 TLSv1.1 TLSv1;
        ssl_session_cache   shared:SSL:10m;
        ssl_session_timeout 10m;
        keepalive_timeout   70;
        ssl_buffer_size 1400;
        spdy_headers_comp 0;

        resolver 8.8.8.8;
        location / {
            set $upstream "";
            rewrite_by_lua '
                 local upstreams = {
                       "http://a.xxx.com",
                       "http://b.xxx.com",
                       "http://c.xxx.com"
                                   }
                 ngx.var.upstream=upstreams[ math.random( #upstreams ) ]
                             ';

            #resolver 8.8.8.8;
            proxy_buffering off;
            proxy_redirect off;
            proxy_set_header Accept-Encoding "";
            proxy_set_header User-Agent $http_user_agent;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_pass       $upstream;
                    }
             }
    }

7.注意事项
我在一台 VPS 上很容易就安装完成了,但是在另外一台 VPS 就死活是 502 错误,找了好久才知道是 DNS 解析的问题,需要添加 resolver 8.8.8.8; 才能正常使用。

解答问题是在这里找到的:

http://stackoverflow.com/questions/17685674/nginx-proxy-pass-with-remote-addr

It seems a bit strange that nginx is failing to resolve the domain name at runtime rather than at configuration time (since the domain name is hard coded). Adding a resolver declaration to the location block usually fixes dns issues experienced at runtime. So your location block might look like:

location ^~ /freegeoip/ {
  #use google as dns
  resolver 8.8.8.8;
  proxy_pass http://freegeoip.net/json/$remote_addr;
}

而我将 resolver 8.8.8.8; 提到 location / 前面去了,作用是一样的。到此为止,将 Google 搜索和 Google Scholar 都配置完成了,解决了在学习中查找资料的问题。由于某同学的建议不要在博客上大肆宣传,所以这将部分配置写出,防止遗忘。

Dec 10

删除Apache2

$ sudo service apache2 stop
$ sudo apt-get purge apache2 apache2-utils apache2.2-bin apache2-common
$ sudo apt-get autoremove
$ whereis apache2
$ sudo rm -rf /etc/apache2

安装nginx

apt-get install build-essential git gcc g++ make
wget "http://nginx.org/download/nginx-1.7.8.tar.gz"
wget "ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.38.tar.gz"
wget "https://www.openssl.org/source/openssl-1.0.1j.tar.gz"
wget "http://zlib.net/zlib-1.2.8.tar.gz"
git clone https://github.com/cuber/ngx_http_google_filter_module
git clone https://github.com/yaoweibin/ngx_http_substitutions_filter_module
tar xzvf nginx-1.7.8.tar.gz
tar xzvf pcre-8.38.tar.gz
tar xzvf openssl-1.0.1j.tar.gz
tar xzvf zlib-1.2.8.tar.gz

cd nginx-1.7.8

./configure \
  --prefix=/usr/local/nginx \
  --with-pcre=../pcre-8.38 \
  --with-openssl=../openssl-1.0.1j \
  --with-zlib=../zlib-1.2.8 \
  --with-http_ssl_module \
  --with-http_stub_status_module \
  --with-http_ssl_module \
  --with-http_spdy_module \
  --with-http_gzip_static_module \
  --with-ipv6 \
  --with-http_sub_module \
  --add-module=../ngx_http_google_filter_module \
  --add-module=../ngx_http_substitutions_filter_module

make
make install

配置nginx

server {
  server_name scholar.google.com;
  resolver 8.8.8.8;
  location / {
    proxy_pass https://scholar.google.com;
  }
}

重启nginx:

/usr/local/nginx/sbin/nginx -s reload

报错代码:

nginx: [error] open() “/usr/local/nginx/logs/nginx.pid” failed (2: No such file or directory)

解决方法:
1.重新运行一下:

/usr/local/nginx/sbin/nginx

2.重启nginx:

/usr/local/nginx/sbin/nginx -s reload

3.运行:

/usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf