Показаны сообщения с ярлыком Nginx. Показать все сообщения
Показаны сообщения с ярлыком Nginx. Показать все сообщения

среда, 25 июля 2018 г.

Nginx Rewrite Rule Examples with Reg-Ex and Flags

Source.

Ability to change (rewrite) incoming URL into a different URL based on your criteria is an essential feature for any webserver.
Nginx rewrite is very powerful and flexible.
In this tutorial, we’ll explain the following examples on nginx rewrite:
  1. Nginx Rewrite Example Using $1, $2, ..
  2. Creating Controller File Using Nginx Rewrite
  3. Rewrite Break Flag in Location Context
  4. Adding Question Mark to Nginx Rewrite Replacement String
  5. If Context and Rewrite Directive
  6. Nginx Rewrite Flags Examples
  7. Capture Nginx Rewrite Hits in Error Log File

The following is the syntax of nginx rewrite:
rewrite reg-ex replacement [flag];
In the above:
  • rewrite directive is part of the ngx_http_rewrite_module module.
  • reg-ex – This is a PCRE regular expression that you’ll specify here. This will be used to match the incoming request URI.
  • replacement – If the reqeust URI matches the reg-ex, then Nginx will use this replacement string to change the request URI accordingly
  • flag – This will decide whether further process of the rewrite directives is required or not. This is explained in details in one of the examples below.
In nginx, the rewrite directive can be specified inside any one of the following three contexts: server, location, if

1. Nginx Rewrite Example Using $1, $2, ..

The following is an example of Nginx rewrite directive:
rewrite ^(/data/.*)/geek/(\w+)\.?.*$ $1/linux/$2.html last;
For example:
  • url/data/distro/geek/test.php will get rewritten as url/data/distro/linux/test.html
  • In this example, when you call the original URL with test.php from the browser, it will get rewritten based on the above rewrite rule and will serve test.html page from /data/distro/linux/
In the above rewrite rule:
  • $1 and $2 will capture the appropriate strings from the original URL that doesn’t change
  • $1 in the replacement string will match whatever is inside the 1st parenthesis ( ) in the reg-ex. In our example, $1 is /data/
  • Similarly $2 will match whatever is inside the 2nd parenthesis ( ) in the reg-ex. So, $2 is (\w+), which is any word that comes after the /geek/ in the original URL. In our example, $2 is test
  • last – This flag will make sure to stop the search of rewrite directive in the current location or block and use the changed URI (i.e rewritten URI) and look for new location for any further rewrite directives that matches.
  • *$ – This indicates the extension in the original URL. Please note that here, the extension from the original URL will be replaced by .html in the replaced URL by rewrite. So, even though you call .php in the original URL, it will only serve the .html file in the rewritten URL.
While Nginx rewrite rules does similar things like Apache, there are still lot of differences in terms of how you write a rewrite rule in Nginx.
Also, if you are new to Nginx, this might help to understand the basics: Nginx Vs Apache: Nginx Basic Architecture and Scalability

2. Creating Controller File Using Nginx Rewrite

Using rewrite, you can route many incoming original URL to a master controller template that will serve those request.
The following rewrite example explains this.
rewrite ^/linux/(.*)$ /linux.php?distro=$1 last;
In the above example, when you call thegeekstuff.com/linux/centos URL, it will get rewritten using the above rule and it will serve the page with this rewritten URL: thegeekstuff.com/linux.php?distro=centos
As you see above, any URL that has matches the pattern here (i.e /linux/ in the URL) will be served by linux.php, but the last portion in the original incoming URL will be used as an value for the distro argument in the linux.php controller.
So, the above rewrite rule will transform the incoming URL like this:
  • linux/centos becomes linux.php?distro=centos
  • linux/debian becomes linux.php?distro=debian
  • linux/redhat becomes linux.php?distro=redhat
  • etc.
Similar to previous example, we are using $1 in the replacement string to capture anything that is inside the 1st parenthesis ( ) in the reg-ex. In this case, this is the last part of the original incoming URL.
We are also using the last flag here to instruct nginx to stop search for further rewrite directives in the current-block and move-on to the next matching location for further search.

3. Rewrite Break Flag in Location Context

In this example, we’ve placed the rewrite condition inside location directive.
In this example, the location directive is /data/, which also matches the $1 in the replacement string given below.
location /data/ {
    rewrite ^(/data/.*)/geek/(\w+)\.?.*$ $1/linux/$2.html break;
    return  403;
}
This is what would’ve happened if you used “last” flag above:
  • So, if you had “last” as the flag, after the initial rewrite of the URL, Nginx will typically look for the next rewrite directive for the new URL.
  • In that case, Nginx will keep redirecting to the same location data and keep processing the same rewrite rule for maximum of 10 times, and finally it will return the 500 error code.
Since, we don’t want the above behavior, we’ve used “break” as the flag here which will just stop processing the rewrite blocks any further.
To use rewrite directive effectively inside location context, you need to understand the details of how location works: 13 Nginx Location Directive Examples including Regular Expression Modifiers

4. Adding Question Mark to Nginx Rewrite Replacement String

If a replacement string includes the new request arguments, the previous request arguments are appended after them. If you don’t want this behavior, putting a question mark at the end of a replacement string avoids having them appended.
In the following example, in the replacement string portion there is no question mark at the end. i.e No question mark after $1
rewrite ^/linux/(.*)$ /linux.php?distro=$1 last;
In the above example, when the replacement string include the incoming request arguments, then the arguments from the previous request are appended after them.
Some times, you probably don’t want that append to happen. In that case, use ? as shown below.
In the following example, in the replacement string portion of the Nginx rewrite, we’ve added ? at the end. i.e There is a question mark after $1
rewrite ^/linux/(.*)$ /linux.php?distro=$1? last;
In the above example, replacement string include the incoming request arguments, then the arguments from the previous request are NOT appended after them.

5. If Context and Rewrite Directive

The following few examples illustrates that we can use rewrite inside the if directive.
You can do a conditional rewrite based by doing some if condition comparison using variables like $scheme, $http_host, $http_user_agent, etc, as shown below:
if ($scheme = "http") {
  rewrite ^ https://www.thegeekstuff.com$uri permanent;
}

if ($http_host = thegeekstuff.com) {
  rewrite  (.*)  https://www.thegeekstuff.com$1;
}

if ($http_user_agent = MSIE) {
    rewrite ^(.*)$ /pdf/$1 break;
}
Please note that there are better ways to achieve the end-result of the above examples. The above examples are just given to show that we can add rewrite directive inside if statement in the nginx config file.
Please note that you can also set the value of the following two parameters to either on or off in your nginx config file:
server_name_in_redirect on 
port_in_redirect off

6. Nginx Rewrite Flags Examples

The following are the 4 different Nginx Rewrite directive flags that you can use.
last: This flag will stop the processing of the rewrite directives in the current set, and will start at the new location that matches the changed URL.
rewrite ^(/data/.*)/geek/(\w+)\.?.*$ $1/linux/$2.html last;
break: This flag will stop the processing of the rewrite directives in the current set.
rewrite ^(/data/.*)/geek/(\w+)\.?.*$ $1/linux/$2.html break;
redirect: This flag will do a temporary redirection using 302 HTTP code. This is mainly used when the replacement string is not http, or https, or $scheme
permanent: This flag will do a permanent redirection using 301 HTTP code
rewrite ^ https://www.thegeekstuff.com$uri permanent;

7. Capture Nginx Rewrite Hits in Error Log File

By default, anytime Nginx does successful rewrite, it doesn’t log it in the error.log.
Initially when you are writing complex rewrite rules, you really want to make sure that Nginx is doing the rewrite as per your requirement.
For this, you should enable the rewrite log, which will write a log entry anytime nginx does a successful rewrite using any one of the rewrite directive in the configuration file.
For this, use the rewrite_log directive and set it to on.
Add the following two lines to your nginx default.conf:
error_log /var/log/nginx/error.log notice;
rewrite_log on;
In the above:
  • The first line indicates the location of the error_log file where we want to write the rewrite messages. Please note that a rewrite message is of type notice. So, you have to add “note” at the end of this line as shown above.
  • rewrite_log on – This line enables logging of all the directives of ngx_http_rewrite_module modules to the error_log file.
After the above change, you’ll started seeing lines like this which clearly shows which specific rewrite rule was used in translating the incoming URL. This also will show the final translated URL in the log entry.
[notice] 14385#14385: *1 "^(/data/.*)/geek/(\w+)\.?.*$" matches "/data/distro/geek/test", client: 192.168.101.1, server: localhost, request: "GET /data/distro/geek/test HTTP/1.1", host: "192.168.101.10"
[notice] 14385#14385: *1 rewritten data: "/data/distro/linux/test.html", args: "", client: 192.168.101.1, server: localhost, request: "GET /data/distro/geek/test HTTP/1.1", host: "192.168.101.10"
In the above:
  • The 1st line shows two things 1) Incoming URL 2) Rewrite rule used
  • In the 1st line, it shows the incoming URL (i.e the request). In this example, the request is: “GET /data/distro/geek/test”
  • In the 1st line, it also shows the Nginx rewrite rule that matched this incoming request. In this example, the rewrite rule used by nginx is: “^(/data/.*)/geek/(\w+)\.?.*$”
  • In the 2nd line, it shows the rewritten translated URL that was used by Nginx after applying the rewrite rule. In this example, the translated rewritten URL is: /data/distro/linux/test.html

пятница, 30 октября 2015 г.

Установка нескольких версий Redmine с конкретными версиями ruby (nginx + passenger + rvm + Ubuntu 14.04)


Статья недоработана, но в плане справочной уже вполне себе :-)

 

Установка Nginx и Passenger


Установим PGP-ключ:
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 561F9B9CAC40B2F7

Добавим поддержку HTTPS для APT (APT репозиторий Phusion Passenger хранится на HTTPS-сервере):
sudo apt-get install apt-transport-https ca-certificates

Добавим сам репозиторий. В /etc/apt/sources.list.d/passenger.list поместим:
deb https://oss-binaries.phusionpassenger.com/apt/passenger trusty main

Затем установим соответствующие права и обновим списки пакетов:
sudo chown root: /etc/apt/sources.list.d/passenger.list
sudo chmod 600 /etc/apt/sources.list.d/passenger.lis
sudo apt-get update

Установим Nginx и Passenger:
sudo apt-get install nginx-extras passenger

Раскомментируем passenger_root и passenger_ruby в конфиге nginx: /etc/nginx/nginx.conf.

Перезапустим nginx:
sudo service nginx restart
 
Настроим nginx (пример): 
cat /etc/nginx/sites-available/redmine-test 
server {
  listen  8080;
  server_name project-t.centrofinans.ru;
  root /opt/redmine-3.1/public;
  passenger_user www-data;
  passenger_group www-data;
  passenger_enabled on;
  passenger_ruby /usr/local/rvm/gems/ruby-1.9.3-p551/wrappers/ruby;
  allow all;
  client_max_body_size      10m; # Max attachemnt size
}


Установка RVM и Rails


RVM нам нужен по нескольким причинам. Во-первых, велика вероятность, что в репозиториях убунты нету версии Ruby, которую вы используете в своём проекте. Во-вторых, через него удобнее установить Rails (чем через gem install rails), т.к. он подтягивает необходимые пакеты автоматически. В-третьих, если на сервере будет висеть несколько проектов, то вероятно, что они будут использовать разные версии Ruby, и тут нам поможет RVM.

gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
\curl -sSL https://get.rvm.io | bash -s stable --rails

После установки нужно выйти и зайти в шелл.

P.S. Если где-то в окружении не хватает rvm или rails, выполним "source /usr/local/rvm/scripts/rvm" или пропишем эту строку в profile (~/.profile или для всех /etc/profile).

 Нам нужна версия руби:


$ rvm install 1.9.3
 
Для удобства создадим для каждого проекта свой набор гемов:
$ rvm use 1.9.3
$ rvm gemset create redmine-3.1
 
Для того, чтобы rvm автоматически переключался на нужную версию руби и набор гемов делаем следующее:

$ echo "rvm use 1.9.3@redmine-3.1" > /opt/redmine-3.1/.rvmrc
 
Теперь в консоли при смене каталога проекта будет автоматически подключена нужная версия руби и набор 
гемов для проекта. 


Установка Redmine


cd /opt
svn co http://svn.redmine.org/redmine/branches/3.1-stable redmine-3.1
cd /opt/redmine-3.1
 
 
Чтобы указать необходимую версию Ruby выполним:

rvm use <версия Ruby>
passenger-config --ruby-command

На строке с префиксом "To use in Nginx :" скопируем оставшуюся часть и добавим в соответствующую запись server конфигурации nginx, завершив точкой с запятой (пример выше).



Add following lines
production:
  adapter: mysql2
  database: redmine
  host: localhost
  username: redmine
  password: redmine
  encoding: utf8
 

Plugins installation

Unarchive plugins to /plugins/ folder
cd /var/data/redmine bundle install

Configuring redmine

Setup redmine folder permissions

cd /var/data/redmine mkdir public/plugin_assets
chown -R www-data:www-data files log tmp public/plugin_assets config.ru chmod -R 755 files log tmp public/plugin_assets

Create database

mysql -u root -p Execute following lines to MySQL
CREATE DATABASE redmine character SET utf8; CREATE user 'redmine'@'localhost' IDENTIFIED BY 'redmine'; GRANT ALL privileges ON redmine.* TO 'redmine'@'localhost'; exit

Migrate database

cd /var/data/redmine bundle exec rake db:migrate bundle exec rake redmine:plugins

Generate session store

bundle exec rake generate_secret_token

Start web server

service nginx start

Restart Redmine

touch /var/data/redmine/tmp/restart.txt
 
 

RVM - несколько версий Ruby на одном сервере

Источник, спасибо автору.

Идея

Идея Ruby Version Manager состоит в разработке, где требуется иметь одновременно несколько версий Ruby и различные варианты Rails под ними.

Общая схема:

Представляет собой стандартную схему frontend + backend. В качестве backend выступает nginx с модулем passenger
  1. nginx (frontend) - проксирование, стандарт;
  2. nginx (backend's) - работа приложений через passenger:
    • каждый backend работает на порту 8<Ruby-version> (например, 8191, 8192);
    • каждое приложение, работая под версией Ruby, может использовать требуемый Rails (через gemset).
Сервер на котором производилась установка: Debian 5.0.8 i686.

Установка

Устанавливаем пакеты необходимые для компиляции:
apt-get install git-core git curl file   \
                gcc make automake autoconf automake1.9 \
                binutils g++ g++-multilib checkinstall
apt-get install libssl-dev libxslt-dev libxml2-dev
apt-get install libcurl4-openssl-dev libmysql++-dev
apt-get install libpcre3 libpcre3-dev

RVM

Установка RVM:
bash < <(curl -s https://rvm.beginrescueend.com/install/rvm)
/etc/rvmrc
umask g+w
export rvm_path="/usr/local/rvm"
export rvm_source_path="${rvm_path}/src"
export rvm_log_path="${rvm_path}/log"
export rvm_bin_path="${rvm_path}/bin"
export rvm_gems_path="$rvm_path/gems"
export rvm_tmp_path="${rvm_tmp_path:-"$rvm_path/tmp"}"
export rvm_install_on_use_flag=0
export rvm_gemset_create_on_use_flag=0
# export rvm_make_flags="-j7"
export rvm_trust_rvmrcs_flag=1
export rvm_pretty_print_flag=1
~/.bashrc
if [ -s "$HOME/.rvm/scripts/rvm" ]; then
  source "$HOME/.rvm/scripts/rvm" # This loads RVM into a shell session.
elif [ -s "/usr/local/rvm/scripts/rvm" ]; then
  source "/usr/local/rvm/scripts/rvm"
fi

Ruby в RVM

Установка Ruby:
rvm package install zlib
rvm install 1.9.1-p378 --with-zlib-dir=$rvm_path/usr
rvm install 1.9.2-p180 --with-zlib-dir=$rvm_path/usr
Проверка выбора Ruby
# rvm --default use 1.9.1
# rvm list

rvm rubies
=> ruby-1.9.1-p378 [ i386 ]
   ruby-1.9.2-p180 [ i386 ]

Системный Passenger

Сборка и установка passenger:
cd /usr/local/src
wget http://rubyforge.org/frs/download.php/74471/passenger-3.0.5.tar.gz
tar -xzvf ./passenger-3.0.5.tar.gz
mv -f passenger-3.0.5 /usr/local/
Passenger установлен в /usr/local/passenger-3.0.5

Поддержка SSL в Ruby

Добавление поддержки SSL:
rvm gem install rack
rvm package install openssl
rvm use ruby-1.9.1-p378
cd /usr/local/rvm/src/ruby-1.9.1-p378/ext/openssl/
ruby extconf.rb
make
make install 
rvm use ruby-1.9.2-p180
cd /usr/local/rvm/src/ruby-1.9.2-p180/ext/openssl/
ruby extconf.rb
make
make install

Сборка backend (nginx + passenger-module)

Собираем nginx с модулем passenger. Исходники помещаем в /usr/local/src :
cd /usr/src
wget http://sysoev.ru/nginx/nginx-1.0.2.tar.gz
tar xzvf nginx-1.0.2.tar.gz
Далее необходимо скачать и развернуть исходники pcre-8.10
cd /usr/local/src/nginx-1.0.2
Сконфигурировать сборку:
sh ./configure \
 --prefix='/usr/local/nginx' \
 --with-http_ssl_module  \
 --with-pcre='/usr/local/src/pcre-8.10' \
 --add-module='/usr/local/passenger-3.0.5/ext/nginx' \
 --user=www-data \
 --group=www-data \
 --error-log-path=/var/log/nginx-rb/error.log \
 --pid-path=/var/run/nginx-rb.pid \
 --lock-path=/var/lock/nginx-rb.lock \
 --http-client-body-temp-path=/var/lib/nginx-rb/body \
 --http-proxy-temp-path=/var/lib/nginx-rb/proxy \
 --http-fastcgi-temp-path=/var/lib/nginx-rb/fastcgi \
 --with-http_ssl_module \
 --with-http_dav_module \
 --with-http_gzip_static_module \
 --with-http_stub_status_module \
 --without-mail_pop3_module \
 --without-mail_smtp_module \
 --without-mail_imap_module
В результате подготовлена сборка nginx-ruby-backend в каталог /usr/local/nginx Выполнить сборку debian-пакета:
checkinstall -D make install
Указать имя пакета "nginx-rb" и описание пакета "Nginx-1.0.2 (backend) with passenger-3.0.5 module"

Настройка

frontend

Стандартная настройка. Варианты проксирования:
  1. по количеству backends/приложений;
  2. "универсально" - в зависимости от $request_uri/$server_name переводить на тот или иной upstream.
Второй вариант потребует меньше конфигураций и изменений во frontend.

backend

На примере backend ruby-1.9.2. Настроим для запуска дополнительный экземпляр nginx-backend:
/usr/local/nginx/etc/nginx-rb-192.conf:
  • пути к pid/лог-файлам;
  • пути к sites-enabled;
  • порт листинга (например, 8192 = 8<ruby-version>);
  • глобальные настройки passenger/ruby:
      http {
          ...
          passenger_root /usr/local/passenger-3.0.5;
          passenger_ruby /path-to-ruby;
          ...
      }
    
Узнать путь к конфигурационному файлу nginx-rb-192.conf (pass-to-ruby):
# rvm use ruby-1.9.2-p180 && which ruby
/etc/init.d/nginx-rb-192:
Включить в автозапуск:
# update-rc.d nginx-rb-192 defaults

Passenger

В конфигурации виртуального хоста backend необходимо указать (минимально):
server {
    listen          8192;
    server_name     <app_name>.domain.tld;
    root            /home/applications/<app_name>/prod/bin/public;
    rails_env       production;
    passenger_user  <app_name_user>;
    passenger_group <app_name_user>;
    passenger_enabled on;
}

Ruby

Gemsets

Принципиально возможно использовать несколько версий Rails под каждой развернутой Ruby. Достигается это манипуляциями с RVM/Gemset. Пример. Для Ruby-1.9.2 необходимо иметь rails-2.3.5 и rails-3.0.5 Формируем 2 gemset'а:
# rvm ruby-1.9.2-p180
# rvm gemset create rails235 rails305
ERROR: Gemset 'rails305' does not exist, rvm gemset create 'rails305' first.
'rails235' gemset created (/usr/local/rvm/gems/ruby-1.9.2-p180@rails235).
'rails305' gemset created (/usr/local/rvm/gems/ruby-1.9.2-p180@rails305).
Ошибка в выводе не принципиальна, появляется не всегда.
В итоге под ruby-1.9.2 имеется 3 gemset'а:
  • global - набор по-умолчанию (gem'ы из него автоматически доступны из всех наборов)
  • rails235
  • rails305
В дальнейшем возможны следующий манипуляции:
  • в global устанавливать gem'ы, необходимые для базовой поддержки приложений (не требующие вариаций по версиям и т.п.);
  • в railsNNN - устанавливать непосредственно rails требуемой версии и связанные с ним gem'ы.
Возможно более разветвленное дерево gemset'ов (в данной статье не рассматривается). Установка gem в gemset (важен выбор назначения — ruby-<version>@<gemset>):
# rvm 1.9.2-p180@rails305
# gem install rails -v 2.3.5
# rvm 1.9.2-p180@rails306
# gem install rails -v 3.0.5
Информация по выбранному gemset'у:
# rvm gemset list
gemsets for ruby-1.9.2-p180 (found in /usr/local/rvm/gems/ruby-1.9.2-p180)
   global
   rails235
=> rails305

Приложения

Для использования возможностей RVM необходимо выдержать следующую схему размещения приложений:
Структура каталогов: /home/applications/<app_name>/<app_type> - корневой каталог (app_root) - домашний каталог пользователя-владельца приложений;
/home/applications/<app_name>/<app_type>/bin - каталог приложения (app_home) В каталогах создать (dot)rvmrc — файлы:
  • для app_root (непосредственно позволяет использовать RVM):
    rvm_path="/usr/local/rvm"
    rvm_trust_rvmrcs_flag=1
    rvm_install_on_use_flag=1
    
  • для app_home (позволяет автоматически выбирать при входе в этот каталог версию Ruby и, при необходимости, gemset под RVM):
    #!/usr/bin/env bash
    environment_id="ruby-1.9.2-p180@rails305"
    if [[ -d "${rvm_path:-$HOME/.rvm}/environments" \
      && -s "${rvm_path:-$HOME/.rvm}/environments/$environment_id" ]] ; then
      \. "${rvm_path:-$HOME/.rvm}/environments/$environment_id"
    
      [[ -s ".rvm/hooks/after_use" ]] && . ".rvm/hooks/after_use"
    else
      rvm --create use "$environment_id"
    fi
    
Собственно выбор версии - environment_id. В app_home, в configure/ поместить файл setup_load_paths.rb:
if ENV['MY_RUBY_HOME'] && ENV['MY_RUBY_HOME'].include?('rvm')
  begin
    rvm_path     = File.dirname(File.dirname(ENV['MY_RUBY_HOME']))
    rvm_lib_path = File.join(rvm_path, 'lib')
    $LOAD_PATH.unshift rvm_lib_path
    require 'rvm'
    RVM.use_from_path! File.dirname(File.dirname(__FILE__))
  rescue LoadError
    # RVM is unavailable at this point.
    raise "RVM ruby lib is currently unavailable."
  end
end
# Select the correct item for which you use below.
# If you're not using bundler, remove it completely.
# If we're using a Bundler 1.0 beta
ENV['BUNDLE_GEMFILE'] = File.expand_path('../Gemfile', File.dirname(__FILE__))
require 'bundler/setup'
# Or Bundler 0.9...
if File.exist?(".bundle/environment.rb")
  require '.bundle/environment'
else
  require 'rubygems'
  require 'bundler'
  Bundler.setup
end

Контроль

Nginx

Штатные методы, с учетом наличия нескольких экземпляров.

Ruby

Вывод текущей версии Ruby:
# rvm list
Вывод текущего gemset:
# rvm gemset list
Вывод списка установленных gem:
# rvm gem list

Rails

Для перезапуска rails-процесса достаточно сделать в каталоге приложения:
# touch tmp/restart.txt

Passenger

Проверка статуса passenger:
  • /usr/local/passenger-3.0.5/bin/passenger-status
    
    Выдает несколько PID (по количеству backends);
  • /usr/local/passenger-3.0.5/bin/passenger-status <PID>
    
    Выдает статус rails-процессов по выбранному backend.

Ограничения

GEM могут быть установлены только от привилегированного пользователя. Для установки gem от имени непривилегированного пользователя требуется доработка.

Литература

  1. Phusion Passenger users guide, Nginx version
  2. NGINX
  3. Using RVM rubies with Passenger