Убрать ^M в конце строк текстового файла

dos2unix filename

sed -i 's/\r$//' filename

sed -i 's/\r//g' file

find . -type f -name '*.php' -exec sed -i -r 's/\r$//g' {} \; - то же самое, но для всех файлов .php в поддиректориях.

Nginx Rewrite Rule Examples with Reg-Ex and Flags

Source.

Ability to change (rewrite) incoming URL into a different URL based on your criteria is an essential feature for any webserver.
Nginx rewrite is very powerful and flexible.
In this tutorial, we’ll explain the following examples on nginx rewrite:

1. Nginx Rewrite Example Using $1, $2, ..
2. Creating Controller File Using Nginx Rewrite
3. Rewrite Break Flag in Location Context
4. Adding Question Mark to Nginx Rewrite Replacement String
5. If Context and Rewrite Directive
6. Nginx Rewrite Flags Examples
7. Capture Nginx Rewrite Hits in Error Log File

The following is the syntax of nginx rewrite:

rewrite reg-ex replacement [flag];

In the above:

• rewrite directive is part of the ngx_http_rewrite_module module.
• reg-ex – This is a PCRE regular expression that you’ll specify here. This will be used to match the incoming request URI.
• replacement – If the reqeust URI matches the reg-ex, then Nginx will use this replacement string to change the request URI accordingly
• flag – This will decide whether further process of the rewrite directives is required or not. This is explained in details in one of the examples below.

In nginx, the rewrite directive can be specified inside any one of the following three contexts: server, location, if

1. Nginx Rewrite Example Using $1, $2, ..

The following is an example of Nginx rewrite directive:

rewrite ^(/data/.*)/geek/(\w+)\.?.*$ $1/linux/$2.html last;

For example:

• url/data/distro/geek/test.php will get rewritten as url/data/distro/linux/test.html
• In this example, when you call the original URL with test.php from the browser, it will get rewritten based on the above rewrite rule and will serve test.html page from /data/distro/linux/

In the above rewrite rule:

• $1 and $2 will capture the appropriate strings from the original URL that doesn’t change
• $1 in the replacement string will match whatever is inside the 1st parenthesis ( ) in the reg-ex. In our example, $1 is /data/
• Similarly $2 will match whatever is inside the 2nd parenthesis ( ) in the reg-ex. So, $2 is (\w+), which is any word that comes after the /geek/ in the original URL. In our example, $2 is test
• last – This flag will make sure to stop the search of rewrite directive in the current location or block and use the changed URI (i.e rewritten URI) and look for new location for any further rewrite directives that matches.
• *$ – This indicates the extension in the original URL. Please note that here, the extension from the original URL will be replaced by .html in the replaced URL by rewrite. So, even though you call .php in the original URL, it will only serve the .html file in the rewritten URL.

While Nginx rewrite rules does similar things like Apache, there are still lot of differences in terms of how you write a rewrite rule in Nginx.
Also, if you are new to Nginx, this might help to understand the basics: Nginx Vs Apache: Nginx Basic Architecture and Scalability

2. Creating Controller File Using Nginx Rewrite

Using rewrite, you can route many incoming original URL to a master controller template that will serve those request.
The following rewrite example explains this.

rewrite ^/linux/(.*)$ /linux.php?distro=$1 last;

In the above example, when you call thegeekstuff.com/linux/centos URL, it will get rewritten using the above rule and it will serve the page with this rewritten URL: thegeekstuff.com/linux.php?distro=centos
As you see above, any URL that has matches the pattern here (i.e /linux/ in the URL) will be served by linux.php, but the last portion in the original incoming URL will be used as an value for the distro argument in the linux.php controller.
So, the above rewrite rule will transform the incoming URL like this:

• linux/centos becomes linux.php?distro=centos
• linux/debian becomes linux.php?distro=debian
• linux/redhat becomes linux.php?distro=redhat
• etc.

Similar to previous example, we are using $1 in the replacement string to capture anything that is inside the 1st parenthesis ( ) in the reg-ex. In this case, this is the last part of the original incoming URL.
We are also using the last flag here to instruct nginx to stop search for further rewrite directives in the current-block and move-on to the next matching location for further search.

3. Rewrite Break Flag in Location Context

In this example, we’ve placed the rewrite condition inside location directive.
In this example, the location directive is /data/, which also matches the $1 in the replacement string given below.

location /data/ {
    rewrite ^(/data/.*)/geek/(\w+)\.?.*$ $1/linux/$2.html break;
    return  403;
}

This is what would’ve happened if you used “last” flag above:

• So, if you had “last” as the flag, after the initial rewrite of the URL, Nginx will typically look for the next rewrite directive for the new URL.
• In that case, Nginx will keep redirecting to the same location data and keep processing the same rewrite rule for maximum of 10 times, and finally it will return the 500 error code.

Since, we don’t want the above behavior, we’ve used “break” as the flag here which will just stop processing the rewrite blocks any further.
To use rewrite directive effectively inside location context, you need to understand the details of how location works: 13 Nginx Location Directive Examples including Regular Expression Modifiers

4. Adding Question Mark to Nginx Rewrite Replacement String

If a replacement string includes the new request arguments, the previous request arguments are appended after them. If you don’t want this behavior, putting a question mark at the end of a replacement string avoids having them appended.
In the following example, in the replacement string portion there is no question mark at the end. i.e No question mark after $1

rewrite ^/linux/(.*)$ /linux.php?distro=$1 last;

In the above example, when the replacement string include the incoming request arguments, then the arguments from the previous request are appended after them.
Some times, you probably don’t want that append to happen. In that case, use ? as shown below.
In the following example, in the replacement string portion of the Nginx rewrite, we’ve added ? at the end. i.e There is a question mark after $1

rewrite ^/linux/(.*)$ /linux.php?distro=$1? last;

In the above example, replacement string include the incoming request arguments, then the arguments from the previous request are NOT appended after them.

5. If Context and Rewrite Directive

The following few examples illustrates that we can use rewrite inside the if directive.
You can do a conditional rewrite based by doing some if condition comparison using variables like $scheme, $http_host, $http_user_agent, etc, as shown below:

if ($scheme = "http") {
  rewrite ^ https://www.thegeekstuff.com$uri permanent;
}

if ($http_host = thegeekstuff.com) {
  rewrite  (.*)  https://www.thegeekstuff.com$1;
}

if ($http_user_agent = MSIE) {
    rewrite ^(.*)$ /pdf/$1 break;
}

Please note that there are better ways to achieve the end-result of the above examples. The above examples are just given to show that we can add rewrite directive inside if statement in the nginx config file.
Please note that you can also set the value of the following two parameters to either on or off in your nginx config file:

server_name_in_redirect on 
port_in_redirect off

6. Nginx Rewrite Flags Examples

The following are the 4 different Nginx Rewrite directive flags that you can use.
last: This flag will stop the processing of the rewrite directives in the current set, and will start at the new location that matches the changed URL.

rewrite ^(/data/.*)/geek/(\w+)\.?.*$ $1/linux/$2.html last;

break: This flag will stop the processing of the rewrite directives in the current set.

rewrite ^(/data/.*)/geek/(\w+)\.?.*$ $1/linux/$2.html break;

redirect: This flag will do a temporary redirection using 302 HTTP code. This is mainly used when the replacement string is not http, or https, or $scheme
permanent: This flag will do a permanent redirection using 301 HTTP code

rewrite ^ https://www.thegeekstuff.com$uri permanent;

7. Capture Nginx Rewrite Hits in Error Log File

By default, anytime Nginx does successful rewrite, it doesn’t log it in the error.log.
Initially when you are writing complex rewrite rules, you really want to make sure that Nginx is doing the rewrite as per your requirement.
For this, you should enable the rewrite log, which will write a log entry anytime nginx does a successful rewrite using any one of the rewrite directive in the configuration file.
For this, use the rewrite_log directive and set it to on.
Add the following two lines to your nginx default.conf:

error_log /var/log/nginx/error.log notice;
rewrite_log on;

In the above:

• The first line indicates the location of the error_log file where we want to write the rewrite messages. Please note that a rewrite message is of type notice. So, you have to add “note” at the end of this line as shown above.
• rewrite_log on – This line enables logging of all the directives of ngx_http_rewrite_module modules to the error_log file.

After the above change, you’ll started seeing lines like this which clearly shows which specific rewrite rule was used in translating the incoming URL. This also will show the final translated URL in the log entry.

[notice] 14385#14385: *1 "^(/data/.*)/geek/(\w+)\.?.*$" matches "/data/distro/geek/test", client: 192.168.101.1, server: localhost, request: "GET /data/distro/geek/test HTTP/1.1", host: "192.168.101.10"
[notice] 14385#14385: *1 rewritten data: "/data/distro/linux/test.html", args: "", client: 192.168.101.1, server: localhost, request: "GET /data/distro/geek/test HTTP/1.1", host: "192.168.101.10"

In the above:

• The 1st line shows two things 1) Incoming URL 2) Rewrite rule used
• In the 1st line, it shows the incoming URL (i.e the request). In this example, the request is: “GET /data/distro/geek/test”
• In the 1st line, it also shows the Nginx rewrite rule that matched this incoming request. In this example, the rewrite rule used by nginx is: “^(/data/.*)/geek/(\w+)\.?.*$”
• In the 2nd line, it shows the rewritten translated URL that was used by Nginx after applying the rewrite rule. In this example, the translated rewritten URL is: /data/distro/linux/test.html

Install Sentry on CentOS 7

Source

This article covers how to install and configure Sentry error tracking software on CentOS 7 system. Assuming you’ve already installed epel repository, if you haven’t you can do it by just typing yum install epel-release. Let’s start installation!
1) As always, we’ll up-to-date system first. If you want, you can reboot your host after doing that.
yum update -y
2) Install pre-requisites that we’ll use for all softwares.
yum install wget python-setuptools.noarch python2-pip.noarch python-devel.x86_64 libxslt.x86_64 libxslt-devel.x86_64 libxml2 libxml2-devel.x86_64 libzip libzip-devel libffi.x86_64 libffi-devel.x86_64 openssl-libs.x86_64 libpqxx libpqxx-devel libyaml libyaml-devel libjpeg libjpeg-devel libpng libpng12 libpng12-devel libpng-devel net-tools gcc gcc-c++ -y
3) Sentry uses Postgresql as database server. Install and initialize Postgresql as follows.
Installing:
yum install postgresql-server.x86_64 postgresql-contrib
Initialize, add to start up and start database:
postgresql-setup initdb
systemctl enable postgresql.service
systemctl start postgresql.service
4) Redis is used for caching and KV storage in Sentry. Install and start it.
yum install redis
systemctl enable redis.service
systemctl start redis.service
5) Install supervisor in this step. We’ll use it to control Sentry and its processes.
yum install supervisor
systemctl enable supervisord.service
we don’t start it now because we’ll to add some config to supervisor, after that we’ll start it.
6) We installed pip already, let’s upgrade it.
pip install --upgrade pip
7) Install virtualenv to create isolated python environment to install Sentry with pip.
pip install -U virtualenv
8) Add Sentry user. We’ll use this account during sentry installation.
useradd sentry
9) Create database and add user to postgres to store Sentry data.
su - postgres
psql template1
create user sentry with password 'type_your_password';
alter user sentry with superuser;
create database sentrydb with owner sentry;
\q
exit
10) Switch to sentry user.
su - sentry
11) Create an environment.
virtualenv /home/sentry/sentry_app
12) To run commands in current shell environment, use source.
source /home/sentry/sentry_app/bin/activate
13) We are finally in the Sentry installation step. Installing Sentry by pip as follows.
pip install -U sentry
14) Initialize Sentry.
/home/sentry/sentry_app/bin/sentry init
15) Update the following two files according to the information provided during the installation process.
/home/sentry/.sentry/sentry.conf.py
/home/sentry/.sentry/config.yml
After updating the first file, it should look like this:

DATABASES = {
'default': {
'ENGINE': 'sentry.db.postgres',
'NAME': 'sentrydb',
'USER': 'sentry',
'PASSWORD': 'your_password',
'HOST': '127.0.0.1',
'PORT': '5432',
'AUTOCOMMIT': True,
'ATOMIC_REQUESTS': False,
}
}

the second file:

redis.clusters:
default:
hosts:
0:
host: 127.0.0.1
port: 6379

Note: please review the other settings in these two files.
16) Update pg_hba.conf file (it locates in /var/lib/pgsql/data/pg_hba.conf) as follows and restart postgres.

# TYPE  DATABASE        USER            ADDRESS                 METHOD

local    all             postgres                                peer

# "local" is for Unix domain socket connections only
local    all             all                                     peer
# IPv4 local connections:
host     all             all             127.0.0.1/32            md5
# IPv6 local connections:
host     all             all             ::1/128                 md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
#local   replication     postgres                                peer
#host    replication     postgres        127.0.0.1/32            ident
#host    replication     postgres        ::1/128                 ident

Restart postgres:
systemctl restart postgresql.service
17) Run upgrade command. Installation will ask you to type your email address in this step. Type and don’t forget to make it superuser.
/home/sentry/sentry_app/bin/sentry upgrade
exit
18) Make following change in /etc/supervisord.conf file at “[include]” section.
files = supervisord.d/*.conf
and then locate to /etc/supervisord.d path. Create a file named sentry.conf and edit it as follows.

[program:sentry-web]
directory=/home/sentry/sentry_app/
environment=SENTRY_CONF="/home/sentry/.sentry"
command=/home/sentry/sentry_app/bin/sentry --config=/home/sentry/.sentry run web
autostart=true
autorestart=true
redirect_stderr=true
user=sentry
stdout_logfile=syslog
stderr_logfile=syslog

[program:sentry-worker]
directory=/home/sentry/sentry_app/
environment=SENTRY_CONF="/home/sentry/.sentry"
command=/home/sentry/sentry_app/bin/sentry --config=/home/sentry/.sentry run worker
autostart=true
autorestart=true
redirect_stderr=true
user=sentry
stdout_logfile=syslog
stderr_logfile=syslog
startsecs=1
startretries=3
stopsignal=TERM
stopwaitsecs=10
stopasgroup=false
killasgroup=true

[program:sentry-cron]
directory=/home/sentry/sentry_app/
environment=SENTRY_CONF="/home/sentry/.sentry"
command=/home/sentry/sentry_app/bin/sentry --config=/home/sentry/.sentry run cron
autostart=true
autorestart=true
redirect_stderr=true
user=sentry
stdout_logfile=syslog
stderr_logfile=syslog

start the supervisor after all these operations:
systemctl start supervisord.service
18) Now we’ve completed all the steps necessary to install the program. Let’s open web browser and type Sentry address.
http://your_ip_address:9000
Fill in all the fields you need and your Sentry is ready!
Additional Steps:
If you want you can use Nginx, here is config for Https connection.
ssl.conf file:

server {
 listen     80;
 return     301 https://$host$request_uri;
}

server {
    listen 443 ssl;
    server_name yoursentry.com;

    ssl_certificate                 /etc/nginx/ssl/yoursentry.crt;
    ssl_certificate_key             /etc/nginx/ssl/yoursentry.key;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS;
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:128m;
    ssl_session_timeout 10m;


    location / {

    proxy_pass         http://sentry;
    proxy_set_header   Host                 $http_host;
    proxy_set_header   X-Forwarded-Proto    $scheme;
    proxy_set_header   X-Forwarded-For      $remote_addr;
    proxy_redirect     off;

    # keepalive + raven.js is a disaster
    keepalive_timeout 30;

    proxy_read_timeout 10s;
    proxy_send_timeout 10s;
    send_timeout 10s;
    resolver_timeout 10s;
    client_body_timeout 10s;

    # buffer larger messages
    client_max_body_size 10m;
    client_body_buffer_size 100k;

    add_header Strict-Transport-Security "max-age=31536000";
    }

  }

upstream sentry {
        keepalive 1024;
        server 0.0.0.0:9000 max_fails=2 fail_timeout=10m;
}