вторник, 3 июля 2018 г.

Install Sentry on CentOS 7

Source

This article covers how to install and configure Sentry error tracking software on CentOS 7 system. Assuming you’ve already installed epel repository, if you haven’t you can do it by just typing yum install epel-release. Let’s start installation!
1) As always, we’ll up-to-date system first. If you want, you can reboot your host after doing that.
yum update -y
2) Install pre-requisites that we’ll use for all softwares.
yum install wget python-setuptools.noarch python2-pip.noarch python-devel.x86_64 libxslt.x86_64 libxslt-devel.x86_64 libxml2 libxml2-devel.x86_64 libzip libzip-devel libffi.x86_64 libffi-devel.x86_64 openssl-libs.x86_64 libpqxx libpqxx-devel libyaml libyaml-devel libjpeg libjpeg-devel libpng libpng12 libpng12-devel libpng-devel net-tools gcc gcc-c++ -y
3) Sentry uses Postgresql as database server. Install and initialize Postgresql as follows.
Installing:
yum install postgresql-server.x86_64 postgresql-contrib
Initialize, add to start up and start database:
postgresql-setup initdb
systemctl enable postgresql.service
systemctl start postgresql.service

4) Redis is used for caching and KV storage in Sentry. Install and start it.
yum install redis
systemctl enable redis.service
systemctl start redis.service

5) Install supervisor in this step. We’ll use it to control Sentry and its processes.
yum install supervisor
systemctl enable supervisord.service

we don’t start it now because we’ll to add some config to supervisor, after that we’ll start it.
6) We installed pip already, let’s upgrade it.
pip install --upgrade pip
7) Install virtualenv to create isolated python environment to install Sentry with pip.
pip install -U virtualenv
8) Add Sentry user. We’ll use this account during sentry installation.
useradd sentry
9) Create database and add user to postgres to store Sentry data.
su - postgres
psql template1
create user sentry with password 'type_your_password';
alter user sentry with superuser;
create database sentrydb with owner sentry;
\q
exit

10) Switch to sentry user.
su - sentry
11) Create an environment.
virtualenv /home/sentry/sentry_app
12) To run commands in current shell environment, use source.
source /home/sentry/sentry_app/bin/activate
13) We are finally in the Sentry installation step. Installing Sentry by pip as follows.
pip install -U sentry
14) Initialize Sentry.
/home/sentry/sentry_app/bin/sentry init
15) Update the following two files according to the information provided during the installation process.
/home/sentry/.sentry/sentry.conf.py
/home/sentry/.sentry/config.yml

After updating the first file, it should look like this:
DATABASES = {
'default': {
'ENGINE': 'sentry.db.postgres',
'NAME': 'sentrydb',
'USER': 'sentry',
'PASSWORD': 'your_password',
'HOST': '127.0.0.1',
'PORT': '5432',
'AUTOCOMMIT': True,
'ATOMIC_REQUESTS': False,
}
}
the second file:
redis.clusters:
default:
hosts:
0:
host: 127.0.0.1
port: 6379
Note: please review the other settings in these two files.
16) Update pg_hba.conf file (it locates in /var/lib/pgsql/data/pg_hba.conf) as follows and restart postgres.

# TYPE  DATABASE        USER            ADDRESS                 METHOD

local    all             postgres                                peer

# "local" is for Unix domain socket connections only
local    all             all                                     peer
# IPv4 local connections:
host     all             all             127.0.0.1/32            md5
# IPv6 local connections:
host     all             all             ::1/128                 md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
#local   replication     postgres                                peer
#host    replication     postgres        127.0.0.1/32            ident
#host    replication     postgres        ::1/128                 ident
Restart postgres:
systemctl restart postgresql.service
17) Run upgrade command. Installation will ask you to type your email address in this step. Type and don’t forget to make it superuser.
/home/sentry/sentry_app/bin/sentry upgrade
exit

18) Make following change in /etc/supervisord.conf file at “[include]” section.
files = supervisord.d/*.conf
and then locate to /etc/supervisord.d path. Create a file named sentry.conf and edit it as follows.
[program:sentry-web]
directory=/home/sentry/sentry_app/
environment=SENTRY_CONF="/home/sentry/.sentry"
command=/home/sentry/sentry_app/bin/sentry --config=/home/sentry/.sentry run web
autostart=true
autorestart=true
redirect_stderr=true
user=sentry
stdout_logfile=syslog
stderr_logfile=syslog

[program:sentry-worker]
directory=/home/sentry/sentry_app/
environment=SENTRY_CONF="/home/sentry/.sentry"
command=/home/sentry/sentry_app/bin/sentry --config=/home/sentry/.sentry run worker
autostart=true
autorestart=true
redirect_stderr=true
user=sentry
stdout_logfile=syslog
stderr_logfile=syslog
startsecs=1
startretries=3
stopsignal=TERM
stopwaitsecs=10
stopasgroup=false
killasgroup=true

[program:sentry-cron]
directory=/home/sentry/sentry_app/
environment=SENTRY_CONF="/home/sentry/.sentry"
command=/home/sentry/sentry_app/bin/sentry --config=/home/sentry/.sentry run cron
autostart=true
autorestart=true
redirect_stderr=true
user=sentry
stdout_logfile=syslog
stderr_logfile=syslog
start the supervisor after all these operations:
systemctl start supervisord.service
18) Now we’ve completed all the steps necessary to install the program. Let’s open web browser and type Sentry address.
http://your_ip_address:9000
Fill in all the fields you need and your Sentry is ready!
Additional Steps:
If you want you can use Nginx, here is config for Https connection.
ssl.conf file:

server {
 listen     80;
 return     301 https://$host$request_uri;
}

server {
    listen 443 ssl;
    server_name yoursentry.com;

    ssl_certificate                 /etc/nginx/ssl/yoursentry.crt;
    ssl_certificate_key             /etc/nginx/ssl/yoursentry.key;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS;
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:128m;
    ssl_session_timeout 10m;


    location / {

    proxy_pass         http://sentry;
    proxy_set_header   Host                 $http_host;
    proxy_set_header   X-Forwarded-Proto    $scheme;
    proxy_set_header   X-Forwarded-For      $remote_addr;
    proxy_redirect     off;

    # keepalive + raven.js is a disaster
    keepalive_timeout 30;

    proxy_read_timeout 10s;
    proxy_send_timeout 10s;
    send_timeout 10s;
    resolver_timeout 10s;
    client_body_timeout 10s;

    # buffer larger messages
    client_max_body_size 10m;
    client_body_buffer_size 100k;

    add_header Strict-Transport-Security "max-age=31536000";
    }

  }

upstream sentry {
        keepalive 1024;
        server 0.0.0.0:9000 max_fails=2 fail_timeout=10m;
}

пятница, 22 июня 2018 г.

CentOS 7 libcurl with OpenSSL backend

When your .Net application fails with the following error: 
 
"The handler does not support custom handling of certificates 
with this combination of libcurl (7.29.0) and its SSL backend ("NSS/3.34")"
 
You could use this very bad solution (do it only for dev environment or in docker). 
 
yum update
yum install openssl-devel gcc
wget https://curl.haxx.se/download/<latest>.tar.gz
tar -zxf <latest>.tar.gz
cd <latest>
./configure --prefix=/usr/local/curl/ --without-nss --with-ssl=/usr/local/ssl/
make && make install 
 
echo '/usr/local/curl/lib' > /etc/ld.so.conf.d/libcurl.conf && ldconfig 

Now you can start your .Net application.

среда, 16 мая 2018 г.

Redis Sentinel: Make your dataset highly available

Source.

At this point the Sentinel jumps in. So what is this Sentinel stuff? The Sentinel process is a Redis Instance which was started with the –sentinel Option (or redis-sentinel binary). It needs a configuration file that tells the Sentinel which Redis master it should monitor.
In short, these are the benefits of using Sentinel:
  • Monitoring: Sentinel constantly checks if the Redis master and its slave instances are working.
  • Notifications: It can notify the system administrators or other tools via an API if something happens to your Redis instances.
  • Automatic Failover: When Sentinel detects a failure of the master node it will start a failover where a slave is promoted to master. The additional slaves will be reconfigured automatically to use the new master. The application/clients that are using the Redis setup will be informed about the new address to use for the connection.
  • Configuration provider: Sentinel can be used for service discovery. That means clients can connect to the Sentinel in order to ask for the current address of your Redis master. After a failover Sentinel will provide the new address.

Configuration and example setup with three nodes

The Sentinel processes are a part of a distributed system. That means your Sentinel processes are working together. For a high availability setup we suggest using more than one Sentinel, as Sentinel itself should not be a single point of failure. It also improves the proper failure detection via quorum.
Before you deploy Sentinel, consider the following facts. More in-depth information on the most important points can be found below:
  • At least Three Sentinel instances are needed for a robust deployment.
  • Separate your Sentinel instances with different VMs or servers.
  • Due to the asynchronous replication of Redis the distributed setup does not guarantee that acknowledged writes are retained during failures.
  • Your client-library needs to support Sentinel.
  • Test your high availability setup from time to time in your test environment and even in production systems.
  • Sentinel, Docker or other NAT/Port Mapping technologies should only be mixed with care.
A three node setup is a good start, so run your Redis Master and two Slaves before setting up the Sentinel processes. On each of your Redis Hosts you create the same Sentinel config (/etc/redis-sentinel.conf, depending on your Linux distribution), like so:
Let’s dig deeper and see what these options do:
  1. This line tells Sentinel which master to monitor (myHAsetup). The Sentinel will find the host at 192.168.1.3:6379. The quorum for this setup is 2. The quorum is only used to detect failures: This number of Sentinels must agree about the fact that the master is not reachable. When a failure is detected, one Sentinel is elected as the leader who authorizes the failover. This happens when the majority of Sentinel processes vote for the leader.
  2. The time in milliseconds an instance is allowed be unreachable for a Sentinel (not answering to PINGs or replies with an error). After this time, the master is considered to be down.
  3. The timeout in milliseconds that Sentinel will wait after a failover before initiating a new failover.
  4. The number of slaves that can sync with the new master at the same time after a failover. The lower the number the longer the failover will need to complete. Using the slaves to serve old data to clients, you maybe don’t want to re-synchronize all slaves with the new master at the same time as there is a very short timeframe in which the slave stops while loading the bulk data from the master. In this case set it to 1.If this does not matter set it to the maximum of slaves that might be connected to the master.
  5. Listen IP, limited to one interface.
After you set up the Sentinel configuration for your instances, start it via init script, systemd unit or simply via its binary (redis-server /path/to/sentinel.conf –sentinel). The Sentinel processes will discover the master, slaves and other connected sentinels and the system is complete. Let’s see now what the setup looks like and what happens during a failover.
redis-sentinel
We have three Redis instances and three Sentinel instances:
M1 = Master
R1 = Replica 1 / Slave 1
R2 = Replica 2 / Slave 2
S1 = Sentinel 1
S2 = Sentinel 2
S3 = Sentinel 3
Let’s check the status of the Sentinel: Via the option -p 26379 you connect directly to the Sentinel API.
As you see the Sentinels are monitoring one master “myHAsetup”, their status is OK and you can see how many slaves and other Sentinels are discovered.
So far so good, everything looks fine. Now let’s see what happens when the master is unresponsive: We can simulate an outage by issuing the following command.
This produces the following log file:
Here a short summary of what happens
  1. Failure is detected
  2. Config-Version Epoch is increased by +1
  3. Leader is elected
  4. Quorum check. 3 Sentinels see the master down
  5. Delay for the next possible failover, after the current one
  6. – 10. Failover to the new master, reconfiguration of the slave nodes (old master 192.168.1.29 is already marked as slave and down at the moment)
  7. Old master comes back to the setup (after hanging in DEBUG for 30 seconds)
  8. Old master is converted to slave and synchronizes to the new master.
Another check with the Sentinel API confirms the new master:
At the end downtime sums up to around 7-8 seconds (6 to detect the failure + 1-2 seconds to fulfill the failover). With a stable network setup without flapping you might be able to reduce the failure detection from 6 seconds to 3 seconds, thus minimizing the downtime, during which no writes are accepted.

Maintainance

So how do you maintain such a setup? In general the procedure is similar to other high availability setups: Before you start, take a backup of your keyspace with an RDB snapshot and copy it to save place, e.g. by updating the redis.conf and increasing the ‚maxmemory‘ parameter or modifying the save parameter for the RDB snapshot engine.
  • stop Redis at one slave node
  • update the config file
  • start Redis
  • wait until the node is synchronized properly
  • repeat for the second slave
  • repeat for the master node (causing a failover)
  • change complete
Another example, where you want to update the redis-package:
  • check the change-log from the new Redis version, if there was no harmful change you can go on
  • backup your keyspace, best would be a RDB snapshot
  • stop Redis and Sentinel
  • copy both config files to a save place on the server
  • update the Redis package
  • copy both configs from save place to the config-dir (/etc/)
  • start Sentinel
  • check whether the updated Sentinel starts and is considered active by the other Sentinels
  • if not, check the logfile for the cause of the issue
  • if yes, start Redis and check the startup and logfile if everything looks fine
  • repeat for the other slave, and then for the master.
  • update complete
After these two examples we inspect the Sentinel with its API:
This is very simple – but shows that the Sentinel works properly. To bring this article to a close, let’s take a look at some advanced commands. They all follow roughly the same pattern:
CommandDescription
sentinel mastersShow a list of all monitored masters + state
sentinel master <master name>Show the state of a specified master
sentinel slaves <master name>Show the slaves of the master + state
sentinel sentinels <master name>Show the Sentinel instances for this master + state
sentinel get-master-addr-by-name <master name>Return ip and port of the master. While a failover is in progress the ip and port of the promoted slave are returned
sentinel reset <pattern>Reset the masters with a matching name. Clears previous state for the master, removes every slave and sentinel discovered. A fresh discovery is started.
sentinel failover <master name>Force a failover, as if the master was not reachable
sentinel ckquorum <master name>Check if the current Sentinel configuration is able to reach quorum and majority
sentinel flushconfigForce Sentinel to rewrite it’s configuration on disk
sentinel monitor <name> <ip> <port> <quorum>During runtime, tell Sentinel to start monitoring a new master
sentinel remove <name>Remove a specified master from monitoring
sentinel set <name>  <option> <value>Similar to config set with Redis, you can change Sentinel options. All options at the Sentinel configuration file can also be set here

пятница, 4 мая 2018 г.

PostgreSQL: How to reload config settings without restarting database

Source

If you are making modifications to the Postgres configuration file postgresql.conf (or similar), and you want to new settings to take effect without needing to restart the entire database, there are two ways to accomplish this.

Option 1: From the command-line shell

su - postgres
/usr/bin/pg_ctl reload

Option 2: Using SQL

SELECT pg_reload_conf();
Using either option will not interrupt any active queries or connections to the database, thus applying these changes seemlessly.

пятница, 20 апреля 2018 г.

Change VmWare SCSI controller type in VM Centos 7 VM

Source

In RHEL 7.x this is a bit different as modprobe.conf doesn't exist.
In my case I needed to change the controller for the root disk from Paravirtual to LSI Logic SAS. As the previous posts suggest, this needs to be done in two places, the regular disk and the RamDisk as both will need to boot with the new driver.
First, if possible, clone your machine, don't snapshot it. Whenever you are working with disks, it's best not to involve snapshots. You may not need to do this second step, I did it in the theory that the disk controller would initialize itself if introduced to the system through an additional disk, just like you'd do for Windows: Second step - Shut down your VM, Attach a 1 GB disk using the SCSI controller type you'd like to change your root disk to and bring the system back up to modprobe discover it. (You might be able to do this hot) Third step - run the following command to add the correct driver to the RamDisk (Remember in my case I was moving from the VMWare Paravirtual to the LSI Logic SAS driver. It's likely you are going the opposite way, but you just need to change the driver type: dracut -f -v --add-drivers mptsas
Other options for drivers are: mptspi mptscsih mptbase
After doing this, shut down and remove the 1 GB temporary disk. Change the controller for the root disk to whatever driver you just added to the ramdisk, and boot up the system.

вторник, 3 апреля 2018 г.

Creating Kerberos Keytab Files Compatible with Active Directory

Source

How to create a keytab file for a Kerberos user logging into Active Directory.  What's a keytab file?  It's basically a file that contains a table of user accounts, with an encrypted hash of the user's password.  Why have a keytab file?  Well, when you want a server process to automatically logon to Active Directory on startup, you have two options:  type the password (in clear text) into a config file somewhere, or store an encrypted hash of the password in a keytab file.  Which is safer?  Well, you can decide.  In any case, you'd better do a good job of protecting the file (be it a config file or a keytab).

Anyway, the accepted way to store a hashed password in Kerberos is to use a keytab file.  Now the file can be created using a number of utilities.  On a Windows machine, you can use ktpass.exe.  On Ubuntu Linux, you can use ktutil.



Before I demonstrate how to create the keytab, a word about encryption.  There are a number of encryption types used for hashing a password.  These include DES-CBC-CRC, DES-CBC-MD5, RC4-HMAC and a few others.  Active Directory uses RC4-HMAC by default.  Back in Windows 2000, you could also use the DES types without any trouble, but since Windows 2003, only RC4-HMAC is supported, unless you make a registry change (to all of your domain controllers).  If you need to use DES for some reason, then refer to the Technet article at the bottom of the page.

Before attempting to create a keytab file, you'll need to know the user's kerberos principal name, in the form of username@MYDOMAIN.COM, and the user's password.

Creating a KeyTab on Windows (tested on Windows Server 2008 R2)
Open a command prompt and type the following command:


ktpass /princ username@MYDOMAIN.COM /pass password /ptype KRB5_NT_PRINCIPAL /out username.keytab
Creating a KeyTab on Ubuntu Linux (tested on Ubuntu 10.10 - Maverick Meerkat)
Open a terminal window and type the following commands:

ktutil
addent -password -p username@MYDOMAIN.COM -k 1 -e RC4-HMAC
- enter password for username -
wkt username.keytab
q

Testing the Keytab File
Now in order to test the keytab, you'll need a copy of kinit.  You can use the version that's on Ubuntu, or if on Windows, you can install the latest Java runtime from Sun (JRE).  In either case, you'll need to setup your /etc/krb5.conf file (on Linux) or c:\windows\krb5.ini (on Windows).  Either file should look something like this:

[libdefaults]
default_realm = MYDOMAIN.COM
krb4_config = /etc/krb.conf
krb4_realms = /etc/krb.realms
kdc_timesync = 1
ccache_type = 4
forwardable = true
proxiable = true

[realms]
MYDOMAIN.COM = {
kdc = mydomain.com:88
admin_server = mydomain.com
default_domain = mydomain.com
}

[domain_realm]
.mydomain.com = MYDOMAIN.COM
mydomain.com = MYDOMAIN.COM

[login]
krb4_convert = true
krb4_get_tickets = false

Once you've got your Kerberos file setup, you can use kinit to test the keytab.  First, try to logon with your user account without using the keytab:

kinit username@MYDOMAIN.COM
- enter the password -

If that doesn't work, your krb5 file is wrong.  If it does work, now try the keytab file:

kinit username@MYDOMAIN.COM -k -t username.keytab

Now you should successfully authenticate without being prompted for a password.  Success!

More Information
If you need to use any other encryption Type than RC4-HMAC, then you'll need to tweak your AD domain controllers.  Please refer to the following TechNet article.

четверг, 14 декабря 2017 г.

How to unlock an user account in Linux?

Source

How to unlock a user account in Linux?

Some times on Linux boxes the user account will be locked due to issues such as wrong password entry, account expiry etc. In this post we will see how to unlock user account with different commands.
Example1: Check if the password is disabled by viewing /etc/shadow file for user entry.
grep ‘username’ /etc/shadow
if you are able to see ! in the second field starting that indicates that password is disabled, you have to enable it back by using passwd with -u option
passwd -u username
Example:
passwd -u surendra
Unlocking password for user temp.
passwd: Success
Example2: Check if the user expiry date is reached or not by using chage command
chage -l username
Example
chage -l surendra
Last password change : Jan 05, 2012
Password expires : never
Password inactive : never
Account expires : Jan 01, 2012
Minimum number of days between password change : 0
Maximum number of days between password change : 99999
Number of days of warning before password expires : 7


If you see that the account expires use usermod or chage command to extend the user expiry time.
usermod -e yyyy-mm-dd username
usermod -e 2012-05-10 surendra
or
chage -E yyyy-mm-dd username
chage -E 2012-05-10 surendra
this will extend user expiry time to 5 more months.
Example3: Check if the user shell is set to a valid shell or not, if it’s not set it to a valid one.
grep ‘username’ /etc/passwd
Example:
grep ‘surendra’ /etc/passwd
If the user shell in seventh feild is set to /sbin/nologin or /bin/false set it back to /bin/bash or /bin/ksh
usermod -s /bin/bash usrename
usermod -s /bin/bash surendra
Share your thoughts on this and let us know if you have other ideas to unlock user accounts in Linux.