вторник, 22 ноября 2022 г.

Convert a X.509 Certificate from Metadata

 Source

There are times that you might need to provide or have access to a X.509 certificate to verify the validity of a signed key or some other piece of data. X.509 Certificates are a standard for public key certificates and are often used to validate signatures on tokens and assertions used during user authentication, for example, when authenticating using SAML (Security Assertion Markup Language).

Recently, I needed to provide an X.509 certificate, provided by an Identity Provider, Azure AD, to an authorization service provider, Auth0. Since the X.509 certificate is a public format, the identity provider makes the certificate available in a long string format from their Federation Metadata Document, which is an .xml file publicly available. The format for the X.509 certificate provided by Azure was encoded in a base64 format, which was not accepted as is by Auth0, I needed to do some conversion prior to uploading to Auth0.

The following are steps taken to obtain and transform the X.509 certificate into a usable format across parties, (not all steps might be necessary for your use case):

  1. Obtain the X.509 certificate from the Identity Provider

Azure presents the X.509 certificate in the Federation Metadata Document which is a publicly available .xml document. In the metadata, there might be more than one X.509 certificate defined, this is because the certificates have differing expiration dates. If one certificate has expired, an application could then try to use another X.509 defined in the metadata for their validation needs.

Example .xml certificate

1
2
3
4
5
6
7
8
9
10
11
<KeyDescriptor use="signing">
<KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
<X509Data>
<X509Certificate>
MIIDBTCCAe2gAwIBAgIQWPB1ofOpA7FFlOBk5iPaNTANBgkqhkiG9w0BAQsFADAtMSswKQYDVQQDEyJhY2NvdW50cy5hY2Nlc3Njb250cm9sLndpbmRvd3MubmV0MB4XDTIxMDIwNzE3MDAzOVoXDTI2MDIwNjE3MDAzOVowLTErMCkGA1UEAxMiYWNjb3VudHMu
....
bmMCnFWuNNahcaAKiJTxYlKDaDIiPN35yECYbDj0PBWJUxobrvj5I275jbikkp8QSLYnSU/v7dMDUbxSLfZ7zsTuaF2Qx+L62PsYTwLzIFX3M8EMSQ6h68TupFTi5n0M2yIXQgoRoNEDWNJZ/aZMY/gqT02GQGBWrh+/vJ
</X509Certificate>
</X509Data>
</KeyInfo>
</KeyDescriptor>

  1. Copy/Paste value of <X509Certificate>…</X509Certificate>

To get the certificate into format we can work with, copy/paste the value in between the <X509Certificate> .xml elements into a new .crt file.

A valid .crt certificate file contains a BEGIN and END certificate declaration. Your file should look something like this:

Example .crt file

1
2
3
4
5
-----BEGIN CERTIFICATE-----
MIIDBTCCAe2gAwIBAgIQWPB1ofOpA7FFlOBk5iPaNTANBgkqhkiG9w0BAQsFADAtMSswKQYDVQQDEyJhY2NvdW50cy5hY2Nlc3Njb250cm9sLndpbmRvd3MubmV0MB4XDTIxMDIwNzE3MDAzOVoXDTI2MDIwNjE3MDAzOVowLTErMCkGA1UEAxMiYWNjb3VudHMu
....
bmMCnFWuNNahcaAKiJTxYlKDaDIiPN35yECYbDj0PBWJUxobrvj5I275jbikkp8QSLYnSU/v7dMDUbxSLfZ7zsTuaF2Qx+L62PsYTwLzIFX3M8EMSQ6h68TupFTi5n0M2yIXQgoRoNEDWNJZ/aZMY/gqT02GQGBWrh+/vJ
-----END CERTIFICATE-----

  1. Make .crt file valid

The current format of the .crt file is not actually a valid certificate format. We need to “fold” the long base64 string into one where each line of the certificate is 64 characters in length. This can be easily done with the fold command.

1
fold -w 64 example.crt > new-example.crt

This command will read our example.crt, perform the fold action to wrap each line after the 64th character, and then write the output to a new file with the new format.

  1. Convert .crt to .pem

Our example.crt might be acceptable, but I’ve found most relying parties need the X.509 certificate in a different format, often a .pem format. To convert the .crt to a .pem format we will use the openssl utility to convert between formats.

1
openssl x509 -in new-example.crt -out example.pem

The converted certificate can now be uploaded to the relying party.

While the steps are a bit manual, they can likely be improved and streamlined with scripting, etc. These certificates are often valid for many years or forever, so this should not be a process that needs to be performed often.

среда, 5 июня 2019 г.

A Read Only Kubernetes Dashboard

Source

the default configuration for the dashboard does not give it permissions to see anything within the cluster. The dashboard page acknowledges this and suggests simply binding the dashboard service account to cluster-admin…YIKES:

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kube-system
 
Ugh! Thats not OK. Better would be a read only user…like the ‘view’ ClusterRole:
But there’s a problem - the view ClusterRole doesn’t actually have permissions for the Cluster level objects like Nodes and Persistent Volume Claims. So we’ll have to create a new RBAC config.
First, we’ll create a new dashboard-viewonly ClusterRole:
 
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: dashboard-viewonly
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - endpoints
  - persistentvolumeclaims
  - pods
  - replicationcontrollers
  - replicationcontrollers/scale
  - serviceaccounts
  - services
  - nodes
  - persistentvolumeclaims
  - persistentvolumes
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - bindings
  - events
  - limitranges
  - namespaces/status
  - pods/log
  - pods/status
  - replicationcontrollers/status
  - resourcequotas
  - resourcequotas/status
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - apps
  resources:
  - daemonsets
  - deployments
  - deployments/scale
  - replicasets
  - replicasets/scale
  - statefulsets
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - autoscaling
  resources:
  - horizontalpodautoscalers
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - batch
  resources:
  - cronjobs
  - jobs
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - extensions
  resources:
  - daemonsets
  - deployments
  - deployments/scale
  - ingresses
  - networkpolicies
  - replicasets
  - replicasets/scale
  - replicationcontrollers/scale
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - policy
  resources:
  - poddisruptionbudgets
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - networkpolicies
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - storage.k8s.io
  resources:
  - storageclasses
  - volumeattachments
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - rbac.authorization.k8s.io
  resources:
  - clusterrolebindings
  - clusterroles
  - roles
  - rolebindings
  verbs:
  - get
  - list
  - watch
And then bind it to the service account:
 
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kubernetes-dashboard 
  namespace: kube-system 
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: kubernetes-dashboard
  labels:
    k8s-app: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: dashboard-viewonly
subjects:
- kind: ServiceAccount
  name: kubernetes-dashboard
  namespace: kube-system

Now we’ve configured a readonly dashboard we can safely share with people that dont even have K8S cluster access (many devs dont - they just push via a CI/CD pipeline). As a side note, this RBAC config does NOT grant access to Secrets, just to be a little safe.

понедельник, 29 апреля 2019 г.

Set AD SPN record for MS SQL service account

 Source
  1. Granting read/write servicePrincipalName permissions to the service account using ADSI Edit, as described in https://support.microsoft.com/en-us/kb/811889
  2. Removing the SPNs that previously existed on the SQL Server computer account (as opposed to the service account) using

    setspn -D MSSQLSvc/HOSTNAME.domain.name.com:1234 HOSTNAME

    where 1234 was the port number used by the instance (mine was not a default instance).
 

пятница, 19 апреля 2019 г.

Linux - Kernel Semaphore parameters

Source

1 - About

On Linux, A semaphore is a System V IPC object that is used to control utilization of a particular process.
Semaphores are a shareable resource that take on a non-negative integer value. They are manipulated by the P (wait) and V (signal) functions, which decrement and increment the semaphore, respectively. When a process needs a resource, a “wait” is issued and the semaphore is decremented. When the semaphore contains a value of zero, the resources are not available and the calling process spins or blocks (as appropriate) until resources are available. When a process releases a resource controlled by a semaphore, it increments the semaphore and the waiting processes are notified.
The Semaphore Kernel parameters
Semaphore Description Minimum
SEMMSL maximum number of semaphores per array 128
SEMMNS maximum semaphores system-wide
SEMOPM maximum operations per semop call
SEMMNI maximum arrays
Advertising

3 - How to

3.1 - Display them ?

This command displays the value of the semaphore parameters:
# /sbin/sysctl -a | grep sem

3.2 - Calculate them ?

  • Calculate the minimum total semaphore requirements using the following formula:
sum (process parameters of all database instances on the system) + system and other application requirements
  • Set semmns (total semaphores systemwide) to this total.
  • Set semmsl (semaphores per set) to 256.
  • Set semmni (total semaphores sets) to semmns / semmsl rounded up to the nearest multiple of 1024.
The following formula can be used as a guide, although in practice, SEMMNS and SEMMNU can be much less than SEMMNI * SEMMSL because not every program in the system needs semaphores.
SEMMNS = SEMMNU = (SEMMNI * SEMMSL)

3.3 - Set them ?

In the file, /etc/sysctl.conf
kernel.sem = 2200 6400 200 25
Where:
kernel.sem = SEMMSL SEMMNS SEMOPM SEMMNI
Then reboot or run this command:
# /sbin/sysctl -p
Advertising

4 - Documentation / Reference

пятница, 8 февраля 2019 г.

Systemd. How to clear journalctl.

Source

1) The self maintenance method is to vacuum the logs by size or time.
Retain only the past two days:
journalctl --vacuum-time=2d
Retain only the past 500 MB:
journalctl --vacuum-size=500M
man journalctl for more information.

 2) You don't typically clear the journal yourself. That is managed by systemd itself and old logs are rotated out as new data comes in. The correct thing to do would be to schedule journald to only keep as much data as you are interested in. The most usual thing to adjust is the total disk space it is allowed to take up. Once it crosses this boundry it will start pitching old entries to stay near this value.
You can set this in /etc/systemd/journald.conf like so:
SystemMaxUse=100M
 
3) To get rid of everything, you need to rotate the files first so that recent entries are moved to inactive files.

So, the complete answer to remove all entries seems to be
journalctl --rotate journalctl --vacuum-time=1s (Note that you cannot combine this into one journalctl command.)
By the way, some distributions have journald configured so that it writes logs to disk (/var/log/journal) while others keep logs in memory (/run/log/journal). I expect that in some cases it may be necessary to use journalctl --flush first to get everything removed.
If you don't have --rotate in your version, you can use the --since argument to filter entries:
--since "2019-01-30 14:00:00" --since today

пятница, 23 ноября 2018 г.

Regex matching in a Bash if statement

 Source
 
#!/bin/bash

# Only continue for 'develop' or 'release/*' branches
BRANCH_REGEX="^(develop$|release//*)"

if [[ $BRANCH =~ $BRANCH_REGEX ]];
then
    echo "BRANCH '$BRANCH' matches BRANCH_REGEX '$BRANCH_REGEX'"
else
    echo "BRANCH '$BRANCH' DOES NOT MATCH BRANCH_REGEX '$BRANCH_REGEX'"
fi

вторник, 23 октября 2018 г.

How to Rebuild Corrupted RPM Database in CentOS

Source

The RPM database is made up of files under the /var/lib/rpm/ directory in CentOS and other enterprise Linux distributions such as RHEL, openSUSE, Oracle Linux and more.
If the RPM database is corrupted, RPM will not work correctly, thus updates cannot be applied to your system, you encounter errors while updating packages on your system via YUM package manager. The worst case scenario is being unable to run any rpm and yum commands successfully.
Read Also: 20 Practical Examples of RPM Command in Linux
There are a number of factors that can lead to the RPM database corruption, such as incomplete previous transactions, installation of certain third-party software, removing specific packages, and many others.
In this article, we will show how to rebuild a corrupted RPM database; this way you can recover from an RPM database corruption in CentOS. This requires root user privileges, otherwise, use the sudo command to gain those privileges.

Rebuild Corrupted RPM Database in CentOS

First start by backing up your current RPM database before proceeding (you might need it in the future), using the following commands.
# mkdir /backups/
# tar -zcvf /backups/rpmdb-$(date +"%d%m%Y").tar.gz  /var/lib/rpm
Backup RPM Database
Backup RPM Database
Next, verify the integrity of the master package metadata file /var/lib/rpm/Packages; this is the file that needs rebuilding, but first remove /var/lib/rpm/__db* files to prevent stale locks using following commands.
# rm -f /var/lib/rpm/__db*  
# /usr/lib/rpm/rpmdb_verify /var/lib/rpm/Packages
Verify RPM Database
Verify RPM Database
In case the above operation fails, meaning you still encounter errors, then you should dump and load a new database. Also verify the integrity of the freshly loaded Packages file as follows.
# cd /var/lib/rpm/
# mv Packages Packages.back
# /usr/lib/rpm/rpmdb_dump Packages.back | /usr/lib/rpm/rpmdb_load Packages
# /usr/lib/rpm/rpmdb_verify Packages
Dump and Load RPM Database
Dump and Load RPM Database
Now to check the database headers, query all installed packages using the -q and -a flags, and try to carefully observe any error(s) sent to the stderror.
# rpm -qa >/dev/null #output is discarded to enable printing of errors only
Last but not least, rebuild the RPM database using the following command, the -vv option allows for displaying lots of debugging information.
# rpm -vv --rebuilddb
Rebuild RPM Database
Rebuild RPM Database

Use dcrpm Tool to Detect and Correct RPM Database

We also discovered the dcrpm (detect and correct rpm) command line tool used to identify and correct well known issues to do with RPM database corruption. It is a simple and easy-to-use tool which you can run without option. For effective and reliable usage, you should run it regularly via cron.
You can install it from source; download the source tree and install it using setup.py (which should grab the psutil dependency from pypi as well), as shown.
# git clone https://github.com/facebookincubator/dcrpm.git
# cd dcrpm
# python setup.py install
Once you have installed dcrpm, run it as shown.
# dcrpm
Finally, try to run your failed rpm or yum command again to see if everything is working fine.
dcrpm Github repository: https://github.com/facebookincubator/dcrpm
You can find more information from RPM database recovery page.
That’s all! In this article, we have explained how to rebuild a corrupted RPM database in CentOS. To ask any questions or share your thoughts about this guide, use the feedback form below.