Difference between revisions of "Einstein"

From Nuclear Physics Group Documentation Pages
Jump to navigationJump to search
 
(26 intermediate revisions by 2 users not shown)
Line 1: Line 1:
Note: This is the page for the NEW EINSTEIN - 8 core server from Microway
+
Einstein is currently a virtual machine running on [[Gourd]]. Go [[old_einstein|here]] for information about the old Einstein hardware.
  
The previous einstein hardware is described in the previous page for Einstein at [[old einstein]]
+
=Virtual Hardware=
  
= New Microway Server =
+
*Memory: 2 GB
 +
*Hard Disk: 20 GB
 +
*Swap Disk: 4 GB
 +
*Network 1 (eth0): Farm-Bridge
 +
*Network 2 (eth1): UNH-Bridge
 +
*SCSI Controller: LSI Logic
  
The new einstein is a 2 quad-CPU server in a 2U rackmount chassis put together nicely by Microway. It arrived at UNH on 11/24/2009. The system has an Areca RAID card with ethernet port and an IPMI card with ethernet port. The motherboard is from Super Micro. Details need to be put here!
+
=Network Settings=
  
 +
*IP Address farm (eth0): 10.0.0.248
 +
*IP Address UNH (eth1):  132.177.88.52
  
 +
=Software and Services=
  
== Important things to remember before this system takes on the identity of Einstein ==
+
==IPTables==
  
# The ssh fingerprint of the old einstein needs to be imported.
+
Einstein uses the standard NPG [[iptables]] firewall. It allows ssh, LDAP, imap, smtp, and http connections.
# Obviously, all important data needs to be moved: Home Directories, Mail, DNS records, ... (what else?)
 
# Fully test functionality before switching!
 
  
== Configurations Needed ==
+
==LDAP==
  
# RAIDs need to be setup on Areca card.
+
Einstein is our [[LDAP]] server. It provides user authentication among other useful services. LDAP configuration is located in /etc/openldap, and the LDAP database is stored in /var/lib/ldap.
# Mail system needs to be setup
 
# Webmail needs to be setup. Uses Apache?
 
# DNS needs to be setup.
 
# Backup needs to be made to work.
 
# rhn_config  - I tried this but our subscriptions seem messed up. (Send message to customer support 11/25/09)
 
  
== Initialization ==
+
==Mail==
  
Server arrived 11/24/2009, was unpacked, placed in the rack and booted on 11/25/2009.
+
Einstein is our [[Email]] server. The mail itself is stored in a raid1 volume on [[Gourd]], and Einstein mounts the NFS share to /var/spool/mail.
  
Initial configuration steps are logged here:
+
Einstein should mount the NFS share at boot, but note that since it mounts by hostname you should make sure that DNS is accessible before starting the Einstein VM. Otherwise mail won't be mounted properly when the system boots. If for some reason DNS is unavailable you can manually mount the NFS share in /mail using gourd's IP address directly.  
* Initial host name is gourd (gourd.unh.edu) with eth0 at 10.0.0.252 and eth0.2 (VLAN) at 132.177.88.75
 
* The ARECA card is set at 10.0.0.152. The password is changed to the standard ARECA card password, user is still ADMIN.
 
* The IPMI card was configured using the SuperMicro ipmicfg utility. The net address is set to 10.0.0.151. Access is verified by IPMIView from Taro. The grub.conf and inittab lines are changed so that SOL is possible at 19200 baud.
 
* The LDAP configuration is copied from Taro. This means it is currently in '''client ldap''' mode, and needs to be change to an '''ldap server''' before production. You can log in as yourself.
 
* The autofs configuration is copied from Taro. The /net/home and /net/data directories work.
 
* The sudoers is copied from Taro, but it does not appear to work - REASON: pam.d/system-auth
 
* Added "auth        sufficient    pam_ldap.so use_first_pass" to /etc/pam.d/system-auth - now sudo works correctly.
 
  
== Disks and Raid Configuration ==
+
===Webmail===
  
'''Current Disk usage Estimates for Einstein:'''
+
Einstein provides web-based access to e-mail using [[Squirrelmail]]. You can access it at [https://einstein.unh.edu/mail].
{| style="wikitable;" border="0"
 
| Mail (/var/spool):
 
| approx. '''30GB'''
 
|-
 
| Home Folders (/home):
 
| approx. '''122GB'''
 
|-
 
| Virtual Machines (/data/VMWare on Taro): 
 
| approx. '''70GB'''
 
|-
 
| LDAP Database (/var/lib/ldap):
 
| approx. '''91MB'''
 
|}
 
<br/>
 
  
Disk sizes in the following tables are based roughly on these current usage estimates with plenty of extra space to grow. They can be adjusted as appropriate to better suit our needs, and to make these designs more cost effective.
+
See also: [[Gourd/Einstein Migration Plan]]
  
<hr/>
+
= Upgrade from RHEL5 to Centos7 =
  
=== Proposed Configuration 1 ===
+
OK, this has not been fun, and taken way too long.
  
This configuration is designed to modularize storage and keep related data on separate mirrors. Could be useful if we have a failover system, such as the old Einstein hardware, because individual components ( Mail, home folders, etc ) could be relocated physically to another machine in the event of some failure rather than copying large amounts of data over the network. This design could be modified to use fewer drives by storing Virtual machines either on the /var array or the /home array, opening up the bays to store spare drives.
+
== Status ==
  
{| style="wikitable;"  border="1"
+
We have a running Centos7 Einstein2 system.
! Drive Bay
+
* It has Named installed and running as a slave to Jalapeño.
! Raid Type
+
* Slapd is installed. Still working on it.
! Contents
 
! Volume Size
 
! Disk Size
 
|-
 
| 1
 
| rowspan="2" | Raid 1
 
| rowspan="2" | Operating System ( / )
 
| rowspan="2" | 250 GB
 
| 250 GB
 
|-
 
| 2
 
| 250 GB
 
|-
 
| 3
 
| rowspan="2" | Raid 1
 
| rowspan="2" | Mail/LDAP ( /var )
 
| rowspan="2" | 250 GB
 
| 250 GB
 
|-
 
| 4
 
| 250 GB
 
|-
 
| 5
 
| rowspan="2" | Raid 1
 
| rowspan="2" | Home Folders
 
| rowspan="2" | 500 GB
 
| 500 GB
 
|-
 
| 6
 
| 500 GB
 
|-
 
| 7
 
| rowspan="2" | Raid 1
 
| rowspan="2" | Virtual Machines <br> Data
 
| rowspan="2" | 250 GB
 
| 250 GB
 
|-
 
| 8
 
| 250 GB
 
|}
 
<br/>
 
<hr/>
 
  
=== Proposed Configuration 2 ===
+
=== LDAP ===
  
This configuration provides a larger amount of contiguous storage space than the previous design with redundancy provided by either a Raid 6 or Raid 5 array. Raid 5 would provide more usable storage, but Raid 6 will withstand more than one disk failure. It may be preferable to err on the side of caution with our user's data and use a Raid 6 for home folders.
+
Follow the installation instruction from https://www.certdepot.net/rhel7-configure-ldap-directory-service-user-connection/
  
{| style="wikitable;"  border="1"
+
Noted differences:
! Drive Bay
+
* Install extra schema:
! Raid Type
+
** core.ldif 
! Contents
+
** inetorgperson.ldif
! Volume Size
+
**  wget https://launchpadlibrarian.net/55451730/autofs.ldif
! Disk Size
+
** /usr/share/doc/samba-4.4.4/LDAP/samba.ldif
|-
+
* for each do: ldapadd -Y EXTERNAL -H ldapi:/// -D "cn=config" -f  <file>
| 1
+
* (Left out misc.schema and RADIUS-LDAPv3.schema )
| rowspan="2" | Raid 1
 
| rowspan="2" | Operating System ( / )
 
| rowspan="2" | 250 GB
 
| 250 GB
 
|-
 
| 2
 
| 250 GB
 
|-
 
| 3
 
| rowspan="2" | Raid 1
 
| rowspan="2" | Mail/LDAP ( /var )
 
| rowspan="2" | 250 GB
 
| 250 GB
 
|-
 
| 4
 
| 250 GB
 
|-
 
| 5
 
| rowspan="4" | Raid 5 '''or''' <br/> Raid 6
 
| rowspan="5" | Home Folders <br/> Virtual Machines <br/> Other data
 
| rowspan="4" | 1000 GB (Raid5) <br/> 1500GB(Raid6)
 
| 500 GB
 
|-
 
| 6
 
| 500 GB
 
|-
 
| 7
 
| 500 GB
 
|-
 
| 8
 
| 500 GB
 
|}
 
<br/>
 
<hr/>
 
  
=== Proposed Configuration 3 ===
+
==== Certificate ====
  
This configuration would create one large data store for all user and system data, which could be stored on separate appropriately sized partitions. This design is less modular, but uses fewer drives than previous designs while leaving bays open to store spares in the event of a drive failure.  
+
You need to configure the TLS certificate for the system. To create a  new one (see: [https://www.lisenet.com/2016/openldap-with-ssl-and-nfs-for-user-home-directories-on-centos-7  certs creating]) :
 +
  cd /etc/pki/tls/private
 +
  DOMAIN=einstein.unh.edu
 +
  openssl genrsa -out "$DOMAIN".key 2048 && chmod 0600 "$DOMAIN".key
 +
  openssl req -new -sha256 -key "$DOMAIN".key -out "$DOMAIN".csr
 +
  openssl x509 -req -days 3650 -sha256 -in "$DOMAIN".csr -signkey "$DOMAIN".key -out "$DOMAIN".crt
 +
  openssl pkcs8 -topk8 -inform pem -in "$DOMAIN".key -outform pem -nocrypt -out "$DOMAIN".pem
 +
  chmod 640 "$DOMAIN".key "$DOMAIN".pem
 +
  chgrp ldap "$DOMAIN".key "$DOMAIN".pem
 +
  mv "$DOMAIN".crt ../certs/
 +
  cd ../certs
 +
  ln -s ${DOMAIN}.crt `/etc/pki/tls/misc/c_hash ${DOMAIN}.crt | cut -d' ' -f1`  # Set a link to the hash of the cert.
  
{| style="wikitable;" border="1"
+
==== Install Certificate ====
! Drive Bay
+
 
! Raid Type
+
Edit an ldif file:
! Contents
+
 
! Volume Size
+
  dn: cn=config
! Disk Size
+
  changetype: modify
|-
+
  replace: olcTLSCACertificatePath
| 1
+
  olcTLSCACertificatePath: /etc/pki/tls/certs
| rowspan="2" | Raid 1
+
  -
| rowspan="2" | Operating System ( / )
+
  replace: olcTLSCACertificateFile
| rowspan="2" | 250 GB
+
  olcTLSCACertificateFile: /etc/pki/tls/certs/ca-bundle.crt
| 250 GB
+
  -
|-
+
  replace: olcTLSCertificateFile
| 2
+
  olcTLSCertificateFile: /etc/pki/tls/certs/einstein.unh.edu.crt
| 250GB
+
  -
|-
+
  replace: olcTLSCertificateKeyFile
| 3
+
  olcTLSCertificateKeyFile: /etc/pki/tls/private/einstein.unh.edu.pem
| rowspan="4" | Raid 6
+
 
| rowspan="4" | Home Folders ( /home ) <br/> Mail/LDAP ( /var ) <br/> Virtual Machines
+
To make a change to the ldap server:
| rowspan="4" | 1000 GB
+
  ldapmodify -Y EXTERNAL -H ldapi:/// -f /etc/openldap/certificate_change_localhost.ldif
| 500 GB
+
 
|-
+
Note that the hostname needs to be correct, so you probably need to set it in /etc/hosts  i.e.  127.0.0.1 einstein localhost
| 4
+
 
| 500 GB
+
Test, add -d1 -d2 or -d3 for increased debug output:
|-
+
  ldapsearch -x -ZZ
| 5
+
  ldapsearch -x -H ldaps://
| 500 GB
+
 
|-
+
Make sure that the "authconfig" line from [[Upgrading to Centos 7]] is executed and restart sssd
| 6
+
 
| 500 GB
+
Import Old Database:
|-
+
* Dump on old einstein, see /var/lib/ldap/dump.ldif  created by /usr/local/bin/ldap_dump.sh
| 7
+
* Import with  ldapadd -x -w 'password' -D cn=root,dc=physics,dc=unh,dc=edu -f  /tmp/dump.ldif
| None
+
 
| None/Spare Drive
+
Allow access over SSL:
| 0
+
* edit: /etc/sysconfig/slapd
| 250/500 
+
* change: SLAPD_URLS="ldapi:/// ldap:/// ldaps:///"
|-
+
* There is still the issue that the TLS is for "einstein" while the testing node is not called "einstein"
| 8
+
 
| None
+
Setup /etc/openssl/ldap.conf:
| None/Spare Drive
+
* echo "tls_reqcert allow" >> /etc/nslcd.conf
| 0
+
** Note that this makes it so that the TLS is not enforced.
| 250/500
+
 
|}
+
==== Allow passwd to work ====
<br/>
+
 
<hr/>
+
This was darn tricky to find out how to do. The issue is properly modifying, or adding olcAccess to the database. The following does the trick. Create a file change_password_access_policy.ldif in /etc/openldap
 +
 
 +
  dn: olcDatabase={2}hdb,cn=config
 +
  add: olcAccess
 +
  olcAccess: {0}to attrs=userPassword
 +
    by dn="cn=root,dc=physics,dc=unh,dc=edu" write
 +
    by anonymous auth
 +
    by self write
 +
    by * none
 +
  olcAccess: {1}to dn.base=""
 +
    by * read
 +
  olcAccess: {2}to *
 +
    by dn="cn=Manager,dc=tuxfixer,dc=com" write
 +
    by * read
 +
 
 +
Then enter the info into LDAP:
 +
 
 +
  ldapmodify -Y EXTERNAL -H ldapi:/// -f /etc/openldap/change_password_access_policy.ldif
 +
 
 +
The documentation for olcAccess is here [https://www.openldap.org/doc/admin24/access-control.html Access Control]
 +
It appears that the entry for olcAccess: {0} gets encrypted. Weird, since I cannot find any documentation on that.
 +
 
 +
= Other Tasks =
 +
 
 +
Move the Mail configuration. See: [[E-mail]]
 +
 
 +
Move the SSH keys from the system.
 +
 
 +
Move the sudoers file.
 +
 
 +
== TO DO ==

Latest revision as of 22:21, 3 January 2018

Einstein is currently a virtual machine running on Gourd. Go here for information about the old Einstein hardware.

Virtual Hardware

  • Memory: 2 GB
  • Hard Disk: 20 GB
  • Swap Disk: 4 GB
  • Network 1 (eth0): Farm-Bridge
  • Network 2 (eth1): UNH-Bridge
  • SCSI Controller: LSI Logic

Network Settings

  • IP Address farm (eth0): 10.0.0.248
  • IP Address UNH (eth1): 132.177.88.52

Software and Services

IPTables

Einstein uses the standard NPG iptables firewall. It allows ssh, LDAP, imap, smtp, and http connections.

LDAP

Einstein is our LDAP server. It provides user authentication among other useful services. LDAP configuration is located in /etc/openldap, and the LDAP database is stored in /var/lib/ldap.

Mail

Einstein is our Email server. The mail itself is stored in a raid1 volume on Gourd, and Einstein mounts the NFS share to /var/spool/mail.

Einstein should mount the NFS share at boot, but note that since it mounts by hostname you should make sure that DNS is accessible before starting the Einstein VM. Otherwise mail won't be mounted properly when the system boots. If for some reason DNS is unavailable you can manually mount the NFS share in /mail using gourd's IP address directly.

Webmail

Einstein provides web-based access to e-mail using Squirrelmail. You can access it at [1].

See also: Gourd/Einstein Migration Plan

Upgrade from RHEL5 to Centos7

OK, this has not been fun, and taken way too long.

Status

We have a running Centos7 Einstein2 system.

  • It has Named installed and running as a slave to Jalapeño.
  • Slapd is installed. Still working on it.

LDAP

Follow the installation instruction from https://www.certdepot.net/rhel7-configure-ldap-directory-service-user-connection/

Noted differences:

  • Install extra schema:
  • for each do: ldapadd -Y EXTERNAL -H ldapi:/// -D "cn=config" -f <file>
  • (Left out misc.schema and RADIUS-LDAPv3.schema )

Certificate

You need to configure the TLS certificate for the system. To create a new one (see: certs creating) :

 cd /etc/pki/tls/private
 DOMAIN=einstein.unh.edu
 openssl genrsa -out "$DOMAIN".key 2048 && chmod 0600 "$DOMAIN".key
 openssl req -new -sha256 -key "$DOMAIN".key -out "$DOMAIN".csr
 openssl x509 -req -days 3650 -sha256 -in "$DOMAIN".csr -signkey "$DOMAIN".key -out "$DOMAIN".crt
 openssl pkcs8 -topk8 -inform pem -in "$DOMAIN".key -outform pem -nocrypt -out "$DOMAIN".pem
 chmod 640 "$DOMAIN".key "$DOMAIN".pem
 chgrp ldap "$DOMAIN".key "$DOMAIN".pem
 mv "$DOMAIN".crt ../certs/
 cd ../certs
 ln -s ${DOMAIN}.crt `/etc/pki/tls/misc/c_hash ${DOMAIN}.crt | cut -d' ' -f1`  # Set a link to the hash of the cert.

Install Certificate

Edit an ldif file:

dn: cn=config
 changetype: modify
 replace: olcTLSCACertificatePath
 olcTLSCACertificatePath: /etc/pki/tls/certs
 -
 replace: olcTLSCACertificateFile
 olcTLSCACertificateFile: /etc/pki/tls/certs/ca-bundle.crt
 -
 replace: olcTLSCertificateFile
 olcTLSCertificateFile: /etc/pki/tls/certs/einstein.unh.edu.crt
 -
 replace: olcTLSCertificateKeyFile
 olcTLSCertificateKeyFile: /etc/pki/tls/private/einstein.unh.edu.pem

To make a change to the ldap server:

 ldapmodify -Y EXTERNAL -H ldapi:/// -f /etc/openldap/certificate_change_localhost.ldif

Note that the hostname needs to be correct, so you probably need to set it in /etc/hosts i.e. 127.0.0.1 einstein localhost

Test, add -d1 -d2 or -d3 for increased debug output:

  ldapsearch -x -ZZ 
  ldapsearch -x -H ldaps://  

Make sure that the "authconfig" line from Upgrading to Centos 7 is executed and restart sssd

Import Old Database:

  • Dump on old einstein, see /var/lib/ldap/dump.ldif created by /usr/local/bin/ldap_dump.sh
  • Import with ldapadd -x -w 'password' -D cn=root,dc=physics,dc=unh,dc=edu -f /tmp/dump.ldif

Allow access over SSL:

  • edit: /etc/sysconfig/slapd
  • change: SLAPD_URLS="ldapi:/// ldap:/// ldaps:///"
  • There is still the issue that the TLS is for "einstein" while the testing node is not called "einstein"

Setup /etc/openssl/ldap.conf:

  • echo "tls_reqcert allow" >> /etc/nslcd.conf
    • Note that this makes it so that the TLS is not enforced.

Allow passwd to work

This was darn tricky to find out how to do. The issue is properly modifying, or adding olcAccess to the database. The following does the trick. Create a file change_password_access_policy.ldif in /etc/openldap

 dn: olcDatabase={2}hdb,cn=config
 add: olcAccess
 olcAccess: {0}to attrs=userPassword
   by dn="cn=root,dc=physics,dc=unh,dc=edu" write
   by anonymous auth
   by self write
   by * none
 olcAccess: {1}to dn.base=""
   by * read
 olcAccess: {2}to *
   by dn="cn=Manager,dc=tuxfixer,dc=com" write
   by * read

Then enter the info into LDAP:

 ldapmodify -Y EXTERNAL -H ldapi:/// -f /etc/openldap/change_password_access_policy.ldif

The documentation for olcAccess is here Access Control It appears that the entry for olcAccess: {0} gets encrypted. Weird, since I cannot find any documentation on that.

Other Tasks

Move the Mail configuration. See: E-mail

Move the SSH keys from the system.

Move the sudoers file.

TO DO