Difference between revisions of "Einstein Status"

From Nuclear Physics Group Documentation Pages
Jump to navigationJump to search
 
(41 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
== <font color="red"> HOT current issues </font> ==
 +
 +
* There is an issue with the SMTP connection for outgoing mail from web-mail aka [[Squirrelmail]].
 +
* Some setup documentation is needed. Start a new page?
 +
 +
== System Check List ==
 +
 
[http://www.redhat.com/docs/manuals/enterprise/RHEL-5-manual/Deployment_Guide-en-US/index.html Massive amount of deployment documentation for RHEL 5]
 
[http://www.redhat.com/docs/manuals/enterprise/RHEL-5-manual/Deployment_Guide-en-US/index.html Massive amount of deployment documentation for RHEL 5]
 
'''Remade the RAID5. Will redo the software, but if the RAID dies again we should think of a different machine to sub for einstein.'''
 
  
 
# Check all services einstein currently provides.  Locate as many custom scripts, etc. as is reasonable and label/copy them.
 
# Check all services einstein currently provides.  Locate as many custom scripts, etc. as is reasonable and label/copy them.
## Network interfaces. eth1 is up as the UNH interface, while eth2 doesn't seem to be working as the farm interface. Pings/traceroutes go out but responses don't come back. I've tried tinkering with iptables but that doesn't seem to be the cause. Configs are as described in [[Servers and Workstations]]. 2/14/2008: I think the best bet for now would be to just use farm over the interface that works, and set up VLAN for unh. I'll switch the cables around and see how it goes. '''After setting it up so that the only ethernet connection is via the switch, with eth0 as the farm and eth0.2 as unh, I'm getting the same symptoms. Even though the unh VLAN has to go through the same physical devices as the farm, it's able to work properly, while the farm isn't. What am I missing?'''
+
## Network interfaces - UNH: ???? Farm: Done!
## [[Iptables]]
+
## [[Iptables]] - Not working. Possibly because of Gourd's weirdness.  -- Done 7/7/2008 = Needed the iptables.config setup to use iptables_npg, and needed iptables_npg started at boot
## [[DNS]]
+
## [[DNS]] - Done!
## [[LDAP]]  
+
## [[LDAP]] - Done!
## [[Postfix]]  
+
## [[Postfix]] - Done! - including TLS!
 
## [[AMaViS]]  
 
## [[AMaViS]]  
 
## [[ClamAV]]
 
## [[ClamAV]]
## [[SpamAssassin]]  
+
## [[SpamAssassin]] - Done!
## <del>[[Cyrus Imap|IMAP]] <code>cyradm localhost</code> gives "cannot connect to server". This all seems to be sasl-related. It'd be probably be easy if there was a way to have cyrus use PAM. [http://www.openldap.org/doc/admin23/sasl.html LDAP and sasl] <ins>Nevermind, that has to do with using SASL to authenticate LDAP</ins><code>saslauthd -v</code> lists pam and ldap as available authentication mechanisms, and /etc/sysconfig/saslauthd has an entry "MECH=pam"&hellip;! What am I missing? '''Tried making a new "mail.physics.unh.edu.crt" for tomato, but couldn't because that would have required revoking einstein's cert of the same name. Tried using the "tomato.unh.edu.crt" and "tomato.unh.edu.key", but is giving the same results as the "mail.physics.unh.edu.*" copied from einstein.''' Tried using tomato's UNH address instead of hostname: same result. '''I'm able to login using the <code>imtest</code> program, but the server doesn't send the same messages as shown [http://cyrusimap.web.cmu.edu/twiki/bin/view/Cyrus/ImtestByHand here].'''</del> Let's try Dovecot instead. It supposed to be simpler to maintain and is supported by RedHat.
+
## [[Dovecot]] - Functioning, have to migrate cyrus e-mails from old Einstein. == Some problems here. The "seen" was not ported properly, but that is OK.
## [[automount|/home]]
+
## [[Squirrelmail]] -- Needed some reconfiguration.
 +
## [[automount|/home]] - Done!
 
## [[Samba]]  If anyone needs samba access, they need to find us and have us make them a samba account. No LDAP integration.
 
## [[Samba]]  If anyone needs samba access, they need to find us and have us make them a samba account. No LDAP integration.
## [[Web Servers|Web]]?
+
## [[Web Servers|Web]]? - This will be handled by roentgen, except squirrelmail, which requires a basic apache setup.
## Fortran compilers and things like that? (Also needs compat libs--'''Nope, tomato is 32-bit.''')
+
## Fortran compilers and things like that? (Also needs compat libs) - Isn't this what pumpkin is for?
# Switch einstein <-> tomato, and then upgrade what was originally einstein
+
# Switch einstein. - Done?
# Look into making an einstein, tomato failsafe setup.
+
 
 +
 
 +
==Current setup on einstein2==
 +
/ is /dev/md0, which is a 3-way mirror comprised of sda1, sdb1, and sdc1. /var/spool/imap (can be changed to match our dovecot configuration) is /dev/md1, which is a 3-way mirror comprised of sda2, sdb2, and sdc2. /home is a 2-way mirror of sdb3 and sdc3. sda3 is the swap partition.
 +
The reason it is set up this way is that the system came installed on a 250gb, and Matt wanted redundancy and space for mail and /home, since they're (some of) the most important things. Two 750gb's were added, and RHEL5 was reinstalled without a hitch. Grub should currently be installed on all three drives, so that if any one (or two!) drives fails, the system can still boot and run. The RAID setup is standard software raid1 using 3 elements for root/mail and 2 elements for home. This will allow us to put the drives in any other system if need be.
 +
 
 +
This setup apparently confused Maurik, who wondered why we need to use the original 250gb drive involved. The answer, in case it ever needs to be known in the future, is that Matt didn't want to waste a perfectly good drive with no other purpose. It's perfectly reasonable to change the drive to 750 and complete the 3-way raiding fun, if desired.
 +
 
 +
==Mail server==
 +
[http://wiki.dovecot.org/Migration/Cyrus This should come in handy moving mail to the new einstein] ("cyrus2courier" seems like the best and not any harder to use than the other two options. No, it isn't. It doesn't work for 2.2+)
 +
 
 +
Dovecot, postfix, squirrelmail, and mailman have all been installed. The logical plan of attack seems to be to get postfix working fully, then dovecot, then squirrel, and finally mailman, in order to satisfy dependencies as well as order of importance.
 +
Because single large files scare me, I'm initially trying to set up postfix with maildir format. Default is mbox, but since mbox stores each user's mail as a single big file, it just looks like it's too easy to lose lots of data from random disk errors. Mbox worked, but once I copied over einstein's configs, things stopped working due to hostname/resolution errors, as can be seen in <code>/var/log/maillog</code>. '''maildir is far more robust anyhow; the Dovecot documentation even recommends it, so I don't know why mbox is the default.'''
 +
 
 +
[http://dovecot.markmail.org/search/?q=cyrus#query:cyrus%20list%3Aorg.dovecot.dovecot+page:1+mid:zyq2ov3pm246py2u+state:results A mailing list message where someone claims to have updated cyrus2courier to work with modern versions of cyrus.] In the message, He points [http://dovecot.org/tools/ here] to download it. The script convert2dovecot.sh doesn't work, but the cyrus2courier program does, and seems to work fine, though it marks all messages as unread.
 +
 
 +
I'm writing a script to intelligently go through all users, move the mail, and make sure the permissions are correct. The current version just checks the /home directory for a list of users, and ignores a certain list of non-users (like domain_admin, npg, farm, lost+found, etc). I realized there's some users buried in the cyrus data, but the only users worth worrying about are in the home directory, which implies an active account.
 +
 
 +
So apparently we have a very nonstandard cyrus setup. Typically, in the users mailbox, there is a file called 'cyrus.seen' that indicates the read/unread status (possibly more) of every mail. On our setup, I've discovered that mine is in /var/lib/imap/user/m/minuti.seen, and others follow the same pattern. Simply moving $USER.seen to cyrus.seen in the user's mailbox appears to allow cyrus2courier to correctly mark and move all mail. I'll update the script to incorporate this discovery.

Latest revision as of 18:04, 3 August 2017

HOT current issues

  • There is an issue with the SMTP connection for outgoing mail from web-mail aka Squirrelmail.
  • Some setup documentation is needed. Start a new page?

System Check List

Massive amount of deployment documentation for RHEL 5

  1. Check all services einstein currently provides. Locate as many custom scripts, etc. as is reasonable and label/copy them.
    1. Network interfaces - UNH: ???? Farm: Done!
    2. Iptables - Not working. Possibly because of Gourd's weirdness. -- Done 7/7/2008 = Needed the iptables.config setup to use iptables_npg, and needed iptables_npg started at boot
    3. DNS - Done!
    4. LDAP - Done!
    5. Postfix - Done! - including TLS!
    6. AMaViS
    7. ClamAV
    8. SpamAssassin - Done!
    9. Dovecot - Functioning, have to migrate cyrus e-mails from old Einstein. == Some problems here. The "seen" was not ported properly, but that is OK.
    10. Squirrelmail -- Needed some reconfiguration.
    11. /home - Done!
    12. Samba If anyone needs samba access, they need to find us and have us make them a samba account. No LDAP integration.
    13. Web? - This will be handled by roentgen, except squirrelmail, which requires a basic apache setup.
    14. Fortran compilers and things like that? (Also needs compat libs) - Isn't this what pumpkin is for?
  2. Switch einstein. - Done?


Current setup on einstein2

/ is /dev/md0, which is a 3-way mirror comprised of sda1, sdb1, and sdc1. /var/spool/imap (can be changed to match our dovecot configuration) is /dev/md1, which is a 3-way mirror comprised of sda2, sdb2, and sdc2. /home is a 2-way mirror of sdb3 and sdc3. sda3 is the swap partition. The reason it is set up this way is that the system came installed on a 250gb, and Matt wanted redundancy and space for mail and /home, since they're (some of) the most important things. Two 750gb's were added, and RHEL5 was reinstalled without a hitch. Grub should currently be installed on all three drives, so that if any one (or two!) drives fails, the system can still boot and run. The RAID setup is standard software raid1 using 3 elements for root/mail and 2 elements for home. This will allow us to put the drives in any other system if need be.

This setup apparently confused Maurik, who wondered why we need to use the original 250gb drive involved. The answer, in case it ever needs to be known in the future, is that Matt didn't want to waste a perfectly good drive with no other purpose. It's perfectly reasonable to change the drive to 750 and complete the 3-way raiding fun, if desired.

Mail server

This should come in handy moving mail to the new einstein ("cyrus2courier" seems like the best and not any harder to use than the other two options. No, it isn't. It doesn't work for 2.2+)

Dovecot, postfix, squirrelmail, and mailman have all been installed. The logical plan of attack seems to be to get postfix working fully, then dovecot, then squirrel, and finally mailman, in order to satisfy dependencies as well as order of importance. Because single large files scare me, I'm initially trying to set up postfix with maildir format. Default is mbox, but since mbox stores each user's mail as a single big file, it just looks like it's too easy to lose lots of data from random disk errors. Mbox worked, but once I copied over einstein's configs, things stopped working due to hostname/resolution errors, as can be seen in /var/log/maillog. maildir is far more robust anyhow; the Dovecot documentation even recommends it, so I don't know why mbox is the default.

A mailing list message where someone claims to have updated cyrus2courier to work with modern versions of cyrus. In the message, He points here to download it. The script convert2dovecot.sh doesn't work, but the cyrus2courier program does, and seems to work fine, though it marks all messages as unread.

I'm writing a script to intelligently go through all users, move the mail, and make sure the permissions are correct. The current version just checks the /home directory for a list of users, and ignores a certain list of non-users (like domain_admin, npg, farm, lost+found, etc). I realized there's some users buried in the cyrus data, but the only users worth worrying about are in the home directory, which implies an active account.

So apparently we have a very nonstandard cyrus setup. Typically, in the users mailbox, there is a file called 'cyrus.seen' that indicates the read/unread status (possibly more) of every mail. On our setup, I've discovered that mine is in /var/lib/imap/user/m/minuti.seen, and others follow the same pattern. Simply moving $USER.seen to cyrus.seen in the user's mailbox appears to allow cyrus2courier to correctly mark and move all mail. I'll update the script to incorporate this discovery.