System Status: Mail server SSL certificate updated; some older mail clients (e.g., Eudora) are having problems. See welltech.374 for more info.

System Status at The WELL

(This page is updated only when the status changes.)

Tue Oct 28 13:13:23 PDT 2014

All systems are up and available.

SECURITY UPDATE

Sometime on Wednesday, Oct 29, we will update the SSL certificate associated with iris.well.com (the WELL's email server, aka Zimbra, aka Iris) to one that uses the SHA-256 (aka SHA-2) hashing algorithm. The old SHA-1 algorithm has been deprecated, and many sources, including our certificate provider, are urging that the old certificates be replaced as soon as possible. (SSL certificates are used by browsers to ensure that the site you are accessing is the site that it claims to be.)

Note that this change could have incompatibility impacts on older browsers and email software. Please see welltech.374.920 for further details.

Sun Oct 19 02:41:13 PDT 2014

All systems are up and available.

At about 11:30pm on Oct 18, there was a configuration glitch that temporarily caused email to be unavailable due to apparent password problems. The glitch was fixed and service restored at about 2:30am. If you tried to change your password during this period, you may find that you now have two different passwords - one for mail, and one for everything else. We recommend updating your password once more to get them back in synch.

Fri Oct 10 10:15:24 PDT 2014

On Monday Oct 13 (Columbus Day) at 9:00 am PDT, the server at well.com (aka user.well.com) will be shut down for maintenance by Amazon Web Services. The maintenance is scheduled to last two hours, though we expect it will be much shorter than that.

During the maintenance period, the UNIX and web interfaces to our conferences, Picospan and Engaged, will not be available. The UNIX shells as well as the Alpine mail program will also be unavailable. However, email will still be available at iris.well.com. Both sending and receiving email will also continue to be available via POP and IMAP, as usual. The www.well.com website will also remain up.

We apologize for any inconvenience this maintenance presents.

Wed Sep 10 17:06:24 PDT 2014

A major update to our web conferencing software, commonly referred to as "Engaged," has been completed. To report any new bugs or to read the scope of the updates please see the System Status Reports topic in News or WellTech.

Some highlights:

  • Improved parsing and formatting of responses
  • Improved shortcut box command feedback
  • Added various shortcut box commands
  • Improved error message handling
  • Improved messaging and handling for slippage, posting without text, and passing with text in the post box
  • The various Forget buttons now require verification
  • CSS and HTML cleanup
  • Spellcheck now works properly when creating topics
  • "New Search" button in Search now works
  • The Settings page now has a Billing tab with info that points to the secure Billing Information Center
  • Blank pseuds no longer crash Engaged
  • Empty conferences no longer crash Engaged
  • Fixed issue that could result in loss of custom CSS
  • Fixed issue that could result in complete loss of all settings
  • Maximum size of a post that will be displayed has been set to 1 megabyte, or about 14000 lines. There's nothing quite like getting hit with a 1,000,000 line post while using your iPhone via cellular data.

For more, including updates to parsing and formatting, please see the System Status Reports topic in News or WellTech. Questions and bug reports are welcome.

All systems remain up and available.

Tue Apr 15 08:06:24 PDT 2014

All systems are up and available.

Tue Apr 8 13:19:51 PDT 2014

The email system received an urgent security update today. It is possible that your browser or mail program will complain about not being able to verify the identity of iris.well.com. Please see welltech.374.334- for info.

Tue Oct 1 14:54:53 PDT 2013

Planned Outage: Email Maintenance and Upgrade Rescheduled for 10/2 - 10/3

03:08 POP access is now available: This completes the Zimbra 8 upgrade: Please see the welltech conference for information about the update, and to report any issues.
03:00 IMAP and Alpine access is now available: you can send and read mail via your IMAP client or the Apline command-line interface. Things will continue to be slow for a while.
02:50 SMTP service is available: you can send mail via your IMAP or POP client, and newly arrived email is being delivered to your mailboxes. Forwarding should also be working at this time.
02:41 Web access to your email is available: you can send and read mail via your browser. Things will be slow for a couple of hours, but they should be working.
01:54 Upgrade complete, starting system
System will be started one component at a time, and may take some time to become completely available.
23:51 Zimbra 8 installed, now configuring.
21:54 Installing Zimbra 8.
21:54 Preparing system for Zimbra 8 update.
20:00 Initial setup for upgrade is underway.

There will be no outbound well.com email service -- and no access to inbound mail -- starting Wednesday 10/2 at 8pm US Pacific Time. while we upgrade the email system to the latest version, Zimbra 8. We expect to restore services at 4am Pacific Time, Thursday, 10/2.

Incoming mail will safely queue on the server until we are back. During the outage, you will not be able to send, receive, or view mail, nor can it forward.

We apologize for any inconvenience. For updates, please watch this page. For discussion, go to the System Status discussions in the News or WELLtech conferences.

Mon Sep 30 9:20:39 PDT 2013

Access has been restored. The upgrade will be rescheduled, due to circumstances beyond our control.

Log in and see the System Status topic in the News or WELLtech conferences if you want to get the details or ask questions.

Mail was queued and then delivered after the system came back up. Sorry for the inconvenience.

Sun Sep 29 22:52:39 PDT 2013

Update: Access will be restored ASAP, due to circumstances beyond our control.

This may be before or after the original target time of 4am. We do not have a new estimated time yet. Log in and see the System Status topic in the News or WELLtech conferences for additional details.

We apologize for any inconvenience.

Sun Sep 29 20:31:22 PDT 2013

Planned Outage: Email Maintenance and Upgrade 9/29 - 9/30

Initiial setup for upgrade is underway. Expect an update to this page at about 10:30pm US Pacific Time.

There will be no outbound well.com email service -- and no access to inbound mail -- starting Sunday 9/29 at 8pm US Pacific Time. while we upgrade the email system to the latest version, Zimbra 8. We expect to restore services at 4am Pacific Time, Monday, 9/30.

Incoming mail will safely queue on the server until we are back. During the outage, you will not be able to send, receive, or view mail, nor can it forward.

We apologize for any inconvenience. For updates, please watch this page. For discussion, go to the System Status discussions in the News or WELLtech conferences.

Tue Apr 9 22:14:46 PST 2013

All systems are running normally.

____

Fri Apr 5 13:09:45 PDT 2013

There will be a brief outage of well.com (the shell/picospan host), user.well.com (web conferencing/Engaged), and www.well.com (the web server) Tuesday at 10pm US Pacific Time (note that this is a rescheduled date). Each system needs to be rebooted so we can enable controls to prevent disk overruns. All systems should be back and running by 10:30pm.

____

Tue Feb 26 22:54:49 PST 2013

All systems are running normally.

____

Tue Feb 26 12:26:10 PST 2013

Mail is currently unavailable as we clean up from last week's upgrade. Mail is expected to return by midnight tonight. See you then!

____

Thu Feb 21 06:09:16 PST 2013

Last night's mail system upgrade appears to have been successful. However, it took 4 hours longer than we scheduled for due to unforeseen issues with the Amazon "Cloud". See welltech.370 for details of what's new.

____

Wed Feb 20 20:00:00 PDT 2013

Mail is currently unavailable as a software upgrade is performed. Mail is expected to be available by 2am Thursday morning. See you then!

Update as of 02:47:00: we're running way behind schedule. Currently, we're expecting to be back online at around 4:30am. We apologize for the inconvenience, and especially the longer than expected outage.

____

Mon Jan 7 14:48:22 PST 2013

Due to an NFS issue, user WEB directories and short-term backup recovery snapshots are unavilable on well.com. User web pages are still available at http://www.well.com, and files can be updated with scp or sftp at www.well.com, but files cannot be viewed or updated from the well.com host.

In addition, you may see problems when trying to login or logout from the Web-based conferencing software (Engaged). To login, go to http://www.well.com. The error on logout can be ignored.

____

Mon Dec 3 16:32:55 PST 2012

A system reboot is scheduled for Tuesday, Dec 4, 11pm US Pacific Time. The reboot will last about 5 minutes, and will effect shell and conferencing access (both via SSH and the Web).

____

Wed Nov 21 22:31:45 PST 2012

The restart of our SMTP server is taking longer than anticipated. We have an open ticket with Amazon Web Services to investigate why the system is not shutting down properly. We don't yet have an ETA.

____

Wed Nov 21 17:10:13 PST 2012

Amazon has alerted us that our SMTP server is running on "degraded hardware". To get things back to normal, we will be rebooting the server later tonight (around 10pm). If all goes well, it should only take about 5 minutes.

Alpine users are most likely to see any effects - while the server is rebooting, Alpine will complain if you try to send email. Just resend the mail a few minutes later if this happens to you. Other customers may seem some minor delays on mail being received or sent, but in almost all cases, the delayed mail should be delivered properly some time after the system comes back up.

Sorry for any inconvenience this reboot will cause...

____

Tue Oct 2 22:03:27 PDT 2012

All systems are up and running.

However, we have changed our DNS authoritative server, and some DNS servers are being slow to pick up this change. If you are having trouble connecting to any WELL machines, you may want to change your DNS server on your router, Mac, or PC to use 8.8.8.8 as the primary server. However, be careful to keep a record of the previous settings, since you will likely want to change these settings back at some point. Your best bet is to wait until your normal DNS server picks up the updates, which should happen within a few hours in most cases.

NOTE: It looks like there have been a few cases of mail bouncing from about 7:40pm-8:50pm, and again from about 10:05pm-10:20pm US Pacific Time. Please ask senders to resend if you are missing mail from this period.

We apologize for any inconvenience resulting from this necessary update.

____

Fri Jun 15 12:18:22 PDT 2012

Amazon EC2 services, the cloud where our servers reside, has called for us to shut down and relaunch well.com and user.well.com, the servers where WELL conferencing lives. Tomorrow, Saturday June 16, those services will be unavailable from approximately 9pm until 11:00pm Pacific Daylight Time while wolfy migrates services to new instances. (Last night there was significant disruption that was probably unrelated to this next action, but which also came from problems at the Amazon cloud. Sorry for the inconvenience.) Thank you for your patience and loyalty.

____

Fri Jun 15 09:38:22 PDT 2012

All systems are up and running.

____

Fri Jun 15 01:22:04 PDT 2012

Email is down due to problems in the Amazon Cloud. We do not have an ETA at this time, but email may not be available until late morning or later.

____



Wed Mar 14 15:04:28 PDT 2012

We will be rebooting well.com and user.well.com Wednesday evening (Mar 14) at about 11pm US Pacific Time to make a minor configuration change. The reboot and maintenance will take about 20 minutes, and will effect access to the conferences via both Picospan and the Web, as well as all shell services (including alpine mail). We apologize for the inconvenience.

____


Thu Mar 1 22:26:11 PST 2012

(ETA changed to midnight - we're running a bit behind schedule)

On Thursday evening between about 9pm and midnight US Pacific Time, WELL conferencing via UNIX shell and the web will be down, with all access to well.com and user.well.com disabled for the duration. During this maintenance, we will be switching over to a new system kernel - we expect that this will put an end to the kernel panics we have seen over the last two months.

Access to your email via https://iris.well.com/, POP, and IMAP will not be affected by this maintenance. However, please note that access to email via the "alpine" mail program will not be available since well.com itself will be unavailable.

We apologize for the inconvenience of this maintenance, but random system crashes are even more inconvenient.

____


Thu Feb 23 21:15:19 PST 2012

ENGAGED IS DOWN - the database was accidentally trashed. We hope to have it back up by 10:30pm US Pacific Time.

____


Wed Jan 11 17:00:35 PST 2012

We did not reboot Tuesday, so we will be rebooting well.com tonight, Jan 11, at about 10pm US Pacific Time. The reboot will take 5 to 10 minutes, and will effect all uses of the conferences.

____


Wed Jan 11 09:37:59 PST 2012

All systems are up and running.

____


Tue Jan 10 12:19:25 PST 2012

We will be rebooting well.com tonight at about 10pm US Pacific Time in an effort to address the repeated system panics. The reboot will take 5 to 10 minutes, and will effect all uses of the conferences.

____


Fri Dec 16 15:43:25 PST 2011

After a reboot, people are back posting. All systems are up and running.

____


Fri Dec 16 14:55:25 PST 2011

We are having a problem with the well.com and user.well.com machines. While we resolve this problem, the conferences -- our forums -- are unreachable. Email at iris.well.com and the www.well.com web page remain up. We are sorry for the inconvenience.

____


Fri Dec 9 10:53:25 PST 2011

All systems are up and running.

____


Thu Dec 8 2011 16:57:24 PST 2011

Our service provider, Amazon, needs to reboot our systems to perform some sort of system update (presumably patches). Fortunately, we can prevent the automatic reboot by rebooting manually. The best time for us to do this is tonight, even though that means a short notice outage.

So, tonight at about 10pm US Pacific Time, each of the various well.com systems will be brought down then rebooted. Each reboot should take less than 5 minutes, though the reboot of the Zimbra machine may take a few minutes longer (it takes several minutes just for Zimbra to shut down, and another few minutes to start up).

We apologize for the short notice for this announcement.

____


Thu Oct 27 14:10:50 PDT 2011

Most remaining bugs appear to have been eradicated. If you encounter any, please report them in the cloud discussion in the welltech conference. Log in, shortcut is "g welltech" to get there.

If you use the PicoSpan interface and you're getting error messages on login, the problem may be configuration in your personal log in files. If you send a message to helpdesk, we can back up and replace your current log in files with basic files so that you can test each line to find the culprit.

____


Wed Oct 26 07:45:03 PDT 2011

The WELL is now back up and running in the Amazon Elastic Computing Cloud (EC2). Please report any bugs by going to the cloud discussion in the welltech conference. Log in, shortcut is "g welltech" to get there. If there is a malformed conference mentioned in an error message on your your Conference List page when you first connect to open a page at "https://user.well.com/engaged.cgi?" you may want to remove the bad conference temporarily by editing your Settings. Reporting a bug could win you one of a bunch of interesting books, so don't hesitate.

____


Mon Oct 24 16:42:39 PDT 2011

Migration down-time planned

On Tuesday night, Oct 25, we finish up our migration into the Amazon Elastic Computing Cloud (EC2). The last machines to be moved will be the main machine (well.com), the conferencing web server (user.well.com), and the web page server (www.well.com).

As part of this migration, conferencing (via both the command line and web browser), as well as all access to The WELL's web pages, will be disabled. Email will remain available, except for the "alpine" mailer. We're anticipating that the migration should be complete in about 6 hours: from 9pm PDT Tuesday evening to 3am PDT Wednesday morning.

Following the migration, you should be able to use conferencing and all web services as before, with only minor changes that most people will not see. If by some chance you do experience problems after the system is restored to service, don't try to change your settings please. If we are still sorting out the move that could make things harder to diagnose. Please just contact helpdesk with any bug reports or post in the System Status topic in the WELL tech or News conference, so a proper diagnosis can be made.

We apologize for any inconvenience this downtime may cause.

____


Tues Aug 30 13:07:22 PDT 2011

Performance of the email system remains sluggish - we are working to restore good service.

We apologize for any inconvenience this slowness may cause.

____


Fri Aug 26 13:43:56 PDT 2011

On Monday night, Aug 29, we begin our migration into the Amazon Elastic Computing Cloud (EC2). The first machines to be moved will be the Zimbra mail server (iris.well.com) and a companion server.

As part of this migration, email will be disabled while the migration is under way. We're anticipating that the migration should be complete in about 6 hours: from 9pm PDT Monday evening to 3am PDT Tuesday morning.

During the migration, incoming email from other sites should be queued at those sites, and re-sent later after we begin accepting incoming email again. WELL customers will not be able to look at previously received email during the outage unless you have already moved that email onto your computer's storage device.

Following the migration, you should be able to continue using email using the settings that have worked for the last few weeks - nothing should change. If you encounter problems after the migration, please avoid changing your mail settings and report the problems to Helpdesk or post in System Status Reports.

We apologize for any inconvenience this downtime may cause.

____


Thu Aug 18 21:37:03 PDT 2011

The WELL is now back up and running. Engaged is still presenting some problems, but for now, you can make your own repairs using the instructions on your Conference List page.

____


Thu Aug 18 14:54:03 PDT 2011

RAID failure knocked out some services, currently being restored

Around 1:00 am on the morning of Thursday, August 18th, home directories became unreachable at The WELL, causing all services except email to fail.

At 9:30 the following morning we we were able to place the following unadorned message on the well.com site:

Most functions of The WELL are currently not available due to a hardware failure of the RAID where most of our data is kept. We are working on restoring full service, but do not yet have an estimated completion time. We apologize for this inconvenience.

Please note that email via IMAP, POP, and the Web should be available at this time.

During that time we went through some replacement hardware options, settled on one, and got it installed at our co-location facility. At about 14:45 PDT logins became possible again.

RAIDs are designed to prevent such problems when a disk fails, and we are in the process of moving well.com to the cloud, so this failure was perplexing and hopefully very soon all questions will become moot. Some services are still being repaired and adjusted at this time. Comments are welcome, please email Helpdesk at the address on the footer of this page, or swing by the welltech conference.

We are sorry for the disruption.

____


Mon Aug 1 16:42:55 PDT 2011

Mail upgrade complete: If you can't get mail you need to change settings

All systems are up and running. We now have IMAP available for all mail customers. Please be advised that we have shut off all access to two older mail servers as of this afternoon as part of the upgrade to email started in 2009. If you are pointing to those servers in your settings, your mail will no longer work.

If you are not getting or sending email, you probably need to update some settings. You no longer have access to smtp.well.com nor to mh.well.com (mail.well.com). All incoming and outgoing mail operations will now involve the iris.well.com server. To get the appropriate Ports, look up your appropriate settings.

1. You should have access to your new email via a direct web browser login at iris.well.com now and henceforth. This is the easy, quick way for most people to see mail.

2. New mail help for all kinds of programs is now up at http://www.well.com/emailhelp.html with an overview of the system, general settings and a link to a page with more settings for specific mobile devices and computer email programs.

3. You can move your old email into the new environment by use of the utilities page if you wish.

4. There is a way to use Alpine for UNIX access to your mail. See the help page first.

5. We attempted to migrate you with settings matching your former preferences. In some situations we were not able to do that! In particular, if you were forwarding elsewhere and leaving a copy you are probably set up with straight forwarding now, and you will want to adjust that with a web login to iris.well.com.

If you have questions, contact helpdesk or ask fellow users who have solved the problem in the welltech conference. Thank you for your patience.

____


Mon Aug 1 09:59:46 PDT 2011

Mail upgrade is in progress.

All remaining mailboxes at The WELL are being transferred to the Zimbra email system at iris.well.com over the next several hours today. Please see the mail help page for instructions on setting up once your mailbox is converted, or use your web browser to go to iris.well.com. (Note that you cannot log in there until the upgrade script converts your mail settings.) Thank you for your patience.

____

Wed Jun 22 12:55:09 PDT 2011

All systems are up and running.

____

Monday, June 20, 2011 05:36:27 PM PDT

The Zimbra email system on iris.well.com will be unavailable Tuesday evening (Jun 21) between the hours of 10pm and midnight so that we can perform a required software update. The legacy email system will remain up during this period. Sorry for the inconvenience.

____


Fri May 13 13:50:40 PDT 2011

Members using the Legacy email server are finding their connections time out when they are trying to access mail from mail.well.com. We are working on the situation.

____


Mon Dec 13 2010

The big Gawker break-in yielded lots of users data including some WELL email addresses, paired with passwords from Gawker and related sites.

This past weekend, the website gawker.com was the victim of a systems break-in. Over 1 million Gawker, Gizmodo, LifeHacker and related sites commenting accounts may have been compromised, with logins, email addresses and passwords grabbed. About 40 well.com email address appeared on the public list of accounts that were compromised.

Since many people use the same password on multiple sites, we have taken the precaution of temporarily disabling those WELL passwords, to prevent malicious access to those accounts here.

If you were locked out for this or other reasons, please generate a new WELL Password. If you were not locked out, but you used an off-WELL email address at Gawker and related sites, and you use the same password at Gawker sites and here at The WELL, please change your password.

You may see some phony emails about this, if your email address was made public for all to use. Sadly, it would be trivial for somebody to make such a message look fairly official in a phishing attempt to get you to go to another site that is unsafe. For that reason we are NOT including a clickable link to change a password in that informational email.

Since you are reading this announcement in the form of a post on our site, and you navigated here on your own, it's safe to put some links up here. (You may also search for the news yourself instead):

If you need help getting back in, or if you want to report a phishing attempt with a fake link, you may call us at the number on the footer section of this web page, between 9am and 4:50pm Pacific Time (Monday - Friday).

We apologize for any inconvenience this action may cause, particularly if you have been careful to avoid using your WELL password on other sites. However, we must take overall security seriously, for everyone's safety and privacy.

Take-aways:

  • Do NOT use the same password on multiple sites. (Or at least don't use your WELL password for more than one site.)
  • Do be sure your off-site email contact info is entered so that the WELL staff may contact you in rare situations like this one where we might suspend your access to your WELL email.

____


Mon Nov 22 13:18:20 PST 2010

We are experiencing a server error for user.well.com and for PicoSpan via ssh connection, each of which serves our conferences. We are working on the situation. Sorry for the inconvenience.

____


Mon Nov 22 13:46:20 PST 2010

Conferencing is back.

____


Mon Nov 22 13:18:20 PST 2010

We are experiencing a server error for user.well.com and for PicoSpan via ssh connection, each of which serves our conferences. We are working on the situation. Sorry for the inconvenience.

____


Wed Nov 3 09:38:20 PDT 2010

And...Mail is back. Sorry for the inconvenience.

____


Fri Wed Nov 3 09:35:10 PDT 2010

Email is temporarily down. We expect it to be restored soon. No mail will be lost.

____


Wed Nov 3 09:35:10 PDT 2010

We're having a problem with our registration pages. If you're trying to sign up for membership on The WELL and you're receiving an error message, please call the number at the bottom of this page or email helpdesk, also listed at the bottom of the page, to receive a notice when sign-ups are being received again. We'll also extend some bonus time on your new account. Sorry for the inconvenience! [Note: offer was valid through 10/5/10]

____


Wed Sep 15 14:39:35 PDT 2010

On Tuesday, September 7, we shut down one more insecure method of accessing The WELL when we disabled FTP on well.com and www.well.com. You now must use SFTP (Secure FTP) to manage the files you upload to your home and Web directories on The WELL.

SFTP is nearly identical to FTP, with the exception that SFTP securely encrypts the connection. The commands are mostly the same, and, if you have a GUI client, you can probably continue to use that client - most GUI FTP clients let you select SFTP or FTP in the settings area.

If you use a command line FTP on your system (such as on a Mac), there's a good chance that there is also a command line SFTP installed. Using SFTP in such cases is usually as simple as just replacing the "ftp well.com" command with "sftp MYLOGIN@well.com". Once you're connected, the commands will likely be the same. Please consult the System Status topic in the News Conference for more information.

____


Mon Aug 23 17:04:01 PDT 2010

The Zimbra email system, iris.well.com, will be down Tuesday Aug 24 from 9pm for a software update. We should be back online no later than 11pm, and likely will be back earlier than that.

During the downtime, iris.well.com will not be available for either sending or receiving of email. Incoming mail will be queued for delivery after the system is restored to service. We are sorry for any inconvenience while we improve our mail service.

(This downtime will not affect users of the legacy email systems (well.com, mh.well.com, smtp.well.com, and webmail.well.com), since they are maintained separately, nor will it affect any other WELL services.)

____


Mon Apr 19 17:35:02 PDT 2010

well.com (picospan, Pine, Unix, etc) is currently unavailable due to a typo. We're working on restoring service as soon as possible. In the meantime, conferencing is available through Engaged, and legacy mail can be accessed through Web Mail. We apologize for the inconvenience.

____


Mon Mar 29 14:06 PDT 2010

The Picospan (well.com) and Engaged (user.well.com) systems will be rebooted Wednesday evening, between about 8pm and 9pm, to bring the system up-to-date with patches. Each system will be down for 15-30 minutes, with each system offline separately. We apologize for any inconvenience caused by these brief outages.

Update as of 21:12 PDT The first reboot took longer than originally scheduled. We are now rebooting Engaged, and should be back by 22:00 PDT

____


Weds Mar 3 2:36 PST 2010

Email has been restored, and email from earlier today is now being delivered. We "downgraded" to an earlier version of VMware, which finally resolved the issues seen through the day.

All systems appear to be fine again. You may see duplicate mails as a rare side-effect due to all the up and down of the servers. We are sorry for the inconvenience.

____


Weds Mar 3 2:36 PST 2010

Email problems continue. You may have had some mail delivered briefly to the legacy email system, and for a longer period to the new iris/Zimbra system, but that has been interrupted again. We are having VMware problems, and we have an expert on the case right now trying the next approach.

Over the morning hours we have done a series of fixes that looked stable but have failed, and we are optimistically taking the next step. More information as soon as it is available. We are sorry for the inconvenience.

____


Weds Mar 3 1:02 PST 2010

We are seeing additional problems, and are on the case.

Looks like we spoke too soon. More information as soon as it is available.

____


Weds Mar 3 12:56 PST 2010

All email systems are working properly now. There will be a little delay as all the queued email is delivered.

Thank you for your patience with this unexpected interruption.

____


Weds Mar 3 12:53 PST 2010

We are hoping to wrap it up within the hour. If you are on the new iris system (powered by Zimbra software) we are now pushing that mail though successfully. If you are on the legacy email system, please hold on while we get you going again.

See the welltech conference for full discussion when we get all the details. We are sorry for the inconvenience.

____


Weds Mar 3 11:35 PST 2010

We are now experiencing some difficulties with both the mh and iris email systems. An initial software reboot to mh was not sufficient, and we began working with the hardware. Now all of our email systems are involved.

More information will be available soon. See the welltech conference for discussion. We are sorry for the inconvenience.

____


Tue Jan 5 16:29:01 PST 2010

The new Zimbra email system (iris.well.com) is back! You may now use the new system for new mail again, via the Web or with your preferred email application.

We are now recovering from the hardware and related problems. The queue of email messages from Monday and early Tuesday is flowing in too, in addition to new mail. You may notice duplicates generated from our emergency actions, if you read some of that mail as suggested below. More information will be available soon. See the welltech conference for discussion.

____


Tues Jan 5 6:24:58 PST 2010

We have not restored access to iris.well.com yet. Work continues. Our efforts at disk repair on Monday were to no avail, and another server was pulled from service, wiped and prepared for restoration of backups. This machine will become the new iris.well.com server soon. Unfortunately, some difficulties cropped up in the backup restoration process. The good news is that we have since gotten what we feel is a credible diagnosis and theory of what that problem is, and we are working on it.

Late yesterday we extended the previously announced forward/copy scheme. This means that you may not only see and respond to copies of mail received since 9:00 am PST at the Squirrelmail web page, but you will also see older mail there, from the time of the hardware crash Sunday morning. There may be some duplicates. We will temporarily keep delivering copies of new email sent to all Zimbra-enabled accounts in the older email systems, until full recovery.

We are simply appalled at this series of machine failures, holiday timing and bad coincidences. As soon as the backed-up email is back, we will work on strategies to make a scenario like this one impossible. We are very sorry for the inconvenience.

____


Mon Jan 4 10:12:58 PST 2010

The partial email outage continues. This is a hardware outage on a new machine, It only impacts Zimbra users, on the new mail system only. Many WELL members have not upgraded to the modern server yet, and will see no effects at all, so if your mail is working, disregard this message.

This morning at 9:00 am Pacific Time we put into effect a forward/copy scheme that will temporarily keep copies of new email sent to all Zimbra-enabled accounts in BOTH email systems. This means if you are a new mail user you may pick up copies of mail received since 9:00 am by fetching them to the older Squirrelmail web page. (Log in at webmail.well.com)

___

Sun Jan 3 12:37:20 PST 2010

The new Zimbra email system (iris.well.com) is currently down for what appears to be a hardware failure. We are working on restoring service as soon as possible, but at this time, it looks like service will not be restored until sometime on Monday. We apologize for the inconvenience caused by this outage.

___

Wed Dec 2 21:49:27 PST 2009

The scheduled maintenance is complete. All systems appear to be up and running normally.

___

Mon Nov 30 13:16:08 PST 2009

We will be applying a variety of security updates and other system patches to all well.com systems on Wednesday evening (Dec 2) between the hours of 7pm and midnight. As each system is updated, the updated system will need to be rebooted. As a result, short outages of between 5 and 15 minutes (and possibly as much as 2 hours in the case of email) can be expected on all systems at different times.

While mail on both the old and new systems will be unavailable during the outages for the respective machines, mail should be delivered as usual when the system is restored to operation.

We apologize for any inconvenience caused by this necessary maintenance.

___

Fri Oct 2 12:08:50 PDT 2009

A little after 11:15am PDT we had problems with our DNS that made the WELL disappear for approximately 16 minutes for anybody who was not logged in at the time. All of our machines were running smoothly, but due to the DNS problem, our services were unavailable to new external connections during that period. Most email sent to addresses in the well.com domain will be re-sent. However, retrying depends on how other services configure their email servers. There is a chance that some email could have bounced back to the sender saying it was a "fatal error".

We are sorry for the inconvenience.

___

Thu Sep 10 09:07:33 PDT 2009

Mail delivery has resumed. It will take some time - several hours, at least - for the backlog of mail to be delivered.

___

Thurs Sep 10 8:16 PDT 2009

The WELL's mail servers were updated last night and rebooted. However, this morning reports from dozens of members confirmed that a further fix is needed. We are working on it right now, and delivery of mail will resume as soon as the fix is completed. We are sorry for the inconvenience.

___

Tue Sep 8 15:48 PDT 2009

The WELL's mail servers will be down for a system software update between 7:30 and 8:30pm Pacific time on Wednesday September 9th. No email will be delivered or sent during this time. The WELL will queue incoming mail, and it will be delivered when the update is completed.

___

Fri Aug 7 09:20 PDT 2009

PicoSpan is unreachable at this time. We're working on the problem. Access to the conferences is available via the Web.

___

Fri Jun 5 13:56:09 PDT 2009

The Web interface to WELL conferences (user.well.com) will be taken out of service Monday June 8th from 8pm till 1am US Pacific Time so we can move the database from one host machine to another. This should prevent certain slowdowns. Complete Plan members may use ssh for PicoSpan (command line) access to conferences.

During the downtime, we will also reboot the mail server (mh.well.com). Mail service should be back up again in fewer than 30 minutes. While the mail server is down, you won't be able to get new mail. Incoming mail will be delivered after the mail server is back online.

We apologize for any inconvenience these preventative moves may cause.

___

Tue Apr 7 14:10:58 PDT 2009

There will be a reboot of the mail server (mh.well.com), Thursday night at 10:00 PM Pacific Time. The system should be back up again in less than 30 minutes. During the downtime, you won't be able to get new mail. Incoming mail will queue up during that time for delivery after the mail server is back online. We apologize for the inconvenience.

Tue Mar 17 16:12:01 PDT 2009

There will be a reboot of the mail server (mh.well.com), Thursday night at 9 Pacific Time. The system should be back up again in less than 30 minutes. During the downtime, you won't be able to get new mail. Incoming mail will queue up during that time for delivery after the mail server is back online.

___

Wed Jan 14 01:39:03 PST 2009

The mail server (mh.well.com) will be taken out of service briefly Sunday at 9pm US Pacific Time so we can add additional memory. This should help prevent some of the mail delays we've been seeing the past 2 weeks. The total downtime should be no more than 30 minutes. Incoming mail will be queued locally, and will be delivered after the reboot. We apologize for the inconvenience.

___

Wed Jan 14 01:39:03 PST 2009

There will be brief outage of well.com Wednesday night at 11pm Pacific Time to correct a system configuration problem. The total downtime should be no more than a few minutes.


For Further Information...

For information about The WELL or for WELL Customer Support use the Helpdesk contact information on the footer of this page.

* The WELL Utilities Page

* SPAM Conference

 
   Join Us
 
Home | Learn About | Conferences | Member Pages | Mail | Store | Services & Help | Password | Join Us

Twitter G+ Facebook