Hi, we are about to do some major service upgrades to our infrastructure. This includes rearranging our main data center, exchanging specific servers to use significantly less power, adding some more capacity, and deleting data from unpaid accounts. This will involve some outages when taking the service offline, so please bear with us over the next week. You will most likely experience it as a degradation in service where some files may not be accessible or the grid may not be writeable for short periods.
We are also expanding our datacenter footprint which then provides geographical redundancy.
Talk soon,
Peter
January 28th, 2010
Hi all,
We’ve been having a few ups and downs in the service lately, so please check our Twitter feed for the latest status.
Thanks,
Peter
October 5th, 2009
Hi all,
The production grid was back up and is fully accessible. Please let us know if you have any issues with it.
If you are interested, here are the details on the outage. We first thought that the scripts to restart the storage nodes were misconfigured and hadn’t restarted the storage nodes after the last power cycle here at the data center. However, it turns out that the scripts were ok (just formatted a little differently) and the actual problem was that a runaway process on each new box had caused out-of-memory errors and hung the box. We’ve restarted them and all is well with the production storage grid. We are diagnosing the situation to make sure it doesn’t happen again, but at this point aren’t sure exactly what caused the runaway processes, so we are monitoring the memory usage on those boxes more closely until we figure it out.
Sorry for the inconvenience to all,
Peter
September 16th, 2009
Hi all,
Long time no talk which is both good and bad. Good because it means there haven’t been any serious issues with the service, bad because we haven’t had any other pro-active communications about our open-source project (doing very well, http://allmydata.org), how the business is doing (breaking even now), or interesting metrics on the storage grid itself (the highest number of files of a given type are images, but the highest aggregate storage is of videos).
For most of the day we have been battling some file access and upload issues with the grid. We’ve finally found the problem (it looks like the scripts to properly restart some of the storage nodes were misconfigured and didn’t restart after a power cycle). At this point, we are confident that we haven’t lost any data and only access has been temporarily denied. To fix the problem, we are going through the storage nodes to make sure no other scripts are misconfigured and fixing the ones we know are.
We should be back up and running by tomorrow morning PDT and will update this blog and Twitter (http://twitter.com/allmydata) if anything changes in the meantime.
Thanks,
Peter
September 16th, 2009
Hi all,
The upload problems were fixed and all should be ok with the production storage grid. The problem was that somehow the number of available storage servers was being incorrectly reported so that when a new file was scheduled for upload, it thought that it was getting a “no space available” from every possible storage server.
Peter
April 21st, 2009
Hi all,
We are having some issues with file uploads at the moment. Please bear with us while we debug and fix it.
Thanks,
Peter
April 21st, 2009
Hi all,
On March 29th our beta FTP service (prodftp.allmydata.com) went down until just a few minutes ago. This was caused by too many open files on the server. We’re looking into why that happened, but this part of the beta cycle helps us prepare for greater loads going forward.
Thanks for trying it out and also to everybody who told us it was down.
Peter
April 5th, 2009
Hi all,
We’ve recently received quite a few requests from people which indicate that despite our best efforts to get in touch with them (emails, blog posts, Twitter updates) they were unaware that the free accounts had expired. This happened on January 19th, 2009 and we have been reclaiming space as necessary for new users and their data. If you do know of anybody using a free account who hasn’t logged in since January 19th, 2009, please let them know as their data will soon be garbage collected if it hasn’t been already.
Thanks,
Peter
March 25th, 2009
Hi all,
We’ve just released a slew of website updates, including the ability to share directories from the web drive, send a message along with your share link, and a few other cosmetic updates. I also wanted to remind you of the “Preview” feature on the web for documents and presentations. I find it very helpful as I browse around my files and want to see what is inside.
If you are interested in a small business plan, let us know (support@allmydata.com). We are testing out some new features that allow you to manage multiple people under one account, add new users, manage quotas, and view utilization of the users and the account.
Have fun!
Peter
February 5th, 2009
Hi all,
Note that all free accounts have expired. We are going to be slowly reclaiming space over the next couple of months, so please let us know soon if you need any of that data, otherwise it will soon be gone. Please, we don’t want to destroy any valuable data, but unfortunately the free business model just isn’t working out and we cannot support that service any longer. We do have a 1GB plan which is only $9.99/year, so sign up for that if you can!
Thanks,
Peter
January 19th, 2009
Previous Posts