Moving servers today

neptronix

Administrator
Staff member
Joined
Jun 15, 2010
Messages
17,405
Location
Utah, USA
Hey all. Just a heads up. I'm going to be moving the server today. There will be a 30min - 1 hr total downtime.
You may experience some slow performance while files crunch along in preparation for the move.

The good news is that we'll be on amazon and have longer backups ( something that was previously very expensive ) and also eliminate the intermittent stuttering issue that our current host has ( digitalocean ).

I will announce the downtime in advance, but it will probably be in a few hours.
 
Prep work done.
The server will go down in approximately an hour and i expect it to be down for approx 45 mins due to the large amount of files that need to be moved.

Hang in there and sorry for the interruption.
 
Okie, time to pull the plug. See you in about an hour.
 
Well that was a pain in the ass and took way longer than i expected.
Ok, this 6 hour job is done...... now you know why i've been putting it off :lol:

Do let me know if you notice anything... off as a result of this change.
 
Are images are still being uncompressed from the backup and it will be another 1-2 hours until missing images re-appear.
Edit: time expectation
 
Wow that happened a lot quicker than i expected.
Images are back and everything seems to work.

Performance may be pokey until tomorrow. OK, i'm done for the day!
 
Glad nobody is seeing issues!
You're welcome also, It's always a little hair raising for me to take down my favorite internet community. So happy that's over.

This server move is actually a big thing. We now have to ability to spawn multiple development environments from a rolling virtual machine image of the server itself. Meaning that more than 1 person can work on ES at a time.

The duration of backups we can keep is also up from 2 days to 1 year. Super good.

A big barrier was removed to getting the Xenforo environment working. The main barrier to deployment is that on a 2 core server like we have, the translation from phpbb to xenforo takes about 12 hours. Then for the data transformation processes, you're looking at 5 hours on top of that. So it becomes a 17 hour job.

Using a local 8-16 core virtual machine copy of the server would be gazillions of times faster, esp if you had a fast SSD.
My idea for this process would be to get the downtime down to 6 hours.
It is now possible :)
 
Computers :roll: always seems take bit longer than expect, cheers for your work :thumb: :bolt:
 
Server tweaking is done. Performance should be a bit snappier.

There will be a reboot in about an hour and images may temporarily go missing, but will be restored.
 
Server needs a reboot and a beating on the command line.
Will be down for 5-10 mins.
 
That was quicker than expected.

Stored images are going to disappear temporarily while i move 50g of data from one disk to another. Hang in there, this is the last of the surgery.
 
Thank you.

Images should all be back, and surgery is finally done, phew!!! :es:
 
Server moves are scary, I've been through a few, the worst one also involved switching from Solaris to Linux. I'm a big fan of virtual machines. I'm very familiar with Vagrant and Virtual Box for dev machines. Let me know if I can help, the one day I don't check the forum and you move the server. I'm glad everything went relatively smooth.
 
Just an update on this.

We've been closer to the edge of a 4gb memory limit on AWS than we were on digitalocean; probably because of the software AWS installs on all the OSes they provide.

Mysql is crashing and restarting on a daily basis because it can occasionally creep over this limit..

I'm aware of this and am looking for small memory cuts to prevent us from having to double our hosting costs over this.
 
I believe i have curred the hiccup. Let me know if you see the forum stutter. Has looked very stable from here.
 
Amascam
The entire cloud is designed to confuse you into paying more.
They intimidate you by constantly adding redundant services that you don't need but don't have
the time to read white papers on. You either give in or you're "too basic to understand"

Its just a money making scheme.
They will spin up so many sub service services...
you'll spend a day with tech support unspinning things
next month your still being billed
another day with tech support - "You didn't see the link on the page with a thousand links"
lol - I'm pretty pretty sure that link is intended to NOT be seen.

they moved on the US government - imagine the billing!?!
Jeff triple lindseys into his pool of gold coins and comes up for air with a splash of sparkly gold -
"Great isn't it?" guffaws

GET OUT its never too late.
 
I understand why you think AWS is a huge pain in the ass. I got thrown into the role of being a AWS server administrator in 2014, and had to learn how to work AWS the hard way. The learning curve is mondo.

But.. knowing how to twiddle these knobs, some setups i'm responsible for *are* cheaper on AWS than anywhere else. And when costs matter, sometimes AWS is the right choice.

I'd prefer to not give amazon a dime. But it makes sense here.
 
So the attachments for pictures limit is 36 as of yesterday when I saw the warning editing a post.
Its plenty, sometimes I just leave blurry pictures in because I am to lazy to delete them. But yeah it makes you think more of what to attach for pictures then minor things.
 
That's why systems like phpbb and xenforo are designed to put limits in at the stage of uploading.
Which can be inconvenient to the user.

That's why i put ~100 hours into writing a platform agnostic, after-the-fact image processor just for our form.

But if a picture is 1000 words then 36 pictures is a novel. See if you can tell the story with less words. :)
 
I'm from the by gone days when pics would have been limited in number much less size. I think the requirements are pretty lax. It's not really the 36 pictures it's the 36 large pictures. If people would size them no larger than needed. Any way good job.

by neptronix » Sep 17 2021 1:50pm

That's why systems like phpbb and xenforo are designed to put limits in at the stage of uploading.
Which can be inconvenient to the user.

That's why i put ~100 hours into writing a platform agnostic, after-the-fact image processor just for our form.

But if a picture is 1000 words then 36 pictures is a novel. See if you can tell the story with less words. :)
 
They all get forced into being below 2048 x 1500 size. So you can upload whatever the hell you want!
Idea stolen from facebook. :)
 
Back
Top