SentinelCraft Legacy Forum 2013 - 2022
Server Stuff => Server Announcements => Topic started by: Migas on October 17, 2020, 11:15:10 AM
-
Hello everyone.
Well... I am sorry to say but this is definitely unavoidable to only keep between staff.
Unfortunately during an update today, I updated Paper to an unstable version, even though it was released to public. As I have never had any issues with Paper before, I thought it would be safe to just update and improve our server's performance as always, alongside other plugins. My issue report is here, if you want to know about it: https://github.com/PaperMC/Paper/issues/4658
This unstable version caused ALL our worlds to become corrupted, which makes our server unable to start. Also adding to this, our backup system was currently not enabled which means there's no way to revert things back.
I'm not sure the extent of the damage but since I had the server whitelisted (might have been the only decent and thoughtful thing I have done this morning), only loaded chunks might have been corrupted. So this might only be the 25 "spawn chunks" for each world. However this is a considerable extent of damage already that at least will require deleting those chunks, if not more than that.
I am deeply sorry for this issue. As you might think, I am deeply upset with this as this is a community I love and I have seen grow up throughout the years, I can't believe that in just a matter of minutes, the "history" of our server has changed drastically.
I just hope this doesn't make you lose interest of staying with us and I hope you understand this issue might take hours or even days to fix (spawn world is the one with most damage due to having NPC's there).
If the damage is bigger than we think, we might have to think of a new approach for the server, because on that scenario, things would be unfixable. This might include finally setting up BungeeCord and revitalize our plugins to fit this new system. ALL the player data is safe, just to let you know. This is only a world issue.
All applications and appeals will be temporarily suspended to focus on this. -Edit- Applications and ban appeals will be processed once again, as normal.
-
I'm so sorry to hear about this unfortunate event.
And Migas, we know server performance are in the sky due to these updates and regular maintenances.
What I wanted to say is don't be sorry even if things goes south. We know you mean good.
Hope everything ok from now on.
-
I'm okay with it, maybe this will even fix bugs that's been hard to fix too. :D We'll keep supporting you Migas and Staffs!! Keep on Fighting!!! XD
-
It's ok Migas even tho theres a chance some of the progress i made for the last 2 month wipe out the drain lets start a new story again and create more memories(that kinda sound gay) as long as we have friend playing with us everything will be ok :) ;)
-
does this that my progress i.e base will be gone ??
-
does this that my progress i.e base will be gone ??
Istmeoo7 it is to early to suspect anything for now lets just wait and see what happens , oky :) <3
-
After understanding the issue i can see it came from chunk loading and i can see that some of the chunks dont have backup therefore they might be lost i know it would be hard for the staff too fix but me and i hope all of the community is with you guys, worst case scenerio if some chunks are lost i hope we can adapt and fix what we have lost and i hope the community can work together as one and fix this issues. Finally i am sure no one will leave and whatever you do migas and the rest of the staff we shall accept and give our support <3
-
Thank you guys for the update..
Lets all hope for the best.
im sure everything will be fine....if not we will pick up the pieces and glue them back together
:D ;D 8)
-
Thanks for keeping us informed and good luck with tackling this horrible situation!
Oh... and I'm not going anywhere else, no matter what the outcome may be! ;)
-
<3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3
<3 "To err is human, to foul things up requires technology." - BHL <3
<3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3 <3
(I'm not leaving! Take your time! Even this blockhead knows you're doing your bestest!)
-
Oh... and I'm not going anywhere else, no matter what the outcome may be! ;)
Same here! Issues may arise, some revisions may happen, and players may leave and never come back, but it's the loving and supportive community that made me stay and feel like I'm at home and loved by a family for almost 6 years already. We're all hoping for the best! We are all with you in this problem <3
-
Hope it turns out well all Admins!
Don't wear yourselves out!
People can leave the server but we cant leace this family-like community
Good luck!! ;) ;)
-
As my father use to say;
"Shit happens, doodoo occurs."
As long as things get fixed, no harm, no foul.
-
well it might be better to start with a fresh server...
Maybe this can be a reset?
-
At this time we are working hard to find fixes to all the world issues we are currently having. Banana and Cryminal have worked hard to narrow down the issues and a potential fix. This does not mean everything is going to be smooth sailing but we want to say we are trying our hardest to ensure the worlds are saved as the last thing we really want to do is reset everything.
-
Go ahead and do what you think is best, you have our support.
-
We managed to get enough done to put the server back up. This was the biggest damage we got, spawn has been restored:
(https://i.postimg.cc/TwNmf18S/2020-10-17-22-27-08.png)
(https://i.postimg.cc/BbdKPznB/2020-10-17-22-27-26.png)
(https://i.postimg.cc/7Pc7R5nJ/2020-10-17-22-27-46.png)
(https://i.postimg.cc/J0yXn8Mh/2020-10-17-22-28-10.png)
Things that still need fixed, probably tomorrow:
- Shop signs
- Crates
- Tokenshop FAQ
- Some decorative details at spawn (NPC's, maps, books, holograms, etc)
- Pern Spawn (Need to find the world file somewhere else)
Banana will write something about this better explaining what kind of data we lost and our process of fixing things today. I am tired and I am going to bed, I am glad things are working out better than expected.
-
Banana will write something about this better explaining what kind of data we lost and our process of fixing things today.
Most importantly, I want to thank everyone who assisted in getting the server back into a working state today. I also want to thank our players for their understanding and patience over the past 12 hours. Thanks to a combined effort, no player builds or data was lost. Everything players have built on the server, their balances, and all of their related data is safe and sound.
Ultimately, the only damage was to server buildings such as Fedguardia spawn or Pern spawn. Staff have been working tirelessly to repair the remaining damage or missing features.
What happened?
As Migas explained in the initial post, our server software Paper was updated to the latest version at the time. This build contained a severe bug. Every chunk loaded with this build is irreversibly corrupted. I cannot emphasize enough that this is an extraordinary occurrence, and we were not the only server affected by this issue.
Sentinel was fortunate enough that no players were online when this version was loaded, so the damage was only to the "spawn chunks" (chunks that are always loaded around the center of each world). We are also fortunate that none of our "spawn chunks" contain player builds. Plugin data, player data, and chunks that were NOT loaded during this time were not affected.
How did we fix it?
First and foremost, we avoided the worst case scenario because the update was performed without any players online. We then tried to ascertain the damage, and to what extent the chunks had been corrupted. After testing separately, we confirmed that only the "spawn chunks" were corrupted.
Initially, the server failed to start but we managed to get it successfully up and running with full functionality. We repaired as much of the Fedgardia spawn island as we could, copying from a creative copy of the island. From there, the server was opened again, and staff set about repairing miscellaneous things (NPC's, shop items, treasure hunt, etc).
At the time of this writing, the server is very nearly back to where it was before all this happened. The remaining issues are all fixable, and will be fixed in the coming days.
What happens now?
Our first priority is completing the remaining tasks to repair the damage. I and many other staff spent the entire day doing so, and I'm sure we will be continuing to do so. We also will be making a proper copy of the spawn island once it is fully repaired, should the need arise for it.
Lastly, we want to make sure that something like this never, ever happens again. We were fortunate that the impact was small enough to be repairable this time. Moving forward, we as staff will be discussing and implementing new systems to make sure the server is running safely and smoothly.
We ask that players be patient with staff, as we are working overtime at the moment. Thank you <3