Hotfix 2.0.0m - Death to the Long Save!

[kinggath]

In theory, this patch finally slays the long save bug!

I say in theory, because we can't reliably trigger the issue, but from everything we understand about it - this should resolve the issue. Even if it doesn't though - we have a guaranteed fix for F4SE players (see my next post in this thread), and the tools we need to further diagnose the issue thanks to Nukem, one of the Bethesda mod communities most knowledgeable folks on engine behavior.

Full patch notes below:

• Added a fix to address the HQ Cell reset bug that will prevent it from occurring to saves started before the 2.0.0L patch. Previous patch notes had listed that the fix would not work on existing saves, this new fix addresses that.
• Added a new tool to the holotape under Tools > Advanced > Quest Fixes > Chapter 2: HQ Reset Issue. Running this while inside of the HQ’s main floor will attempt to recover the save from the HQ Cell reset bug if it had occurred previously.
• Fixed a number of script property issues that would have resulted in papyrus log spam.
• Fixed a bug in the quest “Martial Law” where Mansfield would talk about putting the radio on the table himself after the Gunners already had done so.
• Added Reset Leader Card Cache and Reset Settlement Flag Cache options to Tools > Advanced in the holotape. These will reset the corresponding cache for players who find they are missing options they expect to be available.
• Fixed a bug with the HUD meters that would show shortage of beds and recreation for robot settlers.
• Fixed a bug with the HUD meters that would show shortage of power for plots that were recently scrapped but hadn’t been cleaned up by the game engine yet.
• Fixed a bug where if the quest “The Disappearance of Jake Evans” was played through in Chapter 2, and then the following quest was skipped by console commands or the holotape cheats, The Ron could end up stuck in a weird state, potentially breaking his dialogue in future quests.
• Fixed a bug with the traveling Nightingale medics that prevented them from speaking and offering medical services.
• Fixed a bug in the quest “How to HQ” that would prevent a conversation from completing if Stodge had been killed in the save.
• Fixed a bug part way through the quest “How to HQ”, where the resource values could show negative values.
  • Existing saves should have these negative values removed after starting this patch, which may correct other issues with HQ resources.
• Fixed a bug during the quest “New Connections”, where if the player wasn’t close enough to Circe and Lydia, the conversation could fail to start and the quest would appear stuck.
• Made a change that in theory will greatly reduce the likelihood of a “long save” occurring.
• Fixed several voice files for the CPD.

Xbox players, the versions to watch for when you can safely update are: 44 of SS2, and version 17 of Chapter 2.

If you continue to experience long saves after this patch, please reply below with how long your saves are and what system you're on.

-kinggath

 

[kinggath]

F4SE Garbage Collector Fix

Nukem created this patch for Fallout 4: https://drive.google.com/u/0/uc?id=1UAIXTCy9rtQxyFYc0SMjzuHLU4UlMGKV&export=download
It requires F4SE https://f4se.silverlock.org/, and the address library: https://www.nexusmods.com/fallout4/mods/47327

Presumably this will be released officially as a patch you an download on Nexus, or per our conversation, he may send it to the Buffout 4 creator to have it incorporated there, but in the interim you can keep using it.

Even with SS2 presumably having this issue addressed, this fix is something valuable and may have performance gains beyond correcting long saves.

--
For those of you tech-minded, here's what this fix does, and why the long save issue occurred in the first place:


SS2 often bypasses Bethesda's papyrus limitation against multi-dimensional arrays, by storing scripted objects in our data structures and arrays, this way we can simulate as much depth as we need to and effectively emulate a standard programming language as far as keeping complex data is concerned.

The Creation Engine uses a very simple garbage collector, one that is given a very tiny amount of time each frame to do its thing. Unfortunately, when iterating over arrays, if it finds a piece of data it needs to clean up, it restarts the loop. This in itself is generally not a problem, except that it can't dismantle a struct from an array all at once, it has to do so one data point at a time. So when it comes to one of our emulated multi-dimension arrays, it has to restart the loop over and over again, and will rarely ever finish in its allotted time - so this garbage piles up until just before your game saves, when its finally given the time to do a full clean-up.

Still, none of this is new since SS2 released. We've always used these complex patterns without issue. What changed in the 2.0.0 patch that started causing these long saves, is that we introduced several new plot classes, and one of them was entered twice in an array by mistake.

This then caused an innocuous function I had set up, that was meant to ensure other mods and expansions can inject new building classes one day, to never be able to use its own cache of building class information. The way the cache worked was very simplistic, if the number of entries in the cache matched or exceeded the default class list, the cache would be used, otherwise it would be generated fresh. This was done to handle edge cases where addons were registering building plans before the corresponding class has finished registering.

Since the cache would only accept new unique entries, and there was a duplicate in the default list, the cache was always one entry smaller than the default list, so SS2 treated it like the cache still wasn't finished preparing.

The more plots you had in your save, the more time this building class information cache was being regenerated. In realtime, this didn't really matter, and the code could do it thing, but it would start creating thousands of these data structures over time that took too long for the per frame garbage collector to handle. Then come save time, the GC was given as much time as it needed to clean up the unneeded cache copies.

To fix this for SS2, it was as simple as removing the duplicate entry from our default array so that the cache would be considered valid, and would not be constantly regenerated.

Nukem's fix was very straight-forward, and basically optimizes the per-frame garbage collector so that it doesn't have to repeat the entire loop every time it removes an entry from a struct. Effectively making it so it could correctly dismantle SS2's complicated data structures in the allotted time.


So the issue was one part mistake by me, and one part suboptimal engine code that collided into this epic fail.

 

[NOBODY]

Finally good news!
I've been waiting for LS & Reset HQ issues to be resolved to release my Leader Pack, and here is, Nobody's Leaders 2 is now out too.

Sim Settlements 2 - Nobody's Leaders 2

Sim Settlements 2 Leader Pack, featuring 60+ vanilla & SS2 NPCs available to be city leaders

www.nexusmods.com

 

[VersusXV]

Amazing to see! I have an update for SS2 Superstructures that I can think about doing in the near future. Held off on some things due to the long save bug & HQ Reset.

 

[Dashboarderian]

F4SE Garbage Collector Fix

Nukem created this patch for Fallout 4: https://drive.google.com/u/0/uc?id=1UAIXTCy9rtQxyFYc0SMjzuHLU4UlMGKV&export=download
It requires F4SE https://f4se.silverlock.org/, and the address library: https://www.nexusmods.com/fallout4/mods/47327

Presumably this will be released officially as a patch you an download on Nexus, or per our conversation, he may send it to the Buffout 4 creator to have it incorporated there, but in the interim you can keep using it.

Even with SS2 presumably having this issue addressed, this fix is something valuable and may have performance gains beyond correcting long saves.

--
For those of you tech-minded, here's what this fix does, and why the long save issue occurred in the first place:


SS2 often bypasses Bethesda's papyrus limitation against multi-dimensional arrays, by storing scripted objects in our data structures and arrays, this way we can simulate as much depth as we need to and effectively emulate a standard programming language as far as keeping complex data is concerned.

The Creation Engine uses a very simple garbage collector, one that is given a very tiny amount of time each frame to do its thing. Unfortunately, when iterating over arrays, if it finds a piece of data it needs to clean up, it restarts the loop. This in itself is generally not a problem, except that it can't dismantle a struct from an array all at once, it has to do so one data point at a time. So when it comes to one of our emulated multi-dimension arrays, it has to restart the loop over and over again, and will rarely ever finish in its allotted time - so this garbage piles up until just before your game saves, when its finally given the time to do a full clean-up.

Still, none of this is new since SS2 released. We've always used these complex patterns without issue. What changed in the 2.0.0 patch that started causing these long saves, is that we introduced several new plot classes, and one of them was entered twice in an array by mistake.

This then caused an innocuous function I had set up, that was meant to ensure other mods and expansions can inject new building classes one day, to never be able to use its own cache of building class information. The way the cache worked was very simplistic, if the number of entries in the cache matched or exceeded the default class list, the cache would be used, otherwise it would be generated fresh. This was done to handle edge cases where addons were registering building plans before the corresponding class has finished registering.

Since the cache would only accept new unique entries, and there was a duplicate in the default list, the cache was always one entry smaller than the default list, so SS2 treated it like the cache still wasn't finished preparing.

The more plots you had in your save, the more time this building class information cache was being regenerated. In realtime, this didn't really matter, and the code could do it thing, but it would start creating thousands of these data structures over time that took too long for the per frame garbage collector to handle. Then come save time, the GC was given as much time as it needed to clean up the unneeded cache copies.

To fix this for SS2, it was as simple as removing the duplicate entry from our default array so that the cache would be considered valid, and would not be constantly regenerated.

Nukem's fix was very straight-forward, and basically optimizes the per-frame garbage collector so that it doesn't have to repeat the entire loop every time it removes an entry from a struct. Effectively making it so it could correctly dismantle SS2's complicated data structures in the allotted time.


So the issue was one part mistake by me, and one part suboptimal engine code that collided into this epic fail.

I'd love it if the Garbage Collector fix to become incorporated into Buffout, this is a serious engine-level addition and fix... it isn't just beneficial to Sim Settlements anymore, but any scripted mods really now.

 

[Mythor]

Certainly seems like it's working, didn't get a long save in an admittedly fairly short session, but definitely would have before. Longest save this time was maybe 3 seconds at most, down from occasionally a minute or so. Really good news for the future of SS2, well done to those who helped get it diagnosed and fixed so quickly.

 

[MoreThanKelp]

It worked! No long saves in any settlement.

 

[ElanaLoaded]

Finally good news!
I've been waiting for LS & Reset HQ issues to be resolved to release my Leader Pack, and here is, Nobody's Leaders 2 is now out too.

Sim Settlements 2 - Nobody's Leaders 2

Sim Settlements 2 Leader Pack, featuring 60+ vanilla & SS2 NPCs available to be city leaders

www.nexusmods.com

FYI Also requires Rise of the Commonwealth for SS2. Nexus page corrected.

Spreadsheet is awesome. Is there a way I (we) may download spreadsheet so I can make a textfile version for in game reference?

 

[kinggath]

FYI Also requires Rise of the Commonwealth for SS2.

@NOBODY You don't actually need to make that a requirement. RotC is effectively just a city plan pack, all of the code is part of SS2 itself.

 

[Voltron80]

On Xbox and when I tried to install the update it got rud of all of the mods to SS2 and chapter 2. Now it will not download.

 

[XxA1xX]

On Xbox and when I tried to install the update it got rud of all of the mods to SS2 and chapter 2. Now it will not download.

Give it some time Xbox can be slow on the updates.

 

[kinggath]

On Xbox and when I tried to install the update it got rud of all of the mods to SS2 and chapter 2. Now it will not download.

Bethesda's servers are a pain in the but for Xbox - basically they will flag it as having an update available when I upload, but you won't actually be able to download it for many hours later.

You can tell when its safe to update/download it as on the description section of the mod on Xbox, it will show a version number that matches what I posted above. For the base of SS2 the version number xbox should show is 44 and for Chapter 2, it should show 17.

 

[Voltron80]

Sounds good thanks!

 

[Voltron80]

For the time being, I will just play some Mass Effect.

 

[eMYNOCK]

@NOBODY You don't actually need to make that a requirement. RotC is effectively just a city plan pack, all of the code is part of SS2 itself.

speaking of wich.. is there a way to get the single stages (preferably the finished stage) of some city plans?
not that i want to cheat at all costs.. but i just can't stand haning around in starlight for hours just to make sure no incidents happen and still parts of it litter the enterance area of v88...

 

[Tarkkh]

Omg @kinggath thank you! No more having to build personal shelters at each settlement!
Life is good again!

Honestly very well done job! Now take on that beast called HQ ^^
My honest opinion
HQ» While this is an amazing feature as I stated in a prev post the inner systems seem out of place
Do players need the ability to decide where the bathrooms go? A more linear /hands-free building system would probably serve the intended outcome easily and without all the headaches!

 

[JimJam]

Nice work as always @kinggath.

I don't have time to check right now, but I was having previous issues with speaking to Pastor Edmund in Concord. He had the conversation with Jake, but I haven't been able to initiate conversation with him after that; he just gives me idle dialogue. I understand some other people have been facing the same problem.

Has this been addressed, or is it something dumb on my end?

Thanks again for the mountain of exceptional work you and your team have accomplished thus far!

 

[Agi_]

It works like a charm, no Long Saves even in Starlight, Sanctuary, Slog that are my largest settlements. Bravo to all that worked to fix it

 

[NOBODY]

@NOBODY You don't actually need to make that a requirement. RotC is effectively just a city plan pack, all of the code is part of SS2 itself.

While might have been a unnecessary thing to do, I've only made ROTC a requirement in Nexus, but its not actually a requirement to run the mod ofc as that is SS2 and its dependencies.
I have removed ROTC from Nexus requirements to avoid more confusion.

 

[ElanaLoaded]

@NOBODY You don't actually need to make that a requirement. RotC is effectively just a city plan pack, all of the code is part of SS2 itself.

Good to know - thank you. Glad I may use Leader pack after all. Thank you. Leader Pack = welcome addition.

I was just following thru on list of prerequisites posted on mod's Nexus page. I felt disappointed when I read the Nexus prerequisite - as I'm not using that prerequisite listed mod. So, the Nexus mod page had confusing info - and you fixed it. Gr8!
EDIT = and endorsed!