So I've been rushing to finish some previously planned features as the v469 public beta draws near...
The automatic lzma compression on servers always felt off as it left a bunch of LZMA files sitting all over your server directories.
So I decided to add a more 'contained' way of doing it which combines memory and file sources while making it way
more friendly to server admins, to the point it can be left completely unattended.
When a server finishes loading a map, the 'LZMA server' will receive the package list and it'll attempt to do some
storage managment as it compresses sources.
It uses a fast memory cache reserved for the smallest packages, and a file cache for the larger packages.
- LzmaCache folder.PNG (5.47 KiB) Viewed 1214 times
This would be a sample of the ini file:
Code: Select all
[Server]
MaxMemCacheMegs=8
MaxFileCacheMegs=256
ForceSourceToFileMegs=16
[LzmaCache]
627DAB834FAC01EBC5E278B4CB181480.lzma=../System469/sgUMedia.u
Both cache sizes are limited, the idea is to:
- When package is registered, try to locate an existing file source (can be a LZMA/UZ file next to it, as done via
UCC LZMACOMPRESS or old
bAutoCompressLzma)
- If package has no file source, the LZMA server will being a compression process.
- When compressed:
-- If original file larger than ForceSourceToFileMegs > push to file cache directly.
-- Otherwise keep in memory.
As more packages are pushed into memory, the server will count how much is being used and will push the biggest packages to file cache if the total exceeds
MaxMemCacheMegs.
As more packages are pushed into the file cache, the oldest files will be purged if the total size exceeds
MaxFileCacheMegs
This means a compressed package can potentially have the following lifecycle:
- Compression > Memory > Filesystem > Removal
The LZMA server should automatically delete cache files not referenced in the LzmaCache.ini and delete entries that don't have corresponding files (self maintenance).
Another plan is to temporarily halt download requests from clients to force compression to finish before starting to send data, also, to prioritize
specific packages for compression as more clients request them at the start of the level.
Example logs:
LZMAServer: Autocompressing package ../Maps/CTF-MeinKraft][-v3.unr
ScriptLog: Level change > New Pawn is CTF-MeinKraft][-v3.TBoss0
LZMAServer: Autocompressing package ../Textures/MeinKraftTex.utx
LZMAServer: Autocompressing package ../System469/CacusBlock.u
LZMAServer: Autocompressing package ../System469/XC_Siege_r3.u
LZMAServer: Autocompressing package ../System469/MCHalfSlab.u
LZMAServer: Autocompressing package ../System469/ConformTest.u
LZMAServer: Autocompressing package ../Textures/BossSkins.utx
LZMAServer: Autocompressing package ../System469/FV_Weapons.u
LZMAServer: Autocompressing package ../System469/sgUMedia.u
LZMAServer: Autocompressing package ../System469/SkinPal.u
LZMAServer: Autocompressing package ../System469/BsodPackage.u
LZMAServer: Autocompressing package ../System469/Yoshi.u
LZMAServer: Autocompressing package ../System469/SkeletalChars.u
LZMAServer: Autocompressing package ../System469/LCWeapons_0024.u
LZMAServer: Pushing cache to file [../System469/sgUMedia.u] -> [627DAB834FAC01EBC5E278B4CB181480.lzma]
LZMAServer: Autocompressing package ../System469/CacusBetas0008.u
LZMAServer: All sources processed (40)