Best Compression Methods for 'Specific' Games. Q&A

[dyren]

Assets.zip:

Extract
Create hdiff patch (Assets folder to Assets.zip) (16MB)

For *.png xtool (-mpng+reflate -d1 -dd)+srep+rzs
Note: for some reason compressing with lolz or rzs it will result in a ratio slightly above of 100% (rzs being on the lower end)

For *.jpg brunsli

For *.ogg oggre
Note: The following files will cause oggre to crash during install/unpacking, compress with other tools, i used rzs, or try using oggre by itself, without arc:

Player_Hurt_Drowning_01.ogg
Player_Hurt_Drowning_02.ogg
Player_Hurt_Drowning_03.ogg
Player_Hurt_Drowning_04.ogg
Player_Hurt_Drowning_05.ogg
Player_Hurt_Drowning_06.ogg

For *.bin xtool (-mzstd (1.5.5 lib))+srep+lolz
Note: Couldnt get all the streams, tried multiple libs and levels

For rest of the files srep+lolz

With the hdiff patch file created earlier you can patch the folder back to the original .zip archive.

Game size: 3.61GB
Repack size: 1.2GB

*rzs = RAZOR Archiver 1.0.3.7

[KaktoR]

^

Streams: 1845 / 1845
Time: 00:00:00 (CPU 00:00:00)

Size: 8,43 MB >> 109 MB

zstd 1.5.7

However I don't know if you save anything from this to be honest.

---

The only bad ogg file is Player_Hurt_Drowning_02.ogg in my test if you use oggre without freearc.

[dyren]

Originally Posted by KaktoR

^

Streams: 1845 / 1845
Time: 00:00:00 (CPU 00:00:00)

Size: 8,43 MB >> 109 MB

zstd 1.5.7

However I don't know if you save anything from this to be honest.

---

The only bad ogg file is Player_Hurt_Drowning_02.ogg in my test if you use oggre without freearc.

Oh damn, theres 1.5.7 for zstd? Didn't know that. Where can I find it exactly? Couldn't find it here, don't know if im searching incorrectly.
Usually you can save a bit.

-

Ah, good to know. My setup either errored out or crashed completely with those files. As they werent big in size i didnt check them without freearc.

[KaktoR]

https://github.com/facebook/zstd/releases/tag/v1.5.7

Here is my collection for every library which is compatible with xtool (x64 only).

[dyren]

Thanks ^^

[KaktoR]

Detroit Become Human

Answering this question here: https://fileforums.com/showpost.php?...7&postcount=14

Tools used: QB.Unpacker, HXD, XTool (erase, decode functions)

What I've found out:

Language files are mixed in many game archives.

Example:
BigFile_PC.d23

English -> _ENG -> Offset 16478000
Spanish -> _SPA -> Offset 16899800

and so on.

File names are stored as plain text there. Just go to the offset in HXD (CTRL+G) and read a bit further on the right text until you reach the language code. You can also search for "57 41 56 45 66 6D 74" hex value and read a bit backwards to find the language code. Pattern: CNTR.Hdr - File name - RIFF Wavefmt header

I've extracted the files with QB.Unpacker by EKey from github (you have to compile it with visual studio yourself).

However you have to find out all the files which belongs to what languages yourself. There are thousands of it. The audio files will be stored in COM_SOUND_DATA folder.

In DATA_CONTAINER folder there are video files (KB2j "4B 42 32 6A" header) which you may rip off from the game files with xtool erase function (same for audio files as both are stored as plain text aswell). Unfortunatelly the extracted files are extracted without any file extensions which makes it really hard to tell what files contain what actual data.

Example:
Probably the "Making of" bonus videos for the game (according to the name of the text inside HXD).

File 7612 -> MAKINGOF_MAKING_DETROIT_JPN (Japanese) -> located in "BigFile_PC.d27" at offset 4F843800.
File 7560 -> MAKINGOF_MAKING_DETROIT (all other languages I guess) -> located in "BigFile_PC.d24" at offset 800.

Probably you can make a small tool which reads the file headers and sort the files in different folders automatically.

Have fun.

Edit: If you erase the extracted files out of the game files with xtool, the game files will have the same size as before but with zero'ed data which will lead to <1% compression ratio for that block inside the file.

[wareck]

For World War Z - Aftermath :

Data1 : *.pak except shared_video.pak and shared_sound.pak
xtool:mpreflate:dd3+LOLZ_NORMAL
Data2 : shared_video.pak
xtool:c1792mb:mbik:mbk2:df20p
Data3 : shared_sound.pak
if you know how to => FSB5 pre & post outside diskspan
xtool:mpreflate:dd3+LZMA-MT
data4 : other files
xtool:mpreflate:dd3+razor

[Dunnowho69]

Originally Posted by KaktoR

Detroit Become Human

Answering this question here: https://fileforums.com/showpost.php?...7&postcount=14

Tools used: QB.Unpacker, HXD, XTool (erase, decode functions)

What I've found out:

Language files are mixed in many game archives.

Example:
BigFile_PC.d23

English -> _ENG -> Offset 16478000
Spanish -> _SPA -> Offset 16899800

and so on.

File names are stored as plain text there. Just go to the offset in HXD (CTRL+G) and read a bit further on the right text until you reach the language code. You can also search for "57 41 56 45 66 6D 74" hex value and read a bit backwards to find the language code. Pattern: CNTR.Hdr - File name - RIFF Wavefmt header

I've extracted the files with QB.Unpacker by EKey from github (you have to compile it with visual studio yourself).

However you have to find out all the files which belongs to what languages yourself. There are thousands of it. The audio files will be stored in COM_SOUND_DATA folder.

In DATA_CONTAINER folder there are video files (KB2j "4B 42 32 6A" header) which you may rip off from the game files with xtool erase function (same for audio files as both are stored as plain text aswell). Unfortunatelly the extracted files are extracted without any file extensions which makes it really hard to tell what files contain what actual data.

Example:
Probably the "Making of" bonus videos for the game (according to the name of the text inside HXD).

File 7612 -> MAKINGOF_MAKING_DETROIT_JPN (Japanese) -> located in "BigFile_PC.d27" at offset 4F843800.
File 7560 -> MAKINGOF_MAKING_DETROIT (all other languages I guess) -> located in "BigFile_PC.d24" at offset 800.

Probably you can make a small tool which reads the file headers and sort the files in different folders automatically.

Have fun.

Edit: If you erase the extracted files out of the game files with xtool, the game files will have the same size as before but with zero'ed data which will lead to <1% compression ratio for that block inside the file.

Hopefully someone can make a tool (

[KaktoR]

Here are some other observations.

Language audio files:

Container Header  17 bytes
File size (CSize) 4 bytes
Unknown           4 bytes
Unknown (some ID) 20 bytes
Unknown           8 bytes 
File name         different bytes lenght according to file name lenght
Language code     3 bytes
Unknown           4 bytes
Riff Wavefmt      16 bytes

Audio files (BKHD container):

Container Header   12 bytes
Unknown            25 bytes
Unknown (DC_DATA)  8 bytes
File size (CSize)  4 bytes 
Unknown            4 bytes
Unknown            13 bytes
Unknown (COM_CONT) 8 bytes
Unknown            8 bytes
Unknown (LOADCONT) 8 bytes
Unknown            4 bytes
Unknown (CSNDBKDT) 8 bytes
Unknown            4 bytes
Unknown            4 bytes
BKHD Header        4 bytes

Video files (not required because you could simply use Mesa to extract or just use xtool bink plugin):

Container Header   12 bytes
Unknown            25 bytes
Unknown (DC_DATA)  8 bytes
Unknown            4 bytes
File size (CSize)  4 bytes
Unknown            4 bytes
Unknown            14 bytes
File size (CSize)  4 bytes (some other file size which differ slightly from first file size)
Unknown            4 bytes
Unknown            18 bytes
File size (CSize)  4 bytes (again some other file size which differ slightly from first file size)
Unknown            14 bytes
Unknown (repeat 1) 4 bytes
Unknown (repeat 2) 4 bytes
Unknown            12 bytes
Unknown (COM_CONT) 8 bytes
Unknown            8 bytes
Unknown (VIDEORES) 8 bytes
Unknown            9 bytes
File name          different bytes lenght according to file name lenght
Language code      3 bytes (only JPN)
Unknown            4 bytes
Bink Header        4 bytes

[Dunnowho69]

Originally Posted by L33THAK0R

Source: Clean Steam Files

Version 1.10
Original Size: 27.3 GiB
Final Size: 10.705 GiB (Incl. installer)
Ripped Files: Non EN. loc.

Internal Tools used:

Xtool v0.8.9 (Codecs: reflate, zlib)
Srep
Lolz
Oggre

External Tools used:
SFK
DELZORec
MParallel

Additional notes: Happy with how this turned out, a substantial improvement over my previous results, the approx. ~200 MiB reduction can be attributed to more effective processing of the Vorbis OGG audio, as well as tighter tuning for the lolz model.

How to set DELZORec for installer?

[Masquerade]

^^
Delzorec isn't compatible with FreeArc.

You can generate dunia lzo databases with XTool duniadb maker or bms2xtl.

[Dunnowho69]

Originally Posted by Masquerade

^^
Delzorec isn't compatible with FreeArc.

You can generate dunia lzo databases with XTool duniadb maker or bms2xtl.

Yeah found out about that. Using the xtool plugin now. For installation i just need the lzo2.dll right ?

[KaktoR]

I think delzorec is better but needs more i/o operation.

However if you want to use delzorec:
1. Include delzorec inside your setup file or compressed game archives (if in setup.exe file then make sure that installer extract it in installation folder).
2. Run delzorec.exe with required parameters directly from installer script after installation is success or simply let it run a batch file from installation folder as working dir.

[wrathma]

can someone help me. i have some .wem files i extracted from silent hill f .pak using this tool. the extracted wem files are playable. most of them contain pcm wav. i tried to compress them using msc but it doesnt work (gives same output as input file). i can see fitgirl used oggre on her repack but i cant find any ogg data in .wem files. i tried to compress .wem with oggre_wwise but still gave same output. wemtool also doesnt help much.
edit: attached sample

[KaktoR]

This is audiokinetic wwise format (ogg vorbis), not pcm.

Use wemtool on them.

PS: Skip all wem file below 100kb, otherwise you will get negative ratio.

2.88mb -> 2.64mb

Edit: I extracted all audio files from the pak file and processed them with wemtool.
Result: 8.14GB -> 7.61GB