Tools for Repacking

[KaktoR]

Added new oodle library
- version 2.9.12 (lib3) from "Battlefield 6"
- version 2.8.7 from "Battlefield 2042"

[panker1992]

Oodle Version 2.9.14 Self compiled

[Shegorat]

Hi there. Just a quick note about updating my CLS. I'm too lazy to write a changelog, but you can find it on my Patreon. The changes are mostly aimed at improving stability.

[KaktoR]

Originally Posted by KaktoR

OGG_File_Checker

I updated the tool to v2.
Subfolders are now supported. You can now copy/move entire game folder which includes OGG files and check them.
Removed xdelta patch engine, only hdiff is supported now.

[KaktoR]

Originally Posted by :( Sad8669

Usage:

WAVExtractor <input> <directory> [-verbose]

WAVExtractor (v1.3)

- Now supports input greater than 2 gigabytes. (Used an Extension Method i found on Ekey's Github)
- Added verbose mode that shows information about each stream that is being extracted.
- Added File progress for extraction.
- Fixed a minor stream detection issue.
- Added a logo.

I modified the source code and recompiled it with Visual Studio.
Changes: WavExtractor now extracts the files as WEM extension instead of WAV.

This tool is still usefull because MESA cannot extract some archives (Shegorat already knows about this problem).

[kj911]

Very fresh new competitor from others projects. (FreeArc Next, Diskpan, UCC, etc...)

SYC - Modular Compression Tool

SYC is not a compressor — it's a modular compression orchestrator that chains external tools (zstd, srep, xprecomp, zpaqfranz and more) into a single pipeline, producing .syc archives. Think of it as a conductor: it manages the workflow, archive format, encryption, and progress — the actual compression is done by the external compressors you already have.

Includes a GUI wrapper (sycg) with real-time metrics, dark/white themes, custom icons, and InnoSetup integration. Supports solid mode (-tar), multi-part archives, AES-256/ChaCha20 encryption, CRC32/MD5 hashing, and builds for Windows 7–11 x86/x64 using Python 3.8.

Note: This project isn't meant to reinvent the wheel; it's more of a small personal project I created for my own needs that I'd like to share. I CAN'T WAIT TO HEAR YOUR IDEAS !!!

See this: https://encode.su/threads/4480-SYC-M...mpression-Tool

[kj911]

Separately as a new post!

YadeWira released packJPG Multithreaded versions few hours ago! Windows x86/64 and Linux64 binaries in available!

https://encode.su/threads/4482-packJPG-Multi-threaded

[KaktoR]

Great.

I just tested a bit. It looks like -th0 is actually much slower than without this option.

-th0
Compressed 9 files, 6,362,109 => 4,683,122 bytes. Ratio 73.61%
Compression time: cpu 0.00 sec/real 4.06 sec = 0%. Speed 1.57 mB/s
All OK

no -th option
Compressed 9 files, 6,362,109 => 4,683,122 bytes. Ratio 73.61%
Compression time: cpu 0.02 sec/real 2.12 sec = 1%. Speed 2.99 mB/s
All OK

-th6 is the same as -th0. Am I missunderstand this option?

Also you still have to use solid=0 option for freearc sadly.

Edit: I just saw this is the wrong toppic for such discusions.

[YadeWira]

Here’s what I replied on the encode.su forum: “To be honest, I haven’t tried it on FreeArc yet. It’s not recommended to use the -th option with a single file, since it yields worse results in terms of compression and decompression times. In any case, you should create an option that generates a package (.pjgp?) containing all the already-compressed .jpg files or something similar. I don’t know what you think."

[wrathma]

oodlescan

a simple oodle stream scanner i made. 99% of the logic is taken from xtool source code. built with gcc on windows x64. the only reason i made this and difference between this and Razor12911's oo2scan_7_win64 is that this one uses chunk based data loading. oo2scan_7_win64 tries to occupy the entire size of the file on memory to store and scan it from there, in cases where the data is huge it would fail. oodlescan copies small chunks from the file to memory and scans from there, eliminating the memroy allocation issue. i will add the source code here after some polishing .

 SYMM's Oodle Stream Scanner

 - Scans files for Oodle streams

 Usage: H:\test\ooscan\oodlescan.exe [options] <file>

 Options:
   -cN    size of chunks (e.g. -c16M)
   -v     show valid streams only
   -vv    show valid and invalid streams
   -h     show this help message

edit: added source code

[wrathma]

prl

a simple and very lightweight (11kb binary, lzma 5kb) parallel batch processor. built with msvc x64. it lacks features but gets the job done.
source code moved to codeberg

Usage: H:\test\tools\build\prl.exe <input.bat> [thread_count]

edit: update v1
replaced semaphore with a threadpool.
replaced system() calls with CreateProcessA

[wrathma]

wem(un)pak

This is a more feature packed version of Masquerade's WemTool that i made few months ago to compress wem files from Silent Hill f. As it was made with golang, the binary sizes are huge. I didnt want to port this entire thing ito c, so today i added a feature to generate a batch script to decompress the files. Then you can run the batch script in parallel with prl or Mparallel. It follows same file naming standards as original WemTool. So you can use wempack to decompress WemTool compressed data. Attachments include source code.

• Separate compressor and decompressor.
• Ability to compare input and compressed sizes.
• Works recursively.
• Ability to specify compressor for hdiffz.
• Ability to generate a batch script to decompress.
• Better error handling and processing structure:
• compressor : ww2ogg > oggre_enc > oggre_dec > hdiffz
• decompressor : oggre_dec > hdiffz
• If any of the steps fails, the tool skips that file automatically. Temporary files are deleted in every step of processing, WemTool processes all files and then removes temps at the very end, leaving a huge mess and sometimes filling up the disk.

 SYMM's WEM packer

 - Compresses .wem to .ww + .DIFFZ

 Usage: wem-packer.exe [options] <directory>

 Options:
   -tN    number of threads (e.g. -t4/defaults to cpu count)
   -s     skip if output (.ww + .DIFFZ) is larger than input (.wem)
   -b     keep original .wem files
   -cX    compressor for hdiffz, default = none.
   -h     show this help message

 SYMM's WEM unpacker

 - Decompresses .ww + .DIFFZ to .wem

 Usage: wem-unpacker.exe [options] <directory>

 Options:
   -tN    number of threads (e.g. -t4)
   -b     keep original .ww and .DIFFZ files
   -n     generate a batch script to decompress
   -h     show this help message

Credits:

• Masquerade for the guide
• Sisong for hpatchz/hdiffz
• hsc64 for ww2ogg
• Profrager for oggre

[kj911]

wrathma: This wempak tool, XP-compatible or Vista+ 32bit?? Some parts from Masquerade's WemTool, are yes running from XP.

[wrathma]

Originally Posted by kj911

wrathma: This wempak tool, XP-compatible or Vista+ 32bit?? Some parts from Masquerade's WemTool, are yes running from XP.

the one above builds shouldnt work on 32bit systems, i have attached a 32bit builds for you. not sure if it will work correctly as i dont have a 32bit test system laying around. i replaced the 64bit hdiffz/hpatchz exe with 32bit ones, everything else is same.

[KaktoR]

Originally Posted by wrathma

wem(un)pak

Some small bug

Found 247 .wem files to pack (recursive)
Using 12 threads

[███████████████████████████████████████░] 246/247 (99.6%) ETA: 0s
All files packed successfully!

Also it would be good to give some information on how long the process took.

Another bug in decode mode

Found 389 .ww files to unpack (recursive)
Using 12 threads

[█████████████████████████████████████████████████░] 388/389 (99.7%)
All files unpacked successfully!
[██████████████████████████████████████████████████] 389/389 (100.0%)

Testing:
Surprised how fast this is compared to WemTool

input 339mb
output 316mb

wem-packer
encode 17sec
decode 5sec


wemtool
encode 50sec
decode 16sec

Both tools are running with all cpu threads.

Edit:
Impressive

input 5,12gb
output 4,82gb

wem-packer
encode 10m30s
decode 2m40s


wemtool
encode 30m20s
decode 13min

wempacker is 3x faster than wemtool

Suggestions:
Add -c option for hdiffz compression
Change skip -s option to skip files even if they are same size as input -> equal or greater then