XTool 2020 (Plugins)

[kuyhaa]

Originally Posted by Masquerade

In that case, you will need to make a new database.

You will need VGMToolbox, which you can download here.

Copy this text and save it as PPKG.XML inside the "plugins\AdvancedCutter" folder.

Open the tool and go to Misc Tools ---> Extraction Tools ---> Generic ---> Advanced Cutter / Offset Finder.

In the presets drop down box, select PPKG from the list and then click "Load".

Then, drag the three PKG files into the drag'n'drop box to the left of the preset box.

After all the streams have been extracted, rename the extracted folders to remove the _CUT suffix.

To generate the database, just open a command prompt window in the folder containing the pkg files and do:

for %f in (*.PKG) do xtool generate -c128mb -t100p -mlz4f "%~nf" "%f" TTIoM3.xtl

Now you can delete the folders from VGMToolbox.

works, thanks

Streams: 19102 / 19113
Time: 00:02:28 (CPU 00:00:55)

Size: 8.23 GB >> 13.4 GB

[soulfit15]

looks like a bug in xtool GUI with unreal plugin

XTool is created by Razor12911

Streams: 1430889 / 1675133
Time: 00:21:48 (CPU 00:16:56)

Size: 5.25 GB >> 5.03 GB

unpacking a game with ue5:key and it gets stuck "used the -d1" and cpu keeps spooling at 16%

[Masquerade]

Orcs Must Die! 3 database updated for game version Build 11248349 / v1.2.1.0 (June 5th 2023).

[kj911]

Originally Posted by Masquerade

Bink Video

• Extract the attached archive to the XTool folder and use -mbik or -mbk2 depending on your data set.

Whats wrong or am i lame?? (From testing. See two attached pictures.)

Directly use Binkpack.exe or Freearc usaged whole Bink*.exe file and or CLS-based (de)compressors its works in (de)compress in vice versa... Packed 3.07GB huge BIK file correctly to 2.4GB size with Binkpack.
Use Xtool 0.71 directly or Arc with configurations, don't compress smaller.

[KaktoR]

Letting you know here that the playstation archive plugin doesn't work with xtool 0.7.2(HF) on decompression step.

It works with xtool 0.7.1

[Masquerade]

[Proof of Concept / Work In Progress] Yakuza SLLZ v1

This is a little project I've been working on in an attempt to implement the SLLZ algorithm found in Yakuza games as an XTool plugin. The code used for compression and decompression was written by Kaplas80. The attempt was to make it into a standalone executable. I wanted to share it here despite being unfinished. Requires .NET Framework v4.8.

There are a few problems with the plugin in its current state. Decompression of sllz streams seems to work completely fine. However, compression is slow and is missing the 2x 4 bytes of header information containing the compressed size and decompressed size (because for some reason Visual Studio insists on this being 8 byte rather than 4 byte arrays). They are filled as 00 bytes instead for the time being. The compressor is also bugged in other ways when I tested on samples from Yakuza Kiwami. Reading and writing to stdin/stdout hasn't been added yet so it's even slower since data needs to be extracted to disk.

Here's some information about this algorithm from its header:

First 4 bytes, Magic: SLLZ
Next byte, endianness: 00 (little)
Next byte, version: 01 or 02 (compression version - v2 is zlib I think)
Next 2 bytes is header size (from what I've seen, always 0x1000 because the header is always 16 bytes in my findings)
Next 4 bytes is decompressed size.
Next 4 bytes is compressed size.

Total: 16 bytes

[Stream1]
Name=yakuza
Codec=sllz
BigEndian=0
Signature=0x5A4C4C53
Structure=Signature(4),Endianness(1),Version(1),HeaderSize(2),CSize(4),DSize(4),Stream
StreamOffset=-16
CompressedSize=CSize
DecompressedSize=DSize

xtool precomp -myakuza -df20p -c128mb -t12 hact.par
XTool is created by Razor12911

Streams: 873 / 3397
Time: 00:11:34 (CPU 00:00:17)

Size: 398 MB >> 404 MB

I will probably drop this project for a bit and return to it sometime in the future

[dixen]

Witcher 3: Complete

Used xtool v0.7.3 hotfix

-mzlib+reflate

Streams: 2786 / 2786
Time: 00:01:17 (CPU 00:04:24)
Duplicates: 300 (1.58 MB) [1.80 MB >> 19.1 MB]
Srep decompression memory: 3.00 MB [6.34 MB*]

Size: 508 MB >> 658 MB >> 638 MB >> 607 MB >> 400 MB

100%
Errorlevel=0

Compressed 1 file, 532,191,280 => 419,520,814 bytes. Ratio 78.83%
Compression time: cpu 0.53 sec/real 79.37 sec = 1%. Speed 6.71 mB/s
All OK

-mw3+zlib+reflate

Streams: 3490 / 3854
Time: 00:01:35 (CPU 00:05:40)
Duplicates: 476 (2.20 MB) [3.18 MB >> 226 MB]
Srep decompression memory: 7.00 MB [23.1 MB*]

Size: 508 MB >> 1.12 GB >> 916 MB >> 814 MB >> 303 MB

100%
Errorlevel=0

Compressed 1 file, 532,191,280 => 318,049,218 bytes. Ratio 59.76%
Compression time: cpu 0.47 sec/real 97.06 sec = 0%. Speed 5.48 mB/s
All OK

Needed libliz4.dll (v1.9.3) and doboz.dll near xtool.exe

[Razor12911]

Plugin update available

Changes

- Improved stream detection in unreal engine plugin's ue5 codec (ucas/utoc format)

Notes

I just wished the format of unreal engine file system would stop changing because it's a headache trying to write one library that supports all current and future games and even this plugin has a caveat, mainly the first and last stream left behind and not decrypted because if I tried to grab them using the current method that would mean false positives, I've left R9 just in case R10's detection is worse but good luck.

pakchunk0optional-Windows.ucas (I don't know what game is this but it was a sample I was sent a while ago)

Unreal Engine ucas database maker doesn't work on this file... because they changed they format...

-mue5:k0xCD353CBF6E250AF8ADF4B2B0F79649A27D2BC743AB0AAAEB202DEF7A7C19B2DB+kraken -d1

R9

Tested 1 file, 437,937,960 => 290,461,744 bytes. Ratio 150.77%
Testing time: cpu 0.19 sec/real 11.54 sec = 2%. Speed 25.17 mB/s

R10

Tested 1 file, 578,518,003 => 290,461,744 bytes. Ratio 199.17%
Testing time: cpu 0.25 sec/real 16.30 sec = 2%. Speed 17.82 mB/s

[Masquerade]

The Legend of Heroes Trails into Reverie

• Extract the below archive into XTool folder and use -mReverie.
• This database is for assets.pka. The other pkg files are not compressed.
• Requires liblz4.dll. I included one that gave a good ratio.
• You can discard Reverie.xtl for decoding, but liblz4.dll must be retained.

This database took 13 hours to generate

[Razor12911]

Plugin update available

Changes

- Improved stream detection in unreal engine plugin's ue5 codec (ucas/utoc format)

Notes

Using ue5 codec even on pak files yielded streams which should not have been detected meaning ratio is affected one way or another and this release tackles false positive detections.

[Razor12911]

Plugin update available

Changes

- ue3 codec no longer requires m# parameter to work
- ue3 codec supports lzopro streams (requires lzopro executable plugin)
- ue4 codec speed improvement
- ue4 codec improved detection
- ue5 codec minor bug fixes

Notes

This update is primarily made for the next xtool release (0.7.7) but it should work fine for older versions.

Link

[Razor12911]

Insomniac Engine

Supported games
Marvel's Spider-Man Remastered
Marvel's Spider-Man: Miles Morales
Ratchet & Clank: Rift Apart

Usage
Place insomniac.ini near xtool.exe along with required libraries and use -minsomniac

Notes
Tested on small sample provided by dixen for testing, not sure if it works for the whole game. This is the first advanced configuration plugin so also not sure if this feature in xtool works without bugs.

[Cesar82]

@Razor12911, Thanks for the updates.
As always, once again I fill the topic of questions, but here we go..

1) Can I continue to use the m1, m2, and m3 parameters so the DiskSpan GUI knows what to copy into the xtool folder?

Unreal_Engine3_MK11;         Method: xtool:mue3,m1;
Unreal_Engine3_MELE;         Method: xtool:mue3,m2;
Unreal_Engine3_LZO;          Method: xtool:mue3,m3;

2) Could you share which library and version is used for each of the 3 games supported by the new "Insomniac Engine" plugin?

[Razor12911]

Plugin update available

Changes

- Added support for DirectStorage (gdeflate) streams in Insomniac plugin

Notes

Turns out, not only lz4 was used in this game, after investigations made by Fitgirl, she pointed out the plugin was not working in the initial release and after a few mins of research I managed to create a plugin (gdeflate) to add support for the other streams compressed using the Direct Storage technology and as a result of that, to precompress Ratchet & Clank: Rift Apart, you need dstorage.dll, dstoragecore.dll, gdeflate.dll and liblz4.dll.

tex_char_ratchet0 (sample compressed with gdeflate)
-mxtool:insomniac:rcrads

Compressed 1 file, 135,778,928 => 325,814,638 bytes. Ratio 239.96%
Compression time: cpu 0.11 sec/real 1.36 sec = 8%. Speed 99.86 mB/s

IMPORTANT
With the use of DirectStorage, compression and/or installation will only work on Windows 10 or later, Anything older will not work.

@Cesar82

1) Yes
2) It's the same one that comes with xtool (v1.8.3), a lot of games use this version.

[dixen]

Decompression results

Insomniac R1
39 gb > 48 gb

Insomniac R2
39 gb > 63 gb