Go Back   FileForums > Game Backup > PC Games > PC Games - CD/DVD Conversions > Conversion Tutorials

Reply
 
Thread Tools Display Modes
  #331  
Old 14-12-2022, 12:41
Razor12911's Avatar
Razor12911 Razor12911 is offline
Noob
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,746
Thanks: 2,141
Thanked 11,084 Times in 2,295 Posts
Razor12911 is on a distinguished road
Quote:
Originally Posted by Mini View Post
I managed to create plugin, decompressing the game is possible but recompressing with matching crc is a different story.

Code:
Compressed 1 file, 60,893,070 => 203,772,032 bytes. Ratio 334.64%
Compression time: cpu 0.02 sec/real 1.17 sec = 1%. Speed 52.01 mB/s
Code:
"B.AnimRetarget.cache". File is broken.
Tested 1 file, 203,772,032 => 60,893,070 bytes. Ratio 334.64%
Testing time: cpu 0.09 sec/real 1.06 sec = 9%. Speed 57.33 mB/s
There were 1 warning(s)
Attached Files
File Type: 7z lzfx.7z (38.3 KB, 21 views)

Last edited by Razor12911; 14-12-2022 at 12:43.
Reply With Quote
The Following 4 Users Say Thank You to Razor12911 For This Useful Post:
Gehrman (14-12-2022), Mini (14-12-2022), ScOOt3r (14-08-2023)
Sponsored Links
  #332  
Old 14-12-2022, 18:39
Razor12911's Avatar
Razor12911 Razor12911 is offline
Noob
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,746
Thanks: 2,141
Thanked 11,084 Times in 2,295 Posts
Razor12911 is on a distinguished road
Here's an alternative doboz library plugin, it may have bugs as this is the first xtool plugin written in Visual Studio rather than RAD Studio.

via executable
Code:
Compressed 1 file, 286,247,872 => 1,095,095,861 bytes. Ratio 382.57%
Compression time: cpu 0.20 sec/real 53.75 sec = 0%. Speed 5.33 mB/s

Tested 1 file, 1,095,095,861 => 286,247,872 bytes. Ratio 382.57%
Testing time: cpu 0.22 sec/real 33.81 sec = 1%. Speed 8.47 mB/s
via library
Code:
Compressed 1 file, 286,247,872 => 1,095,095,861 bytes. Ratio 382.57%
Compression time: cpu 0.23 sec/real 2.62 sec = 9%. Speed 109.33 mB/s

Tested 1 file, 1,095,095,861 => 286,247,872 bytes. Ratio 382.57%
Testing time: cpu 0.17 sec/real 27.38 sec = 1%. Speed 10.45 mB/s
Refer to the executable version if you have issues which can be found here.
Attached Files
File Type: 7z doboz_lib.7z (40.5 KB, 33 views)
File Type: 7z doboz_src.7z (12.1 KB, 21 views)

Last edited by Razor12911; 14-12-2022 at 18:43.
Reply With Quote
The Following 2 Users Say Thank You to Razor12911 For This Useful Post:
Gehrman (14-12-2022), L0v3craft (14-12-2022)
  #333  
Old 15-12-2022, 16:38
L33THAK0R's Avatar
L33THAK0R L33THAK0R is offline
Registered User
 
Join Date: Feb 2021
Location: Saudi Arabia
Posts: 344
Thanks: 117
Thanked 57 Times in 37 Posts
L33THAK0R is on a distinguished road
Quote:
Originally Posted by Razor12911 View Post
Here's an alternative doboz library plugin, it may have bugs as this is the first xtool plugin written in Visual Studio rather than RAD Studio.
Do any titles other than "The Witcher 3: Wild Hunt" or titles running on the PS Vita use Doboz? Only done a quick skim through, but cant seem to find many cases.
Reply With Quote
  #334  
Old 20-12-2022, 04:03
L33THAK0R's Avatar
L33THAK0R L33THAK0R is offline
Registered User
 
Join Date: Feb 2021
Location: Saudi Arabia
Posts: 344
Thanks: 117
Thanked 57 Times in 37 Posts
L33THAK0R is on a distinguished road
Quote:
Originally Posted by Razor12911 View Post
I managed to create plugin, decompressing the game is possible but recompressing with matching crc is a different story.

Code:
Compressed 1 file, 60,893,070 => 203,772,032 bytes. Ratio 334.64%
Compression time: cpu 0.02 sec/real 1.17 sec = 1%. Speed 52.01 mB/s
Code:
"B.AnimRetarget.cache". File is broken.
Tested 1 file, 203,772,032 => 60,893,070 bytes. Ratio 334.64%
Testing time: cpu 0.09 sec/real 1.06 sec = 9%. Speed 57.33 mB/s
There were 1 warning(s)
Bit of a silly sounding question but how do you use this plugin? Does xtool.ini need to be modified for this library to be accepted? Interested in seeing if despite the CRC issues anything can be done with the decompressed data either as it is or via an xdelta/hdiffz patch.
Reply With Quote
  #335  
Old 20-12-2022, 07:33
Razor12911's Avatar
Razor12911 Razor12911 is offline
Noob
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,746
Thanks: 2,141
Thanked 11,084 Times in 2,295 Posts
Razor12911 is on a distinguished road
Quote:
Originally Posted by L33THAK0R View Post
Do any titles other than "The Witcher 3: Wild Hunt" or titles running on the PS Vita use Doboz? Only done a quick skim through, but cant seem to find many cases.
I don't know either

Quote:
Originally Posted by L33THAK0R View Post
Bit of a silly sounding question but how do you use this plugin? Does xtool.ini need to be modified for this library to be accepted? Interested in seeing if despite the CRC issues anything can be done with the decompressed data either as it is or via an xdelta/hdiffz patch.
The sample he sent over has a bms script, meaning you first need to use bms2xtl, the comtype used in the game is evolution/lzfx so I just make a plugin that acts a evolution precompressor, however no matter what parameters you use for lzfx, you simply cannot recompress the data with matching crc. I revisited the data set again today after updating the library to patch the streams but the diff files are just too big and wouldn't make sense to precompress this game.

Last edited by Razor12911; 20-12-2022 at 07:35.
Reply With Quote
  #336  
Old 20-12-2022, 08:04
KaktoR's Avatar
KaktoR KaktoR is offline
Lame User
 
Join Date: Jan 2012
Location: From outer space
Posts: 4,250
Thanks: 1,069
Thanked 6,831 Times in 2,572 Posts
KaktoR is on a distinguished road
LEGO Star Wars - The Skywalker Saga

My first xtool plugin.

GAME6.DAT
Code:
-mxtool:legoswtws
Compressed 1 file, 1,643,604,387 => 3,766,491,875 bytes. Ratio 229.16%
Compression time: cpu 1.22 sec/real 82.54 sec = 1%. Speed 19.91 mB/s
All OK

-mxtool:kraken,l6
Compressed 1 file, 1,643,604,387 => 3,629,442,699 bytes. Ratio 220.82%
Compression time: cpu 1.23 sec/real 92.98 sec = 1%. Speed 17.68 mB/s
All OK

-moo2reck
Compressed 1 file, 1,643,604,387 => 3,696,758,986 bytes. Ratio 224.92%
Compression time: cpu 1.80 sec/real 434.00 sec = 0%. Speed 3.79 mB/s
All OK
GAME.DAT
Code:
-mxtool:legoswtws
Compressed 1 file, 4,062,980,016 => 9,814,174,667 bytes. Ratio 241.55%
Compression time: cpu 3.77 sec/real 229.08 sec = 2%. Speed 17.74 mB/s
All OK

-mxtool:kraken,l6
Compressed 1 file, 4,062,980,016 => 9,451,336,766 bytes. Ratio 232.62%
Compression time: cpu 3.39 sec/real 246.06 sec = 1%. Speed 16.51 mB/s
All OK

-moo2reck
Compressed 1 file, 4,062,980,016 => 9,595,617,928 bytes. Ratio 236.17%
Compression time: cpu 5.73 sec/real 1036.90 sec = 1%. Speed 3.92 mB/s
All OK

Thanks to Razor12911 for the education and sorry for the headaches
Attached Files
File Type: 7z legoswtws.7z (299 Bytes, 31 views)
__________________
Haters gonna hate

Last edited by KaktoR; 20-12-2022 at 08:12.
Reply With Quote
The Following 9 Users Say Thank You to KaktoR For This Useful Post:
Cesar82 (24-12-2022), dixen (20-12-2022), Gehrman (20-12-2022), kuyhaa (24-12-2022), L0v3craft (07-05-2023), L33THAK0R (20-12-2022), Razor12911 (20-12-2022), ScOOt3r (20-12-2022), shazzla (20-12-2022)
  #337  
Old 01-01-2023, 09:27
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 1,123
Thanks: 284
Thanked 1,307 Times in 595 Posts
Masquerade is on a distinguished road
Watch Dogs [XCompress]
  • Extract the contents of the attached archive to XTool folder and use -mwatchdogs.
  • Requires XCompress plugin, you can get that here.
  • Previous database shared by Razor12911 does not work with current XTool which is why I am sharing a new database.
Code:
Compressing common.dat, 109,476,518 bytes

Compressed 1 file, 109,476,518 => 405,191,786 bytes. Ratio 370.12%
Compression time: cpu 0.11 sec/real 20.00 sec = 1%. Speed 5.47 mB/s
All OK
Attached Files
File Type: 7z WatchDogs.7z (2.97 MB, 88 views)
Reply With Quote
The Following 7 Users Say Thank You to Masquerade For This Useful Post:
Cesar82 (01-01-2023), dixen (01-01-2023), L0v3craft (02-01-2023), L33THAK0R (01-01-2023), Razor12911 (01-01-2023), ScOOt3r (01-01-2023), shazzla (01-01-2023)
  #338  
Old 19-01-2023, 12:01
Razor12911's Avatar
Razor12911 Razor12911 is offline
Noob
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,746
Thanks: 2,141
Thanked 11,084 Times in 2,295 Posts
Razor12911 is on a distinguished road
Crilayla

Supported games
Japanese games

Usage
Place crilayla.ini near xtool.exe in conjunction with executable version of the plugin and use -mcrilayla

Persona 3 Portable\data_EN\umd0.cpk
Code:
Compressed 1 file, 147,942,064 => 845,153,964 bytes. Ratio 571.27%
Compression time: cpu 0.14 sec/real 41.55 sec = 0%. Speed 3.56 mB/s
Attached Files
File Type: 7z crilayla_R1.7z (275 Bytes, 28 views)
Reply With Quote
The Following 8 Users Say Thank You to Razor12911 For This Useful Post:
BKR-TN (22-01-2023), Cesar82 (19-01-2023), Gehrman (20-01-2023), hdneo (20-01-2023), L33THAK0R (20-01-2023), Masquerade (19-01-2023), ScOOt3r (19-01-2023), shazzla (19-01-2023)
  #339  
Old 21-01-2023, 06:48
deepshit deepshit is offline
Registered User
 
Join Date: Dec 2013
Location: Abroad
Posts: 7
Thanks: 6
Thanked 3 Times in 3 Posts
deepshit is on a distinguished road
Hey There
Can somebody help me with NFS Shift conversion?

Game uses xmemcompress. I used the bms script to generate xtl db but looks like xtool cant decompress the file.
I also put the required libraries beside xtool.( exe and dll)
sample and script and db are attached.
Attached Files
File Type: 7z SetupPackTest.7z (18.69 MB, 17 views)

Last edited by deepshit; 21-01-2023 at 06:50.
Reply With Quote
  #340  
Old 21-01-2023, 09:52
Razor12911's Avatar
Razor12911 Razor12911 is offline
Noob
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,746
Thanks: 2,141
Thanked 11,084 Times in 2,295 Posts
Razor12911 is on a distinguished road
Quote:
Originally Posted by deepshit View Post
Hey There
Can somebody help me with NFS Shift conversion?

Game uses xmemcompress. I used the bms script to generate xtl db but looks like xtool cant decompress the file.
I also put the required libraries beside xtool.( exe and dll)
sample and script and db are attached.
Refer to the Project Cars 1 configuration by Masquerade for this game, xmemcompress must have these parameters.

Code:
[xmemcompress]
Encode=xcompress.exe d -window=131072 -partition=524288 <library>
Decode=xcompress.exe e -window=131072 -partition=524288 <library>
Code:
Compressed 1 file, 25,522,432 => 84,558,796 bytes. Ratio 331.31%
Compression time: cpu 0.05 sec/real 4.22 sec = 1%. Speed 6.05 mB/s
Reply With Quote
The Following 5 Users Say Thank You to Razor12911 For This Useful Post:
BKR-TN (22-01-2023), Cesar82 (21-01-2023), deepshit (21-01-2023), Gehrman (28-02-2023), ScOOt3r (22-01-2023)
  #341  
Old 17-02-2023, 09:34
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 1,123
Thanks: 284
Thanked 1,307 Times in 595 Posts
Masquerade is on a distinguished road
Bink Video
  • XTool plugin adding support for Bink Video compression.
  • This plugin uses Bink Pack library by ProFrager.
  • Thanks to Razor12911 for the little help.
Advantages of using this plugin versus vanilla Bink Pack
  • Multithreading support.
  • No CRC check fail since XTool will patch the videos as they are unpacked.
  • Incompatible Bink 2 videos (KB2n) are automatically ignored.
Special Notes
  • For Bink 2 videos, you may need to make use of the --diff=* parameter as some patches between CRC imperfect videos to the original state can be larger than XTool's default threshold for patches.
  • You may need to use a large chunk size to ensure all videos are processed. See example below.
Example Usage
  • Extract the attached archive to the XTool folder and use -mbik or -mbk2 depending on your data set.
Test
  • Test on the videos from Assassin's Creed 2 (27x Bink 1 videos).
    Code:
    xtool precomp -mbik "AC2Videos\"
    XTool is created by Razor12911
    
    Streams: 13/13
    Time: 00:00:09 (00:00:01)
    Memory: 148 MB (148 MB)
    
    Results: 937 MB >> 928 MB
    Code:
    xtool precomp -c128mb -mbik "AC2Videos\"
    XTool is created by Razor12911
    
    Streams: 26/26
    Time: 00:02:12 (00:00:04)
    Memory: 391 MB (391 MB)
    
    Results: 937 MB >> 776 MB
    Code:
    xtool precomp -c256mb -mbik "AC2Videos\"
    XTool is created by Razor12911
    
    Streams: 27/27
    Time: 00:02:25 (00:00:06)
    Memory: 677 MB (677 MB)
    
    Results: 937 MB >> 725 MB
  • Test on 05_DAM folder from Shadow Warrior 3 Defintive Edition (6x Bink 2 videos).
    Code:
    xtool precomp -c256mb -mbk2 "SW3\Content\Movies\Cutscenes\05_DAM"
    XTool is created by Razor12911
    
    Streams: 6/6
    Time: 00:01:06 (00:00:03)
    Memory: 406 MB (406 MB)
    
    Results: 214 MB >> 187 MB
Attached Files
File Type: 7z XTool_Bink.7z (199.8 KB, 111 views)
File Type: 7z XTool_Bink_STDIO.7z (256.1 KB, 97 views)

Last edited by Masquerade; 06-03-2023 at 01:34.
Reply With Quote
The Following 10 Users Say Thank You to Masquerade For This Useful Post:
Cesar82 (17-02-2023), dixen (17-02-2023), Gehrman (18-02-2023), hdneo (17-02-2023), KaktoR (17-02-2023), kj911 (17-02-2023), L0v3craft (14-03-2023), Razor12911 (17-02-2023), ScOOt3r (14-05-2023), shazzla (17-02-2023)
  #342  
Old 17-02-2023, 19:36
Cesar82's Avatar
Cesar82 Cesar82 is offline
Registered User
 
Join Date: May 2011
Location: Brazil
Posts: 970
Thanks: 1,660
Thanked 2,114 Times in 713 Posts
Cesar82 is on a distinguished road
Quote:
Originally Posted by Masquerade View Post
For Bink 2 videos, you may need to make use of the --diff=* parameter as some patches between CRC imperfect videos to the original state can be larger than XTool's default threshold for patches.

For me if I use --diff=* in arc.ini it doesn't work, return:
ERROR: write error (disk full?) in compression algorithm xtool:c256mb:mbk2
Code:
Arc.ini

--cache=0
[External compressor:xtool]
header    = 0
default   = -c64mb -t100p
packcmd   = "{compressor}.exe" precomp { -option} --diff=* - - <stdin> <stdout>
unpackcmd = "{compressor}.exe" decode -t100p - - <stdin> <stdout>

pack.bat
arc.exe a -ep1 -dses --dirs -s; -lc- -di -i2 -r -w.\temp -m"xtool:c256mb:mbk2" data.arc "pack\*"
I also tested using it directly from the command line after removing --diff=* from arc.ini:
xtool:c256mb:mbk2:-diff=*
xtool:c256mb:mbk2:df*

Code:
Arc.ini

--cache=0
[External compressor:xtool]
header    = 0
default   = -c64mb -t100p
packcmd   = "{compressor}.exe" precomp { -option} - - <stdin> <stdout>
unpackcmd = "{compressor}.exe" decode -t100p - - <stdin> <stdout>

pack.bat
arc.exe a -ep1 -dses --dirs -s; -lc- -di -i2 -r -w.\temp -m"xtool:c256mb:mbk2:df*" data.arc "pack\*"
or
arc.exe a -ep1 -dses --dirs -s; -lc- -di -i2 -r -w.\temp -m"xtool:c256mb:mbk2:-diff=*" data.arc "pack\*"
If I don't use --diff=* it compresses normally.
Code:
FreeArc 0.67 (March 15 2014) using additional options: --cache=0
Creating archive: data.arc using xtool:c256mb:mbk2
Memory for compression 0b, decompression 0b, cache 0b
Compressing 5 files, 336,582,396 bytes
  Compressing health_screen.bk2
  Compressing ms_logo.bk2
  Compressing ps_studios_long_strawberry_4k30_006.bk2
  Compressing lt_outro_1080p.bk2
  Compressing lt_intro_1080p.bk2
Compressed 5 files, 336,582,396 => 291,316,936 bytes. Ratio 86.55%
Compression time: cpu 0.20 sec/real 29.95 sec = 1%. Speed 11.24 mB/s
All OK
Am I doing something wrong?
Reply With Quote
  #343  
Old 17-02-2023, 23:48
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 1,123
Thanks: 284
Thanked 1,307 Times in 595 Posts
Masquerade is on a distinguished road
Quote:
Originally Posted by Cesar82 View Post
Am I doing something wrong?
The --diff parameter in XTool sets a max size in bytes or percentage size of the stream in order for XTool to accept the stream.

The * character is just a placeholder (a wildcard effectively meaning anything/everything). My explanation meant you'd need to change this to an integer.

By default, XTool will not accept a stream if the generated XDelta patch is greater in size than 5% of the stream, but using --diff= allows you to increase this threshold.

You can use --diff=20p to set the max patch size to be less than or equal to 20% of the stream size for XTool to accept the stream. You can also use an integer to specify a max patch size in bytes (e.g. --diff=50000).
Reply With Quote
The Following User Says Thank You to Masquerade For This Useful Post:
Cesar82 (18-02-2023)
  #344  
Old 18-02-2023, 00:15
dixen dixen is offline
Registered User
 
Join Date: Sep 2014
Location: Russia
Posts: 383
Thanks: 438
Thanked 428 Times in 196 Posts
dixen is on a distinguished road
XCOM Enemy Unknown

*.bik *.bk2 (With XTool 0.69 + Bink Plugin)

Quote:
FreeArc 0.67 (March 15 2014) testing archive: data.arc
Tested 168 files, 6,433,251,603 => 7,859,788,952 bytes. Ratio 81.85%
Testing time: cpu 8.44 sec/real 344.19 sec = 2%. Speed 22.84 mB/s
All OK

Last edited by dixen; 18-02-2023 at 05:19.
Reply With Quote
The Following 2 Users Say Thank You to dixen For This Useful Post:
Gehrman (18-02-2023), L0v3craft (14-03-2023)
  #345  
Old 22-02-2023, 00:05
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 1,123
Thanks: 284
Thanked 1,307 Times in 595 Posts
Masquerade is on a distinguished road
Labyrinth of Galleria: The Moon Society [LZ4]
  • Requires LZ4.DLL from the game files. Rename it to liblz4.dll for use with XTool.
  • Drop the configuration into XTool folder and use -mLoGTMS
By Razor12911
Attached Files
File Type: 7z LoGTMS.7z (301 Bytes, 5 views)
Reply With Quote
The Following User Says Thank You to Masquerade For This Useful Post:
Razor12911 (22-02-2023)
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
XTool 2020 (Main Project) Razor12911 Conversion Tutorials 681 28-04-2024 05:19
XTool - Successor of ZTool Razor12911 Conversion Tutorials 584 22-04-2023 18:16
[Dev]XTool Razor12911 Conversion Tutorials 180 23-10-2020 06:26
XTool 2019 (Plugins) Razor12911 Conversion Tutorials 50 16-05-2020 06:14



All times are GMT -7. The time now is 21:35.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.
Copyright 2000-2020, FileForums @ https://fileforums.com