Go Back   FileForums > Game Backup > PC Games > PC Games - CD/DVD Conversions > Conversion Tutorials

Reply
 
Thread Tools Display Modes
  #481  
Old 18-03-2022, 03:02
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 776
Thanks: 175
Thanked 849 Times in 414 Posts
Masquerade is on a distinguished road
Quote:
Originally Posted by dixen View Post
Is it possible to implement this with DELZOREC? I mean Far Cry 5
For example

[fc5dec]
Encode=dlzo.exe u <filein>.fat <filein>.dat <fileout>.pack
Decode=dlzo.exe r <filein>.pack <fileout>.dat <fileout>.fat
Afaik no, because DELZORec requires non solid + 2 input files whereas FA can only deliver 1 input file.
Reply With Quote
Sponsored Links
  #482  
Old 20-03-2022, 17:41
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,645
Thanks: 2,026
Thanked 10,399 Times in 2,210 Posts
Razor12911 is on a distinguished road
Quote:
Originally Posted by dixen View Post
Is it possible to implement this with DELZOREC? I mean Far Cry 5
For example

[fc5dec]
Encode=dlzo.exe u <filein>.fat <filein>.dat <fileout>.pack
Decode=dlzo.exe r <filein>.pack <fileout>.dat <fileout>.fat
no, it's not possible because xtool can't be made to give two inputs nor accept two outputs
Reply With Quote
  #483  
Old 22-03-2022, 14:49
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,645
Thanks: 2,026
Thanked 10,399 Times in 2,210 Posts
Razor12911 is on a distinguished road
Update available

Changes

- generate database feature fixed
- fixed external executable support issues
- fixed lz4f level setting bug
Reply With Quote
The Following 5 Users Say Thank You to Razor12911 For This Useful Post:
elit (23-03-2022), Gehrman (22-03-2022), ScOOt3r (23-03-2022), shazzla (22-03-2022), vint56 (22-03-2022)
  #484  
Old 28-03-2022, 10:57
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,645
Thanks: 2,026
Thanked 10,399 Times in 2,210 Posts
Razor12911 is on a distinguished road
Update available

Changes

- updated oodle scanner
- updated external executable support
- updated configuration based plugin support to add depth information
- updated verbose mode

Notes

Given that very small number of people are able to make their own plugins and codecs to use in xtool, I've added an example that imports codecs from QuickBMS after you have generated your database and have no codec to use with it.

For compression:
Code:
[cpk,snappy,rfpk]
Encode=quickbms.exe -s "comtype <codec> ; clog <fileout> 0 <insize> <outsize>" "" <filein>
Decode=quickbms.exe -s "comtype <codec>_COMPRESS ; clog <fileout> 0 <insize> <outsize>" "" <filein>
For encryption:
Code:
[blowfish,xxtea]
Encode=quickbms.exe -s "open FDDE <fileres> 1 ; getdstring X_KEY <ressize> 1; encryption <codec> X_KEY "" 0 <ressize>; log <fileout> 0 <insize>" "" <filein>
Decode=quickbms.exe -s "open FDDE <fileres> 1 ; getdstring X_KEY <ressize> 1; encryption <codec> X_KEY "" 1 <ressize>; log <fileout> 0 <insize>" "" <filein>
You'll need to check http://aluigi.altervista.org/papers/quickbms.txt for the list of supported compressors built in so that you don't pick something like lz2k and expect it to work (quickbms itself has to be able to both compress and decompress). You can add more codecs like I have done here "cpk,snappy,rfpk", if you wanted to add lzss for example, it then becomes "cpk,snappy,rfpk,lzss" then from there you can use Bms2Xtl for scripts that have this comtype and possibly expect xtool to use quickbms with no real issues (hopefully).

Also note that if for whatever reason there is crc mismatch, xtool will still help you by applying xdelta automatically.

Also also note that this generate temps files meaning the process is slower because it has to both read and write to disk therefore, only use this approach if there is nothing else that you can to do to improve the situation.

Last edited by Razor12911; 28-03-2022 at 11:00.
Reply With Quote
The Following 10 Users Say Thank You to Razor12911 For This Useful Post:
Cesar82 (28-03-2022), elit (29-03-2022), Gehrman (28-03-2022), kuyhaa (28-03-2022), L33THAK0R (28-03-2022), Masquerade (28-03-2022), Pantsi (21-07-2022), ScOOt3r (29-03-2022), seryogakms (28-03-2022), shazzla (28-03-2022)
  #485  
Old 29-03-2022, 09:42
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 255
Thanks: 180
Thanked 315 Times in 113 Posts
elit is on a distinguished road
OMG ok with this I think you opened can for a whole lot of more codecs/games to be able to process.
Reply With Quote
  #486  
Old 03-04-2022, 23:57
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,645
Thanks: 2,026
Thanked 10,399 Times in 2,210 Posts
Razor12911 is on a distinguished road
Update available

Changes

- fixed issue with storing correct recompression information when stream patching is performed

Notes

This issue was brought up by Masquerade and L0v3craft when they were trying to precompress WRC 10 and upon decoding xtool would produce an error so thanks to them, it has been resolved however I'd like to add more useful information regarding this game and why even with xtool helping you with deltas will not allow you to process all streams based on how xtool was designed.

We all know that xtool patches streams if they cannot be restored correctly however it sets its own boundaries, by default the patch file cannot be 5% larger than the original size else the patch fails as highlighted here with some streams being missed. https://fileforums.com/showpost.php?...&postcount=248

This 5% can be changed by user via the command --diff=5p, with 5p being 5%, so to capture all these other streams, you'll have to increase this percentage.

CHUNK_57.PKG
using -mwrc10
Code:
Compressed 1 file, 99,435,664 => 189,185,421 bytes. Ratio 190.26%
Compression time: cpu 0.14 sec/real 3.71 sec = 4%. Speed 26.82 mB/s
using -mwrc10 --diff=20p
Code:
Compressed 1 file, 99,435,664 => 263,029,277 bytes. Ratio 264.52%
Compression time: cpu 0.09 sec/real 3.61 sec = 3%. Speed 27.53 mB/s
Verbose information displays
Code:
[0] Processing lz4f stream at 0000000000005476 (514 >> 893 >> 514) using l9:b4:d0 has failed
[0] - Patching stream at 0000000000005476 (514 >> 514) [26] has failed
Patching this 514 byte stream produced a 26 byte patch file, which is 5.1% and that is larger than the 5% limit so increasing this means you are allowing xtool to accept this stream.

--diff=20p

Code:
[0] Processing lz4f stream at 0000000000005476 (514 >> 893 >> 514) using l9:b4:d0 has failed
[0] - Patched stream at 0000000000005476 (514 >> 514) [26] successfully
Reply With Quote
The Following 8 Users Say Thank You to Razor12911 For This Useful Post:
:( Sad8669 (04-04-2022), elit (07-04-2022), ffmla (05-05-2022), Gehrman (06-04-2022), L0v3craft (04-04-2022), L33THAK0R (05-04-2022), Masquerade (04-04-2022), seryogakms (05-04-2022)
  #487  
Old 05-04-2022, 11:43
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 776
Thanks: 175
Thanked 849 Times in 414 Posts
Masquerade is on a distinguished road
LEGO® Star Wars™: The Skywalker Saga uses Kraken

Code:
XTool is created by Razor12911

Streams: 284071/284071
Time: 00:13:43 (00:48:35)
Memory: 512 MB (512 MB)

100.0%
Errorlevel=0

Compressed 1 file, 4,061,948,534 => 9,598,998,234 bytes. Ratio 236.32%
Compression time: cpu 4.22 sec/real 1032.25 sec = 0%. Speed 3.94 mB/s
All OK
Reply With Quote
The Following 3 Users Say Thank You to Masquerade For This Useful Post:
dixen (05-04-2022), Mypko (06-04-2022), ScOOt3r (05-04-2022)
  #488  
Old 05-04-2022, 12:21
GamerfromPoland GamerfromPoland is offline
Registered User
 
Join Date: Apr 2022
Location: Poland
Posts: 1
Thanks: 0
Thanked 0 Times in 0 Posts
GamerfromPoland is on a distinguished road
OMG it's a great tool! Thanks for it!
https://radaryonline.pl/gdzie-jest-burza/
Reply With Quote
  #489  
Old 27-04-2022, 18:59
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,645
Thanks: 2,026
Thanked 10,399 Times in 2,210 Posts
Razor12911 is on a distinguished road
Update available

Changes

- added IO functions (erase, replace)
- fixed external executable support bugs

Notes

Xtool has introduced IO functions, which should help you perform file and folder operations such as erase and replace (for now, more will be added). I actually wanted to add these functions a long time ago as they could be useful for repacking however I delayed them time after time because precompression needed more attention but as there have been no bug reports for the past 3 weeks I decided to start working on them.

To summarise, Erase is a feature that fills a given input with zeroes in case you wanted to repack certain data separately and Replace is a feature that replaces existing data within certain files with another.
Xtool will search for locations of the extracted streams and store these positions for when you use decode function to revert the changes (yes, the process is reversible)

Erase
Code:
xtool erase extracted_streams original_data [decode_data]
xtool decode decode_data extracted_streams original_data
Use cases for erase is the removal of language/videos files from archives in games after using another extraction tool, so rather than manually searching for positions and storing them, erase will make this process automated.

An example is after extracting video files from an archive and then wanting to remove credits video, the syntax would be

Code:
xtool erase credits.bk2 Gobi\Content\Paks\pakchunk33-WinGDK.pak credits.xtl
xtool decode credits.xtl credits.bk2 Gobi\Content\Paks\pakchunk33-WinGDK.pak
decode will fill the zeroes with the original data

Replace
Code:
xtool replace old_streams new_streams original_data [decode_data]
xtool decode decode_data extracted_streams original_data
The use cases for replace is possibly game file modification.

An example is having several modified files and the original files and you wanted to bulk replace these files, you'd have to prepare two folders with the same file structure and the syntax would be

Code:
xtool replace "textures\original\*" "textures\modified\*" "game\*" mod_tex.xtl
xtool decode mod_tex.xtl "textures\original\*" "game\*"
decode will fill the modified data sectors with the original to restore the file its original form.

PS

If you are using these features for a repack in which people would select languages or video credits for example, if the users did not select any of these to be installed then there would be missing files. Xtool's decode command would still work and will try to restore the original data using whatever data that was selected and available without problems.

Last edited by Razor12911; 27-04-2022 at 19:15.
Reply With Quote
The Following 10 Users Say Thank You to Razor12911 For This Useful Post:
:( Sad8669 (28-04-2022), Cesar82 (28-04-2022), ffmla (05-05-2022), Gehrman (28-04-2022), KaktoR (28-04-2022), L0v3craft (28-04-2022), Masquerade (27-04-2022), ScOOt3r (28-04-2022), seryogakms (29-04-2022), shazzla (27-04-2022)
  #490  
Old 13-05-2022, 02:29
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,645
Thanks: 2,026
Thanked 10,399 Times in 2,210 Posts
Razor12911 is on a distinguished road
Update available

Changes

- added IO functions (find, extract, patch)
- generate database feature and IO functions now can search for streams larger than chunk size

Notes

More IO functions introduced. Find simply helps you track positions of extracted files from a given input, while extract uses a generated decode file data input to extract streams in case if you wanted another person to extract their very own streams using your own findings and patch, well patch compares two inputs and generates a diff file which can allow you to patch an input to make it similar to the new data.

Find

Code:
xtool find [parameters] extracted_streams original_data [decode_data]
Find can be used to search for streams to then produce decode data which you can upload here on the forum in cases where you wanted to inform people what files should be extracted if they wanted for example to separate audio files per language from a game.

An example where I wanted to make a guide of how to separate english, german, italian, russian and spanish languages from the game would be something like:

Code:
xtool find "extracted\en\*" "game\*" en.xtl
xtool find "extracted\de\*" "game\*" de.xtl
xtool find "extracted\it\*" "game\*" it.xtl
xtool find "extracted\es\*" "game\*" es.xtl
xtool find "extracted\ru\*" "game\*" ru.xtl
These files are then uploaded for people if they wanted to extract the same data by themselves by using xtool via the "extract" function or for later use by yourself.

Extract

Code:
xtool extract decode_data original_data extracted_streams
Extract works mostly hand in hand with with the find function as its job is to use the decode_data to extract the data that was used for its generation.

A use case for it would be similar to the find example but from another user's perspective, so if someone uploaded decode data to use to extract my very own streams from their investigations, rather than them giving me position ranges for languages or the tools to use, we can just use their uploaded decode data to do it ourselves

Code:
xtool extract en.xtl "game\*" "extracted\en\*"
xtool extract de.xtl "game\*" "extracted\de\*"
xtool extract it.xtl "game\*" "extracted\it\*"
xtool extract es.xtl "game\*" "extracted\es\*"
xtool extract ru.xtl "game\*" "extracted\ru\*"
Then from here, we use these extracted files, to even use the erase function introduced in the version prior to blank out the sectors to prepare our own repack

Code:
xtool erase "extracted\*" "game\*" languages.xtl
which we can then restore after installation using

Code:
xtool decode languages.xtl "extracted\*" "game\*"
Patch

Code:
xtool patch [parameters] old_data new_data patch_data
xtool decode patch_data old_data
Patch's use case can be the generation of update patches in cases where you already made a repack but for whatever reason refuse to re-repack the game so instead choose to generate a patch file which can be used to update the old repack with newer files

Code:
xtool patch "Sims4\*" "Sims4_updated\*" sims4_wedding_stories.patch
Then after the installation of the original repack, you can add an additional archive that has sims4_wedding_stories.patch which will be used to update the game

Code:
xtool decode sims4_wedding_stories.patch "Sims4\*"
I noticed after compiling xtool.exe that xdelta is set to compress these patch files which means that they cannot be precompressed nor compressed with something better, however this compression in the next version will be removed.

Last edited by Razor12911; 13-05-2022 at 02:32.
Reply With Quote
The Following 5 Users Say Thank You to Razor12911 For This Useful Post:
Cesar82 (13-05-2022), dixen (13-05-2022), elit (04-07-2022), Gehrman (13-05-2022), L33THAK0R (13-05-2022)
  #491  
Old 18-05-2022, 04:19
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,645
Thanks: 2,026
Thanked 10,399 Times in 2,210 Posts
Razor12911 is on a distinguished road
Update available

Changes

- added IO functions (archive, execute)
- fixed issue in patch io function
- removed compression on patch diff files

Notes

More IO functions added. Archive behaves like -m0 but allows you to the archive to stdout and read from stdin when decoding (if you ever need that). Then there is execute which allows you execute several instances of another executable in parallel mode while all their inputs are fed from one source and all their output is fed to one destination.

Archive

Code:
xtool archive files1 files2... archive
There is nothing to add here as I have personal uses for this but the example would be

Code:
xtool archive game\* mygame.xtl
xtool decode mygame.xtl extracted\*
Execute

Code:
xtool execute [parameters] input output [exec_syntax]
Too lazy to write description but, here's an example

Code:
xtool.exe execute -c64mb -t8 UI.sb UI.bin bcm.exe -b64 [filein] [fileout]
xtool.exe decode -t100p UI.bin UI_dec.sb bcm.exe -d [filein] [fileout]
The left side of the syntax is to command xtool and after specifying input and output files, the right side begins and here you can write the command line that should be used to perform execution.

[filein], [fileout], [stdin], [stdout] can be used and denote what IO the program being executed uses.
[] is used to avoid conflicting with Freearc's <>

Freearc example would look like this

Code:
[External compressor:xbcm]
header    = 0
packcmd   = xtool.exe execute { -option} - - <stdin> <stdout> bcm.exe -b64 [filein] [fileout]
unpackcmd = xtool.exe decode -t100p - - <stdin> <stdout> bcm.exe -d [filein] [fileout]
and the method would be -mxbcm:c64mb:t75p
Reply With Quote
The Following 7 Users Say Thank You to Razor12911 For This Useful Post:
Cesar82 (18-05-2022), dixen (18-05-2022), elit (03-07-2022), Gehrman (18-05-2022), KaktoR (18-05-2022), L33THAK0R (18-05-2022), Masquerade (18-05-2022)
  #492  
Old 22-05-2022, 13:56
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,645
Thanks: 2,026
Thanked 10,399 Times in 2,210 Posts
Razor12911 is on a distinguished road
Update available

Changes

- added png stream preprocessor
- removed grittibanzli codec (since nobody uses it)

Notes

Xtool is able to process deflate/zlib streams and png images do contain these streams however, they are split up into several blocks which at times does prevent the program from being able to process them. The png stream preprocessor's job is to concatenate (rejoin all these blocks into a single stream) which can then be processed by xtool, so if you know your input contains these images then it's best to include -mpng into the method chain and use -d1 to first preprocess the streams then process them using zlib, reflate or preflate (preflate is the preferred method to use).

Results

without png preprocessor:
Code:
Compressed 1 file, 7,232,549 => 8,291,632 bytes. Ratio 114.64% -mreflate
Compressed 1 file, 7,232,549 => 7,232,655 bytes. Ratio 100.00% -mpreflate
with png preprocessor:
Code:
Compressed 1 file, 7,232,549 => 26,075,119 bytes. Ratio 360.52% -mpng+reflate -d1
Compressed 1 file, 7,232,549 => 25,289,529 bytes. Ratio 349.66% -mpng+preflate -d1
Reply With Quote
The Following 6 Users Say Thank You to Razor12911 For This Useful Post:
Cesar82 (22-05-2022), elit (03-06-2022), ffmla (28-05-2022), KaktoR (23-05-2022), L0v3craft (25-05-2022), Masquerade (01-06-2022)
  #493  
Old 22-05-2022, 18:44
Cesar82's Avatar
Cesar82 Cesar82 is offline
Registered User
 
Join Date: May 2011
Location: Brazil
Posts: 813
Thanks: 1,348
Thanked 1,749 Times in 591 Posts
Cesar82 is on a distinguished road
@Razor12911, can i use the parameters in that order in freearc commandline?
Code:
xtool:mpreflate:mpng:d1
Reply With Quote
  #494  
Old 22-05-2022, 20:09
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,645
Thanks: 2,026
Thanked 10,399 Times in 2,210 Posts
Razor12911 is on a distinguished road
yes but you'll have to change arc.ini from { -moption} to { -option}
Reply With Quote
  #495  
Old 22-05-2022, 21:16
Cesar82's Avatar
Cesar82 Cesar82 is offline
Registered User
 
Join Date: May 2011
Location: Brazil
Posts: 813
Thanks: 1,348
Thanked 1,749 Times in 591 Posts
Cesar82 is on a distinguished road
Quote:
Originally Posted by Razor12911 View Post
yes but you'll have to change arc.ini from { -moption} to { -option}
Thanks.
Yes, I use { -option} in Arc.ini of the DSG.
I tested precompressing pngs with just "xtool:mpreflate" or with "xtool:mpreflate:mpng:d1" and got similar (near)I figured I was using the command order wrong.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[Dev]XTool Razor12911 Conversion Tutorials 180 23-10-2020 06:26
Project Cars Digital Edition (3xDVD5) (srep+lzma) GTX590 PC Games - CD/DVD Conversions 10 28-08-2017 08:34
Project IGI Anthology 1xCD700 CIUV2 2039 mausschieber PC Games - CD/DVD Conversions 0 24-07-2017 15:12
Space Channel 5 Part 2 Translation Project Christuserloeser DC Games 0 21-06-2004 18:16



All times are GMT -7. The time now is 16:40.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Copyright 2000-2020, FileForums @ https://fileforums.com