Go Back   FileForums > Game Backup > PC Games > PC Games - CD/DVD Conversions > Conversion Tutorials

Reply
 
Thread Tools Display Modes
  #1  
Old 13-01-2018, 15:07
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 244
Thanks: 176
Thanked 306 Times in 106 Posts
elit is on a distinguished road
LZ_Scan

LZ_Scan is a small scripted utility that consist of several tools and is primarily based on Razor's xtool. It is a reliable alternative to GFS(Game File Scan). Output size difference in this tool will be exactly what you get when using xtool on game data with same settings. It is also more stable than GFS. I posted it once somewhere, now I decided to update script and post it in its own thread. Supported formats are same as: zlib/deflate, lz4, lzo, zstd, crilayla as well as unreal and unity engines.

Usage(for 'other codecs'):
Run .bat file, type one of available codecs(z-xxx is based on ztool, x-xxx is based on xtool), enter extra parameters or skip with ENTER, enter file or directory path to scan, wait for finish, compare size differences.
Remember you can use file masks to scan only specific files, eg: "C:\gamefolder\*.pak"

EDIT2:
New update. Include uelr(Unreal decmp tool) and AFR(Anvil forge decmp tool). These however write temporarily to disk within lz_scan directory(therefore consider where to place it). AFR only need input file as output is written into NULL, because it display all necessary info on its own. Uelr need both input and output tmp file for lz_scan to be able to measure size differences. Temp files are automatically deleted.

EDIT3:
Updated to include latest version of xtool(v0.4.4). This version contain additional batch scripts you can use to test lz4, zstd and oodle on all levels, methods and libraries included.
For example:
Code:
test_lz4.bat file.ext
In case of oodle, all methods supported by xtool(kraken, selkie, mermaid, lzna, hydra and leviathan) will be tested on all included libraries.
In case of zstd, all libs will be tested as well.
In case of lz4, again all libs will be tested in both standard and lz4hc mode.

Additional libraries can be copied and added to batch script. Included ones are all that were available on github in compiled dll form(for both lz4 and zstd).
In fact, with these codecs this is now the recommended method to find out compression, default .bat is mainly for zlib/deflate/crilayla or specific engines.

EDIT4:
New and rewamped version! Lot of changes. This version make it easier to figure out compression with support of deep scan based on xtool's 'verbose' command and default scanner. Contain ALL available zstd and lz4 libraries(I compiled missing ones myself) from github to this day! Readme.txt will keep track of version and info, I decided to reset this to v1.0 despite previously releasing higher number for many reasons.

EDIT5:
New version 1.1, fixes x-lzo in cli script and add 'deepscan' script for it.

lz_scan_v1.1.7z

Last edited by elit; 18-03-2022 at 04:32.
Reply With Quote
The Following 22 Users Say Thank You to elit For This Useful Post:
-NORO- (14-01-2018), 78372 (13-01-2018), BLACKFIRE69 (12-09-2019), Cesar82 (17-03-2022), COPyCAT (24-01-2018), dixen (07-07-2019), doofoo24 (13-01-2018), EzzEldin16 (14-01-2018), ffmla (15-01-2018), Jiva newstone (13-01-2018), KaktoR (15-02-2018), L0v3craft (30-08-2018), MagixZ (06-08-2019), nickwazhere2012 (14-01-2018), oltjon (13-01-2018), PsYcHo_RaGE (09-07-2019), rambohazard (13-01-2018), ramydarkness (19-03-2022), ScOOt3r (13-03-2022), shazzla (14-01-2018), Simorq (13-01-2018), vint56 (14-01-2018)
Sponsored Links
  #2  
Old 16-01-2018, 14:34
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 244
Thanks: 176
Thanked 306 Times in 106 Posts
elit is on a distinguished road
Replace start.bat with this one, its better.

[Removed = obsolete.]

Last edited by elit; 06-07-2019 at 06:28.
Reply With Quote
The Following 5 Users Say Thank You to elit For This Useful Post:
78372 (17-01-2018), COPyCAT (24-01-2018), EzzEldin16 (17-01-2018), oltjon (17-01-2018), Simorq (16-01-2018)
  #3  
Old 17-01-2018, 12:28
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 244
Thanks: 176
Thanked 306 Times in 106 Posts
elit is on a distinguished road
You start with "start.bat". Then:
procedure.png
- First you type one of the available methods, like I did "pzlib:high" for this game file as example.
- Then path to file or directory as shown(use quotes for paths with spaces, also tab completition works just like in shell here so you dont have to type everything).
- Then you wait and when done you compare 2 size differences as shown in picture example. If second number is bigger it mean inflation worked for this. If same or too close you either got wrong method, game is not compressed or is not supported by ztool.
- You can also see overall time it took in seconds as below.
- Finally you can start all over again or quit.

Last edited by elit; 17-01-2018 at 12:30.
Reply With Quote
The Following 2 Users Say Thank You to elit For This Useful Post:
COPyCAT (24-01-2018), EzzEldin16 (18-01-2018)
  #4  
Old 12-02-2018, 01:12
ZakirAhmad ZakirAhmad is offline
Registered User
 
Join Date: Sep 2015
Location: Kmr
Posts: 290
Thanks: 56
Thanked 54 Times in 37 Posts
ZakirAhmad is on a distinguished road
Can we mofify it for afr, uelr etc.
Reply With Quote
  #5  
Old 14-02-2018, 06:14
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 244
Thanks: 176
Thanked 306 Times in 106 Posts
elit is on a distinguished road
Quote:
Originally Posted by ZakirAhmad View Post
Can we mofify it for afr, uelr etc.
I want to include them as well, when I have more time - in next version. This will also make it more universal not just for ztool.
Reply With Quote
The Following 3 Users Say Thank You to elit For This Useful Post:
oltjon (14-02-2018), Simorq (14-02-2018), ZakirAhmad (14-02-2018)
  #6  
Old 14-02-2018, 08:41
ZakirAhmad ZakirAhmad is offline
Registered User
 
Join Date: Sep 2015
Location: Kmr
Posts: 290
Thanks: 56
Thanked 54 Times in 37 Posts
ZakirAhmad is on a distinguished road
Quote:
Originally Posted by elit View Post
I want to include them as well, when I have more time - in next version. This will also make it more universal not just for ztool.
this would be great step.
Reply With Quote
  #7  
Old 14-02-2018, 18:28
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 244
Thanks: 176
Thanked 306 Times in 106 Posts
elit is on a distinguished road
Unfortunately I found that neither uelr nor afr support stdio/pipelining, which without further coding is impossible to implement AFAIK.
EDIT: I am working on it already, it will be done but will use disk writes.

Last edited by elit; 15-02-2018 at 06:17.
Reply With Quote
  #8  
Old 15-02-2018, 10:32
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 244
Thanks: 176
Thanked 306 Times in 106 Posts
elit is on a distinguished road
New version released. Include uelr, AFR and script enhancements. Don't say I never loved you .
Reply With Quote
The Following 4 Users Say Thank You to elit For This Useful Post:
78372 (15-02-2018), KaktoR (15-02-2018), romulus_ut3 (16-02-2018), ZakirAhmad (15-02-2018)
  #9  
Old 15-02-2018, 18:57
ZakirAhmad ZakirAhmad is offline
Registered User
 
Join Date: Sep 2015
Location: Kmr
Posts: 290
Thanks: 56
Thanked 54 Times in 37 Posts
ZakirAhmad is on a distinguished road
Good job bro.
Reply With Quote
The Following User Says Thank You to ZakirAhmad For This Useful Post:
elit (16-02-2018)
  #10  
Old 19-02-2018, 14:07
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 244
Thanks: 176
Thanked 306 Times in 106 Posts
elit is on a distinguished road
Fixed non recursing subdirectories on mask. New file upped.
EDIT: There is still problem with parsing, will update soon.
EDIT2: Done. Hopefully now for good.
EDIT3: Turned out 7z already have cleaner solution with -r0 option. Upped again.

Last edited by elit; 20-02-2018 at 12:45.
Reply With Quote
  #11  
Old 16-03-2018, 05:33
KaktoR's Avatar
KaktoR KaktoR is offline
Lame User
 
Join Date: Jan 2012
Location: From outer space
Posts: 3,584
Thanks: 943
Thanked 5,848 Times in 2,159 Posts
KaktoR is on a distinguished road
It would be good if you could add output.txt which lists all files contain streams.
__________________
Haters gonna hate
Reply With Quote
  #12  
Old 17-03-2018, 08:40
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 244
Thanks: 176
Thanked 306 Times in 106 Posts
elit is on a distinguished road
Quote:
Originally Posted by KaktoR View Post
It would be good if you could add output.txt which lists all files contain streams.
You dont need this tool for that, you can simply "for /r %i in ("*.*") do (ztool ... "%i" 1>> file.log) or something like that

To know which file have stream individually I would have to process them one by one and compare output. That would be too much for a simple shell script that use tar to process all in one stream. Also slower.

You can use file mask to check files with specific extension only, or check single file as that mean all others with same extension will be processable.
Reply With Quote
  #13  
Old 06-07-2019, 06:31
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 244
Thanks: 176
Thanked 306 Times in 106 Posts
elit is on a distinguished road
New version 2.0 in the first post. Added x-tool 0.12 support. I have not tested oodle and crilayla and although it should work, let me know if it doesn't.
Reply With Quote
The Following User Says Thank You to elit For This Useful Post:
ffmla (06-07-2019)
  #14  
Old 07-07-2019, 08:09
IgorKolesnik IgorKolesnik is offline
Registered User
 
Join Date: Dec 2018
Location: Kiev
Posts: 31
Thanks: 27
Thanked 7 Times in 6 Posts
IgorKolesnik is on a distinguished road
can I check the file immediately for all possible encryption? or each method separately only?
Reply With Quote
  #15  
Old 07-07-2019, 09:56
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 244
Thanks: 176
Thanked 306 Times in 106 Posts
elit is on a distinguished road
Quote:
Originally Posted by IgorKolesnik View Post
can I check the file immediately for all possible encryption? or each method separately only?
This tool scan for compression, not encryption. If you mean several types of compression at once, no. You select single compression and see if it inflate. You can do 2 things though:
1, test single smaller file known to be likely compressed first to save time
2, run several scripts at once in parallel, each doing its own specific cmp test
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump



All times are GMT -7. The time now is 20:59.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Copyright 2000-2020, FileForums @ https://fileforums.com