Go Back   FileForums > Game Backup > PC Games > PC Games - CD/DVD Conversions > Conversion Tutorials

Reply
 
Thread Tools Display Modes
  #256  
Old 30-05-2018, 08:52
ZakirAhmad ZakirAhmad is offline
Registered User
 
Join Date: Sep 2015
Location: Kmr
Posts: 290
Thanks: 56
Thanked 54 Times in 37 Posts
ZakirAhmad is on a distinguished road
xtool v 0.7 Tried it on a file from AC orgins

it didnt expanded.
sorry to bother oodle was missing from arc.ini setting.
this sorted the issue.

afr expaned it to 163.15% while xtool to 162.91% xtool
restored it in only 14sec while afr took 18 seconds in my system

Now my only problem is with Dunia engine. i wrote dat file address in xtool.ini still no change.

BTW is farcry primal and James Camerons avatar supported.

Last edited by ZakirAhmad; 30-05-2018 at 09:21.
Reply With Quote
Sponsored Links
  #257  
Old 30-05-2018, 09:15
KaktoR's Avatar
KaktoR KaktoR is offline
Lame User
 
Join Date: Jan 2012
Location: From outer space
Posts: 3,555
Thanks: 935
Thanked 5,799 Times in 2,140 Posts
KaktoR is on a distinguished road
Yep, didn't expand with :lzo

Maybe wrong codec in arc.ini? lzo2.dll is in place

Edit: You have to use
Code:
:oodle
for Origins

Code:
DataPC.forge (ACO)

xtool v0.7 oodle: 273.088.512 Bytes => 574.853.120 Bytes
AFR:              273.088.512 Bytes => 574.849.024 Bytes
From this example xtool and AFR have pretty much the same compression speed (AFR is a little bit faster, but just a few secs)
__________________
Haters gonna hate

Last edited by KaktoR; 30-05-2018 at 11:24.
Reply With Quote
  #258  
Old 30-05-2018, 13:52
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 224
Thanks: 163
Thanked 287 Times in 99 Posts
elit is on a distinguished road
Quote:
Originally Posted by Razor12911 View Post
Update available
In terms of the Dunia engine, I advise that you use the latest liblz4.dll to avoid some streams being skipped due to their major difference, this could be because even I myself am not sure if I used the correct parameters for recompression, if you have an idea of what is used, please comment. (I used lz4hc, level 9)
I dont know if you want to go this route, but the way I did my tests when developing lz4 packer for "Raiders of the Broken Planet" game was that I used both original and self-modified quickbms script for unpacking game archive first. I unpacked thus twice - for each.

In modified script only difference was that I disabled decompression by taking out "comtype" command and used "log" instead of "clog" to dump files raw & compressed.

So I had both compressed and decompressed individual files from archive. Then it was only matter of finding right/best cmp settings through hex editor and binary comparison(first on few files and then all) to get best setting. And even wrong(but close) settings were still compatible with 80%+ of all archive.
Reply With Quote
The Following User Says Thank You to elit For This Useful Post:
Simorq (30-05-2018)
  #259  
Old 30-05-2018, 20:42
ShivShubh ShivShubh is offline
Registered User
 
Join Date: Dec 2015
Location: India
Posts: 94
Thanks: 311
Thanked 101 Times in 43 Posts
ShivShubh is on a distinguished road
Quote:
Originally Posted by elit View Post
I dont know if you want to go this route, but the way I did my tests when developing lz4 packer for "Raiders of the Broken Planet" game was that I used both original and self-modified quickbms script for unpacking game archive first. I unpacked thus twice - for each.

In modified script only difference was that I disabled decompression by taking out "comtype" command and used "log" instead of "clog" to dump files raw & compressed.

So I had both compressed and decompressed individual files from archive. Then it was only matter of finding right/best cmp settings through hex editor and binary comparison(first on few files and then all) to get best setting. And even wrong(but close) settings were still compatible with 80%+ of all archive.
So by saying all that what were you trying to say by this "I dont know if you want to go this route" ? That Razor should find the correct parameters ? I don't think he has that much of an interest in far cry 5 or wants to waste time doing all that.
Reply With Quote
  #260  
Old 30-05-2018, 21:28
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,630
Thanks: 2,003
Thanked 10,318 Times in 2,198 Posts
Razor12911 is on a distinguished road
Quote:
Originally Posted by Sergey3695 View Post
https://yadi.sk/d/WNKW73Ht3UpyVN
data.cpk (393 mb)
NARUTO SHIPPUDEN Ultimate Ninja STORM 2

[External compressor:xprecomp]
header = 0
packcmd = xtool.exe e:precomp:c32mb,t1:crilayla - - <stdin> <stdout>
unpackcmd = xtool.exe d:precomp:c32mb,t1:crilayla - - <stdin> <stdout>

Extracting archive: data.arc
Extracting data.cpk (413123988 bytes)
ERROR: file _TEST\data.cpk failed CRC check
:confused:
i'll check :)

Last edited by pakrat2k2; 31-05-2018 at 10:07.
Reply With Quote
The Following 2 Users Say Thank You to Razor12911 For This Useful Post:
Sergey3695 (31-05-2018), ShivShubh (24-11-2019)
  #261  
Old 31-05-2018, 03:20
Edison007's Avatar
Edison007 Edison007 is offline
Registered User
 
Join Date: Mar 2014
Location: EU
Posts: 24
Thanks: 0
Thanked 48 Times in 15 Posts
Edison007 is on a distinguished road
compr:
Code:
afr:v0+srep:m5f:l512:d512m:hash-+lolz:d64m - 37,6 MB (39 463 094 bytes)
xtool+srep:m5f:l512:d512m:hash-+lolz:d64m  - 37,8 MB (39 671 049 bytes)
dec time/speed:
Code:
cls-afr, v019 (x86)
t1 - 199.28 sec; ~  5.37 mB/s
t7 -  44.07 sec; ~ 24.29 mB/s
t8 -  42.89 sec; ~ 24.96 mB/s

xtool_x86, v07
t1 - 237.07 sec; ~  4.52 mB/s
t7 -  53.84 sec; ~ 19.88 mB/s
t8 -  51.19 sec; ~ 20.91 mB/s

xtool_x64, v07
t1 - 210.85 sec; ~  5.08 mB/s
t7 -  49.35 sec; ~ 21.69 mB/s
t8 -  46.91 sec; ~ 22.82 mB/s
Code:
1'070'530'560 ->   afr: 1'767'925'585 bytes
              -> xtool: 1'768,021'857 bytes

i7-4700MQ, 8gb ram, ram-disk, win7x64.

Last edited by Edison007; 31-05-2018 at 04:19.
Reply With Quote
The Following 4 Users Say Thank You to Edison007 For This Useful Post:
-NORO- (31-05-2018), KaktoR (31-05-2018), Razor12911 (31-05-2018), Simorq (31-05-2018)
  #262  
Old 31-05-2018, 04:14
shazzla shazzla is offline
Registered User
 
Join Date: Nov 2010
Location: Hunnia
Posts: 222
Thanks: 408
Thanked 86 Times in 63 Posts
shazzla is on a distinguished road
Similar happens on Agony's .PAK files(zlib,v0.7, v0.6 works good)
First file processed well,but the second one looks like simply copied.
Cant upload files because they are more than 10 gigs. :/
Reply With Quote
The Following 3 Users Say Thank You to shazzla For This Useful Post:
doofoo24 (31-05-2018), oltjon (31-05-2018), Razor12911 (31-05-2018)
  #263  
Old 31-05-2018, 08:50
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 224
Thanks: 163
Thanked 287 Times in 99 Posts
elit is on a distinguished road
Quote:
Originally Posted by doofoo24 View Post
i think it may be better for Razor12911 if he develop the tool separately like pzlib/plz4/pzstd/plzo for games...
less of a headache
How exactly is that less headache? This way at least he can re-use a lot of same code/routines for each compressor instead of duplicating and maintaining separate projects with a lot of same code.
Reply With Quote
  #264  
Old 31-05-2018, 09:31
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,630
Thanks: 2,003
Thanked 10,318 Times in 2,198 Posts
Razor12911 is on a distinguished road
Update available

Changes

- fixed crilayla bug on naruto games
- fixed issue detecting lzo2a, lzo1c streams

@doofoo24
I have to agree with elit, It's more headache separating the codecs. An example would be ztool itself, normally when I make changes to pzlib, if it's not related to precompression itself, I had to make the same changes in plz4, plzo... sometimes I forget doing this and end up with different sources of each, in one, a bug was fixed then another bug appears in another, now there are bugs all over the codecs, different bugs, you end up trying to fix one and another pops up and so forth. The idea of putting everything together means one code, if one codec is affected, then all will be affected then I'll know the source of the bug.

Quote:
Originally Posted by Edison007 View Post
compr:
Code:
afr:v0+srep:m5f:l512:d512m:hash-+lolz:d64m - 37,6 MB (39 463 094 bytes)
xtool+srep:m5f:l512:d512m:hash-+lolz:d64m  - 37,8 MB (39 671 049 bytes)
dec time/speed:
Code:
cls-afr, v019 (x86)
t1 - 199.28 sec; ~  5.37 mB/s
t7 -  44.07 sec; ~ 24.29 mB/s
t8 -  42.89 sec; ~ 24.96 mB/s

xtool_x86, v07
t1 - 237.07 sec; ~  4.52 mB/s
t7 -  53.84 sec; ~ 19.88 mB/s
t8 -  51.19 sec; ~ 20.91 mB/s

xtool_x64, v07
t1 - 210.85 sec; ~  5.08 mB/s
t7 -  49.35 sec; ~ 21.69 mB/s
t8 -  46.91 sec; ~ 22.82 mB/s
Code:
1'070'530'560 ->   afr: 1'767'925'585 bytes
              -> xtool: 1'768,021'857 bytes

i7-4700MQ, 8gb ram, ram-disk, win7x64.
I wonder if there is still room for improvement.

Last edited by Razor12911; 31-05-2018 at 09:39.
Reply With Quote
The Following 5 Users Say Thank You to Razor12911 For This Useful Post:
doofoo24 (31-05-2018), elit (02-06-2018), KaktoR (31-05-2018), Sergey3695 (31-05-2018), ShivShubh (24-11-2019)
  #265  
Old 31-05-2018, 09:52
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,630
Thanks: 2,003
Thanked 10,318 Times in 2,198 Posts
Razor12911 is on a distinguished road
Quote:
Originally Posted by doofoo24 View Post
@Razor12911 xtool 0.7 seems to skip file's test it on ME A with zstd and before with lzo on ac 1...
xtool 0.6 work fine...
any idea ?
i will test with 0.8...
I didn't touch zstd codec in between the updates

ztool_08

Code:
Process ID       : 12240
Thread ID        : 19080
Process Exit Code: 0
Thread Exit Code : 0

User Time        :          24.500s
Kernel Time      :           0.734s
Process Time     :          25.234s
Clock Time       :           7.764s

Working Set      :          193016 KB
Paged Pool       :             114 KB
Nonpaged Pool    :              13 KB
Pagefile         :          314972 KB
Page Fault Count : 311084

IO Read          :           43332 KB (in             683 reads )
IO Write         :          101353 KB (in            1584 writes)
IO Other         :               3 KB (in             178 others)
ztool_06

Code:
Process ID       : 12748
Thread ID        : 14488
Process Exit Code: 0
Thread Exit Code : 0

User Time        :          24.890s
Kernel Time      :           1.125s
Process Time     :          26.015s
Clock Time       :           8.160s

Working Set      :          177480 KB
Paged Pool       :             115 KB
Nonpaged Pool    :              12 KB
Pagefile         :          229352 KB
Page Fault Count : 316978

IO Read          :           43262 KB (in             677 reads )
IO Write         :          101353 KB (in            1584 writes)
IO Other         :               5 KB (in             270 others)
Reply With Quote
The Following User Says Thank You to Razor12911 For This Useful Post:
ShivShubh (24-11-2019)
  #266  
Old 31-05-2018, 10:02
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,630
Thanks: 2,003
Thanked 10,318 Times in 2,198 Posts
Razor12911 is on a distinguished road
Quote:
Originally Posted by Edison007 View Post
compr:
Code:
afr:v0+srep:m5f:l512:d512m:hash-+lolz:d64m - 37,6 MB (39 463 094 bytes)
xtool+srep:m5f:l512:d512m:hash-+lolz:d64m  - 37,8 MB (39 671 049 bytes)
dec time/speed:
Code:
cls-afr, v019 (x86)
t1 - 199.28 sec; ~  5.37 mB/s
t7 -  44.07 sec; ~ 24.29 mB/s
t8 -  42.89 sec; ~ 24.96 mB/s

xtool_x86, v07
t1 - 237.07 sec; ~  4.52 mB/s
t7 -  53.84 sec; ~ 19.88 mB/s
t8 -  51.19 sec; ~ 20.91 mB/s

xtool_x64, v07
t1 - 210.85 sec; ~  5.08 mB/s
t7 -  49.35 sec; ~ 21.69 mB/s
t8 -  46.91 sec; ~ 22.82 mB/s
Code:
1'070'530'560 ->   afr: 1'767'925'585 bytes
              -> xtool: 1'768,021'857 bytes

i7-4700MQ, 8gb ram, ram-disk, win7x64.
Ran a test using the samples I got from forum. Very small input but got these results:

Input: 278 MB (291,605,504 bytes)
DataPC_Map_Menu.forge (ACI)
DataPC_LoadingRoom2.forge (ACI)
DataPC.forge (ACII)
DataPC.forge (ACIV)

Quote:
command line:
xtool_x86: d:precomp:t# %1.out1 %1.res
afr19_x86: d -v0 -t# %1.out2 %1.res

Thread(1) = 1 result:
xtool: 17.898s, 18.050s, 17.901s, 18.056s, 17.942s
afr19: 17.853s, 17.875s, 18.173s, 17.865s, 17.879s

Thread(s) = 2 result:
xtool: 9.120s, 9.156s, 9.035s, 9.067s, 9.008s
afr19: 9.394s, 9.356s, 9.318s, 9.248s, 9.291s

Thread(s) = 3 result:
xtool: 6.325s, 6.412s, 6.249s, 6.412s, 6.264s
afr19: 6.603s, 6.511s, 6.696s, 6.517s, 6.691s
Not sure in terms of results here, I ran test on ramdisk using my old core 2 extreme, I didn't include t4 since those are mostly irrelevant.

As for final output after compression, I'd be taking a gamble when I say it's headers since they include stuff like what codec was used since xtool is not just made for one game engine but I'll check in terms of the number of streams found by xtool vs afr
Attached Files
File Type: 7z batch_file_4_tests.7z (2.1 KB, 35 views)

Last edited by pakrat2k2; 31-05-2018 at 10:08.
Reply With Quote
The Following 2 Users Say Thank You to Razor12911 For This Useful Post:
ShivShubh (24-11-2019), ZakirAhmad (31-05-2018)
  #267  
Old 31-05-2018, 10:08
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,630
Thanks: 2,003
Thanked 10,318 Times in 2,198 Posts
Razor12911 is on a distinguished road
Quote:
Originally Posted by doofoo24 View Post
tried xtool 0.8 and 0.7 on multiple files at once like shazzla First file processed well,but the second one looks like simply copied.
lz4 / lzo /zlib
upload files and command line used
Reply With Quote
The Following User Says Thank You to Razor12911 For This Useful Post:
ShivShubh (24-11-2019)
  #268  
Old 31-05-2018, 10:09
pakrat2k2's Avatar
pakrat2k2 pakrat2k2 is offline
Moderator
 
Join Date: Apr 2005
Location: Canada
Posts: 7,210
Thanks: 3,039
Thanked 8,975 Times in 3,082 Posts
pakrat2k2 is on a distinguished road
ALWAYS choose disable smilies, when posting/ replying ( checkbox below message box )

pain to have to come in & clean up posts & quotes.
Reply With Quote
The Following 2 Users Say Thank You to pakrat2k2 For This Useful Post:
elit (02-06-2018), Razor12911 (31-05-2018)
  #269  
Old 31-05-2018, 10:40
Edison007's Avatar
Edison007 Edison007 is offline
Registered User
 
Join Date: Mar 2014
Location: EU
Posts: 24
Thanks: 0
Thanked 48 Times in 15 Posts
Edison007 is on a distinguished road
Quote:
Not sure in terms of results here
It's normal, because i using differents decode methods in cls and exe. In cls i use some tricks, which allow me to get a higher speed. exe-decoder, yes, slower.

Quote:
I'll check in terms of the number of streams found by xtool vs afr
I think, that numbers of streams found is the same. But maybe in AFR the output container is more successfully made

Quote:
I wonder if there is still room for improvement.
Yep, but I'm not interested in this engine anymore.
Reply With Quote
  #270  
Old 31-05-2018, 10:56
KaktoR's Avatar
KaktoR KaktoR is offline
Lame User
 
Join Date: Jan 2012
Location: From outer space
Posts: 3,555
Thanks: 935
Thanked 5,799 Times in 2,140 Posts
KaktoR is on a distinguished road
Quote:
Originally Posted by Edison007 View Post
Yep, but I'm not interested in this engine anymore.
So, what's your next clue?
__________________
Haters gonna hate
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Information about ZTool 78372 Conversion Tutorials 73 10-10-2019 06:27
ztool info doofoo24 Conversion Tutorials 2 24-12-2017 22:08
ZTool (Ultimate Precompressor) Razor12911 Conversion Tutorials 16 20-07-2017 07:56



All times are GMT -7. The time now is 00:35.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Copyright 2000-2020, FileForums @ https://fileforums.com