Go Back   FileForums > Game Backup > PC Games > PC Games - CD/DVD Conversions > Conversion Tutorials

Reply
 
Thread Tools Display Modes
  #241  
Old 04-12-2020, 12:40
FitGirl FitGirl is offline
Registered User
 
Join Date: Dec 2014
Location: Riga
Posts: 191
Thanks: 16
Thanked 231 Times in 124 Posts
FitGirl is on a distinguished road
Regarding the crilayla algo - it's speed heavily depends on data and data size. Big files with RAW-like data compress insanely slowly, even with original algo (not xtool). So when xtool+cri is slow - it's not because of xtool, but because of crilayla. And with big files/chunks sometimes only one thread works in xtool at any given second and doesn't start next chunks until the big one is finished.
Reply With Quote
The Following User Says Thank You to FitGirl For This Useful Post:
elit (04-12-2020)
Sponsored Links
  #242  
Old 04-12-2020, 13:41
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 244
Thanks: 176
Thanked 306 Times in 106 Posts
elit is on a distinguished road
^Yes that is to be expected as threads have to wait for each other at some point to advance It should still be faster than one by one step of bms tool, although using your style where you dump raw files and use ppx to parallel them is of course even better.
But I think in those Persona 5 packs some files are not compressed(in the same file). They(some) don't have "CRILAYLA" header and dealing with all that really add to work. I could just dump ones with header of course as those without are likely uncompresable like audio and videos anyway.
But with xtool this saves so much hassle for just little extra time.


EDIT:I tried another cpk file with FA GUI and c32mb, I am getting about ~9mb/s inflating and ~10mb/s recoding it. ~10mb/s is IMO more than ok and I saw someone's oodle giving only slightly higher speed. Deflate gives me about 20mb/s(decompression) so 10mb/s for crilayla is great.

Last edited by elit; 04-12-2020 at 16:09.
Reply With Quote
  #243  
Old 04-12-2020, 15:46
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 244
Thanks: 176
Thanked 306 Times in 106 Posts
elit is on a distinguished road
Regarding '+' and ',' characters issues, why not just use ':' for everything and dissect the parameters in the code instead? I mean, for example:
Code:
xtool -mzlib:c32mb:reflate:l9
^This is perfectly easy to scan because names like zlib, kraken, reflate etc are unique and cannot appear as codec flags. So in your code you would just look for a word that represent a codec in the list and then all other params would be flags until next codec name is detected. So for example:
Code:
-mzlib:kraken:{param}
program would still know that zlib is with default parameters and kraken with whatever you put behind it simply because it knows about reserved words.

Also regarding versioning, whats wrong with simple v1.032, v1.033, v1.034 or anything like that?

Just a suggestions of course, I don't want to feel ungrateful or anything.
Reply With Quote
The Following User Says Thank You to elit For This Useful Post:
shazzla (11-12-2020)
  #244  
Old 04-12-2020, 16:23
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 244
Thanks: 176
Thanked 306 Times in 106 Posts
elit is on a distinguished road
Sorry for spamming but another zip file test:
Code:
orig zip size: 31.4mb

zlib: 31.4mb
reflate: 85.9mb
preflate: 49.5mb
grittibanzli: 101mb

then srep+xlzma:

reflate: 26.3mb
preflate: 29.8mb
  grittibanzli: 25.4mb
3.4% better final compression of grittibanzli vs reflate. Preflate look weak so far in every test for me. Looks like zlib+reflate and/or grittibanzli are 2 best general options.

EDIT:tried on another 2g file, grittibanzli took a freaking 20(!) minutes and loaded all cpu cores to max nonstop, pzlib+deflate took ~2min with much smaller cpu activity. Final compression was around 860mb vs 844mb only. That leave pzlib+deflate the only and best viable method for me.

Last edited by elit; 07-12-2020 at 16:13.
Reply With Quote
  #245  
Old 04-12-2020, 20:22
78372 78372 is offline
Registered User
 
Join Date: Dec 2016
Location: Bangladesh
Posts: 650
Thanks: 772
Thanked 936 Times in 309 Posts
78372 is on a distinguished road
Quote:
Originally Posted by elit View Post

-zlib codec did not inflated zip file at all!(I thought it was supposed to handle deflate?)
Yes, zlib is supposed to handle deflate but here's the catch, most of the standard zip files does not use regular deflate levels. In order to precompress, the zlib codec tries standard deflate compression levels (9levels * 9 memlevels = 81 tries) and not the tuned deflate options. On the other hand, preflate and reflate does not go with the hit&trial method, it has it's own algo for processing deflate streams. Games in general does not use deflate tuning which is why the zlib codec works good for games, but in other cases like zip, preflate/reflate works better.

Using zlib on where it does not need preflate/reflate is better though, as zlib codec does not produce additional hif files which makes compression a bit worse. When you are not sure, zlib+reflate is good to go as zlib will precompress what it can and pass whatever it can't to reflate which will maximize precompression. Except a few complex deflate streams, zlib+preflate is also a good option as hif files produced by preflate are significantly smaller than the files produced by reflate.
__________________
NOT AVAILABLE

Last edited by 78372; 04-12-2020 at 20:27.
Reply With Quote
The Following User Says Thank You to 78372 For This Useful Post:
elit (05-12-2020)
  #246  
Old 07-12-2020, 07:12
dixen dixen is online now
Registered User
 
Join Date: Sep 2014
Location: Russia
Posts: 296
Thanks: 356
Thanked 332 Times in 152 Posts
dixen is on a distinguished road
Cyberpunk 2077 (Preload)

Test on basegame_1_engine.archive

xtool2020 (mkraken)

pack

Quote:
Compressed 1 file, 764,993,536 => 1,949,886,395 bytes. Ratio 254.89%
Compression time: cpu 0.83 sec/real 575.53 sec = 0%. Speed 1.33 mB/s
All OK
unpack

Quote:
Tested 1 file, 1,949,886,395 => 764,993,536 bytes. Ratio 254.89%
Testing time: cpu 0.53 sec/real 25.08 sec = 2%. Speed 30.50 mB/s
All OK
Reply With Quote
The Following 3 Users Say Thank You to dixen For This Useful Post:
KaktoR (07-12-2020), shazzla (07-12-2020), ZAZA4EVER (07-12-2020)
  #247  
Old 07-12-2020, 07:19
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 748
Thanks: 168
Thanked 796 Times in 394 Posts
Masquerade is on a distinguished road
@dixen
Which oodle core used? One from game file?
Reply With Quote
  #248  
Old 07-12-2020, 07:40
dixen dixen is online now
Registered User
 
Join Date: Sep 2014
Location: Russia
Posts: 296
Thanks: 356
Thanked 332 Times in 152 Posts
dixen is on a distinguished road
Quote:
Originally Posted by masquerade View Post
@dixen
which oodle core used? One from game file?
oo2core_7_win64.zip

Last edited by dixen; 07-12-2020 at 07:42.
Reply With Quote
The Following 5 Users Say Thank You to dixen For This Useful Post:
hwang4 (07-12-2020), Masquerade (07-12-2020), Mortal Lord (09-12-2020), shazzla (07-12-2020), ZAZA4EVER (07-12-2020)
  #249  
Old 10-12-2020, 22:18
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,636
Thanks: 2,013
Thanked 10,344 Times in 2,202 Posts
Razor12911 is on a distinguished road
Update available

Changes

- added oo2ext* dll support
- updated search support

To everyone

If you left a post and I didn't reply to or acknowledge, I'm a bit busy and I spend way less time on forum.

Regarding the project. I don't plan on making a 2021 release of xtool, the 2020 is more of a final release and it will only receive minor updates and bug fixes with no new features. If there is a game that requires support, you need to check the plugins that are designed for xtool. I'll be focusing more on this. For the next few days I shall be giving out updates until Christmas so if there is a major feature that you need to be added in xtool leave a post and I'll probably add it.
Reply With Quote
The Following 6 Users Say Thank You to Razor12911 For This Useful Post:
elit (11-12-2020), ffmla (11-12-2020), Gehrman (11-12-2020), Masquerade (11-12-2020), shazzla (11-12-2020), ZAZA4EVER (11-12-2020)
  #250  
Old 11-12-2020, 04:24
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 748
Thanks: 168
Thanked 796 Times in 394 Posts
Masquerade is on a distinguished road
I do recall having a few issues with new XTool whereit would stop during decompression of ue4 games.

I will do a test and see if that's still an issue.
Reply With Quote
  #251  
Old 11-12-2020, 04:25
Ele's Avatar
Ele Ele is offline
Registered User
 
Join Date: Nov 2020
Location: Near Mars
Posts: 93
Thanks: 36
Thanked 90 Times in 37 Posts
Ele is on a distinguished road
Mr. Razor
Is lz4, lzo and zstd still not supported for xtool 2020?
Reply With Quote
  #252  
Old 11-12-2020, 06:37
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 244
Thanks: 176
Thanked 306 Times in 106 Posts
elit is on a distinguished road
Razor, pls check my above post No. 243 and consider that idea for parameters parsing to finalize specification. It would make xtool compatible with FA including GUI.
Finally, please consider including all parameters list with their description within command line help instead of external file.

EDIT: Also, provide full documentation on plugin system reserved keywords and their exact meaning(like if offset jump is meant from current position or beginning, if reading string advance position or not and so on), their full list etc.

Thank you for everything.

Last edited by elit; 11-12-2020 at 13:12.
Reply With Quote
The Following User Says Thank You to elit For This Useful Post:
L0v3craft (14-12-2020)
  #253  
Old 15-12-2020, 16:20
Cesar82's Avatar
Cesar82 Cesar82 is offline
Registered User
 
Join Date: May 2011
Location: Brazil
Posts: 800
Thanks: 1,318
Thanked 1,709 Times in 580 Posts
Cesar82 is on a distinguished road
@Razor12911, is there any change from XTool 2020 to have a 32 bit version too?
Perhaps in the thin version l of the XTool 2020, it would be great.
Most games require 64-bit systems, but there are still some games that support 32-bit.
So if there is only 64-bit XTool it would not be recommended to compress this type of games using it, having to choose to use the old XTool.
Also if the user has a 32-bit system installed he will not be able to use Xtool, even if it is for a game that will be installed on another system in the future.
Thanks!
Reply With Quote
  #254  
Old 16-12-2020, 01:39
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 244
Thanks: 176
Thanked 306 Times in 106 Posts
elit is on a distinguished road
Quote:
Originally Posted by Cesar82 View Post
Most games require 64-bit systems, but there are still some games that support 32-bit.
So if there is only 64-bit XTool it would not be recommended to compress this type of games using it
Why? What would happen, according to you? 64/32bit xtool have absolutely no relation to 64/32bit games. Only libraries used in xtool, hence why crilayla was dropped.
Reply With Quote
  #255  
Old 16-12-2020, 02:49
Ele's Avatar
Ele Ele is offline
Registered User
 
Join Date: Nov 2020
Location: Near Mars
Posts: 93
Thanks: 36
Thanked 90 Times in 37 Posts
Ele is on a distinguished road
Quote:
Originally Posted by elit View Post
Why? What would happen, according to you? 64/32bit xtool have absolutely no relation to 64/32bit games. Only libraries used in xtool, hence why crilayla was dropped.
I think people with 32-bit operating systems use 32-bit versions of games. If they want to back up their games, they must use the 32-bit compression tool. Yes, xtool's architecture (32 or 64) is for the operating system only and not for the game.
Reply With Quote
The Following User Says Thank You to Ele For This Useful Post:
Cesar82 (16-12-2020)
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[Dev]XTool Razor12911 Conversion Tutorials 180 23-10-2020 06:26
Project Cars Digital Edition (3xDVD5) (srep+lzma) GTX590 PC Games - CD/DVD Conversions 10 28-08-2017 08:34
Project IGI Anthology 1xCD700 CIUV2 2039 mausschieber PC Games - CD/DVD Conversions 0 24-07-2017 15:12
Space Channel 5 Part 2 Translation Project Christuserloeser DC Games 0 21-06-2004 18:16



All times are GMT -7. The time now is 00:34.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Copyright 2000-2020, FileForums @ https://fileforums.com