Go Back   FileForums > Game Backup > PC Games > PC Games - CD/DVD Conversions > Conversion Tutorials

Reply
 
Thread Tools Display Modes
  #151  
Old 22-08-2020, 01:35
humorlesslumber humorlesslumber is offline
Registered User
 
Join Date: Aug 2020
Location: us
Posts: 1
Thanks: 0
Thanked 0 Times in 0 Posts
humorlesslumber is on a distinguished road
Good but can we somehow compress modded android games like dragon city?

Last edited by humorlesslumber; 06-10-2020 at 08:41.
Reply With Quote
Sponsored Links
  #152  
Old 24-08-2020, 00:59
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,636
Thanks: 2,013
Thanked 10,343 Times in 2,202 Posts
Razor12911 is on a distinguished road
Quote:
Originally Posted by shazzla View Post
@Razor12911 :

Few days ago i started playing with XTool 2008. Great job ! Thanks !
But i found this :

Frostpunk - On the edge (Is its streams ZLIB? It must be IMHO)

It cant precompress those .dat files. (For example ,common.dat and languages.dat were tested).
Common.dat nearly a 600 MB file,languages.dat is a much smaller.
In both case ,the output is the same.
Older generation of XTool (AFAIR V0.9) precompresses commond.dat well to ~1.5 GB, languages.dat ,too.
XTOOLs (older) setting was : 128mb
XTOOLs (2008) setting was : 128mb ,d0 and d9 ,after this 256mb. (doesnt matter)

In my other test i faced with this (dont remember what game,maybe Hellbound):
XTool 2008 inflation was 2 gigs AFAIR ,compressed with LOLZ ,after that compressed the game without XTOOL2008 and the same LOLZ parameters and it was smaller with some MBs ! (maybe a special case ,dont know)

ATM im away from my computer for the next...lot of days ,i cant send example files.

Keep up the good work !
will be waiting for the sample
Reply With Quote
The Following User Says Thank You to Razor12911 For This Useful Post:
shazzla (24-08-2020)
  #153  
Old 24-08-2020, 02:41
shazzla shazzla is offline
Registered User
 
Join Date: Nov 2010
Location: Hunnia
Posts: 223
Thanks: 410
Thanked 86 Times in 63 Posts
shazzla is on a distinguished road
On it.. maybe 10 hrs from now... maybe.
Reply With Quote
  #154  
Old 24-08-2020, 07:13
FitGirl FitGirl is offline
Registered User
 
Join Date: Dec 2014
Location: Riga
Posts: 191
Thanks: 16
Thanked 231 Times in 124 Posts
FitGirl is on a distinguished road
Quote:
Originally Posted by Razor12911 View Post
will be waiting for the sample
Check Discord for Halo sample
Reply With Quote
The Following User Says Thank You to FitGirl For This Useful Post:
Razor12911 (24-08-2020)
  #155  
Old 24-08-2020, 11:21
shazzla shazzla is offline
Registered User
 
Join Date: Nov 2010
Location: Hunnia
Posts: 223
Thanks: 410
Thanked 86 Times in 63 Posts
shazzla is on a distinguished road
Quote:
Originally Posted by Razor12911 View Post
will be waiting for the sample
As i promised :

https://mega.nz/file/11RUTKIb#SCuzYA...LWlLeP5mgiu6EM
Reply With Quote
The Following User Says Thank You to shazzla For This Useful Post:
Razor12911 (24-08-2020)
  #156  
Old 25-08-2020, 18:51
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,636
Thanks: 2,013
Thanked 10,343 Times in 2,202 Posts
Razor12911 is on a distinguished road
@FitGirl
using

-mzlib+reflate
Code:
Tested 1 file, 92,646,762 => 48,529,408 bytes. Ratio 190.91%
Testing time: cpu 0.06 sec/real 2.62 sec = 2%. Speed 18.56 mB/s
-mzlib+preflate
Code:
Tested 1 file, 90,385,191 => 48,529,408 bytes. Ratio 186.25%
Testing time: cpu 0.08 sec/real 3.60 sec = 2%. Speed 13.46 mB/s
@shazzla

using

-mzlib+reflate
Code:
"localizations.dat". File is broken.
Tested 1 file, 67,329,029 => 9,663,657 bytes. Ratio 696.72%
Testing time: cpu 0.00 sec/real 1.10 sec = 0%. Speed 8.75 mB/s
-mzlib+preflate
Code:
Tested 1 file, 9,663,731 => 9,663,657 bytes. Ratio 100.00%
Testing time: cpu 0.02 sec/real 0.59 sec = 3%. Speed 16.34 mB/s
I remember mentioning somewhere that you sometimes need to include reflate/preflate along with zlib because this is no longer automated like in previous iterations.

in the sample FitGirl provided reflate gave better output, not sure if it's because of bigger hifs but this is why preflate doesn't replace reflate and why both are used, you just need to pick one to work with if the default zlib method doesn't process streams by itself.

that said, reflate no longer gets updates so at times, it breaks crc of files like in thr sample provided by shazzla so this means you need to use preflate.

I hope this clears up a few things about the program's usage.

Tests were performed using xtool_2008_R1 (the current version),

arc.ini
Code:
[External compressor:xtool]
header    = 0
packcmd   = xtool.exe precomp -mzlib+preflate -c32mb -t100p-1 -d0 - - <stdin> <stdout>
unpackcmd = xtool.exe decode -t100p-1 - - <stdin> <stdout>
Edit:

the next version of xtool will have reflate automatically verify the streams it processes to avoid checksum errors and if it fails but preflate was enabled, it will jump to preflate. may take longer to process but I guess there is no other alternative at this point.

Last edited by Razor12911; 25-08-2020 at 18:53.
Reply With Quote
The Following 3 Users Say Thank You to Razor12911 For This Useful Post:
Harsh ojha (25-08-2020), L0v3craft (03-09-2020), shazzla (25-08-2020)
  #157  
Old 26-08-2020, 17:45
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,636
Thanks: 2,013
Thanked 10,343 Times in 2,202 Posts
Razor12911 is on a distinguished road
Update available

Changes

- fixed command line parser
- updated deflate scanner
- added stream deduplication
- added stream database
- added decompression memory limiter
- added grittibanzli (also handles deflate stream but slow af)

Notes

stream deduplication doesn't allow you to set max decompression memory so it uses VM file, this will be in future versions.
stream deduplication also produces a file that is required for decompression, this file must always exist else xtool will always fail restoring the data

Code:
[External compressor:xtool]
header    = 0
packcmd   = xtool.exe precomp -mzlib+preflate -c32mb -t100p-1 --dbase --dedup=xtool.bin - - <stdin> <stdout>
unpackcmd = xtool.exe decode -t100p-1 --dedup=xtool.bin - - <stdin> <stdout>
xtool.bin must be put along with the installation (setup.exe) and extracted to temp like you would with xtool.exe and other decompression exe/dll/cls files

if you have multiple archives, the same xtool.bin file can be used. meaning if you used xtool in data1.bin and data2.bin, xtool.bin can be used for both, the program itself will decide which section of the xtool.bin belongs to which archive. This is better than having xtool1.bin and xtool2.bin for each archive

Last edited by Razor12911; 26-08-2020 at 17:56.
Reply With Quote
The Following 14 Users Say Thank You to Razor12911 For This Useful Post:
Cesar82 (26-08-2020), darkwolves (27-08-2020), DiCaPrIo (27-08-2020), dixen (26-08-2020), elit (04-12-2020), ffmla (27-08-2020), Harsh ojha (26-08-2020), L0v3craft (03-09-2020), Masquerade (28-08-2020), oltjon (27-08-2020), Perykles (27-08-2020), shazzla (26-08-2020), vint56 (26-08-2020), ZAZA4EVER (29-08-2020)
  #158  
Old 27-08-2020, 21:50
nicholas2006 nicholas2006 is offline
Registered User
 
Join Date: Aug 2020
Location: USA
Posts: 1
Thanks: 0
Thanked 0 Times in 0 Posts
nicholas2006 is on a distinguished road
Thanks for the awesome tool.

It works wonderfully in 99% cases but I recently came across with a game called UnderMine and xtool seems to not do anything..
GFS shows zlib 135 MB > 226 MB and I'm using the following method: -mzlib+preflate -c128mb -t2

Any guidance would be greatly appreciated.

*edit: I used different keywords to search and found my answer in a different thread (issue being it was dynamic stream). But any other input may be helpful!

Last edited by nicholas2006; 27-08-2020 at 22:01.
Reply With Quote
  #159  
Old 28-08-2020, 08:32
Snapppr6 Snapppr6 is offline
Registered User
 
Join Date: Jun 2020
Location: libya
Posts: 32
Thanks: 18
Thanked 16 Times in 14 Posts
Snapppr6 is on a distinguished road
????

hi razor12911
in any data type we can use xtool 2008 and 1908
Reply With Quote
  #160  
Old 28-08-2020, 11:51
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 747
Thanks: 168
Thanked 792 Times in 394 Posts
Masquerade is on a distinguished road
Thank you for the dedup mode, it is working well.

Mad Max = 43kb deduplication file made

I expect that with more duplicated streams, the file size of course will increase - is there a limit on how large the file could reach? or could it reach astronomical sizes (20+mb), in which case, packaging it inside a setup would appear irregular.
Reply With Quote
  #161  
Old 28-08-2020, 15:07
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,636
Thanks: 2,013
Thanked 10,343 Times in 2,202 Posts
Razor12911 is on a distinguished road
Quote:
Originally Posted by nicholas2006 View Post
Thanks for the awesome tool.

It works wonderfully in 99% cases but I recently came across with a game called UnderMine and xtool seems to not do anything..
GFS shows zlib 135 MB > 226 MB and I'm using the following method: -mzlib+preflate -c128mb -t2

Any guidance would be greatly appreciated.

*edit: I used different keywords to search and found my answer in a different thread (issue being it was dynamic stream). But any other input may be helpful!
these were probably false positives but you can try using -mzlib+reflate to see what happens.

Quote:
Originally Posted by Snapppr6 View Post
hi razor12911
in any data type we can use xtool 2008 and 1908
2008 currently only supports deflate compressed data.

Quote:
Originally Posted by Masquerade View Post
Thank you for the dedup mode, it is working well.

Mad Max = 43kb deduplication file made

I expect that with more duplicated streams, the file size of course will increase - is there a limit on how large the file could reach? or could it reach astronomical sizes (20+mb), in which case, packaging it inside a setup would appear irregular.
these deduplication files are actually compressible, try compressing any of the ones you have generated so far and see what happens. Inno Setup compresses resources it uses so...

Last edited by Razor12911; 28-08-2020 at 15:10.
Reply With Quote
  #162  
Old 29-08-2020, 15:05
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,636
Thanks: 2,013
Thanked 10,343 Times in 2,202 Posts
Razor12911 is on a distinguished road
Update available

Changes

- fixed deduplication memory calculation error
- added virtual memory support for deduplication
- added --mem=# parameter to control deduplication memory usage

the virtual memory files xtool creates look like this and they are placed in same temp directory freearc creates, this way you can change where they are supposed to be created by setting temp/work directory
Attached Images
File Type: png 74.PNG (9.6 KB, 327 views)
Reply With Quote
The Following 9 Users Say Thank You to Razor12911 For This Useful Post:
78372 (29-08-2020), DiCaPrIo (29-08-2020), dixen (30-08-2020), ffmla (30-08-2020), Harsh ojha (30-08-2020), sajmon83 (05-05-2022), shazzla (29-08-2020), vint56 (29-08-2020), ZAZA4EVER (29-08-2020)
  #163  
Old 29-08-2020, 21:15
Gupta Gupta is offline
Registered User
 
Join Date: Aug 2016
Location: India
Posts: 397
Thanks: 139
Thanked 705 Times in 229 Posts
Gupta is on a distinguished road
good old GTA 5

compressing x64a.rpf...x64k.rpf with xtool+srep+lolz, crc error at x64d.rpf

compress x64a.rpf...x64d.rpf with xtool decompression never finishes

xtool precomp -mzlib+reflate -c64mb -t100p-1 --dbase=gtav.xdb --dedup=xtool.bin
Attached Images
File Type: jpg z.JPG (42.9 KB, 320 views)
__________________
XD
Reply With Quote
The Following User Says Thank You to Gupta For This Useful Post:
Razor12911 (29-08-2020)
  #164  
Old 29-08-2020, 21:50
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,636
Thanks: 2,013
Thanked 10,343 Times in 2,202 Posts
Razor12911 is on a distinguished road
try without reflate
Reply With Quote
  #165  
Old 29-08-2020, 23:31
Gupta Gupta is offline
Registered User
 
Join Date: Aug 2016
Location: India
Posts: 397
Thanks: 139
Thanked 705 Times in 229 Posts
Gupta is on a distinguished road
here is offended part of file https://drive.google.com/file/d/1HA-...ew?usp=sharing

I didn't test much but other method seems to be working fine on this file atleast
__________________
XD

Last edited by Gupta; 29-08-2020 at 23:35.
Reply With Quote
The Following User Says Thank You to Gupta For This Useful Post:
Razor12911 (01-09-2020)
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[Dev]XTool Razor12911 Conversion Tutorials 180 23-10-2020 06:26
Project Cars Digital Edition (3xDVD5) (srep+lzma) GTX590 PC Games - CD/DVD Conversions 10 28-08-2017 08:34
Project IGI Anthology 1xCD700 CIUV2 2039 mausschieber PC Games - CD/DVD Conversions 0 24-07-2017 15:12
Space Channel 5 Part 2 Translation Project Christuserloeser DC Games 0 21-06-2004 18:16



All times are GMT -7. The time now is 15:54.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Copyright 2000-2020, FileForums @ https://fileforums.com