Go Back   FileForums > Game Backup > PC Games > PC Games - CD/DVD Conversions > Conversion Tutorials

Reply
 
Thread Tools Display Modes
  #16  
Old 18-04-2018, 08:09
doofoo24 doofoo24 is offline
Registered User
 
Join Date: Nov 2016
Location: canada
Posts: 408
Thanks: 138
Thanked 459 Times in 227 Posts
doofoo24 is on a distinguished road
Quote:
Originally Posted by joekkerr View Post
Holy shit, thank you so so so so much man! you were so informative.

how do you identify which game has which.
Quote:
Originally Posted by 78372 View Post
use Razor12911's GFS for testing what to use...
Reply With Quote
The Following 3 Users Say Thank You to doofoo24 For This Useful Post:
78372 (18-04-2018), joekkerr (18-04-2018), Spexxos (18-04-2018)
Sponsored Links
  #17  
Old 18-04-2018, 08:23
78372 78372 is offline
Registered User
 
Join Date: Dec 2016
Location: Bangladesh
Posts: 649
Thanks: 772
Thanked 935 Times in 308 Posts
78372 is on a distinguished road
Quote:
Originally Posted by Spexxos View Post

@78372 About xtool vs precomp. Precompv046 gave me better results than xtool on my testfiles (Witcher1Savegames) I probably should scan the files so see what they are made of but for now I am wondering if my xtool settings are wrong. To be honest I have no clue how to properly configure xtool/arc.ini

From precomp 046, it's a complete compressor, not only a precompressor, so it will give better result on small files and when you have compression on. To use xtool, you must use xtool+srep+lzma(or xtool+srep+lolz) to get better ratio, else it will just inflate files, not compress, plus xtool isn't complete yet, it's a beta tool, it has bugs.
__________________
NOT AVAILABLE
Reply With Quote
The Following 2 Users Say Thank You to 78372 For This Useful Post:
EzzEldin16 (20-04-2018), Spexxos (18-04-2018)
  #18  
Old 18-04-2018, 08:30
Spexxos Spexxos is offline
Registered User
 
Join Date: Apr 2017
Location: Germany
Posts: 13
Thanks: 21
Thanked 1 Time in 1 Post
Spexxos is on a distinguished road
Quote:
Originally Posted by 78372 View Post
From precomp 046, it's a complete compressor, not only a precompressor, so it will give better result on small files and when you have compression on. To use xtool, you must use xtool+srep+lzma(or xtool+srep+lolz) to get better ratio, else it will just inflate files, not compress, plus xtool isn't complete yet, it's a beta tool, it has bugs.
My bad I should have been more detailed. I used precompv46 only as a precompressor, so with these parameters ("precomp -cn -intense").
The file that precomp delivered was about %10 bigger than the one from xtoo. Which means precomp found more to decompress?

Aferwards I put both the testfiles from precomp and from xtool into srep393a (-mx), then I proceeded to compress them with 7z(7za.exe a -t7z -m0=LZMA2 -mmt=1 -mx9 -md=1536m -mfb=273 -ms=on -mqs=on)

Result is the precomp testfile is half the size of the xtool one. That makes me think I have to configure xtool differently.
Reply With Quote
  #19  
Old 18-04-2018, 08:35
78372 78372 is offline
Registered User
 
Join Date: Dec 2016
Location: Bangladesh
Posts: 649
Thanks: 772
Thanked 935 Times in 308 Posts
78372 is on a distinguished road
May be xtool didn't work good on that, it's a beta tool after all.
Try pZlib v3 instead
__________________
NOT AVAILABLE
Reply With Quote
The Following User Says Thank You to 78372 For This Useful Post:
Spexxos (18-04-2018)
  #20  
Old 18-04-2018, 08:39
Spexxos Spexxos is offline
Registered User
 
Join Date: Apr 2017
Location: Germany
Posts: 13
Thanks: 21
Thanked 1 Time in 1 Post
Spexxos is on a distinguished road
Quote:
Originally Posted by 78372 View Post
May be xtool didn't work good on that, it's a beta tool after all.
Try pZlib v3 instead
what settings do you suggest for pzlib?


EDIT: pzlibv3, christmas edition, gives the same result as precomp v046 (pZLib3a.exe e -c128m -t75p). It is also noticeably faster. Is there any sense in checking out older versions of pzlib? I have seen older versions have more switches/options. Is that because the needed more manual input from the user to give the best result, and pzlib3 does that automatically or is there an other reason for that?


EDIT2: After Precomp/pzlib what defines the order in which you apply further tools like DELTA and Srep, msc etc? Which one comes first, and why? Or does the order not matter at all?

Last edited by Spexxos; 18-04-2018 at 09:09.
Reply With Quote
  #21  
Old 18-04-2018, 09:13
78372 78372 is offline
Registered User
 
Join Date: Dec 2016
Location: Bangladesh
Posts: 649
Thanks: 772
Thanked 935 Times in 308 Posts
78372 is on a distinguished road
MSC should be used *ONLY* for multimedia data and delta should be used *ONLY* for exe and wav files, applying delta for normal game files will cause worse ratio, and without multimedia data, msc can't do anything.
MSC should come first, then pzlib, then srep and at last lzma/lolz
__________________
NOT AVAILABLE
Reply With Quote
The Following 4 Users Say Thank You to 78372 For This Useful Post:
flaek1337 (03-06-2018), joekkerr (18-04-2018), Prince4 (28-03-2021), Spexxos (18-04-2018)
  #22  
Old 18-04-2018, 09:18
Spexxos Spexxos is offline
Registered User
 
Join Date: Apr 2017
Location: Germany
Posts: 13
Thanks: 21
Thanked 1 Time in 1 Post
Spexxos is on a distinguished road
Quote:
Originally Posted by 78372 View Post
MSC should be used *ONLY* for multimedia data and delta should be used *ONLY* for exe and wav files, applying delta for normal game files will cause worse ratio, and without multimedia data, msc can't do anything.
MSC should come first, then pzlib, then srep and at last lzma/lolz
Thank you. The thing about DELTA is that, that on my witchersave testfile I got better compression with precompv46+srep393a+DELTA+lzma2 compared to precompv46+srep393a+lzma2. Any Idea why? Or is that the exception that confirms the rules?
Reply With Quote
  #23  
Old 18-04-2018, 09:32
78372 78372 is offline
Registered User
 
Join Date: Dec 2016
Location: Bangladesh
Posts: 649
Thanks: 772
Thanked 935 Times in 308 Posts
78372 is on a distinguished road
That's a save file, not a game file and probably delta works better for it. Not everywhere delta works and using them on executables is purely safe and that's why I mentioned about that. But in general, delta doesn't work better on game files(such as textures).
__________________
NOT AVAILABLE
Reply With Quote
The Following 2 Users Say Thank You to 78372 For This Useful Post:
EzzEldin16 (20-04-2018), Spexxos (18-04-2018)
  #24  
Old 18-04-2018, 09:38
Spexxos Spexxos is offline
Registered User
 
Join Date: Apr 2017
Location: Germany
Posts: 13
Thanks: 21
Thanked 1 Time in 1 Post
Spexxos is on a distinguished road
Quote:
Originally Posted by 78372 View Post
That's a save file, not a game file and probably delta works better for it. Not everywhere delta works and using them on executables is purely safe and that's why I mentioned about that. But in general, delta doesn't work better on game files(such as textures).
I see. My main interest for now is compressing my savegames. 500gb unpacked/ 190gb packed. They got quite large recently. I think its time to change that. Anyway. Thank you very much for your help.

Its a shame that there is no "Best Compression Methods for 'Specific' SaveGames INDEX" Maybe its time to do something about that


EDIT:

ok, two more questions popped up: Where can I find lolz, the only thread I found was Lolzx which is apparently a private compressor?
And second whats the matter with pzlib3 according to razor there is this command "-x" which is supposed to give the best output. Cant find it anywhere else mentioned though, not even in the instructions for pzlib. What does this command do? Also what is the use case for pzlibs "-r" deflate. Its different decompression mode I assume?

Last edited by Spexxos; 18-04-2018 at 10:04.
Reply With Quote
  #25  
Old 18-04-2018, 11:02
Andu21 Andu21 is offline
Registered User
 
Join Date: Feb 2016
Location: Impossible Planet
Posts: 114
Thanks: 179
Thanked 48 Times in 38 Posts
Andu21 is on a distinguished road
Quote:
Originally Posted by Spexxos View Post
After Precomp/pzlib what defines the order in which you apply further tools like DELTA and Srep, msc etc? Which one comes first, and why? Or does the order not matter at all?
Hard to tell, sometimes delta before srep improves compression, same with msc.

Quote:
Originally Posted by Spexxos View Post
Where can I find lolz, the only thread I found was Lolzx which is apparently a private compressor?
http://fileforums.com/showthread.php?t=99953

Last edited by Andu21; 18-04-2018 at 11:07.
Reply With Quote
The Following User Says Thank You to Andu21 For This Useful Post:
Spexxos (18-04-2018)
  #26  
Old 18-04-2018, 13:53
Spexxos Spexxos is offline
Registered User
 
Join Date: Apr 2017
Location: Germany
Posts: 13
Thanks: 21
Thanked 1 Time in 1 Post
Spexxos is on a distinguished road
According to this page [SPOILER]https://translate.googleusercontent....ygj-rpHOhRlC3g[/SPOILER]

these setting should provide good compression without sacrificing it for a faster compression. Am I right? Results are ~7% better than with lzma2 in my testing. Is there room for any improvement? lolz has a lot of options I havent even touched yet.

"lolz -mmt0 -mt1 -mtb64 -d2032 input output"


How do you acutally unpack lolz archives?

Last edited by Spexxos; 18-04-2018 at 13:57.
Reply With Quote
  #27  
Old 19-04-2018, 17:45
Spexxos Spexxos is offline
Registered User
 
Join Date: Apr 2017
Location: Germany
Posts: 13
Thanks: 21
Thanked 1 Time in 1 Post
Spexxos is on a distinguished road
So, I have been testing alot the last two days. Compressors like lolz, lz4,5, mcm, rzm nz etc. I found that Nanozip gives me the best compression ratio. Do you guys think that Nanozip is reliable enough for a longterm use?
Reply With Quote
  #28  
Old 20-04-2018, 17:33
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 224
Thanks: 163
Thanked 287 Times in 99 Posts
elit is on a distinguished road
Here is how it work today:

You install the game to get its individual files in folder. First, nowadays we have many specialized compressors, we have bpk for compressing bik/bk2 (bink) videos, we have mpz for mp3 files, we have oggre for ogg files, we have packjpg for jpg files and some others but these count most.

So you use their respective compressors for those. Next we replaced internal lzma of FA for 64bit variant(to allow bigger dictionaries etc). XZ/7zip is one choice, I made xnlz by modifying plzip to get as close to FA:xlzma experience as possible in terms of speed and features, however you can also use FA's default its not that big difference.

Bigger difference is from lolz which can pull ~10% better to lzma. Lolz is a general purpose compressor but specialized on textures using lzma with additional cm modeling for textures/images, whose can detect within files, so its good for games. Its slower so you may combine it with lzma for different files. Its not really better outside of textures than lzma thats why.

As you may noticed, I spoke lot about specialization and individual files, thats the game now. If game files are in packs and compressed, then(and only then) you need tools like ztool, xtool,punity or precomp. In fact for precomp I dont see much purpose anymore, it doesnt support stdio and is generally slower, really tools from Razor is all about. Also, often you may need to go deeper and extract/repack game files using quickbms scripts and other tools, for example you can only compress XCOM2 WotCh that way as it use oodle compression.

I forgot, there are also specialized precomps for certain engines, uelr for Unreal's lzo, afr for anvil(Ubisoft's .forge) lzo, xtool for CRI, punity for Unity's lz4. Ztool is for older lzo/zstd/lz4 but work always for pzlib/deflate.

To scan files you can use GFS scanner or my lz_scan, GFS have nice UI but is buggy and unstable in anything other than zlib/deflate. lz_scan also support afr/uelr scan.

Srep you use latest 393a 64bit, FA 0.67 as it fixed *lot* of bugs and added features, dont even think about anything lower.

Finally, you generally use default internal delta of FA on all but images/textures. As for xdelta, this has to do with recomposing CRC matched files back. For example, before xtool you would have to use quickbms with cpk.bms to extract .cpk files. When you repack them back they were not same because of header data change so you would use xdelta for that final touch. Same goes for recompressing .wem files. Other than that I dont see much point for xdelta compression itself, its all about canceling duplicates and srep is already doing that AFAIK(FitGirl you may want to step here to explain better if I am wrong).


1. What is DELTA and when to use it?
A: delta type compression for canceling duplicates and get diffs, use always except on textures, or never, usually no big deal
2. When to use what precompressor or any of the programs following? (precomp, srep, pzlib, ztool, xtool, PrecompMT, msc )
I have seen different versions of precompressors delivering different results. Sometimes older versions worked better than newer ones?
A: first forget precomp(MT),pzlib,msc and all that crap, just use:
ztool for deflate/pzlib,lzo,lz4,zstd(last 3 will likely not work on newest games)
xtool for deltate/pzlib,CRI
punity for *some* unity games
uelr for most unreal games(.tfc, .upk...)
afr for anvil games
srep always, its for data deduplication not a recompressor
3. Why is FreeArc alsways the recommended tool? Depending on my files I got better results with also 7z (lzma2) and rar? Is it because of the ease of use for external compressors?
A: Because FA is so much more that that. I dont think you can use bpk+oggre+mpz+packjpg+uelr+srep+lzma in one go, on files sorted with one of many advanced ways best fitting its game files structure, in anything else other than FA or based on FA, can you? You can use 7zip as external LZMA within FA if you like.
4. Whats are the benefits of xz? (In my unterstanding its just a container, am i right?)
A: Same as 7zip or xnlz, 64bit support
5. Is there any benefit in using FreeArcs successor (FreeArcNext(?)?)
A: Its still early alpha/beta and lack feature I believe. Its also not (supposed) to be free once finished. My personal opinion is that Bulat pretty much killed his own legacy, almost nobody will use it and he is wasting his time.
6. Most importantly: Are there any already written guides that I have missed that have answers to my questions?
A: Start reading threads on all mentioned tools upside down both here and on encode.ru. In fact, read this whole site(and encode). Then learn Russian and read whole Krinkels.org website. Then (and only then), you will become Jedi.


So thats about it. As you see its not anymore old times when you could just precomp+(srep)+FA and thought you done. Today its much more complicated and often exotic practices are required. Good luck.

Last edited by elit; 20-04-2018 at 17:44.
Reply With Quote
The Following 5 Users Say Thank You to elit For This Useful Post:
EzzEldin16 (20-04-2018), flaek1337 (08-06-2018), mubbii (12-11-2018), PsYcHo_RaGE (28-04-2018), Spexxos (20-04-2018)
  #29  
Old 28-04-2018, 23:57
PsYcHo_RaGE's Avatar
PsYcHo_RaGE PsYcHo_RaGE is offline
Registered User
 
Join Date: Mar 2018
Location: DarckSide
Posts: 239
Thanks: 194
Thanked 157 Times in 77 Posts
PsYcHo_RaGE is on a distinguished road
Quote:
Originally Posted by elit View Post
Here is how it work today:

You install the game to get its individual files in folder. First, nowadays we have many specialized compressors, we have bpk for compressing bik/bk2 (bink) videos, we have mpz for mp3 files, we have oggre for ogg files, we have packjpg for jpg files and some others but these count most.

So you use their respective compressors for those. Next we replaced internal lzma of FA for 64bit variant(to allow bigger dictionaries etc). XZ/7zip is one choice, I made xnlz by modifying plzip to get as close to FA:xlzma experience as possible in terms of speed and features, however you can also use FA's default its not that big difference.

Bigger difference is from lolz which can pull ~10% better to lzma. Lolz is a general purpose compressor but specialized on textures using lzma with additional cm modeling for textures/images, whose can detect within files, so its good for games. Its slower so you may combine it with lzma for different files. Its not really better outside of textures than lzma thats why.

As you may noticed, I spoke lot about specialization and individual files, thats the game now. If game files are in packs and compressed, then(and only then) you need tools like ztool, xtool,punity or precomp. In fact for precomp I dont see much purpose anymore, it doesnt support stdio and is generally slower, really tools from Razor is all about. Also, often you may need to go deeper and extract/repack game files using quickbms scripts and other tools, for example you can only compress XCOM2 WotCh that way as it use oodle compression.

I forgot, there are also specialized precomps for certain engines, uelr for Unreal's lzo, afr for anvil(Ubisoft's .forge) lzo, xtool for CRI, punity for Unity's lz4. Ztool is for older lzo/zstd/lz4 but work always for pzlib/deflate.

To scan files you can use GFS scanner or my lz_scan, GFS have nice UI but is buggy and unstable in anything other than zlib/deflate. lz_scan also support afr/uelr scan.

Srep you use latest 393a 64bit, FA 0.67 as it fixed *lot* of bugs and added features, dont even think about anything lower.

Finally, you generally use default internal delta of FA on all but images/textures. As for xdelta, this has to do with recomposing CRC matched files back. For example, before xtool you would have to use quickbms with cpk.bms to extract .cpk files. When you repack them back they were not same because of header data change so you would use xdelta for that final touch. Same goes for recompressing .wem files. Other than that I dont see much point for xdelta compression itself, its all about canceling duplicates and srep is already doing that AFAIK(FitGirl you may want to step here to explain better if I am wrong).


1. What is DELTA and when to use it?
A: delta type compression for canceling duplicates and get diffs, use always except on textures, or never, usually no big deal
2. When to use what precompressor or any of the programs following? (precomp, srep, pzlib, ztool, xtool, PrecompMT, msc )
I have seen different versions of precompressors delivering different results. Sometimes older versions worked better than newer ones?
A: first forget precomp(MT),pzlib,msc and all that crap, just use:
ztool for deflate/pzlib,lzo,lz4,zstd(last 3 will likely not work on newest games)
xtool for deltate/pzlib,CRI
punity for *some* unity games
uelr for most unreal games(.tfc, .upk...)
afr for anvil games
srep always, its for data deduplication not a recompressor
3. Why is FreeArc alsways the recommended tool? Depending on my files I got better results with also 7z (lzma2) and rar? Is it because of the ease of use for external compressors?
A: Because FA is so much more that that. I dont think you can use bpk+oggre+mpz+packjpg+uelr+srep+lzma in one go, on files sorted with one of many advanced ways best fitting its game files structure, in anything else other than FA or based on FA, can you? You can use 7zip as external LZMA within FA if you like.
4. Whats are the benefits of xz? (In my unterstanding its just a container, am i right?)
A: Same as 7zip or xnlz, 64bit support
5. Is there any benefit in using FreeArcs successor (FreeArcNext(?)?)
A: Its still early alpha/beta and lack feature I believe. Its also not (supposed) to be free once finished. My personal opinion is that Bulat pretty much killed his own legacy, almost nobody will use it and he is wasting his time.
6. Most importantly: Are there any already written guides that I have missed that have answers to my questions?
A: Start reading threads on all mentioned tools upside down both here and on encode.ru. In fact, read this whole site(and encode). Then learn Russian and read whole Krinkels.org website. Then (and only then), you will become Jedi.


So thats about it. As you see its not anymore old times when you could just precomp+(srep)+FA and thought you done. Today its much more complicated and often exotic practices are required. Good luck.
Can i have your lz_scan??
Reply With Quote
The Following User Says Thank You to PsYcHo_RaGE For This Useful Post:
mubbii (12-11-2018)
  #30  
Old 29-04-2018, 00:08
PsYcHo_RaGE's Avatar
PsYcHo_RaGE PsYcHo_RaGE is offline
Registered User
 
Join Date: Mar 2018
Location: DarckSide
Posts: 239
Thanks: 194
Thanked 157 Times in 77 Posts
PsYcHo_RaGE is on a distinguished road
Unfortunately i am too here a newbie i mean newbie to all these things that but i have been reading every post here since 3 months, dude only 3 months

i am new to all these things only 3 months, i was just wandering around looking for games on repack sites and i did't know what was repacking and what r repacks my only target was to find the game download it and play, this was my life after that i was curious whats it different sites different game sizes i feel very curious about what it was and then after searching a lot of stuff on internet i found these forums joined em and start reading posts, using discord chat with new friends and now its still going, now i can repack most of games on my PC, still i am noob i want to be a jedi as @elit mentioned
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
KaOs Krew compressing method PanosBabo PC Games - CD/DVD Conversions 5 10-04-2021 04:41
Simple Tutorial of Compressing GTAV Razor12911 Conversion Tutorials 26 29-01-2019 01:31
Tutorial about compressing games faster using UltraArc 78372 Conversion Tutorials 2 29-04-2017 17:25
NASCAR The Game 2013 (inno setup script) sentinelks PC Games - CD/DVD Conversions 28 08-09-2013 20:30
problem with compressing .bik game video files -sbx- General Gaming 6 23-07-2010 12:20



All times are GMT -7. The time now is 12:50.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Copyright 2000-2020, FileForums @ https://fileforums.com