Go Back   FileForums > Game Backup > PC Games > PC Games - CD/DVD Conversions > Conversion Tutorials

Reply
 
Thread Tools Display Modes
  #3181  
Old 29-09-2022, 14:19
Dragonis40 Dragonis40 is offline
Registered User
 
Join Date: Mar 2021
Location: italy
Posts: 37
Thanks: 0
Thanked 1 Time in 1 Post
Dragonis40 is on a distinguished road
Quote:
Originally Posted by Masquerade View Post
Don't use XLolz - there's absolutely no need for it to exist. Lolz decompression speed through cls-lolz plugin is imensely quick and already much faster to the point where your limiting factor is the speed of your storage medium, not the algo.

Your decompression speed will be limited by any precompressor you have used and what algo it's trying to recompress. I think you were talking about CyberPunk 2077, right? That game uses Oodle Kraken compression and XTool is working it's ass off to compress that for you as quick as possible.
I'm talking about Cyberpunk 2077. I've used Oodle Kraken (the one you can find into bin\x64 folder) to compress most of .archive files. I've also noticed two of them can be decompressed and files inside can be compressed with other methods.
Of course I've used Oodle Kraken file for decompression as well.
For decompression, I've used cls-srep, cls-lolz, xtool_2020, precomp and something else to compress decompressed .archive files.
I'm still unexperienced in compressing and decompressing world, but how can Fitgirl version be 200 mb smaller then my final compressed .bin file and taking almost 20 minutes less (installation time) then mine?
Reply With Quote
Sponsored Links
  #3182  
Old 29-09-2022, 14:32
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 888
Thanks: 224
Thanked 993 Times in 475 Posts
Masquerade is on a distinguished road
OK now you've completely lost me. The archives are already compressed with kraken. You use the precompressor (XTool) on them. There's absolutely no need to use precomp here. It's not needed.
Reply With Quote
  #3183  
Old 29-09-2022, 15:07
Dragonis40 Dragonis40 is offline
Registered User
 
Join Date: Mar 2021
Location: italy
Posts: 37
Thanks: 0
Thanked 1 Time in 1 Post
Dragonis40 is on a distinguished road
Quote:
Originally Posted by Masquerade View Post
OK now you've completely lost me. The archives are already compressed with kraken. You use the precompressor (XTool) on them. There's absolutely no need to use precomp here. It's not needed.
Let me better explain: I've used precomp only for "Bonus content" files, not for .archive ones. Two .archive files, once decompressed, give better compression result rather than original .archive files. That's the reason why I've decompressed two .archive files.
Reply With Quote
  #3184  
Old 29-09-2022, 15:12
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 888
Thanks: 224
Thanked 993 Times in 475 Posts
Masquerade is on a distinguished road
Brewmaster: Beer Brewing Simulator

Code:
xzlib+srep+lolz/$jpg=brunsli
  • 3.2GB ---> 1.5GB
Reply With Quote
  #3185  
Old 29-09-2022, 19:19
L33THAK0R's Avatar
L33THAK0R L33THAK0R is offline
Registered User
 
Join Date: Feb 2021
Location: Saudi Arabia
Posts: 283
Thanks: 98
Thanked 28 Times in 24 Posts
L33THAK0R is on a distinguished road
Whats the best method for processing unencrypted UE4 ".pak" files? I've tried just applying xtool's zlib+reflate / oodle codecs but whilst the streams are picked up I always have issues decompressing, with the archives being reported as corrupted. I used to just process the pak's by unpacking and compressing the loose files but with larger titles this isn't very efficient for the user's disk I/O and with some titles I'm investigating currently (such as "Phantom Doctrine"), seemingly random files fail to decompress, pushing me to investigate again into packing the compressed ".pak" files as they're originally presented instead.
Reply With Quote
  #3186  
Old 29-09-2022, 19:28
L33THAK0R's Avatar
L33THAK0R L33THAK0R is offline
Registered User
 
Join Date: Feb 2021
Location: Saudi Arabia
Posts: 283
Thanks: 98
Thanked 28 Times in 24 Posts
L33THAK0R is on a distinguished road
Also quick second question, has anyone else had issues with using AFR on AnvilEngine 1.0 titles (or whatever the engine was named prior to AnvilNext)? I'm getting some ".forge" titles being processed with a warning at the end of processing them with AFR: "WARNING! In file found corrupted LZO-streams. Check your data!" and I have to manually test each files or a batch of files to determine which produce this error. packing the files that produce this error separately to the ones that don't results in a consistent successful decompression however the size is inflated due to them not being de-duplicated together. Currently affected titles that I've noticed are "Assassin's Creed" "III", "Black Flag", "Freedom Cry" & "Rogue". I'll note that I've been using AFR v0.0.1.9, using the "-vr" parameter as well as Srep v0.392, since other versions have produced issues with the "*.forge" files from my experience.
Reply With Quote
  #3187  
Old 30-09-2022, 03:40
KaktoR's Avatar
KaktoR KaktoR is offline
Lame User
 
Join Date: Jan 2012
Location: From outer space
Posts: 3,817
Thanks: 996
Thanked 6,237 Times in 2,311 Posts
KaktoR is on a distinguished road
Quote:
Originally Posted by L33THAK0R View Post
Also quick second question, has anyone else had issues with using AFR on AnvilEngine 1.0 titles (or whatever the engine was named prior to AnvilNext)? I'm getting some ".forge" titles being processed with a warning at the end of processing them with AFR: "WARNING! In file found corrupted LZO-streams. Check your data!" and I have to manually test each files or a batch of files to determine which produce this error. packing the files that produce this error separately to the ones that don't results in a consistent successful decompression however the size is inflated due to them not being de-duplicated together. Currently affected titles that I've noticed are "Assassin's Creed" "III", "Black Flag", "Freedom Cry" & "Rogue". I'll note that I've been using AFR v0.0.1.9, using the "-vr" parameter as well as Srep v0.392, since other versions have produced issues with the "*.forge" files from my experience.
I don't know where your "problem" is exactly now. You can ignore the AFR message about corrputed LZO-Streams. This just means that some streams aren't compressed with lzo, use another codec or lzo version or whatever else. Just ignore it.
__________________
Haters gonna hate
Reply With Quote
  #3188  
Old 30-09-2022, 03:42
KaktoR's Avatar
KaktoR KaktoR is offline
Lame User
 
Join Date: Jan 2012
Location: From outer space
Posts: 3,817
Thanks: 996
Thanked 6,237 Times in 2,311 Posts
KaktoR is on a distinguished road
Quote:
Originally Posted by L33THAK0R View Post
Whats the best method for processing unencrypted UE4 ".pak" files? I've tried just applying xtool's zlib+reflate / oodle codecs but whilst the streams are picked up I always have issues decompressing, with the archives being reported as corrupted. I used to just process the pak's by unpacking and compressing the loose files but with larger titles this isn't very efficient for the user's disk I/O and with some titles I'm investigating currently (such as "Phantom Doctrine"), seemingly random files fail to decompress, pushing me to investigate again into packing the compressed ".pak" files as they're originally presented instead.
I took your example "Phantom Doctrine" and it works fine with xtool+srep+lzma, so... ?
__________________
Haters gonna hate
Reply With Quote
  #3189  
Old 30-09-2022, 08:02
L33THAK0R's Avatar
L33THAK0R L33THAK0R is offline
Registered User
 
Join Date: Feb 2021
Location: Saudi Arabia
Posts: 283
Thanks: 98
Thanked 28 Times in 24 Posts
L33THAK0R is on a distinguished road
Quote:
Originally Posted by KaktoR View Post
I took your example "Phantom Doctrine" and it works fine with xtool+srep+lzma, so... ?
Ah cheers, maybe it's lolz then thats acting up, I'll definitely have to do a bit of testing I reckon, thanks for testing it on your end!

Quote:
Originally Posted by KaktoR View Post
I don't know where your "problem" is exactly now. You can ignore the AFR message about corrputed LZO-Streams. This just means that some streams aren't compressed with lzo, use another codec or lzo version or whatever else. Just ignore it.
Ah alright, I'll definitely give recompressing with another codec a shot, since that might be whats messing things up on my end, honestly cant think of what else it could be considering I'm getting decompression issues when just packing with afr+srep (to test).
Reply With Quote
  #3190  
Old 30-09-2022, 08:05
KaktoR's Avatar
KaktoR KaktoR is offline
Lame User
 
Join Date: Jan 2012
Location: From outer space
Posts: 3,817
Thanks: 996
Thanked 6,237 Times in 2,311 Posts
KaktoR is on a distinguished road
There is anvil plugin for xtool, you should give it a try.
__________________
Haters gonna hate
Reply With Quote
  #3191  
Old 30-09-2022, 13:44
hwang4 hwang4 is offline
Registered User
 
Join Date: Sep 2007
Location: Algeria
Posts: 79
Thanks: 40
Thanked 10 Times in 4 Posts
hwang4 is on a distinguished road
Ebola 3

Masquerade , thank you for your help!

Finally , I found the right oo2core.dll!

oo2reck (7) + srep+ lolz

23gb ------> 16.6gb
Attached Files
File Type: rar oo2core_7_win64.rar (223.8 KB, 8 views)
Reply With Quote
The Following User Says Thank You to hwang4 For This Useful Post:
Masquerade (30-09-2022)
  #3192  
Old 01-10-2022, 04:59
KaktoR's Avatar
KaktoR KaktoR is offline
Lame User
 
Join Date: Jan 2012
Location: From outer space
Posts: 3,817
Thanks: 996
Thanked 6,237 Times in 2,311 Posts
KaktoR is on a distinguished road
Pathfinder Wrath of the Righteous
Enhanced Edition

Code:
 10:58:21 - Selected ARC/DS method for Data1a-01.bin was: xtool:munity,lz4hc,l12:lz4_193+srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
 10:58:21 - Selected ARC/DS method for Data1b-01.bin was: srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
-------------------------------------------------------------------------------------------------------------------------------------------
 10:58:22 - Compression initialized in 01.10.2022 at 10:58:22.
 10:58:23 - Starting Data1a-01.bin archive creation.
 12:46:03 - Moving Data1a-01.bin file to media folder.
 12:46:04 - Starting Data1b-01.bin archive creation.
 12:59:09 - Moving Data1b-01.bin file to media folder.
 12:59:10 - Compression finished in 01.10.2022 at 12:59:10.
-------------------------------------------------------------------------------------------------------------------------------------------
 12:59:10 - Getting size of the game: Pathfinder - Wrath of the Righteous
 12:59:10 - » Installation folder size: 37.88 GB
 12:59:10 - » Data1a-01.bin file size: 14.17 GB
 12:59:10 - » Data1b-01.bin file size: 3.85 GB
 12:59:10 - » Compressed game size: 18.02 GB (Ratio 47.58%)
-------------------------------------------------------------------------------------------------------------------------------------------
 12:59:10 - Overall input size: 37.88 GB
 12:59:10 - Overall output size: 18.02 GB (Ratio 47.58%)
 12:59:10 - Overall conversion time: 02:00:38
__________________
Haters gonna hate
Reply With Quote
The Following 2 Users Say Thank You to KaktoR For This Useful Post:
JustFun (06-10-2022), ScOOt3r (01-10-2022)
  #3193  
Old 01-10-2022, 08:02
KaktoR's Avatar
KaktoR KaktoR is offline
Lame User
 
Join Date: Jan 2012
Location: From outer space
Posts: 3,817
Thanks: 996
Thanked 6,237 Times in 2,311 Posts
KaktoR is on a distinguished road
Command & Conquer Remastered Collection

I made this as a test and showcase with use of quickbms + bink patching. In this case use of quickbms is possible because game files are not compressed (or at least not seem).

Steps:
  1. Make Backup of original archives (Data\movies_*.meg)
  2. Unpack original archives (Data\movies_*.meg) with quickbms
    Code:
    for %%f in ("MOVIES_*.MEG") do quickbms_4gb_files -d -o -Y petroglyph.bms "%%f"
  3. Compress binks and make patches (bink_pack.exe, bink_unpack.exe, hdiffz.exe, hpatchz.exe)
    There are some bk2 files which even bink_unpack.exe has problems with, so patch size is same as original size (416mb diff size). Take care of them.
  4. Null unpacked bk2 files (fill contents with zeros)
  5. Reimport nulled bk2 files with quickbms into original archives (petroglyph.bms)
    Code:
    quickbms_4gb_files -w -r petroglyph.bms "input .meg file" "input folder containing nulled bk2 files"
    Before reimporting you have to separate all the files in the unpacked folders, you only need bk2 files for reimporting
    Tbh I don't know if quickbms has a option to just import defined file types lol
    Remove Data\movies_*.meg.bak files if there are any (quickbms creates these files I guess)
  6. Delete all (nulled) bk2 files, so you have just compressed binks and patch files + nulled .meg archives left
    These nulled .meg archives have a compression ratio by nearly 0%, so don't mind the size of these (14.8 GB -> some KB)
  7. Compress the game
  8. Include required files in Installer or game folder (before compress the game)
  9. Let your Installer run a batch file to do it's job after extracting all files (example batch)

Installation/batch stuff will take some time, .meg archives are crc perfect at the end.

Results:
Code:
 15:11:02 - Selected ARC/DS method for Data1a-01.bin was: srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
-------------------------------------------------------------------------------------------------------------------------------------------
 15:35:12 - Overall input size: 36.90 GB
 15:35:12 - Overall output size: 15.67 GB (Ratio 42.47%)
 15:35:12 - Overall conversion time: 00:24:08
Results without quickbms + bink patching stuff:
https://fileforums.com/showpost.php?...postcount=1772
__________________
Haters gonna hate

Last edited by KaktoR; 01-10-2022 at 08:15.
Reply With Quote
The Following 3 Users Say Thank You to KaktoR For This Useful Post:
Gehrman (01-10-2022), JustFun (06-10-2022), kuyhaa (02-10-2022)
  #3194  
Old 02-10-2022, 16:42
Wanterlude's Avatar
Wanterlude Wanterlude is offline
Registered User
 
Join Date: Apr 2017
Location: Russia
Posts: 8
Thanks: 62
Thanked 12 Times in 3 Posts
Wanterlude is on a distinguished road
Terra Invicta (v.0.3.18)

Code:
XTool (Unity plugin + liblz4 1.9.3)+srep+lolz

Compressed 979 files, 20,418,359,013 => 7,908,853,159 bytes. Ratio 38.73%
Compression time: cpu 149.16 sec/real 9259.21 sec = 2%. Speed 2.21 mB/s
Overall conversion time: 02:34:21

19.02 GB => 7.37 GB

Last edited by Wanterlude; 02-10-2022 at 16:48.
Reply With Quote
The Following 3 Users Say Thank You to Wanterlude For This Useful Post:
dixen (03-10-2022), Gehrman (28-10-2022), JustFun (06-10-2022)
  #3195  
Old 02-10-2022, 20:33
angeldarkRS's Avatar
angeldarkRS angeldarkRS is offline
Registered User
 
Join Date: May 2014
Location: Philippines
Posts: 3
Thanks: 0
Thanked 0 Times in 0 Posts
angeldarkRS is on a distinguished road
Quote:
Originally Posted by Wanterlude View Post
Terra Invicta (v.0.3.18)

Code:
XTool (Unity plugin + liblz4 1.9.3)+srep+lolz

Compressed 979 files, 20,418,359,013 => 7,908,853,159 bytes. Ratio 38.73%
Compression time: cpu 149.16 sec/real 9259.21 sec = 2%. Speed 2.21 mB/s
Overall conversion time: 02:34:21

19.02 GB => 7.37 GB
What specs your PC/Laptop?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Games Are For Kids JoyBoy Chit Chat 83 16-11-2021 15:28
CorePack Compression methods ? Help ramazan19833 Conversion Tutorials 34 09-08-2018 07:51
i got the email in the mail about copying ps2 games EVOEvo PS2 Games 7 21-11-2003 08:56
Self-booting games on an older Japanese DC? Named DC Games 1 26-09-2003 18:48
Having Problems Burning Numerous *Recent* Games....Help is Appreciated Protosstic PSX Games 2 15-01-2002 22:27



All times are GMT -7. The time now is 01:42.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Copyright 2000-2020, FileForums @ https://fileforums.com