Go Back   FileForums > Game Backup > PC Games > PC Games - CD/DVD Conversions > Conversion Tutorials

Reply
 
Thread Tools Display Modes
  #3196  
Old 02-10-2022, 20:36
Wanterlude's Avatar
Wanterlude Wanterlude is offline
Registered User
 
Join Date: Apr 2017
Location: Swamp in Russia
Posts: 34
Thanks: 208
Thanked 60 Times in 20 Posts
Wanterlude is on a distinguished road
Quote:
Originally Posted by angeldarkRS View Post
What specs your PC/Laptop?
11th Gen Intel(R) Core(TM) i5-11400F
16,0 GB RAM
M.2 SSD
Reply With Quote
Sponsored Links
  #3197  
Old 03-10-2022, 19:11
L33THAK0R's Avatar
L33THAK0R L33THAK0R is offline
Registered User
 
Join Date: Feb 2021
Location: Saudi Arabia
Posts: 350
Thanks: 119
Thanked 58 Times in 37 Posts
L33THAK0R is on a distinguished road
Quote:
Originally Posted by L33THAK0R View Post
Also quick second question, has anyone else had issues with using AFR on AnvilEngine 1.0 titles (or whatever the engine was named prior to AnvilNext)? I'm getting some ".forge" titles being processed with a warning at the end of processing them with AFR: "WARNING! In file found corrupted LZO-streams. Check your data!" and I have to manually test each files or a batch of files to determine which produce this error. packing the files that produce this error separately to the ones that don't results in a consistent successful decompression however the size is inflated due to them not being de-duplicated together. Currently affected titles that I've noticed are "Assassin's Creed" "III", "Black Flag", "Freedom Cry" & "Rogue". I'll note that I've been using AFR v0.0.1.9, using the "-vr" parameter as well as Srep v0.392, since other versions have produced issues with the "*.forge" files from my experience.
For anyone interested or who's encountered similar issues. I found out the cause of my troubles with both this issue and the immediate post prior to this one. Recently I streamlined and revamped my whole pipeline for packing titles and in the process accidentally included an outdated version of Srep somehow, using an updated version has fixed all my issues thankfully.
Reply With Quote
The Following User Says Thank You to L33THAK0R For This Useful Post:
ScOOt3r (04-10-2022)
  #3198  
Old 04-10-2022, 22:00
L33THAK0R's Avatar
L33THAK0R L33THAK0R is offline
Registered User
 
Join Date: Feb 2021
Location: Saudi Arabia
Posts: 350
Thanks: 119
Thanked 58 Times in 37 Posts
L33THAK0R is on a distinguished road
Sleeping Dogs Update

Quote:
Originally Posted by L33THAK0R View Post
Hi all,

Been looking into Sleeping Dogs, and after a bit of discussion with some more experienced/technical tool engineers, I've got to say I'm quite intrigued. It uses a very aggressive "LZ" algorithm, with a given ".BIG" (archive format for the title in question) decompressing to over twice its original size! It also seems the WWise ".pck" files used for audio within the title have the potential for additional gains using "OGGRE"/"MSC". Unfortunately for the former I don't possess the skill-set (yet) to adapt the provided compression/decompression algorithm to be able to parse inputs for use with a chain of compressors, but none the less I think its still interesting enough for others to see.

Attached is a tool suite for processing the aforementioned ".BIG" archives, as well as a sample archive.
Came back to this one after a bit of a break and after learning about xtool's "verbose" mode, turns out LZO is used on the archives, however it seems that no matter what level depth setting or library I use no streams are able to be processed, does anyone know the reason behind this? Attached is an image supporting my findings.
Reply With Quote
  #3199  
Old 05-10-2022, 01:19
kuyhaa's Avatar
kuyhaa kuyhaa is offline
Registered User
 
Join Date: Jun 2019
Location: metro
Posts: 54
Thanks: 88
Thanked 8 Times in 4 Posts
kuyhaa is on a distinguished road
hi, how to TEXTURE Rebuild and compression .dds file use "crunch" ?
Reply With Quote
  #3200  
Old 05-10-2022, 05:21
KaktoR's Avatar
KaktoR KaktoR is offline
Lame User
 
Join Date: Jan 2012
Location: From outer space
Posts: 4,398
Thanks: 1,077
Thanked 7,004 Times in 2,653 Posts
KaktoR is on a distinguished road
Dakar Desert Rally

bk2 (kb2n) = no bpk
pak = kraken (oo2core 7)

Code:
Streams: 17099/17099
Time: 00:12:38 (00:12:36)
Memory: 245 MB (245 MB)

Compressed 1 file, 2,125,021,534 => 4,260,018,174 bytes. Ratio 200.47%
Compression time: cpu 1.53 sec/real 125.97 sec = 1%. Speed 16.87 mB/s
All OK
__________________
Haters gonna hate

Last edited by KaktoR; 05-10-2022 at 05:38.
Reply With Quote
  #3201  
Old 05-10-2022, 15:18
Edison007's Avatar
Edison007 Edison007 is offline
Registered User
 
Join Date: Mar 2014
Location: Solar System - Planet Earth
Posts: 30
Thanks: 0
Thanked 56 Times in 19 Posts
Edison007 is on a distinguished road
Quote:
Came back to this one after a bit of a break and after learning about xtool's "verbose" mode, turns out LZO is used on the archives, however it seems that no matter what level depth setting or library I use no streams are able to be processed, does anyone know the reason behind this? Attached is an image supporting my findings.
No, in the game does not use LZO.
Тhis is a private(?) LZ algorithm. You can read this thread: https://forum.xentax.com/viewtopic.php?f=10&t=9481


Quote:
hi, how to TEXTURE Rebuild and compression .dds file use "crunch" ?
crunch = lossy, not suitable for repacking
Reply With Quote
  #3202  
Old 05-10-2022, 17:39
L33THAK0R's Avatar
L33THAK0R L33THAK0R is offline
Registered User
 
Join Date: Feb 2021
Location: Saudi Arabia
Posts: 350
Thanks: 119
Thanked 58 Times in 37 Posts
L33THAK0R is on a distinguished road
Quote:
Originally Posted by Edison007 View Post
No, in the game does not use LZO.
Тhis is a private(?) LZ algorithm. You can read this thread: https://forum.xentax.com/viewtopic.php?f=10&t=9481
I remember reading through that entire thread when I was first researching this bugger of a title, had a hope perhaps the lads over at the Xentax forums had been wrong, and that the PMCQ algorithim was just a fork of LZO. I actually managed to talk to Ekey (a very talented developer who's quite experienced with developing various decompression suites for a variety of titles), but he unfortunately lost his source code for his Sleeping Dogs decompression tool. I'm not sure if you'd be able to answer this but do you know why xtool might be reporting lzo1x streams? Is this just as simple as being an error on xtool's part? Honestly might try again to adapt his original decompression function (which we still thankfully have the source code for) as an xtool plugin, but given my lack of success in previous attempts I doubt I'll make much progress.

Last edited by L33THAK0R; 05-10-2022 at 17:46.
Reply With Quote
  #3203  
Old 06-10-2022, 02:11
Edison007's Avatar
Edison007 Edison007 is offline
Registered User
 
Join Date: Mar 2014
Location: Solar System - Planet Earth
Posts: 30
Thanks: 0
Thanked 56 Times in 19 Posts
Edison007 is on a distinguished road
Quote:
ad a hope perhaps the lads over at the Xentax forums had been wrong, and that the PMCQ algorithim was just a fork of LZO.
No This is a different algorithm.

Quote:
Is this just as simple as being an error on xtool's part?
Yep. False positive detect.

Quote:
Honestly might try again to adapt his original decompression function
You also need a compress function.
Reply With Quote
  #3204  
Old 06-10-2022, 06:51
L33THAK0R's Avatar
L33THAK0R L33THAK0R is offline
Registered User
 
Join Date: Feb 2021
Location: Saudi Arabia
Posts: 350
Thanks: 119
Thanked 58 Times in 37 Posts
L33THAK0R is on a distinguished road
Quote:
Originally Posted by Edison007 View Post
No This is a different algorithm.
Drat! It's always bizarre how complex some titles can be to properly process!

Quote:
Originally Posted by Edison007 View Post
You also need a compress function.
Thats going to be a fun time trying to figure out, heres hoping its lying around somewhere on the web!
Reply With Quote
  #3205  
Old 07-10-2022, 10:07
kj911 kj911 is offline
Registered User
 
Join Date: Apr 2010
Location: world
Posts: 182
Thanks: 132
Thanked 63 Times in 44 Posts
kj911 is on a distinguished road
Best compression methods from Stalker: Shadow of Chernobyl?? The game compressibble in smaller than 2-2.5GB's size? (with or without MP files. Multilang version installed size in has ~9.5GB's. Installer smaller than ~3.2GB with INNO only.)

Tested one gamedata archive file (*.db5), in few methods. This db5 archive contents extracted in specialized tools and extracted files compressed again into separated 3 archives and results its bigger than compress the one *.db5 file. Not understood!

File: gamedata.db5
Size: 677 500kB

CAB/LZX:21: 520 571kB
7z: 498 055kB
7z/PPMD: 529 581kB
SQX2: 492 547kB
DLZ only: 456 021kB
OGGRE+DLZ: 462 367kB

MSC only (BMP/DDSRAW/DXT=1): 550 356kB* (archive broken! Any data portions its missing??) --> Crunch in DLZ: 387 155kB (probably false...)
MSC only (DDSRAW=0, DXT=1): 677 377kB --> Crunch in DLZ: 478 327kB

Extracted the db5 file:

*.OGM: 68 290kB (6 file) --> DLZ: 59 064kB (Note: OGGRE+DLZ: 59 061kB)
*.OGG: 174 445kB (2052 file) --> OGGRE: 146 989kB
*.DDS: 446 886kB (1249 file) --> MSC+DLZ: 317 108kB / DLZ: 294 630kB
other: 908kB (17 file) --> MSC+DLZ: 103kB
__________________
Total: 690 528kB --> 523 264kB / 500 786kB (WTF!)
Reply With Quote
  #3206  
Old 11-10-2022, 12:47
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 1,161
Thanks: 284
Thanked 1,354 Times in 614 Posts
Masquerade is on a distinguished road
Overcooked! 2

Code:
xtool_unity+srep+lolz
7.9GB ----> 3.7GB

liblz4 v1.8.0 used for best ratio.
Reply With Quote
The Following 4 Users Say Thank You to Masquerade For This Useful Post:
Cesar82 (11-10-2022), JustFun (11-12-2022), ScOOt3r (12-10-2022), Wanterlude (12-10-2022)
  #3207  
Old 11-10-2022, 12:49
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 1,161
Thanks: 284
Thanked 1,354 Times in 614 Posts
Masquerade is on a distinguished road
Prison Architect
  • Paradox Launcher files ripped.
  • All DAT files unpacked with WinRAR.
Code:
precomp:d0+srep+lolz/$ogg=oggre
670MB ---> 306MB

Thanks Nicolas Comerci for precomp STDIO fork.
Reply With Quote
The Following 5 Users Say Thank You to Masquerade For This Useful Post:
Cesar82 (11-10-2022), dixen (11-10-2022), JustFun (11-12-2022), ScOOt3r (12-10-2022), Wanterlude (12-10-2022)
  #3208  
Old 12-10-2022, 11:50
L33THAK0R's Avatar
L33THAK0R L33THAK0R is offline
Registered User
 
Join Date: Feb 2021
Location: Saudi Arabia
Posts: 350
Thanks: 119
Thanked 58 Times in 37 Posts
L33THAK0R is on a distinguished road
Grim Dawn LZ4 dropping streams

Hey all,

I've been giving "Grim Dawn" a looksie and was wondering if anyone might be able to clear something up
Code:
FreeArc 0.67 (March 15 2014) creating archive: Data.arc
Compressing 1 file, 145,766,328 bytes. Processed   0%
Compressing 145,766,328 bytes with "C:\Compressor_Resources\XTool_2020\xtool_0.6.5\XTool.exe" precomp -mgrimTEST -c512mb -t95p $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
XTool is created by Razor12911

Streams: 1396/14028
Time: 00:00:10 (00:00:36)
Memory: 160 MB (160 MB)

100%
Errorlevel=0
Compressed 1 file, 145,766,328 => 160,143,671 bytes. Ratio 109.86%
Compression time: cpu 0.09 sec/real 10.60 sec = 1%. Speed 13.75 mB/s
All OK
I've tried levels 1 - 9 so far and I only get maybe ~1 additional stream for each level increment, and I've tested essentially every version of LZ4 under the sun, (other versions only resulting in ~<100 streams being picked up), would anyone happen to know if its possible to pick up more streams or is it more likely that LZ4 was used very lightly and the other streams are inconsequential.
Reply With Quote
  #3209  
Old 12-10-2022, 15:23
KaktoR's Avatar
KaktoR KaktoR is offline
Lame User
 
Join Date: Jan 2012
Location: From outer space
Posts: 4,398
Thanks: 1,077
Thanked 7,004 Times in 2,653 Posts
KaktoR is on a distinguished road
Sample?
__________________
Haters gonna hate
Reply With Quote
  #3210  
Old 12-10-2022, 19:21
L33THAK0R's Avatar
L33THAK0R L33THAK0R is offline
Registered User
 
Join Date: Feb 2021
Location: Saudi Arabia
Posts: 350
Thanks: 119
Thanked 58 Times in 37 Posts
L33THAK0R is on a distinguished road
Quote:
Originally Posted by KaktoR View Post
Sample?
Ah whoops I forgot to attach it, within the following attachment is a sample, a BMS script to decompress "Grim Dawn"'s assets (since the native LZ4 stream detection fails to pick up any at all), as well as the LZ4 library I've found success with.
Attached Files
File Type: rar GrimDawnGoodies.rar (11.68 MB, 50 views)
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Games Are For Kids JoyBoy Chit Chat 83 16-11-2021 15:28
CorePack Compression methods ? Help ramazan19833 Conversion Tutorials 34 09-08-2018 07:51
i got the email in the mail about copying ps2 games EVOEvo PS2 Games 7 21-11-2003 08:56
Self-booting games on an older Japanese DC? Named DC Games 1 26-09-2003 18:48
Having Problems Burning Numerous *Recent* Games....Help is Appreciated Protosstic PSX Games 2 15-01-2002 22:27



All times are GMT -7. The time now is 06:20.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.
Copyright 2000-2020, FileForums @ https://fileforums.com