Go Back   FileForums > Game Backup > PC Games > PC Games - CD/DVD Conversions > Conversion Tutorials

Reply
 
Thread Tools Display Modes
  #16  
Old 25-11-2022, 23:49
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 928
Thanks: 241
Thanked 1,036 Times in 489 Posts
Masquerade is on a distinguished road
Quote:
Originally Posted by Crimzan View Post
Also how do you know it's compressed with zlib? Are you an experienced file magician and / or do you know of a scanner software that can test if a file is compressed, and what it is compressed with?
Razor made drop + scan 4 zlib which is sometimes useful.

There's also this Zenhax page where Aluigi explains how to recognise different algorithms by their headers or magic numbers: https://zenhax.com/viewtopic.php?t=27

This requires a hex editor to view the data. I like HxD.
Reply With Quote
The Following User Says Thank You to Masquerade For This Useful Post:
Crimzan (27-11-2022)
Sponsored Links
  #17  
Old 27-11-2022, 06:28
Crimzan Crimzan is offline
Registered User
 
Join Date: Dec 2020
Location: Germany
Posts: 4
Thanks: 6
Thanked 0 Times in 0 Posts
Crimzan is on a distinguished road
I am happy you guys putting in the time to answer my questions, thanks a lot!

Quote:
Originally Posted by Masquerade View Post
There's also this Zenhax page where Aluigi explains how to recognise different algorithms by their headers or magic numbers: https://zenhax.com/viewtopic.php?t=27
I don't know much about Hexadicmal, I do have HxD and tried a 7z (LZMA2) and a standard windows zip file (I suspect LZMA because it says so in the link?); however, none of the Headers shown in the link are actually seen in the Hexadecimal view of these files for some reason. Why is that? Could it be that the header is just offset or are some diferences in the Bytes to be expected? I expected to find pretty much the exact same Header in my file as in the link. Do you know why it differs on my end?


Quote:
Originally Posted by KaktoR View Post
To check a file, you can use xtool's verbose mode
Would you mind explaining in a bit more detail how to use the XTool Verbose mode? I created a standard zip and also a 7zip file (LZMA2) and created a batch file based on what you showed in the link you provided (also changed the algorithm from mzlib to lzma2 (hoping that it works that way), but no matter what file I yeet onto xtool, all I get is this:

Code:
[0] Performing scan from block 0000000000000000 to 0000000000502C9D (5254302)

Streams: 0/0
Time: 00:00:00 (00:00:00)
Memory: 128 MB (128 MB)
I didn't expect it to not find any lzma2 streams in an lzma2 file, or other files I tried out with the adjusted algorithm parameter. I expected, similar to the "Drop and Scan 4 ZLib" Application from Razor to have different values here, that recognize how many parts of a file actually correspond to the compression algorithm. Am I misunderstanding the function and expecting a result that this tool is not supposed to give? If it isn't too much to ask, could you maybe show me a working result on a file?

Quote:
Originally Posted by KaktoR View Post
After you find out, pick the file and compress it with different combinations
So, let's keep things simple for now and let's say I found a file is actually compressed with LZMA2 / 7zip -- I would use xtool to decompress it (precomp?), receive a larger file, and then use a more efficient combination to get a better compression ratio, right?

Quote:
Originally Posted by KaktoR View Post
my choice is in most cases 4x4:lzma.
What does the 4x4: stand for? are these parameters in xtool or where does it come from?

Once again, I really appreciate you answering all these noob-ish questions, I really hope they aren't too bothersome!
Have a nice day guys!

- Crim
Reply With Quote
  #18  
Old 27-11-2022, 07:00
KaktoR's Avatar
KaktoR KaktoR is offline
Lame User
 
Join Date: Jan 2012
Location: From outer space
Posts: 3,869
Thanks: 1,009
Thanked 6,319 Times in 2,340 Posts
KaktoR is on a distinguished road
Quote:
Originally Posted by Crimzan View Post
Would you mind explaining in a bit more detail how to use the XTool Verbose mode? I created a standard zip and also a 7zip file (LZMA2) and created a batch file based on what you showed in the link you provided (also changed the algorithm from mzlib to lzma2 (hoping that it works that way), but no matter what file I yeet onto xtool, all I get is this:

Code:
[0] Performing scan from block 0000000000000000 to 0000000000502C9D (5254302)

Streams: 0/0
Time: 00:00:00 (00:00:00)
Memory: 128 MB (128 MB)
I didn't expect it to not find any lzma2 streams in an lzma2 file, or other files I tried out with the adjusted algorithm parameter. I expected, similar to the "Drop and Scan 4 ZLib" Application from Razor to have different values here, that recognize how many parts of a file actually correspond to the compression algorithm. Am I misunderstanding the function and expecting a result that this tool is not supposed to give? If it isn't too much to ask, could you maybe show me a working result on a file?
You cannot use -mlzma2 directly. As far as I know, zip/7z files contain always reflate streams (correct me if I'm wrong).

So for zip/7z files you have to use -mreflate if they are compressed.

Example for what it should look like:
Code:
...
...
[0] Processed reflate stream at 00000000022750C2 (304787 >> 349571 >> 304787) using l9:w15 successfully
[0] Processed reflate stream at 00000000022BF7D2 (33359 >> 43715 >> 33359) using l9:w15 successfully
[0] Processed reflate stream at 00000000022C7B42 (9895 >> 20251 >> 9895) using l9:w15 successfully
[0] Processed reflate stream at 00000000022CA262 (219985 >> 349563 >> 219985) using l9:w15 successfully
[0] Processed reflate stream at 00000000022FFEC2 (155628 >> 278096 >> 155628) using l9:w15 successfully
[0] Processed reflate stream at 0000000002325F32 (141976 >> 174791 >> 141976) using l9:w15 successfully
[0] Processed reflate stream at 0000000002348A42 (1270475 >> 1398167 >> 1270475) using l9:w15 successfully
[0] Processed reflate stream at 000000000247ED92 (491179 >> 699087 >> 491179) using l9:w15 successfully
[0] Processed reflate stream at 00000000024F6CC2 (2433413 >> 2796295 >> 2433413) using l9:w15 successfully
[0] Processed reflate stream at 0000000002748EC2 (965084 >> 1398165 >> 965084) using l9:w15 successfully

Streams: 167/167
Time: 00:00:04 (00:00:03)
Memory: 175 MB (175 MB)
Here are some possible options:
Code:
zlib / reflate / preflate
kraken / mermaid / selkie / leviathan (part of oodle compression family, requires "oo2core_#_win64.dll" library near xtool.exe)
flac (wave audio)
packjpg / brunsli / jojpeg (for JPG images, requires corresponding libraries near xtool.exe)
png (PNG image precompression, requires -d1 option and preflate option -> -mpng:d1:preflate)
lz4 / lz4f / lz4hc (requires liblz4.dll library near xtool.exe)
zstd (requires libzstd.dll library near xtool.exe)


Quote:
Originally Posted by Crimzan View Post
So, let's keep things simple for now and let's say I found a file is actually compressed with LZMA2 / 7zip -- I would use xtool to decompress it (precomp?), receive a larger file, and then use a more efficient combination to get a better compression ratio, right?
Yes.



Quote:
Originally Posted by Crimzan View Post
What does the 4x4: stand for? are these parameters in xtool or where does it come from?
4x4 is part of freearc. It utilizes all cores (max 16) for compression. This gives a good speed boost, but cpu usage is 100% most of the time.
__________________
Haters gonna hate
Reply With Quote
  #19  
Old 28-11-2022, 05:07
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 928
Thanks: 241
Thanked 1,036 Times in 489 Posts
Masquerade is on a distinguished road
Quote:
Originally Posted by KaktoR View Post
You cannot use -mlzma2 directly. As far as I know, zip/7z files contain always reflate streams (correct me if I'm wrong).
7z archives most commonly use LZMA2 since it's the default option when you select 7z container in 7Zip creation window.

Zip files most commonly use deflate for their compression, although you can use any algo with a zip container (zip header is PK). For example, Crysis 2 Remastered uses LZ4 on its zip game archives. We Are Football uses ZSTD on its zip archive.

Afair there's no current easy precompressor (e.g. xtool) for lzma[2]. I know pLZMA exists on encode forum but it's not a precompressor like XTool.
Reply With Quote
The Following User Says Thank You to Masquerade For This Useful Post:
KaktoR (28-11-2022)
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump



All times are GMT -7. The time now is 04:46.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2023, vBulletin Solutions Inc.
Copyright 2000-2020, FileForums @ https://fileforums.com