new to this

[KaktoR]

Depends on how much you want to size down your games. Generally spoken: The stronger the compression, the more time will it take and also CPU/RAM resources, to be simple.

To get the "quick and easy way" you can use DiskSpan_GUI to compress your files.
You can also unpack your compressed archives with it too (Tools -> Unpacker).

In your example for "Sniper Elite 4" I guess you can just use srep+lolz if you want it strong, with cost of speed and pc resources, or srep+lzma with cost of compression ratio, but requires way less pc resources. But I didn't checked the game so I cannot tell you for sure.

Some games requires that you first unpack the files and then compress them. In most cases you use xtool to unpack the game files, but you should know what compression method the game is compressed with. For example "Sniper Elite 5" is compressed with zlib (if I remember well), so you have to use xtool first to unpack the files, then apply srep to get rid of duplicates, then use lolz or lzma to use final compression.

Maybe you have noticed the problem for "newbies" here. That's nothing to explain in a few sentences.

[Neuromancer815]

thanks kaktor for the reply, helped understand a few things i was held up on, i got "srep+7zip multi volume" and "lzma" from the stickies as well as diskspan GUI, but i followed the link to lolz to krinkles.org and it seems i need to post stuff there before i can get download righs which sucks because i dont know anything so what would i post about. also diskpan gui is being flagged as malware, is that a false flag?

[Cesar82]

Originally Posted by Neuromancer815

thanks kaktor for the reply, helped understand a few things i was held up on, i got "srep+7zip multi volume" and "lzma" from the stickies as well as diskspan GUI, but i followed the link to lolz to krinkles.org and it seems i need to post stuff there before i can get download righs which sucks because i dont know anything so what would i post about. also diskpan gui is being flagged as malware, is that a false flag?

If DiskSpan GUI was flagged as a virus it is a false positive.
You can download without worries.
If you need to, disable your antivirus and after extracting the zip file from DiskSpan GUI, enable your antivirus again and check the extracted folder from DiskSpan GUI. Let me know exactly which file was captured by your antivirus so that if there is an alternative version maybe I can replace the captured file.

NOTES: DiskSpan GUI already includes all compressors reported by KaktoR, such as LOLZ.

EDIT: DiskSpan GUI is only False positive for viruses.
No virus detection using Kasperky Internet Security (See attached video below).

[Crimzan]

Hello! Thank you for these explanations!

I'm also really new to this and I have read a lot but it is a lot of arbitrary seeming detailed information in these forums that make it incredibly hard to categorize and make sense of, so I have a lot of questions and I hope it's ok to just dump them all at once

I downloaded DiskSpan GUI, it seems like it's one of the more customizeable, yet very easy to use ones. Do you think this is a good tool to use or would I be better off using these exe's separately, or even a completely different software?

Originally Posted by KaktoR

In your example for "Sniper Elite 4" I guess you can just use srep+lolz if you want it strong, with cost of speed and pc resources, or srep+lzma with cost of compression ratio, but requires way less pc resources.

You said "srep+lolz" and "srep+lzma", does this mean, in DiskSpan GUI (if you are familiar with the software), I can select both srep first and then lolz / lzma and then it is basically what you had described?

To understand the logic behind this, when I select a folder to compress with DiskSpanGUI, "srep+lolz" would basically mean using srep first to deduplicate, and then compress the srep output with lolz?

I don't have Sniper Elite 4, but this is interesting -- how do you know srep+lolz is strong? Is it one of the strongest general combinations? Or how do you know, or in other words, how can I find out what compressors and thingies work the best?

Originally Posted by KaktoR

Some games requires that you first unpack the files and then compress them. In most cases you use xtool to unpack the game files, but you should know what compression method the game is compressed with. For example "Sniper Elite 5" is compressed with zlib (if I remember well), so you have to use xtool first to unpack the files, then apply srep to get rid of duplicates, then use lolz or lzma to use final compression.

Super interesting!
So, to make sure I understand what you said at the end, in DiskSpanGUI I would basically have xTool_zlib+srep+lolz (or lzma), is that correct?

Also how do you know it's compressed with zlib? Are you an experienced file magician and / or do you know of a scanner software that can test if a file is compressed, and what it is compressed with?

One final question, which is hopefully not too broad: What is your workflow finding 'the best' compression combinations (that is, not as slow as paq compression for instance but still really good compression ratio)? Is it Trial and Error? Like, do you just select a couple of files to compress, and try each and every combination? Because that's what I'm doing currently and this kinda sounds super inefficient so I suspect there's a way to find out what works with scanners or anything

Anyways, I'm eager to learning more about compression but I don't think I'm getting far without asking some dedicated beginner questions. I feel like I know some stuff but they are absolutely random bits of information in different fields and it's incredibly hard and really confusing trying to make sense of them

Thanks!
- Crimzan

[KaktoR]

Originally Posted by Crimzan

I downloaded DiskSpan GUI, it seems like it's one of the more customizeable, yet very easy to use ones. Do you think this is a good tool to use or would I be better off using these exe's separately, or even a completely different software?

I would say this tool is handy, but have it's limitations aswell. Most repackers use their own tool set anyway, but in the end all of them are using the same compressors, with some additions and techniques.

Originally Posted by Crimzan

You said "srep+lolz" and "srep+lzma", does this mean, in DiskSpan GUI (if you are familiar with the software), I can select both srep first and then lolz / lzma and then it is basically what you had described?

To understand the logic behind this, when I select a folder to compress with DiskSpanGUI, "srep+lolz" would basically mean using srep first to deduplicate, and then compress the srep output with lolz?

Yes.

Originally Posted by Crimzan

I don't have Sniper Elite 4, but this is interesting -- how do you know srep+lolz is strong? Is it one of the strongest general combinations? Or how do you know, or in other words, how can I find out what compressors and thingies work the best?

lolz is mainly good for texture compression. You know, games use textures. Save your time and don't try to use lolz for audio/video files and such.

For instance, took some DDS image files, and compare ratios between srep+lolz and srep+lzma.

Spoiler: lolz will be superior on these files in terms of ratio. It's just a matter on how long you want to run the process and how small the output you want.

Originally Posted by Crimzan

Super interesting!
So, to make sure I understand what you said at the end, in DiskSpanGUI I would basically have xTool_zlib+srep+lolz (or lzma), is that correct?

Yes.

Originally Posted by Crimzan

Also how do you know it's compressed with zlib? Are you an experienced file magician and / or do you know of a scanner software that can test if a file is compressed, and what it is compressed with?

It's like common sense sometimes. If you are in the process for years, you will get a "sense" of it. For example take any Unreal Engine 4 game. Most of these games are compressed with zlib/reflate, many others with oodle, and a few even with lz4 or zstd.

To check a file, you can use xtool's verbose mode, or manually check the file with HxD or similar for magic numbers and such (this requires that you know about these things).

I'm not a magican or something, I still have questions sometimes too

Originally Posted by Crimzan

One final question, which is hopefully not too broad: What is your workflow finding 'the best' compression combinations (that is, not as slow as paq compression for instance but still really good compression ratio)? Is it Trial and Error? Like, do you just select a couple of files to compress, and try each and every combination? Because that's what I'm doing currently and this kinda sounds super inefficient so I suspect there's a way to find out what works with scanners or anything

On larger archives (like in UE4 games the *.pak files) use xtool's verbose mode to find out. After you find out, pick the file and compress it with different combinations until you are happy with the outcome (ratio/time). There is no "magic tool" to give you all answers about a file in general.

Personally I don't use lolz because it is time consuming af and it tooks literally all PC resources while running, and I want it to be as fast as possible with a decent outcome in ratio, therefore for me my choice is in most cases 4x4:lzma.

[Masquerade]

Originally Posted by Crimzan

Also how do you know it's compressed with zlib? Are you an experienced file magician and / or do you know of a scanner software that can test if a file is compressed, and what it is compressed with?

Razor made drop + scan 4 zlib which is sometimes useful.

There's also this Zenhax page where Aluigi explains how to recognise different algorithms by their headers or magic numbers: https://zenhax.com/viewtopic.php?t=27

This requires a hex editor to view the data. I like HxD.