Helping thread :)

[FitGirl]

arc.exe -m4x4:lzma

[VickNet]

Thx, but is it possible to combine 4x4:lzma with other parameters? Like this -m4x4:lzma:a1:mfbt4:d400m:fb128...
- I tested like this but the process of packing is not starting

[FitGirl]

The example string works just fine
srep+4x4:t4:b128m:lzma:128m

Never use large dict size for 4x4
4x4 means you're running 4 instances of LZMA, each with 400Mb dict size, that leads to 400*4*11 = 17600 MB of RAM to compress.

On user machine ISDone will require four continuous memory blocks of 400 MB each.

Also, using 4x4 hurts compression ratio due to blocking. That's why "some guy" repacks are larger than any other. You can try to use LZMA2 instead of LZMA1 (gives you 2 decompression threads by default), but it has it's own downsides.

I've personally used 4x4:LZMA in some of my early repacks, but gave it up. Number of SSD users is not that high, and unpacking in four streams on HDD often leads to overload.

[aswadd]

as fitgirl said 4x4 is not good to use for public repacks you can use it for your personnel repacks it require 4X what normal lzma require but also faster ,
one more thing RLD didn't use 4x4:lzma they used 4x4:tor you don't have to use 4x4 for lzma only but it will be a good option for a fast storing with a light compression ratio

[MCWRX]

I tried this string

srep+4x4:t4:b128m:lzma:128m

but I am not able to unpack it.

[aswadd]

Originally Posted by MCWRX

I tried this string

srep+4x4:t4:b128m:lzma:128m

but I am not able to unpack it.

why you're making it like this 4x4:t4:b128m:lzma:128m
why not 4x4:lzma:b128m:128m

if you used fazip for 4x4 , lzma then you must use it again for unpacking
see (arc.ini)

[MCWRX]

Originally Posted by aswadd

why you're making it like this 4x4:t4:b128m:lzma:128m
why not 4x4:lzma:b128m:128m

if you used fazip for 4x4 , lzma then you must use it again for unpacking
see (arc.ini)

thanks, I will test.

I use LZMA x64

[aswadd]

Originally Posted by MCWRX

thanks, I will test.

I use LZMA x64

one here mentioned that don't use lzmax64.exe it's not good i don't remember why but he said that as he tested use 7z lzma2 (xz) or fazip for (4x4,lzma)

[MCWRX]

Hi everyone. What can the problem that after using precomp, the decompression process starts very slow? I press "INSTALL" and it stays at 0% for 2-3 minutes then precomp starts. It is not very good I think. Is there any solution?

[Bulat]

Originally Posted by MCWRX

Hi everyone. What can the problem that after using precomp, the decompression process starts very slow? I press "INSTALL" and it stays at 0% for 2-3 minutes then precomp starts. It is not very good I think. Is there any solution?

it's ok - precomp is in beginning of your compression string, i.e. smth like "precomp+srep+lzma", and this means that on decompression it's last algo in the line, so all other compressora are done when precomp starts

still, you can use cls_precomp and cls_srep to make all decompressors work simultaneously

[MCWRX]

Originally Posted by Bulat

it's ok - precomp is in beginning of your compression string, i.e. smth like "precomp+srep+lzma", and this means that on decompression it's last algo in the line, so all other compressora are done when precomp starts

still, you can use cls_precomp and cls_srep to make all decompressors work simultaneously

Thanks for your response!

I've got something else - what can cause this error: Not enough memory. unarc.dll returned error code - 5. It appears for some users. It appears when files compressed with 4x4:lzma start to unpack. Yes, I know, you guys told me it's not very ok to use it but what alternatives do I have? It's really fast comparing to default compression. I personally haven't got this error, it's rare, but the fact that it exists, annoys me and I can't move on without making it clear. Have any of you got ideas?

[Bulat]

well, it may be bug in my program (unarc.dll) or yiu use it wrong way. so, let's start with lt command output and computer that fails. can you make some checks on this computer so i can understand the situation?

write me to Bulat.Ziganshin********com since i rarely read this forum

[Bulat]

Originally Posted by FitGirl

The example string works just fine
srep+4x4:t4:b128m:lzma:128m

Never use large dict size for 4x4
4x4 means you're running 4 instances of LZMA, each with 400Mb dict size, that leads to 400*4*11 = 17600 MB of RAM to compress.

On user machine ISDone will require four continuous memory blocks of 400 MB each.

Also, using 4x4 hurts compression ratio due to blocking. That's why "some guy" repacks are larger than any other. You can try to use LZMA2 instead of LZMA1 (gives you 2 decompression threads by default), but it has it's own downsides.

I've personally used 4x4:LZMA in some of my early repacks, but gave it up. Number of SSD users is not that high, and unpacking in four streams on HDD often leads to overload.

i agree with almost everything you said except for two things:

1. 4x4 doesn't mean "use exactly 4 threads both for compression and decompression". let's see - data are split into independently compressed blocks, so you can use any number of threads at compression stage and, independent of that, any number of threads at decompression stage. both arc and unarc find largest contiguous block available (in 64-bit windows it will be 2047 MB), divide its size by the memory required for single (de)compression thread, and run so many threads. so, on 64-bit windows it will use more threads, on 32-bit windows it will use less threads

2. lzma2 is different from lzma in two things:
- it stores incompressible chunks of data
- like 4x4, it can split input data into independent blocks. unfortunately, it always uses only 1 thread for decompression. When manual says "LZMA2 uses: 1 thread for each chunk in x1 and x3 modes; and 2 threads for each chunk in x5, x7 and x9 modes" it means only comperssion stage

So, compared to 4x4:lzma, lzma2 may give slightly better ratio and single-thread speed, but it can't decompress even in 2 threads. but probably you meant compression w/o splitting into independent blocks, where lzma2 indeed is slightly better, but still single-threaded on decompression side. unfortunately, it's inherent property of lzma algorithm - you can't use morte than 1 thread to decompress single lzma compressed stream

3. that said, inability to decompress data compressed with 4x4:lzma is a bug in my program. I already have a better algorithm for InsertTempfile procedure, but ATM it's not yet ideal and still wrtitten in Lua. But i will fix that sooner or later.

4. these days, most of user computers run Win64, so probably it's time to switch to 64-bit unarc.dll in most repacks? i will try to build one.

[Bulat]

Originally Posted by aswadd

if you used fazip for 4x4 , lzma then you must use it again for unpacking
see (arc.ini)

it's opposite - fazip can implement any internal arc.exe algorithm as external one. this means that you can compress with fazip and decompress internally, or vice versa.

in practice, i recommend to use fazip for large compression methods (rep, lzma, ppmd). you may also prefer to use fazip for decompression (even in installer) since it's yet another way to break 2/4 GB barrier of 32-bit programs

[MCWRX]

Bulat, send me please a PM with your email.