Exdupe with stdout ?!

[shazzla]

Hi guys !

Few days ago i tried to use Exdupe with stdout in arc.ini ,but i failed.
Anyone succeeded with this ?

Yes ,i know,there is SREP. Its okay,but in some case,it eats insanely hughe amount of RAM,even with -dngb parameter.

Any help appreciated !
Thanks.
(Atm away from PC,im on tablet)

[Bulat]

increase -l parameter in srep, it should be always better than exdupe

[shazzla]

Thank Bulat!

I know about the - l - parameter ,but it doesnt decrease the decompression memory req. Only a little bit!
Real example :
Approx 40 Gb dataset with L512 ,decomp memory 4.8 Gb
l1024-l4096 ,decomp memory around 3.9 gb.
-dngb n=1-4 ,M3F-M5F used in both case !
Those decomp.mem.reqs are too much for me.
Thats why i want to use exdupe without tempfile.
Btw i love SREP...

Any idea to decrease decomp mem reqs?

[Bulat]

-m0 -d1g will definitely use less than 1 GB

[78372]

m0 is the rep method, so it's memory usage is obviously low.


BTW, in case of x86 srep, how can we prevent srep from creating virtual temp file(I have only 4GB RAM)?

[FD_RLT]

Originally Posted by 78372

m0 is the rep method it's memory usage is obviously low.


BTW, in case of x86 srep, how can we prevent srep from creating virtual temp file(I have only 4GB RAM)?

Why you don't want it to create virtual temp file if you only have 4 GB RAM ?

[78372]

Because of the massive read and writes in hdd

[shazzla]

uses razor's CLS-SREP.DLL.
it uses memory for TMP ,but u need to keep decomp memory req within considerable values.

[78372]

That's the problem. I know razor's cls is helpful, but hey it's only useful if you use 64bit srep because it allows you to use that. But the problem is, I use x86 editions and when you use that, you need the virtual memory.

Srep is a big problem for me when repacking games bigger as GTA V, because I use solid compression which requires a lot of ram to compress and decompress

[ChronoCross]

[External compressor:exdupe]
packcmd   = exdupe.exe  -x0 -m512  $$arcdatafile$$.tmp $$arcpackedfile$$.tmp  
unpackcmd = exdupe.exe  -R  $$arcpackedfile$$.tmp $$arcdatafile$$.tmp   
datafile   = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp

[shazzla]

thanks ,its okay ,but how can i use STDOUT ?

[ChronoCross]

Originally Posted by shazzla

thanks ,its okay ,but how can i use STDOUT ?

why you make so complicated?, go for the most simple way(freearc ->arc.ini)

7z.exe -an -ttar a -bd "D:\TEST\*" -so | exdupe.exe -x0 -m512 -stdin - -stdout | zstd.exe -f -19 -v - -o test.zst

[shazzla]

Sorry for the noob question,but how can i use this in arc.ini?
I mean pack and unpack command..

i tried several combination to pack/unpack. sadly without success.

Arc.ini PACK command :

packcmd = exdupe.exe -x1 -m512 -stdin -p -stdout <stdin> <stdout>

working on UNPACK......

[shazzla]

2 years passed and no success. Anyone ? :/

[shazzla]

hmmm..
tried many versions,but what is the unpack command for this ?

7z.exe -an -ttar a -bd "D:\TEST\*" -so | exdupe.exe -x0 -m512 -stdin - -stdout | zstd.exe -f -19 -v - -o test.zst

As the topic describes ,the main goal is use exdupe with stdio with freearc.
It would be nice.
Please ,anybody ?