Fast Zlib Recompressor by Razor12911 (*Test*)

[panker1992]

This is as title says a fast zlib recompressor, works like antiz and precomp but it features better speed.

at the moment this doesn't feature a decompression this was only made yesterday.

*Important*
please consider to post feedback, No feedback means the program will never see the light of the world just as it happened with reflate.


[External compressor:fzr]
header = 0
packcmd = fzr <stdin> <stdout>

In short this tool takes input directly from Freearc, it doesnt slow down the process at all, it runs real life on processor to provide exactly copy paste speeds (Photo below with marked speed)

[RamiroCruzo]

Yet another point for Zee-Panker Team... Nice work hermano...

FreeArc 0.67 (March 15 2014) creating archive: data.rcf
Compressed 1 file, 48,699,392 => 48,742,902 bytes. Ratio 100.09%
Compression time: cpu 0.20 sec/real 1.34 sec = 15%. Speed 36.23 mB/s
All OK


E:\Reflate\Testing>pause
Press any key to continue . . .

Hell fast...Just like F12

[RamiroCruzo]

Hmm...Did some more tests....

mfzr+reflate

Errorlevel=0
Compressed 1 file, 48,699,392 => 115,716,669 bytes. Ratio 237.61%
Compression time: cpu 0.39 sec/real 80.36 sec = 0%. Speed 0.61 mB/s
All OK

Without Fzr....
mreflate

Errorlevel=0
Compressed 1 file, 48,699,392 => 155,495,319 bytes. Ratio 319.30%
Compression time: cpu 0.33 sec/real 91.58 sec = 0%. Speed 0.53 mB/s
All OK

Its awesome...

[Razor12911]

It does not work on GTAV files. Try another input

[RamiroCruzo]

Originally Posted by Razor12911

It does not work on GTAV files. Try another input

Pardon me...In camp so using my PC via RDP client....

Also, I tested it on my traditional 30K textures zip....

Free Reflate Time: 24 min 16 sec
Antiz time: 1 min 18 sec
Skylake time: 59 sec 49 millisec
Reflate m1 time: 40 sec 23 millisec
FZR time: 26 sec 12 millisec

[Razor12911]

Some screenshots, but as Panker said, tool was created yesterday and we really haven't started with the decompression of files this thing makes, we still have to sort out the compression part so that we can prepare it for decompression.

[gozarck]

WOW guys Nice team and good work this is just the begining. keep going.
Speed is very good (really awesome )

Compressing 692 files of 729,000,732 bytes: 0.11 sec
  Using fzr
  Memory for compression 0b, decompression 0b
Compressing 0 bytes with fzr
100%
Errorlevel=0

  Solid block compression results (4.321 seconds)
    fzr: 729,420,748 bytes in 43.587 seconds

  Writing directory: 43.83 sec
  Found 36 directory names: 43.84 sec
  Directory written: 43.85
Compressed 692 files, 729,000,732 => 729,420,748 bytes. Ratio 100.06%
Compression time: cpu 4.32 sec/real 43.92 sec = 10%. Speed 16.60 mB/s
All OK

[ChronoCross]

Hey ramirocruzo reflate always win in size because detects valid and invalid zlib streams. Also if fzr works like precomp and antiz those programs discard invalid zlib streams. I made some test with diferent texture and i cant see good result. So what kind of files suggest to test?

[panker1992]

definitely not textures

texture files are by no means zlib files.

you will need to test ZLIB compatible files. for example dragon age CAS extension files can do it

[felice2011]

Hey Panker finally back to action, great job as precompressor, in addition to good speed, also work on the compression threshold and increase the compatibility of file extensions, I think the interest of all.
Also thanks to Razor for the work done.

[panker1992]

this is a free addition to the freearc exe free performance boost.

how it can work is

-mfzr+precomp+srep+delta+exe2+xz

it performs a light scan. so that precomp is abit speedy try it

[Mini]

very good! we wait unpack

[Razor12911]

Originally Posted by ChronoCross

Hey ramirocruzo reflate always win in size because detects valid and invalid zlib streams. Also if fzr works like precomp and antiz those programs discard invalid zlib streams. I made some test with diferent texture and i cant see good result. So what kind of files suggest to test?

Invalid zlib streams? Maybe you referring to headerless zlib streams.
reflate doesn't always win in size too, there is a file from NFS The Run, UI.sb
with reflate, you get 361MB, precomp gets 373MB, fzr gets 348MB and some other methods of reflate that I have created get 308MB.
If a zlib stream is invalid then it is not a zlib stream, not even reflate can do anything about it.

There are 2 problems with zlib recompressors
1: detection, finding all streams a file has is not easy but finding common is easy which is what fzr, precomp and reflate, the difference with reflate is it takes it too another level, it doesn't detect common zlib, it knows what programs that use zlib as compression for zip, for example 7-Zip or KZip, all those use zlib, detecting the zlib stream using common headers is not possible so the only thing that reflate (rawdet) does I think is detect the header of the zip file created by an archiver, e.g. 7-Zip then it knows that there is a stream here and there but precomp, antiz don't do any of that.

2: is after finding that zlib stream, after unpacking it, can you restore it the way it was (matching CRC) if so then precomp, fzr or reflate will unpack it but if you can't then it is ignored.

precomp, fzr and other recompressors have these problems, if it is not detecting then it is restoration luckily with reflate, it isn't much of a problem because it is a program that is mastered to detect normal zlib streams and zlib streams that have been tuned to work specifically for archivers. PNG's use zlib but I myself have problems restoring the zlib stream because of the variations done to the source of zlib.

**Note, this is what I know so far.

[ChronoCross]

No no im right. 1 detection of zlib headers.

2 in zlib headers detect valid zlib streams.


So invalid zlib streams in reflate are false positive. Reflate dont have a valid or invalid detection of zlib streams (like antiz or precomp) in that case reflate always wins. (correct or incorrect detection and this is not good for reflate). So i prefers do not make a comparation frz with reflate because use diferent algo and still in alpha. (like ramirocomparation) i made some test with precomp and frz. I have good result with precomp in size and with frz i have only speed.

[felice2011]

The optimum speed at the expense of the compression...

P.S :"-mfzr+precomp+srep+delta+exe2+xz" (Arc in Crash - Stops Working)...