Go Back   FileForums > Game Backup > PC Games > PC Games - CD/DVD Conversions > Conversion Tutorials

Reply
 
Thread Tools Display Modes
  #1  
Old 13-04-2016, 07:36
panker1992's Avatar
panker1992 panker1992 is offline
Registered User
 
Join Date: Oct 2015
Location: Always Somewhere
Posts: 540
Thanks: 113
Thanked 768 Times in 300 Posts
panker1992 is on a distinguished road
Fast Zlib Recompressor by Razor12911 (*Test*)

This is as title says a fast zlib recompressor, works like antiz and precomp but it features better speed.

at the moment this doesn't feature a decompression this was only made yesterday.

*Important*
please consider to post feedback, No feedback means the program will never see the light of the world just as it happened with reflate.


[External compressor:fzr]
header = 0
packcmd = fzr <stdin> <stdout>

In short this tool takes input directly from Freearc, it doesnt slow down the process at all, it runs real life on processor to provide exactly copy paste speeds (Photo below with marked speed)
Attached Images
File Type: jpg fzr.jpg (418.5 KB, 473 views)
Attached Files
File Type: 7z fzr.7z (262.5 KB, 291 views)
__________________
My projects : Masked Compression, lzma2(xz) on Freearc, Zstd compressor for windows
My optimizations : packjpg.exe, zstd, lzham, precomp-dev-0.45.
Reply With Quote
The Following 10 Users Say Thank You to panker1992 For This Useful Post:
arkantos7 (14-04-2016), COPyCAT (22-04-2016), felice2011 (14-04-2016), gozarck (13-04-2016), Ichiraku2001 (13-04-2016), kenzo34 (13-04-2016), Lucas65 (16-04-2016), Mini (13-04-2016), RamiroCruzo (13-04-2016), Simorq (13-04-2016)
Sponsored Links
  #2  
Old 13-04-2016, 09:23
RamiroCruzo's Avatar
RamiroCruzo RamiroCruzo is offline
Registered User
 
Join Date: Jul 2015
Location: India
Posts: 184
Thanks: 386
Thanked 169 Times in 75 Posts
RamiroCruzo is on a distinguished road
Yet another point for Zee-Panker Team... Nice work hermano...

Quote:
FreeArc 0.67 (March 15 2014) creating archive: data.rcf
Compressed 1 file, 48,699,392 => 48,742,902 bytes. Ratio 100.09%
Compression time: cpu 0.20 sec/real 1.34 sec = 15%. Speed 36.23 mB/s
All OK


E:\Reflate\Testing>pause
Press any key to continue . . .
Hell fast...Just like F12
__________________
We are such stuff as dreams are made on, and our little life is rounded with a sleep.
Reply With Quote
  #3  
Old 13-04-2016, 09:30
RamiroCruzo's Avatar
RamiroCruzo RamiroCruzo is offline
Registered User
 
Join Date: Jul 2015
Location: India
Posts: 184
Thanks: 386
Thanked 169 Times in 75 Posts
RamiroCruzo is on a distinguished road
Hmm...Did some more tests....

mfzr+reflate
Quote:
Errorlevel=0
Compressed 1 file, 48,699,392 => 115,716,669 bytes. Ratio 237.61%
Compression time: cpu 0.39 sec/real 80.36 sec = 0%. Speed 0.61 mB/s
All OK
Without Fzr....
mreflate
Quote:
Errorlevel=0
Compressed 1 file, 48,699,392 => 155,495,319 bytes. Ratio 319.30%
Compression time: cpu 0.33 sec/real 91.58 sec = 0%. Speed 0.53 mB/s
All OK
Its awesome...
__________________
We are such stuff as dreams are made on, and our little life is rounded with a sleep.
Reply With Quote
  #4  
Old 13-04-2016, 09:46
Razor12911's Avatar
Razor12911 Razor12911 is offline
Noob
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,746
Thanks: 2,141
Thanked 11,099 Times in 2,295 Posts
Razor12911 is on a distinguished road
It does not work on GTAV files. Try another input
Reply With Quote
  #5  
Old 13-04-2016, 09:57
RamiroCruzo's Avatar
RamiroCruzo RamiroCruzo is offline
Registered User
 
Join Date: Jul 2015
Location: India
Posts: 184
Thanks: 386
Thanked 169 Times in 75 Posts
RamiroCruzo is on a distinguished road
Smile

Quote:
Originally Posted by Razor12911 View Post
It does not work on GTAV files. Try another input
Pardon me...In camp so using my PC via RDP client....

Also, I tested it on my traditional 30K textures zip....

Free Reflate Time: 24 min 16 sec
Antiz time: 1 min 18 sec
Skylake time: 59 sec 49 millisec
Reflate m1 time: 40 sec 23 millisec
FZR time: 26 sec 12 millisec
__________________
We are such stuff as dreams are made on, and our little life is rounded with a sleep.
Reply With Quote
  #6  
Old 13-04-2016, 09:59
Razor12911's Avatar
Razor12911 Razor12911 is offline
Noob
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,746
Thanks: 2,141
Thanked 11,099 Times in 2,295 Posts
Razor12911 is on a distinguished road
Some screenshots, but as Panker said, tool was created yesterday and we really haven't started with the decompression of files this thing makes, we still have to sort out the compression part so that we can prepare it for decompression.
Attached Images
File Type: png 2016-04-13_17-56-03.png (570.3 KB, 458 views)
Reply With Quote
The Following 8 Users Say Thank You to Razor12911 For This Useful Post:
-XCX- (22-12-2016), arkantos7 (14-04-2016), COPyCAT (22-04-2016), felice2011 (14-04-2016), gozarck (13-04-2016), Mini (13-04-2016), panker1992 (13-04-2016), RamiroCruzo (13-04-2016)
  #7  
Old 13-04-2016, 19:51
gozarck's Avatar
gozarck gozarck is offline
Registered User
 
Join Date: Nov 2013
Location: Argentina
Posts: 163
Thanks: 254
Thanked 494 Times in 114 Posts
gozarck is on a distinguished road
WOW guys Nice team and good work this is just the begining. keep going.
Speed is very good (really awesome )
Code:
Compressing 692 files of 729,000,732 bytes: 0.11 sec
  Using fzr
  Memory for compression 0b, decompression 0b
Compressing 0 bytes with fzr
100%
Errorlevel=0

  Solid block compression results (4.321 seconds)
    fzr: 729,420,748 bytes in 43.587 seconds

  Writing directory: 43.83 sec
  Found 36 directory names: 43.84 sec
  Directory written: 43.85
Compressed 692 files, 729,000,732 => 729,420,748 bytes. Ratio 100.06%
Compression time: cpu 4.32 sec/real 43.92 sec = 10%. Speed 16.60 mB/s
All OK
__________________
-the science of the useless-
Reply With Quote
  #8  
Old 13-04-2016, 21:10
ChronoCross's Avatar
ChronoCross ChronoCross is offline
Registered User
 
Join Date: Sep 2014
Location: Acacia Dragons
Posts: 254
Thanks: 145
Thanked 203 Times in 115 Posts
ChronoCross is on a distinguished road
Hey ramirocruzo reflate always win in size because detects valid and invalid zlib streams. Also if fzr works like precomp and antiz those programs discard invalid zlib streams. I made some test with diferent texture and i cant see good result. So what kind of files suggest to test?
Reply With Quote
  #9  
Old 13-04-2016, 21:17
panker1992's Avatar
panker1992 panker1992 is offline
Registered User
 
Join Date: Oct 2015
Location: Always Somewhere
Posts: 540
Thanks: 113
Thanked 768 Times in 300 Posts
panker1992 is on a distinguished road
definitely not textures

texture files are by no means zlib files.

you will need to test ZLIB compatible files. for example dragon age CAS extension files can do it
__________________
My projects : Masked Compression, lzma2(xz) on Freearc, Zstd compressor for windows
My optimizations : packjpg.exe, zstd, lzham, precomp-dev-0.45.
Reply With Quote
The Following User Says Thank You to panker1992 For This Useful Post:
RamiroCruzo (14-04-2016)
  #10  
Old 14-04-2016, 01:00
felice2011's Avatar
felice2011 felice2011 is offline
Registered User
 
Join Date: Feb 2011
Location: italy
Posts: 836
Thanks: 354
Thanked 1,152 Times in 388 Posts
felice2011 is on a distinguished road
Hey Panker finally back to action, great job as precompressor, in addition to good speed, also work on the compression threshold and increase the compatibility of file extensions, I think the interest of all.
Also thanks to Razor for the work done.
__________________
≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈ ≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈
« I Mediocri Imitano, I Geni Copiano, Dio Crea & Distrugge » (Io Ridefinisco & Perfeziono le Loro Opere Rendendole Uniche)
≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈ ≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈
« Mediocrities Imitate, Genius Copy, God Creates & Destroys » (I Reconsider & Improve Their Works, Rending Them One And Only)

Last edited by felice2011; 14-04-2016 at 01:14.
Reply With Quote
  #11  
Old 14-04-2016, 07:19
panker1992's Avatar
panker1992 panker1992 is offline
Registered User
 
Join Date: Oct 2015
Location: Always Somewhere
Posts: 540
Thanks: 113
Thanked 768 Times in 300 Posts
panker1992 is on a distinguished road
this is a free addition to the freearc exe free performance boost.

how it can work is

-mfzr+precomp+srep+delta+exe2+xz

it performs a light scan. so that precomp is abit speedy try it
__________________
My projects : Masked Compression, lzma2(xz) on Freearc, Zstd compressor for windows
My optimizations : packjpg.exe, zstd, lzham, precomp-dev-0.45.
Reply With Quote
  #12  
Old 14-04-2016, 10:15
Mini Mini is offline
Registered User
 
Join Date: Nov 2013
Location: -
Posts: 125
Thanks: 171
Thanked 76 Times in 43 Posts
Mini is on a distinguished road
very good! we wait unpack
Reply With Quote
  #13  
Old 14-04-2016, 13:43
Razor12911's Avatar
Razor12911 Razor12911 is offline
Noob
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,746
Thanks: 2,141
Thanked 11,099 Times in 2,295 Posts
Razor12911 is on a distinguished road
Quote:
Originally Posted by ChronoCross View Post
Hey ramirocruzo reflate always win in size because detects valid and invalid zlib streams. Also if fzr works like precomp and antiz those programs discard invalid zlib streams. I made some test with diferent texture and i cant see good result. So what kind of files suggest to test?
Invalid zlib streams? Maybe you referring to headerless zlib streams.
reflate doesn't always win in size too, there is a file from NFS The Run, UI.sb
with reflate, you get 361MB, precomp gets 373MB, fzr gets 348MB and some other methods of reflate that I have created get 308MB.
If a zlib stream is invalid then it is not a zlib stream, not even reflate can do anything about it.

There are 2 problems with zlib recompressors
1: detection, finding all streams a file has is not easy but finding common is easy which is what fzr, precomp and reflate, the difference with reflate is it takes it too another level, it doesn't detect common zlib, it knows what programs that use zlib as compression for zip, for example 7-Zip or KZip, all those use zlib, detecting the zlib stream using common headers is not possible so the only thing that reflate (rawdet) does I think is detect the header of the zip file created by an archiver, e.g. 7-Zip then it knows that there is a stream here and there but precomp, antiz don't do any of that.

2: is after finding that zlib stream, after unpacking it, can you restore it the way it was (matching CRC) if so then precomp, fzr or reflate will unpack it but if you can't then it is ignored.

precomp, fzr and other recompressors have these problems, if it is not detecting then it is restoration luckily with reflate, it isn't much of a problem because it is a program that is mastered to detect normal zlib streams and zlib streams that have been tuned to work specifically for archivers. PNG's use zlib but I myself have problems restoring the zlib stream because of the variations done to the source of zlib.

**Note, this is what I know so far.

Last edited by Razor12911; 14-04-2016 at 13:46.
Reply With Quote
The Following 2 Users Say Thank You to Razor12911 For This Useful Post:
Mini (14-04-2016), RamiroCruzo (14-04-2016)
  #14  
Old 14-04-2016, 14:57
ChronoCross's Avatar
ChronoCross ChronoCross is offline
Registered User
 
Join Date: Sep 2014
Location: Acacia Dragons
Posts: 254
Thanks: 145
Thanked 203 Times in 115 Posts
ChronoCross is on a distinguished road
No no im right. 1 detection of zlib headers.

2 in zlib headers detect valid zlib streams.


So invalid zlib streams in reflate are false positive. Reflate dont have a valid or invalid detection of zlib streams (like antiz or precomp) in that case reflate always wins. (correct or incorrect detection and this is not good for reflate). So i prefers do not make a comparation frz with reflate because use diferent algo and still in alpha. (like ramirocomparation) i made some test with precomp and frz. I have good result with precomp in size and with frz i have only speed.
Reply With Quote
The Following User Says Thank You to ChronoCross For This Useful Post:
felice2011 (14-04-2016)
  #15  
Old 14-04-2016, 15:55
felice2011's Avatar
felice2011 felice2011 is offline
Registered User
 
Join Date: Feb 2011
Location: italy
Posts: 836
Thanks: 354
Thanked 1,152 Times in 388 Posts
felice2011 is on a distinguished road





The optimum speed at the expense of the compression...

P.S :"-mfzr+precomp+srep+delta+exe2+xz" (Arc in Crash - Stops Working)...
__________________
≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈ ≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈
« I Mediocri Imitano, I Geni Copiano, Dio Crea & Distrugge » (Io Ridefinisco & Perfeziono le Loro Opere Rendendole Uniche)
≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈ ≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈
« Mediocrities Imitate, Genius Copy, God Creates & Destroys » (I Reconsider & Improve Their Works, Rending Them One And Only)
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Processing Zlib streams without headers RamiroCruzo Conversion Tutorials 3 21-09-2015 01:24
NG:DEV.DIRECT announced Fast Striker for Dreamcast Basile DC Games 0 31-08-2010 15:52
Borderlands v1.20 [MULTI5] No-DVD/Fixed EXE crash when fast travel to Jakobs Cove mikehd PC Games 24 10-04-2010 10:32
PS2 Tray Ejects/Closes Too Fast!!! cobraxt PS2 Games 2 16-06-2002 15:21
Fast error skipping settings for Safedisc 2 Bandiet CD/DVD Copy Protections & Utilities 16 22-03-2002 14:58



All times are GMT -7. The time now is 11:04.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.
Copyright 2000-2020, FileForums @ https://fileforums.com