Masked Compression (Ultimate compression in one go)

[panker1992]

~~~~~~~~~~~{Intro}~~~~~~~~~~~

In compression its all about algorithms
there are tons of algorithms that are suited all for a specific different purpose

this project is all about utilizing the correct compressor on specific type of files thus gaining the upper hand on compression and decompression

(example) there are types of data that compress better or doesnt compress at all
so why use lzma or even reflate and precomp even if you gain nothing from doing so ??

masked compression will use specific compressors per file types
it will gather all text files using extensions *.ini *.rtf *.txt
and it will perform the right compressor to compress the better of them

the same will happen with movies files and precomp file
in the latter using precomp.exe to process only files that benefit from it
on movies zstd will be used for maximum decompression

~~~~~~~~~~~{Tiny SFX Packing}~~~~~~~~~~~

1)Choose you Program or Game make sure it is small to begin with
: Tiny Tools cannot support large number of Files
2) Put it inside Tiny_SFX_Compressor.bat and choose whatever compression method you want
: As I said all works Including Precomp, you can leave it as is its very Optimal
3) Run the bat and wait for magic
4) after you have the data.bin run make sfx.bat
5) the exe that is produced is ready to work without anything it will automatically Unpack whatever method i have provided so far
6) This tool is provided as is, it worked for me and i gladly share, further than that i cant guarantee that it work for you!!!

~~~~~~~~~~~{Bik, Bk2 Re-encoding}~~~~~~~~~~~

Read the Readme.log i provided, it has everything you need to know


~~~~~~~~~~~{Update log}~~~~~~~~~~~

[Update 1]

Added new Zstd Compressor for windows, I splited this into 2 different compression settings
Zstd ultra compression = srep + zstd used a pipeline and offer great compression similar to lzma2 max and even bypass ultra on >5gb files
Zstd fast compression = pure zstd offering very very fast compression ideal for fast travel data
Important: If you used my previous tools, you need to download Zstd Ultra Compression to maintain support,
all your data that you have already compressed by this tool will be usable only with this option

[Masked Compression 2.5]

masks 2.5 are here but this time a minor release

1) third mask was beta last time, now it aint ...it works out of the box with great success

2)precomp 038 remade, and optimized from scratch, precomp 0.38 is custom
and it is faster than all other precomp, on my test precomp 0.45 took 9 minues almost
and precomp0.38 took 5 minutes and 35 seconds ( test file was 700 mb file )

precomp038 set to slow mode and prrecomp 0.43 for brute mode, more of this in arc.ini

3) legacy2 works again since precomp038 is added again

4) msc is disabled, but you can enable it yourself, i did a proper test with this and reflate doesnt work with msc

plus the ratio was increased and not decreased (if you dont believe me i can post results and you can always test yourself)

5) zstd doesnt work with my config anymore... it took errors i had to reformat entire pc and again didnt ( that was enough for me to cast it aside tho i left the sonido )
and the option for you to re-enable it

6) msrsolid mask and void mask are better optimized !!!

Masked Compression Archive Added including all masks up to 2.5

Masked Compression x86 Support

Masked Compression Manual
Stay tuned



~~~~~~~~~~~{Special Thanks}~~~~~~~~~~~

Special thanks to Razor for reflate and Inno(WPI) and Gozarck for his inspiration and tweaks

[felice2011]

I understood, in fact we all understand that your methods "LZ77", "XZ" are best of the old and obsolete method LZMA, thou hast repeated a hundred times in my and in your thread, but the file you posted www.fileforums.com/showthread.php?t=97303&page=5#67 changing folder "resource" sends error in my utility with different methods of choice, if we want to change something in my utility we can do together I hope you understand this you.

[panker1992]

i know what you mean Felice the file i uploaded was to show you the removed dll files and the updated srep that went to 3.93 just keep those files ..... i didnt update overall you work :P

I dont know if you need the dlls for your exe to work by the way ..... because i use commandline and you interface

[felice2011]

My utility uses a variety of combinations for different methods, and the files placed in the folder 'resource' are not included by chance..
I checked the files included in your 'Masked Compression' method bat console and arc.ini files modified, it is not difficult to insert and update all in my utility, but the combinations of choice increased considerably, and this may be a problem ...

[panker1992]

Felice if each compressor uses from 2~5 different switches then if you combine 5 programs
the amount of switches dramatically change and even a single modification is chained :P

i suggest you toy with my bat file before you try to combine it in DLG :P

I want to hear some thoughts on it ..... did you find it better ?? or worse ?? something that will help me too

as you can see clearly decompression is still the same nothing will change there at this point
masks only help the compression part hope you like this far

i have some nasty ideas on compression but only razor can help me on this because i work solo i dont have any help besides this community.
packjpg that is included in there is the exe from the latest release, i can also compile it in dll but cannot inject in precomp in order to upgrade it, if razor can inject it or better make the code use multithreading it would be great
packjpg is GPL program by the way all can get its source !!

[felice2011]

Ok I will be working on the 'Masked Compression' I have a couple of titles to be compressed and are curious to know how it acts..
Meantime remove the zip file that you posted in my thread, if it is used by users and give error, by me come to break the balls for to malfunction..

[panker1992]

ops you are right i assume you got the updates especially on srep you will need them

[felice2011]

Originally Posted by panker1992

ops you are right i assume you got the updates especially on srep you will need them

Do not worry...

[oltjon]

truly the best so far

[panker1992]

@oltjon if you have weak computer the compression speed will be like half or even better
the same goes for high end computers but it can run parallel

hope you like it

[Amsal]

@PANKER1992, Really THANKS A LOT FOR THIS! ONLY one question, how to extract the archives in inno script?
And Bro, tell me with reflate and without reflate. I know how to compress with reflate and with reflate, only I am asking, how to decompress the files with an inno script.
Please answer in detail!

Again thanks for the Masked Compressor, it is really very fast

[panker1992]

@Amsal really thanks, i will update decomp tomorrow so no worries

generally in decomp nothing changes, masks only exist to instruct how a compressor behaves


example: all text files are compressed next to each other and movies the same and so on
thus gaining an upper hand in everything, trying to apply the best settings in each case
so it doesnt affect decompression because decomp doesnt care what it extracts as long as it extracts


DETAILS TOMORROW its 4 morning here :P

[Amsal]

Originally Posted by panker1992

@Amsal really thanks, i will update decomp tomorrow so no worries

generally in decomp nothing changes, masks only exist to instruct how a compressor behaves


example: all text files are compressed next to each other and movies the same and so on
thus gaining an upper hand in everything, trying to apply the best settings in each case
so it doesnt affect decompression because decomp doesnt care what it extracts as long as it extracts


DETAILS TOMORROW its 4 morning here :P

Bro, here is also morning, from INDIA, you know I compressed from 1.61gb to only 98.4mb with this method, and from compressing in ultraarc it gives 1.2gb!!

That mean, your compressor is amazing, i am shocked with results!! Please post the decompression method soon as possible? I will go on vacation in 2 days, so please post as soon as possible!

[panker1992]

now that is what i want all of you to do

very nice feedback

copy all files from my 7z somewhere and make a unpack.bat file

write inside unpack.bat

arc.exe x data1.bin -y -pYourpassword

and save this


now see how fast truly gets


Inno is coming up tomorrow ..... it works ok so no worries

[Amsal]

Originally Posted by panker1992

now that is what i want all of you to do

very nice feedback

copy all files from my 7z somewhere and make a unpack.bat file

write inside unpack.bat

arc.exe x data1.bin -y -pYourpassword

and save this


now see how fast truly gets


Inno is coming up tomorrow ..... it works ok so no worries

Bro, I used this method, but it only creates the folder, it doesn't extract all the files, I tried, but it only made folder! Listen, Complete your decompression process/script/bat and then give all the required bat/files to us with the method to use in inno! But bro, if possible complete these all test till today night