View Single Post
  #16  
Old Today, 05:36
panker1992's Avatar
panker1992 panker1992 is online now
Registered User
 
Join Date: Oct 2015
Location: Always Somewhere
Posts: 597
Thanks: 124
Thanked 965 Times in 340 Posts
panker1992 is on a distinguished road
The internal naming is the name of the project i am replacing, for example addex is replacing srep and the internal name has many srep counterparts in it.

Addex is also heavily inspired from srep. And Tarz is also heavily inspired from Lolz.

I am fixing Decompression because Tarz stdio decompression has some bugs.
CLS should work.

Let me explain how my multithreading works, the data is split in exactly as many threads as you request.

For example if you have an input of 16GB and you request 16 threads. this automatically means you split the data in 16 parts of 1gb each, you create 16 sliding windows, 16 pairs of models and of course you will need exactly 16 threads to decompress the data.

The reason i hard limited the threads to 16 is because of that because in the beginning i decied that even 8 will be alot.

Tarz doesn't advertise encoding speed, however the decode speed is boosted when using multithreading because the load is decoded evenly.

i don't yet have code to make this a block based encoding possible. something which lolz can do. there is a possibility to add this later.

let me explain how ram works.

each dictionary you create, needs * 10 the ram + number of block + 64mb for LRM.

if you use 128mb dictionary and one thread, this means around 1600 mb of ram. to decompress this you need 128mb + 64mb for decoding.

which if you ask me is quite efficient. it's highly assymetric.
__________________
My projects : Masked Compression, lzma2(xz) on Freearc, Zstd compressor for windows
My optimizations : packjpg.exe, zstd, lzham, precomp-dev-0.45.
Reply With Quote
The Following User Says Thank You to panker1992 For This Useful Post:
shazzla (Today)
Sponsored Links