Exdupe with stdout ?!

[Razor12911]

I'll try to help but no promises okay?

[shazzla]

WoW !
Thank you very much !

[78372]

Does this support stdin/stdout at default or we need to make a patcher?

[shazzla]

In theory ,yes. It supports :

eXdupe 0.5.0 BETA. Copyright 2010 - 2013 eXdupe.com. UNSTABLE UNSTABLE

Full backup:
[-vxarctpmgl] [-f] [-s] <sources> <destination>
[-vxarctpmgl] <-stdin> <filename to assign> <destination>

<destination> can either be -stdout or a file. Use .full as file extension

Differential backup:
-D[vxarctpl] [-f] [-s] <sources> <.full file> <destination>
-D[vxarctpl] <-stdin> <filename to assign> <.full file> <destination>

<destination> can either be -stdout or a file. Use .diff as file extension

Restore full backup:
-R[vo] <.full file> <destination directory | -stdout> [files]
-R[vo] <-stdin> <destination directory>

[files] is one or more files, drives or directories to restore, typed as
provided by -L flag

Restore differential backup:
-RD[vo] <.full> <.diff file> <destination directory | -stdout> [files]
-RD[vo] <-stdin> <.diff file> <destination directory>

[files] is one or more files, drives or directories to restore, typed as
provided by -L flag

List contents:
-L <.full file | .diff file>

Flags:
-r Do not include sub directories
-o Overwrite existing files (default is to abort)
-c Continue if a source file cannot be read or does not exist (default is
to abort)
-xn Use compression level n for traditional data compression applied after
deduplication. Set n = 0 for none (lets you apply your own), x = 1 for
QuickLZ, x = 2 for zlib or x = 3 for bzip2 (default = 1)
-p Include named pipes
-l On *nix: Follow symlinks (default is to store link only). On Windows:
symlinks are always followed - storing link only is not supported
-a Store absolute and complete paths (default is to remove the common
parent path of items passed on the command line)
-s"d:" Use Volume Shadow Copy Service for local drive d: (Windows only)
-f"s" Filter files using a script, s, written in the Lua language
-gn Use n GB memory for a hash table (default = 1). Use -mn to specify
number of MB instead. Use 1 GB per TB data for best compression ratio
Differential backups will use the same memory as the full backup
-tn Use n threads (default = 8)
-vn Use verbose level n for n = 0...9 (default = 8)
-- Prefix items in the <sources> list with "--" to exclude them.

Quick example of backup, differential backups and a restore:
exdupe z:\database\ database.full
exdupe -D z:\database\ database.full database.diff1
exdupe -D z:\database\ database.full database.diff2
exdupe -RD database.full database.diff2 z:\database\restored\

More examples:
exdupe -stdin database.mdf -stdout < database.mdf > database.full
exdupe -m256t2 z:\vmdk\win\ z:\vmdk\mac\ z:\vmdk\hpux\ servers.full
exdupe -RD servers.full servers.diff z:\restored mac win
exdupe -c -f"return(dir or size < 1000000)" z:\stuff stuff.full
exdupe -s"c:" c:\ --c:\pagefile.sys system.full
exdupe -R data.full -stdout > data.xml

[felice2011]

The version recommended to be used is "eXdupe 0.3.5" configured in this way:

[External compressor:eXdupe]
packcmd    = "EXDUPE\eXdupe.exe" -x3 -g1 -t2 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp  
unpackcmd  = "EXDUPE\eXdupe.exe" -R $$arcpackedfile$$.tmp $$arcdatafile$$.tmp   
datafile   = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp

All other next versions create problems with FA

[Razor12911]

I tried to make a patch for this thing but it is half done. The main issue is that exdupe seeks while in stdio, kinda like razor compressor, it is a bit harder to patch something like this plus we have no idea how much it needs to reread from input.

If you test this via Freearc, it will fail but via Cmd it doesn't because Cmd allows seeking while FA doesn't , so if I have free time I'll probably make a full patch for it, a bit busy lately

[kj911]

eXdupe v1.0.0 Final released!

Joe: I know about it! Version 1.10beta1 and beta2 are already out. (BETA1 its bigger EXE than BETA2 and 1.0.0) Separately attached BETA1 version. Beta2 not tested in last time. Mini test results included compared to srep and srep+exdupe mixed precompression before via LOLZ.

https://github.com/rrrlasse/exdupe/r...xdupe-beta.exe

UPD: 2023-11-14 - Results updated!

[Joe Forster/STA]

GitHub: https://github.com/rrrlasse/exdupe.

[Carldric Clement]

I did some full research on the eXdupe pre-process file. It looks like that was difficult to make it both stdin and stdout. So I did this to make it work by using the latest version of eXdupe like this:

[External compressor:exdupe]
header    = 0
packcmd   = exdupe -x0 -g1 -t2 $$arcdatafile$$.tmp -stdout <stdout>
unpackcmd = exdupe -R $$arcpackedfile$$.tmp -stdout <stdout>

Hope you guys can try if it works or not.