Go Back   FileForums > Game Backup > PC Games > PC Games - CD/DVD Conversions > Conversion Tutorials

Reply
 
Thread Tools Display Modes
  #1  
Old 15-08-2022, 19:37
ncomerci ncomerci is offline
Registered User
 
Join Date: Aug 2022
Location: Uruguay
Posts: 3
Thanks: 3
Thanked 16 Times in 3 Posts
ncomerci is on a distinguished road
My Precomp fork with stdin/stdout support

Hi, I've been working on a precomp fork for a few weeks now.
There is still a lot of stuff I want to do with it, but I think right now its on a place where it might be cool to let you guys try it.

Interesting features so far:
  • No changes to PCF header format, meaning at least for now, there should still be full compatibility with PCF files created from main precomp project (v0.4.8) lastest executable and this fork
  • Temp files have a random tag prepended on their filename, meaning you can run multiple instances working on files on the same folder and they should not interfere with each other
  • stdout support for precompression (stdin impractical, at least without heavy reengineering as we do seek on the input file many times during recompression)
  • stdin/stdout support for recompression

Usage:
Mostly the same as regular precomp, the only differences are for using stdin/stdout
  • (example of decompression using stdout) precomp -ostdout -cn -intense -brute file.dat > file.dat.pcf
  • (example of using stdin and stdout for recompression) srep -v0 -d file.tar.pcf.srep - | precomp -ostdout -r stdin | 7za x -si -ttar
so essentially, just use stdout as "output filename" or stdin as "input filename"

This fork should make precomp more practical to use in contexts where only xtool was practical before.

If anybody wants to see the code its at https://github.com/nicolas-comerci/precomp-cpp
Code is still on the compatible_refactor branch

You can find attached 32/64 bits zipped executables.
Please feel free to test and use and report any errors.
For now I am mostly interested on making sure I haven't introduced any new bugs, so I probably won't be fixing anything that also fails on mainline Precomp 0.4.8

EDIT 08/21/22: Added v1.0 of my fork, has been tested a little more and I fixed some OTF compression/decompression problems
Attached Files
File Type: rar precomp fork (broken OTF compression).rar (2.18 MB, 77 views)
File Type: 7z precomp stdin-stdout v1.0 32bit.7z (602.5 KB, 47 views)
File Type: 7z precomp stdin-stdout v1.0 64bit.7z (678.7 KB, 102 views)

Last edited by ncomerci; 21-08-2022 at 15:17. Reason: x64 binary was outdated, included newly compiled one
Reply With Quote
The Following 10 Users Say Thank You to ncomerci For This Useful Post:
Balaji007 (04-09-2022), BLACKFIRE69 (16-08-2022), ffmla (16-08-2022), Gehrman (16-08-2022), kj911 (10-12-2022), Lord.Freddy (15-08-2022), Masquerade (15-08-2022), Razor12911 (16-08-2022), ScOOt3r (16-08-2022), Vladislav989 (09-09-2022)
Sponsored Links
  #2  
Old 16-08-2022, 03:16
Lord.Freddy's Avatar
Lord.Freddy Lord.Freddy is offline
Registered User
 
Join Date: Apr 2022
Location: In Forest
Posts: 46
Thanks: 194
Thanked 30 Times in 21 Posts
Lord.Freddy is on a distinguished road
There seem to be some problems
Attached Images
File Type: png Screenshot 2022-08-16 144122.png (129.7 KB, 325 views)
Reply With Quote
  #3  
Old 16-08-2022, 04:21
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 1,155
Thanks: 284
Thanked 1,346 Times in 610 Posts
Masquerade is on a distinguished road
Quote:
Originally Posted by Lord.Freddy View Post
There seem to be some problems
Output is being redirected to STDERR (console window)
Reply With Quote
The Following User Says Thank You to Masquerade For This Useful Post:
Lord.Freddy (16-08-2022)
  #4  
Old 16-08-2022, 04:35
Gehrman's Avatar
Gehrman Gehrman is offline
Registered User
 
Join Date: Jan 2020
Location: Save Palestine
Posts: 78
Thanks: 1,267
Thanked 87 Times in 47 Posts
Gehrman is on a distinguished road
Thanks ncomerci

Good Work
Code:
[External compressor:precomp048f]
header = 0
packcmd   = precomp64 -cn -intense0 -ostdout <stdout> $$arcdatafile$$.tmp 
unpackcmd = precomp64 -ostdout -r stdin <stdin> <stdout>
Reply With Quote
The Following 4 Users Say Thank You to Gehrman For This Useful Post:
Lord.Freddy (16-08-2022), Masquerade (16-08-2022), ncomerci (16-08-2022), TheGh0stie13 (10-12-2022)
  #5  
Old 16-08-2022, 04:59
Gehrman's Avatar
Gehrman Gehrman is offline
Registered User
 
Join Date: Jan 2020
Location: Save Palestine
Posts: 78
Thanks: 1,267
Thanked 87 Times in 47 Posts
Gehrman is on a distinguished road
Code:
Compressed 2 files, 314,572,800 => 511,394,374 bytes. Ratio 162.57%
Compression time: cpu 0.31 sec/real 75.05 sec = 0%. Speed 4.19 mB/s
All OK
Code:
Tested 2 files, 511,394,374 => 314,572,800 bytes. Ratio 162.57%
Testing time: cpu 0.94 sec/real 30.75 sec = 3%. Speed 10.23 mB/s
All OK
Reply With Quote
The Following 3 Users Say Thank You to Gehrman For This Useful Post:
Lord.Freddy (16-08-2022), Masquerade (16-08-2022), TheGh0stie13 (10-12-2022)
  #6  
Old 16-08-2022, 20:04
Lord.Freddy's Avatar
Lord.Freddy Lord.Freddy is offline
Registered User
 
Join Date: Apr 2022
Location: In Forest
Posts: 46
Thanks: 194
Thanked 30 Times in 21 Posts
Lord.Freddy is on a distinguished road
I got this error in test archive
Attached Images
File Type: jpg Screenshot 2022-08-16 222638.jpg (67.3 KB, 295 views)
Reply With Quote
The Following User Says Thank You to Lord.Freddy For This Useful Post:
ncomerci (16-08-2022)
  #7  
Old 16-08-2022, 20:43
ncomerci ncomerci is offline
Registered User
 
Join Date: Aug 2022
Location: Uruguay
Posts: 3
Thanks: 3
Thanked 16 Times in 3 Posts
ncomerci is on a distinguished road
Quote:
Originally Posted by Lord.Freddy View Post
I got this error in test archive
it says liblzma error, you are using precomp's included compression?
it wouldn't surprise me if I broke it because I didn't test much with it, will look into it.

however for now you can use -cn option to disable the included compression (this I did test thoroughly) and just use some other compression method with freearc, or pipe it through to xz/7za...

In fact, I might even completely delete the included compression capabilities in a future version on this fork... seems to me like stdin/stdout support makes that obsolete and just unnecessarily complicates the code
Reply With Quote
The Following 3 Users Say Thank You to ncomerci For This Useful Post:
Gehrman (17-08-2022), Lord.Freddy (17-08-2022), Razor12911 (18-08-2022)
  #8  
Old 21-08-2022, 15:18
ncomerci ncomerci is offline
Registered User
 
Join Date: Aug 2022
Location: Uruguay
Posts: 3
Thanks: 3
Thanked 16 Times in 3 Posts
ncomerci is on a distinguished road
Quote:
Originally Posted by Lord.Freddy View Post
I got this error in test archive
Added new version on the original post, on my tests OTF compression/decompression now works correctly
Reply With Quote
The Following 3 Users Say Thank You to ncomerci For This Useful Post:
Gehrman (22-08-2022), Lord.Freddy (21-08-2022), Masquerade (21-08-2022)
  #9  
Old 10-12-2022, 01:04
Gehrman's Avatar
Gehrman Gehrman is offline
Registered User
 
Join Date: Jan 2020
Location: Save Palestine
Posts: 78
Thanks: 1,267
Thanked 87 Times in 47 Posts
Gehrman is on a distinguished road
Compatible with Windows XP
Attached Files
File Type: rar precomp 048 stdin-stdout v1.0 32bit_WinXP.rar (2.08 MB, 10 views)
Reply With Quote
The Following User Says Thank You to Gehrman For This Useful Post:
kj911 (10-12-2022)
  #10  
Old 10-12-2022, 02:35
kj911 kj911 is offline
Registered User
 
Join Date: Apr 2010
Location: world
Posts: 179
Thanks: 130
Thanked 61 Times in 42 Posts
kj911 is on a distinguished road
Gehrman: Thanks from Win XP compatible executables!

One problem from precompf: Totally ignore the external INI file and settings, located in main EXE folder. See the picture.



Second smaller problems: More huge external dll's. (45 file!)

Solutions? 1. Recompile again or make one executable file via embedding more resources. Created my Enigma Vbox-ed versions the package. (Used in last v9.9.0) Attached in and testing now.

UPD: Added size optimized versions the package from smaller EXE's size fans.

Precomp(f).exe files and few dll's packed in UPX and packed all files to one Enigma VBoxed EXE file. More AV in falsely alarms!! Use at your own risk! This one positive the package: 5.19/5.20MB vs. 1.94MB's EXE file size in HDD. Compressibility via to 7Z in get ~0.1MB's smaller size usage from installers. (One EXE file packed to 7z archive.) See TXT file from more infos. Safer ways the many AV's conflicts to use bigger EXE files.

Compresing via stdin/out mode testing now in next days.

UPD2: The quick test results added comprare the Precomp0.4.8 stdio vs. xtool 0.12 x86 versions. This make few MB's archive and decompressing, its precomp now send ~1kB's binary input data from console window without stdout. Decompression its finished without any extra issues. 2nd decompression, not have stderr issues. More bigger files test its OK. No problems or any moe. CRC's its OK tested in 2nd decompression stage will finished and extracted files overwriting. See TXT file from test results from comparing.

Last edited by kj911; 10-12-2022 at 10:12. Reason: Added size optimized PreUPX'ed package
Reply With Quote
The Following User Says Thank You to kj911 For This Useful Post:
Gehrman (10-12-2022)
  #11  
Old 10-12-2022, 13:57
Gehrman's Avatar
Gehrman Gehrman is offline
Registered User
 
Join Date: Jan 2020
Location: Save Palestine
Posts: 78
Thanks: 1,267
Thanked 87 Times in 47 Posts
Gehrman is on a distinguished road
@kj911
The following tools are used.
Code:
Winpp
HXD
kernelxp.dll
QTfiles32 DLL Files
The following example was made using the above tools
Attached Files
File Type: rar zstd-v1.5.2-win32_WinXP.rar (471.9 KB, 7 views)
Reply With Quote
The Following 2 Users Say Thank You to Gehrman For This Useful Post:
kj911 (12-12-2022), ScOOt3r (11-12-2022)
  #12  
Old 12-12-2022, 04:59
kj911 kj911 is offline
Registered User
 
Join Date: Apr 2010
Location: world
Posts: 179
Thanks: 130
Thanked 61 Times in 42 Posts
kj911 is on a distinguished road
Gehrman: Two quick questions.

1. What would that be "QTfiles32 DLL Files" ?? No found reasonable results.
2. WinPP: Would that be it? -> https://www.winholdem.net/winpp/

HXD use patching EXE's running to older OS? (Win7 to WinXP) See more post in the topic --> https://encode.su/threads/1838-Comma...Profiling-Tool

This precomp EXE files, compiled in GCC or use any compiler?
Reply With Quote
  #13  
Old 12-12-2022, 05:23
Gehrman's Avatar
Gehrman Gehrman is offline
Registered User
 
Join Date: Jan 2020
Location: Save Palestine
Posts: 78
Thanks: 1,267
Thanked 87 Times in 47 Posts
Gehrman is on a distinguished road
Winpp
QTfiles
We use HXD to change KERNEL32.dll to KERNELXP.dll.


Winpp can also add CLS-LOLZ_x64.exe and CLS-LOLZ_x86.exe compatibility on Windows XP.
Attached Images
File Type: jpg 1.jpg (174.4 KB, 177 views)
Attached Files
File Type: rar winpp20.rar (217.8 KB, 16 views)

Last edited by Gehrman; 12-12-2022 at 05:28.
Reply With Quote
  #14  
Old 12-12-2022, 11:03
kj911 kj911 is offline
Registered User
 
Join Date: Apr 2010
Location: world
Posts: 179
Thanks: 130
Thanked 61 Times in 42 Posts
kj911 is on a distinguished road
Gehrman: Thanks from replies!

In last cls-lolz_x86.exe file natively works and running from Windows XP. (To my knowledge, this original EXE CRC: 97A4C611) Older v21a5/6/7 versions required from exe patching. (And OGGRE and DLZ.) In better ways posted from external site use "MakeLX and MakeXP" mini tools, using patched exe from XP system, without using external dll's. See test it.... and attached.

MakeXP-x86.exe -> XP compatibility patcher.
MakeLAA-x86.exe -> Add large-address-aware functions. (Or just use 4GB memory patcher tools from available via net.)

Usage from via cmd: MakeXP/LAA-x86.exe anyfile.exe or drop any exe/dll file to EXE file icon.

Installed 64bit Win XP not available in my own circle of acquaintance. Not have available any x64 XP install disk and hardware from testing purpose.

UPD: Comparing the MakeLX and WinPP exe patching. No size difference from two patched executables. WinPP patched to OS and Subsystem version to 4.0 its original higher than 6.0 or any values. MakeLX patched to OS/Subsystems version to 5.0... No se any differences. (Except patcher exe size. 520kB vs. 2kB )

UPD2: Quickly compare MakeLX and 4GB patch EXE tools differences... MakeLX not change 3bytes in exe file, located in hexa: 00120 lines. Hexa values: "00 00 00 00 03 00 00 81" 4GB pated version hex: "AF 23 03 00 03 00 00 81" OS/Subsystem line located in: 00108 and 00110 and first hexa byte changed 06 to 05 or 04 values.
Attached Files
File Type: 7z MakeLX.7z (2.2 KB, 10 views)

Last edited by kj911; 12-12-2022 at 11:35. Reason: More infos added
Reply With Quote
The Following User Says Thank You to kj911 For This Useful Post:
Gehrman (12-12-2022)
  #15  
Old 27-04-2023, 16:56
panker1992's Avatar
panker1992 panker1992 is offline
Registered User
 
Join Date: Oct 2015
Location: Always Somewhere
Posts: 539
Thanks: 113
Thanked 766 Times in 299 Posts
panker1992 is on a distinguished road
Hello, Mr Nomerci. I have been testing your stuff secretly

I have found some bugs regarding your version of precomp
Game that i Found a bug: Resident Evil HD Remaster
Folder and files: nativePC\arc\id\e_rom\text\fre\mes\*.*
(They are proccessed and resotred but CRC doesnt remain the same. the test was outside freearc)

Furthermore, I have seen in your github you uploaded a neo Releases series.
and you have removed the ability to keep data decompressed thus making the data lzma-ed by default. srep cant be used and lolz cant be used

any fixes? in the making?
__________________
My projects : Masked Compression, lzma2(xz) on Freearc, Zstd compressor for windows
My optimizations : packjpg.exe, zstd, lzham, precomp-dev-0.45.
Reply With Quote
Reply

Tags
precomp

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[CIUv3] Custom Installer Ultimate v3 KaktoR Conversion Tutorials 895 14-08-2024 10:36
Ultimate Conversion Compressor (UCC) vollachr Conversion Tutorials 55 26-04-2021 09:27
Precomp 0.44 and NEW packjpg.exe (self optimized) panker1992 Conversion Tutorials 7 14-01-2016 09:23
Information about Precomp Razor12911 Conversion Tutorials 6 02-09-2015 16:10



All times are GMT -7. The time now is 12:33.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.
Copyright 2000-2020, FileForums @ https://fileforums.com