balrogg: lossless Vorbis/Opus recompressor

[_contextmodel]

balrogg losslessly recompresses Ogg Vorbis and Opus files. Archives are typically 8-12% smaller than .ogg files and 3-8% smaller than .opus files. After decompression they are reconstructed identically.

The source code (GPLv3) and binaries are available here: https://github.com/iczelia/balrogg.

Hopefully some find this useful. The Windows binaries are attached to the thread (64-bit, 32-bit modern Windows, 32-bit Windows 95 and newer).

Edit (05/09/26):
- v1.1 is now attached. It slightly improves compression and decompression speed and memory usage. It also accepts more valid Vorbis and Opus files. Until v2.0 is reached, the releases will be neither forwards- nor backwards-compatible

[kj911]

Very good work!

In the case of the OGG container format, 5 different encoding profiles were tested (see sample files, description), where in the case of small >30s sample files, it processed the files flawlessly, while on a ~2 hour test file (different source material), it only partially succeeded, with various error messages or endless encoding loops. The memory usage was always between ~200,000KB and ~270,000KB, slowly increasing, over and over again, without writing an output file. Compared to OGGRE, it consumes 2-3-4* more memory, but often with better results. It also brought more than 10% size reduction on the tested OPUS file.

[_contextmodel]

kj911, thank you for the test. Would you mind sharing the corpus used, so that the failures can be pinned down? The high memory usage, among other issues that you observed, is unintended (unless working in parallel).

[kj911]

The test corpus is a bit problematic (and the net too, up: <20kB/s!), at least in terms of public and copyright.

The small samples are from ESB, I'm now testing this quickly to see how the program reacts to converting the entire 2-hour file to OGG. (Source It's in 256kbps AC3.) The problematic cases written in TXT occurred on the ANH audio. (It's in PCM as part of a 10+ GB MKV file.) What is questionable is whether the error appears depending on the given audio and/or just the time length. I also tested it on a 100+ minute OPUS file, without error. (With audio copied from a WEBM video, taken from a video hosting site.)

The conversion/copying was done using XMedia Recode 3.2.7.1 x86 (last XP-compatible release).

UPD: Infinite loop again!

Source: ESB / 2hr / 256kbps AC3

128kbps, Vorbis/ABR OGG via l1: Works!
128kbps, Quality (q4) OGG via l9: Infinite compression loop. Level4 works. (watch longer than 2-4minutes times the Windows Process Explorer.)

The errors you experienced can be easily reconstructed if you have ~2 hours (SW, etc...) of audio files and test all the OGG encoding settings that XMediaRecode allows when encoding audio files to OGG.

UPD2: Infinite loop = Various compression methods used, forced it??

First 10minutes piece from ANH audio, encoded to 128kbps/q4 OGG file: Memory fluctuation are present it avg. 8-12 or upto times (~18-24MB), compression finished! Default cases, 100MB's OGG handled it 1-2minutes times. And one compression steps. Level5 only two memory fluctuation and finished it 17 seconds. (5.8MB's OGG) Level1 in less than 2sec.

Other bugs, not easily reproducible. (packet ends/unpaired bits or any.)

[_contextmodel]

Can you try v1.1 with the progress tracking enabled, and re-measure the encoding/decoding time and memory usage? I have done some tests on larger and more complex Vorbis files.

[kj911]

Recreating problematic files again (few hours ago deleted it):

Encoding times avg. 13-15min per file from Xeon PC..

First test "packet ends" errors via v1.1 are fixed! (128kbps ABR Vorbis OGG case) ~25% reductions! (level1)

The "18 unparsed audio bits" error has also been fixed!

It looks like the error reported as "infinite loop" was most likely a mistake!
Running v1.1 once at level 9 on a 2-hour OGG file took about 20 minutes, if not longer, to compress.

The resulting file sizes, where I compared them, are a few hundred bytes larger for v1.1 than for files compressed by v1.0. (if this was the price of the bug fix, then OK!)

The circumstances of the following case in the first test are a bit confusing:

"128kbps Quality Vorbis OGG:
balrogg-i686.exe -1 e track3.ogg track3.blr
Works! Level 5 to 9 infinite looping compress, no output file. (not finished in below 5min)"

There is such a line in the first TXT: "89 206 180 track3.ogg"

For this file, it did not appear, it finished with it quickly. (both v1.0/v1.1 tested)

Maybe this was "75 731 994 track4.ogg" that problematic file??? I have not managed to reproduce it byte-by-byte so far.

I have not tested the OPUS file now.

[KaktoR]

Looks like --jobs=N does not have any effect? Or how is it supposed to be used?

Edit: Here are some files for which balrogg does not work. They are ogg vorbis but something is off with them. You can check them out if you want.
https://pixeldrain.com/u/TD8wjGrG

Edit: Never mind, I used it wrong. --jobs=N is working.

[KaktoR]

Looks like -5 is the sweet spot here, at least for vorbis set I have.

Input 90.419.200 Bytes
Setting --jobs=6 -b e

-1
Clock Time       :           2.907s
Output 81.047.039 Bytes

-5
Clock Time       :          25.681s
Output 79.436.915 Bytes

-9
Clock Time       :         109.501s
Output 79.430.984 Bytes

[kj911]

Kaktor: Does OGGRE process the problematic files? Have you tested the program on OPUS files (compressed or not)?

A more general question, regarding games...

Since you are the main tester of the forum, you have already tested many games: How common is it in games of the last 20 years (ex: GTA: SA) and are there any examples of it, that OGG or OPUS audio files of 1-2 CDs (or even more) are found? Which is the most glaring case?

_contextmodel: Would it be possible to add streaming/stdio support in the later ones, so that they can be used like, say, OGGRE, via FreeArc? Can the files be unpacked with CLS? (without writing temporary files)

[_contextmodel]

kj911, KaktoR,

I am working on v1.2 (to make the tool a bit faster, work better on Windows, and so on). In principle due to the specifics of how OGG files work, without seekable stream support, they have to be buffered. Perhaps there is a way around this. I have never used FreeArc and I don't know what CLS is.

[KaktoR]

Originally Posted by kj911

Kaktor: Does OGGRE process the problematic files? Have you tested the program on OPUS files (compressed or not)?

A more general question, regarding games...

Since you are the main tester of the forum, you have already tested many games: How common is it in games of the last 20 years (ex: GTA: SA) and are there any examples of it, that OGG or OPUS audio files of 1-2 CDs (or even more) are found? Which is the most glaring case?

Oggre does not work on them either.

I already tested opus and it works.

I think vorbis is more common than opus.

Test on the attachment. It is opus.

[kj911]

Originally Posted by _contextmodel

In principle due to the specifics of how OGG files work, without seekable stream support, they have to be buffered. Perhaps there is a way around this. I have never used FreeArc and I don't know what CLS is.

Here in this post, there is a small archive that you can use to test in practical conditions how to use OGGRE with FreeArc when repackaging games.

Also study the DiskSpan GUI tool, you will find a lot of interesting things.

[KaktoR]

It already works with freearc. However it looks like you have to use solid=0, therefore you cannot use --jobs option I guess.

[External compressor:balrogg]
header    = 0
solid      = 0
packcmd   = balrogg64.exe -1 e $$arcdatafile$$.ogg $$arcdatafile$$.blr
unpackcmd = balrogg64.exe d $$arcdatafile$$.blr $$arcdatafile$$.ogg
datafile   = $$arcdatafile$$.ogg
packedfile = $$arcdatafile$$.blr