Results 1 to 29 of 29

Thread: Quo Vadis JPEG - Another update (3)

  1. #1
    Member
    Join Date
    Apr 2012
    Location
    Stuttgart
    Posts
    430
    Thanks
    1
    Thanked 93 Times in 54 Posts

    Quo Vadis JPEG - Another update (3)

    Hi folks, just released a new version of the experimental(!) jpeg coder demonstrating the ideas our new JPEG standard will provide. The sources are here: https://github.com/thorfdbg/libjpeg Only minor modifications were made in this release. The Huffman coder now fills undefined bits at the end of the entropy coded segment by ones, and potentially inserts a stuffed zero byte. Note that the standard defines this only in an informative note, so it is likely not required. The insertion of a stuffed zero byte at this position might also be unnecessary as a parser should identify the first 0xff byte ahead of the marker as "fill byte". Residual coding has been reworked and uses now a simple progressive scan with the special DC case removed. The performance does not differ significantly, and it makes integration into legacy software or hardware easier. Default Huffman tables are now also provided for progressive and lossless scans, thus optimizing the Huffman tables just for these scan types is no longer necessary. Residual or refinement scans still require this argument, though. Usage of the codec does not differ from previous releases, though codestreams using the residual coding mode are not compatible to those generated by 0.5. Note that this is still experimental software and the codestream might still change until the ISO committee decides on the technology. Finally, a couple of workarounds for earlier releases of the g++ compiler have been added.

  2. #2
    Member
    Join Date
    Apr 2009
    Location
    here
    Posts
    200
    Thanks
    161
    Thanked 108 Times in 64 Posts
    a win32 compile here.

    http://ompldr.org/vZnY1NA

  3. #3
    Member
    Join Date
    Apr 2012
    Location
    Stuttgart
    Posts
    430
    Thanks
    1
    Thanked 93 Times in 54 Posts
    Quote Originally Posted by load View Post
    a win32 compile here. http://ompldr.org/vZnY1NA
    Thanks!

  4. #4
    Member Fallon's Avatar
    Join Date
    May 2008
    Location
    Europe - The Netherlands
    Posts
    154
    Thanks
    14
    Thanked 10 Times in 5 Posts
    I suggest to use mediafire for small legal files, or a google account. Or others.
    ompldr.org is known to have spread malicious stuff like the ransom virus.
    (Sorry load).

  5. #5
    Member
    Join Date
    Apr 2009
    Location
    here
    Posts
    200
    Thanks
    161
    Thanked 108 Times in 64 Posts

  6. #6
    Member Fallon's Avatar
    Join Date
    May 2008
    Location
    Europe - The Netherlands
    Posts
    154
    Thanks
    14
    Thanked 10 Times in 5 Posts
    But there's a warning if you try to open the site in Opera browser.
    https://sites.google.com/site/fallon...2012-10-14.png
    Okay, I probably should have left it at that.
    Last edited by Fallon; 14th October 2012 at 21:42.

  7. #7
    Member Alexander Rhatushnyak's Avatar
    Join Date
    Oct 2007
    Location
    Canada
    Posts
    231
    Thanks
    36
    Thanked 78 Times in 42 Posts
    Guess there's a bug in this version.
    It fails to decompress well the attached image
    (after it's converted from png back to ppm).
    Most likely because of the header processing code.

    Code:
    jpeg.exe -p -c -a fuji15.ppm fuji15.ari
    jpeg.exe fuji15.ari fuji15.dec
    fc/b fuji15.ppm fuji15.dec |more
    Comparing files fuji15.ppm and FUJI15.DEC
    0000000F: 0B 01
    00000010: 01 03
    00000011: 03 09
    00000012: 09 02
    00000013: 02 03
    00000014: 03 0A
    00000015: 0A 02
    00000016: 02 03
    00000017: 03 0B
    00000018: 0B 01
    00000019: 01 03
    0000001A: 03 0B
    0000001B: 0B 02
    0000001C: 02 03
    0000001D: 03 0B
    0000001E: 0B 02
    0000001F: 02 03
    00000020: 03 0C
    ...
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	fuji15.png 
Views:	250 
Size:	73.9 KB 
ID:	2064  

    This newsgroup is dedicated to image compression:
    http://linkedin.com/groups/Image-Compression-3363256

  8. #8
    Member
    Join Date
    Apr 2012
    Location
    Stuttgart
    Posts
    430
    Thanks
    1
    Thanked 93 Times in 54 Posts
    Quote Originally Posted by Alexander Rhatushnyak View Post
    Guess there's a bug in this version. It fails to decompress well the attached image (after it's converted from png back to ppm). Most likely because of the header processing code.
    Thanks, could very well be. Could you please sent the image to thor at math dot tu dash berlin dot de?

  9. #9
    Member
    Join Date
    Apr 2012
    Location
    Stuttgart
    Posts
    430
    Thanks
    1
    Thanked 93 Times in 54 Posts
    Quote Originally Posted by Alexander Rhatushnyak View Post
    Guess there's a bug in this version. It fails to decompress well the attached image
    Thanks, found and fixed. Interestingly, actually not a bug of the core library. This is a side effect of writing a primitive PPM reader in a rush and not reading fscanf() properly. The latter gobbled up any character at the start of the image that is a valid white space character (bummer!) and thus shifted the image and the components within the image.

  10. #10
    Member
    Join Date
    Apr 2012
    Location
    Stuttgart
    Posts
    430
    Thanks
    1
    Thanked 93 Times in 54 Posts
    Please find a new release at its usual place https://github.com/thorfdbg/libjpeg/ Besides the fix already mentioned above, I also found that the support for grey-scale images was broken, which was again not a library but a front-end problem. I also fixed a bug in the hierarchical coder which, depending on the image height, sometimes computed the residual signal incorrectly.

  11. #11
    Member caveman's Avatar
    Join Date
    Jul 2009
    Location
    Strasbourg, France
    Posts
    190
    Thanks
    8
    Thanked 62 Times in 33 Posts
    Quote Originally Posted by thorfdbg View Post
    Please find a new release at its usual place https://github.com/thorfdbg/libjpeg/ Besides the fix already mentioned above, I also found that the support for grey-scale images was broken, which was again not a library but a front-end problem. I also fixed a bug in the hierarchical coder which, depending on the image height, sometimes computed the residual signal incorrectly.
    Just a stupid question, does your lib write all the DHT (Huffman tables) and DQT (Quantization tables) in a single marker (Adobe Photoshop does it this way and it saves a few bytes) or does it only enclose one table per marker (IJG libjpg does it this way)?

    Sample files attached.

    IJG libjpeg (4 Huffman tables in 4 different DHT markers):

    Offset 0x00b1 Marker 0xffc4 DHT Define Huffman Table(s) length variable 0x1c
    JPEG_DHT_Parameters:
    TableClass = 0
    HuffmanTableIdentifier = 0
    nHuffmanCodesOfLength 0 = 1
    ValueOfHuffmanCode 0 = 0
    nHuffmanCodesOfLength 1 = 0
    nHuffmanCodesOfLength 2 = 0
    nHuffmanCodesOfLength 3 = 7
    ValueOfHuffmanCode 0 = 1
    ValueOfHuffmanCode 1 = 2
    ValueOfHuffmanCode 2 = 3
    ValueOfHuffmanCode 3 = 5
    ValueOfHuffmanCode 4 = 6
    ValueOfHuffmanCode 5 = 7
    ValueOfHuffmanCode 6 = 8
    nHuffmanCodesOfLength 4 = 1
    ValueOfHuffmanCode 0 = 4
    nHuffmanCodesOfLength 5 = 0
    nHuffmanCodesOfLength 6 = 0
    nHuffmanCodesOfLength 7 = 0
    nHuffmanCodesOfLength 8 = 0
    nHuffmanCodesOfLength 9 = 0
    nHuffmanCodesOfLength 10 = 0
    nHuffmanCodesOfLength 11 = 0
    nHuffmanCodesOfLength 12 = 0
    nHuffmanCodesOfLength 13 = 0
    nHuffmanCodesOfLength 14 = 0
    nHuffmanCodesOfLength 15 = 0

    Offset 0x00cf Marker 0xffc4 DHT Define Huffman Table(s) length variable 0x5b
    JPEG_DHT_Parameters:
    TableClass = 1
    HuffmanTableIdentifier = 0
    nHuffmanCodesOfLength 0 = 0
    nHuffmanCodesOfLength 1 = 0
    nHuffmanCodesOfLength 2 = 5
    ValueOfHuffmanCode 0 = 0
    ValueOfHuffmanCode 1 = 1
    ValueOfHuffmanCode 2 = 2
    ValueOfHuffmanCode 3 = 3
    ValueOfHuffmanCode 4 = 4
    nHuffmanCodesOfLength 3 = 2
    ValueOfHuffmanCode 0 = 5
    ValueOfHuffmanCode 1 = 18
    nHuffmanCodesOfLength 4 = 2
    ValueOfHuffmanCode 0 = 6
    ValueOfHuffmanCode 1 = 17
    nHuffmanCodesOfLength 5 = 3
    ValueOfHuffmanCode 0 = 7
    ValueOfHuffmanCode 1 = 19
    ValueOfHuffmanCode 2 = 34
    nHuffmanCodesOfLength 6 = 8
    ValueOfHuffmanCode 0 = 8
    ValueOfHuffmanCode 1 = 20
    ValueOfHuffmanCode 2 = 33
    ValueOfHuffmanCode 3 = 49
    ValueOfHuffmanCode 4 = 50
    ValueOfHuffmanCode 5 = 65
    ValueOfHuffmanCode 6 = 81
    ValueOfHuffmanCode 7 = 180
    nHuffmanCodesOfLength 7 = 13
    ValueOfHuffmanCode 0 = 21
    ValueOfHuffmanCode 1 = 24
    ValueOfHuffmanCode 2 = 35
    ValueOfHuffmanCode 3 = 40
    ValueOfHuffmanCode 4 = 55
    ValueOfHuffmanCode 5 = 66
    ValueOfHuffmanCode 6 = 82
    ValueOfHuffmanCode 7 = 85
    ValueOfHuffmanCode 8 = 97
    ValueOfHuffmanCode 9 = 117
    ValueOfHuffmanCode 10 = 148
    ValueOfHuffmanCode 11 = 161
    ValueOfHuffmanCode 12 = 212
    nHuffmanCodesOfLength 8 = 7
    ValueOfHuffmanCode 0 = 36
    ValueOfHuffmanCode 1 = 51
    ValueOfHuffmanCode 2 = 98
    ValueOfHuffmanCode 3 = 113
    ValueOfHuffmanCode 4 = 114
    ValueOfHuffmanCode 5 = 145
    ValueOfHuffmanCode 6 = 182
    nHuffmanCodesOfLength 9 = 6
    ValueOfHuffmanCode 0 = 22
    ValueOfHuffmanCode 1 = 23
    ValueOfHuffmanCode 2 = 129
    ValueOfHuffmanCode 3 = 146
    ValueOfHuffmanCode 4 = 162
    ValueOfHuffmanCode 5 = 210
    nHuffmanCodesOfLength 10 = 9
    ValueOfHuffmanCode 0 = 52
    ValueOfHuffmanCode 1 = 54
    ValueOfHuffmanCode 2 = 83
    ValueOfHuffmanCode 3 = 99
    ValueOfHuffmanCode 4 = 130
    ValueOfHuffmanCode 5 = 163
    ValueOfHuffmanCode 6 = 164
    ValueOfHuffmanCode 7 = 177
    ValueOfHuffmanCode 8 = 178
    nHuffmanCodesOfLength 11 = 10
    ValueOfHuffmanCode 0 = 37
    ValueOfHuffmanCode 1 = 38
    ValueOfHuffmanCode 2 = 39
    ValueOfHuffmanCode 3 = 101
    ValueOfHuffmanCode 4 = 102
    ValueOfHuffmanCode 5 = 118
    ValueOfHuffmanCode 6 = 132
    ValueOfHuffmanCode 7 = 150
    ValueOfHuffmanCode 8 = 194
    ValueOfHuffmanCode 9 = 213
    nHuffmanCodesOfLength 12 = 7
    ValueOfHuffmanCode 0 = 67
    ValueOfHuffmanCode 1 = 69
    ValueOfHuffmanCode 2 = 116
    ValueOfHuffmanCode 3 = 147
    ValueOfHuffmanCode 4 = 166
    ValueOfHuffmanCode 5 = 179
    ValueOfHuffmanCode 6 = 226
    nHuffmanCodesOfLength 13 = 0
    nHuffmanCodesOfLength 14 = 0
    nHuffmanCodesOfLength 15 = 0

    Offset 0x012c Marker 0xffc4 DHT Define Huffman Table(s) length variable 0x1c
    JPEG_DHT_Parameters:
    TableClass = 0
    HuffmanTableIdentifier = 1
    nHuffmanCodesOfLength 0 = 1
    ValueOfHuffmanCode 0 = 0
    nHuffmanCodesOfLength 1 = 0
    nHuffmanCodesOfLength 2 = 2
    ValueOfHuffmanCode 0 = 1
    ValueOfHuffmanCode 1 = 2
    nHuffmanCodesOfLength 3 = 3
    ValueOfHuffmanCode 0 = 3
    ValueOfHuffmanCode 1 = 4
    ValueOfHuffmanCode 2 = 5
    nHuffmanCodesOfLength 4 = 1
    ValueOfHuffmanCode 0 = 6
    nHuffmanCodesOfLength 5 = 1
    ValueOfHuffmanCode 0 = 7
    nHuffmanCodesOfLength 6 = 1
    ValueOfHuffmanCode 0 = 8
    nHuffmanCodesOfLength 7 = 0
    nHuffmanCodesOfLength 8 = 0
    nHuffmanCodesOfLength 9 = 0
    nHuffmanCodesOfLength 10 = 0
    nHuffmanCodesOfLength 11 = 0
    nHuffmanCodesOfLength 12 = 0
    nHuffmanCodesOfLength 13 = 0
    nHuffmanCodesOfLength 14 = 0
    nHuffmanCodesOfLength 15 = 0

    Offset 0x014a Marker 0xffc4 DHT Define Huffman Table(s) length variable 0x46
    JPEG_DHT_Parameters:
    TableClass = 1
    HuffmanTableIdentifier = 1
    nHuffmanCodesOfLength 0 = 0
    nHuffmanCodesOfLength 1 = 2
    ValueOfHuffmanCode 0 = 0
    ValueOfHuffmanCode 1 = 1
    nHuffmanCodesOfLength 2 = 1
    ValueOfHuffmanCode 0 = 2
    nHuffmanCodesOfLength 3 = 2
    ValueOfHuffmanCode 0 = 3
    ValueOfHuffmanCode 1 = 17
    nHuffmanCodesOfLength 4 = 2
    ValueOfHuffmanCode 0 = 4
    ValueOfHuffmanCode 1 = 33
    nHuffmanCodesOfLength 5 = 5
    ValueOfHuffmanCode 0 = 5
    ValueOfHuffmanCode 1 = 18
    ValueOfHuffmanCode 2 = 19
    ValueOfHuffmanCode 3 = 49
    ValueOfHuffmanCode 4 = 240
    nHuffmanCodesOfLength 6 = 7
    ValueOfHuffmanCode 0 = 6
    ValueOfHuffmanCode 1 = 21
    ValueOfHuffmanCode 2 = 34
    ValueOfHuffmanCode 3 = 50
    ValueOfHuffmanCode 4 = 65
    ValueOfHuffmanCode 5 = 81
    ValueOfHuffmanCode 6 = 209
    nHuffmanCodesOfLength 7 = 8
    ValueOfHuffmanCode 0 = 20
    ValueOfHuffmanCode 1 = 113
    ValueOfHuffmanCode 2 = 129
    ValueOfHuffmanCode 3 = 145
    ValueOfHuffmanCode 4 = 161
    ValueOfHuffmanCode 5 = 177
    ValueOfHuffmanCode 6 = 193
    ValueOfHuffmanCode 7 = 241
    nHuffmanCodesOfLength 8 = 7
    ValueOfHuffmanCode 0 = 7
    ValueOfHuffmanCode 1 = 51
    ValueOfHuffmanCode 2 = 66
    ValueOfHuffmanCode 3 = 82
    ValueOfHuffmanCode 4 = 97
    ValueOfHuffmanCode 5 = 130
    ValueOfHuffmanCode 6 = 194
    nHuffmanCodesOfLength 9 = 5
    ValueOfHuffmanCode 0 = 35
    ValueOfHuffmanCode 1 = 36
    ValueOfHuffmanCode 2 = 114
    ValueOfHuffmanCode 3 = 162
    ValueOfHuffmanCode 4 = 210
    nHuffmanCodesOfLength 10 = 8
    ValueOfHuffmanCode 0 = 22
    ValueOfHuffmanCode 1 = 52
    ValueOfHuffmanCode 2 = 98
    ValueOfHuffmanCode 3 = 99
    ValueOfHuffmanCode 4 = 115
    ValueOfHuffmanCode 5 = 146
    ValueOfHuffmanCode 6 = 225
    ValueOfHuffmanCode 7 = 242
    nHuffmanCodesOfLength 11 = 3
    ValueOfHuffmanCode 0 = 178
    ValueOfHuffmanCode 1 = 211
    ValueOfHuffmanCode 2 = 226
    nHuffmanCodesOfLength 12 = 1
    ValueOfHuffmanCode 0 = 227
    nHuffmanCodesOfLength 13 = 0
    nHuffmanCodesOfLength 14 = 0
    nHuffmanCodesOfLength 15 = 0



    Photoshop (4 Huffman tables in single DHT marker):

    Offset 0x00d8 Marker 0xffc4 DHT Define Huffman Table(s) length variable 0xd3
    JPEG_DHT_Parameters:
    TableClass = 0
    HuffmanTableIdentifier = 0
    nHuffmanCodesOfLength 0 = 1
    ValueOfHuffmanCode 0 = 0
    nHuffmanCodesOfLength 1 = 0
    nHuffmanCodesOfLength 2 = 0
    nHuffmanCodesOfLength 3 = 7
    ValueOfHuffmanCode 0 = 1
    ValueOfHuffmanCode 1 = 2
    ValueOfHuffmanCode 2 = 3
    ValueOfHuffmanCode 3 = 5
    ValueOfHuffmanCode 4 = 6
    ValueOfHuffmanCode 5 = 7
    ValueOfHuffmanCode 6 = 8
    nHuffmanCodesOfLength 4 = 1
    ValueOfHuffmanCode 0 = 4
    nHuffmanCodesOfLength 5 = 0
    nHuffmanCodesOfLength 6 = 0
    nHuffmanCodesOfLength 7 = 0
    nHuffmanCodesOfLength 8 = 0
    nHuffmanCodesOfLength 9 = 0
    nHuffmanCodesOfLength 10 = 0
    nHuffmanCodesOfLength 11 = 0
    nHuffmanCodesOfLength 12 = 0
    nHuffmanCodesOfLength 13 = 0
    nHuffmanCodesOfLength 14 = 0
    nHuffmanCodesOfLength 15 = 0
    TableClass = 0
    HuffmanTableIdentifier = 1
    nHuffmanCodesOfLength 0 = 1
    ValueOfHuffmanCode 0 = 0
    nHuffmanCodesOfLength 1 = 0
    nHuffmanCodesOfLength 2 = 2
    ValueOfHuffmanCode 0 = 1
    ValueOfHuffmanCode 1 = 2
    nHuffmanCodesOfLength 3 = 3
    ValueOfHuffmanCode 0 = 3
    ValueOfHuffmanCode 1 = 4
    ValueOfHuffmanCode 2 = 5
    nHuffmanCodesOfLength 4 = 1
    ValueOfHuffmanCode 0 = 6
    nHuffmanCodesOfLength 5 = 1
    ValueOfHuffmanCode 0 = 7
    nHuffmanCodesOfLength 6 = 1
    ValueOfHuffmanCode 0 = 8
    nHuffmanCodesOfLength 7 = 0
    nHuffmanCodesOfLength 8 = 0
    nHuffmanCodesOfLength 9 = 0
    nHuffmanCodesOfLength 10 = 0
    nHuffmanCodesOfLength 11 = 0
    nHuffmanCodesOfLength 12 = 0
    nHuffmanCodesOfLength 13 = 0
    nHuffmanCodesOfLength 14 = 0
    nHuffmanCodesOfLength 15 = 0
    TableClass = 1
    HuffmanTableIdentifier = 0
    nHuffmanCodesOfLength 0 = 0
    nHuffmanCodesOfLength 1 = 0
    nHuffmanCodesOfLength 2 = 5
    ValueOfHuffmanCode 0 = 0
    ValueOfHuffmanCode 1 = 1
    ValueOfHuffmanCode 2 = 2
    ValueOfHuffmanCode 3 = 3
    ValueOfHuffmanCode 4 = 4
    nHuffmanCodesOfLength 3 = 2
    ValueOfHuffmanCode 0 = 18
    ValueOfHuffmanCode 1 = 5
    nHuffmanCodesOfLength 4 = 2
    ValueOfHuffmanCode 0 = 17
    ValueOfHuffmanCode 1 = 6
    nHuffmanCodesOfLength 5 = 3
    ValueOfHuffmanCode 0 = 34
    ValueOfHuffmanCode 1 = 19
    ValueOfHuffmanCode 2 = 7
    nHuffmanCodesOfLength 6 = 8
    ValueOfHuffmanCode 0 = 33
    ValueOfHuffmanCode 1 = 49
    ValueOfHuffmanCode 2 = 65
    ValueOfHuffmanCode 3 = 81
    ValueOfHuffmanCode 4 = 50
    ValueOfHuffmanCode 5 = 20
    ValueOfHuffmanCode 6 = 180
    ValueOfHuffmanCode 7 = 8
    nHuffmanCodesOfLength 7 = 14
    ValueOfHuffmanCode 0 = 97
    ValueOfHuffmanCode 1 = 161
    ValueOfHuffmanCode 2 = 66
    ValueOfHuffmanCode 3 = 82
    ValueOfHuffmanCode 4 = 98
    ValueOfHuffmanCode 5 = 35
    ValueOfHuffmanCode 6 = 148
    ValueOfHuffmanCode 7 = 212
    ValueOfHuffmanCode 8 = 21
    ValueOfHuffmanCode 9 = 85
    ValueOfHuffmanCode 10 = 117
    ValueOfHuffmanCode 11 = 55
    ValueOfHuffmanCode 12 = 24
    ValueOfHuffmanCode 13 = 40
    nHuffmanCodesOfLength 8 = 6
    ValueOfHuffmanCode 0 = 113
    ValueOfHuffmanCode 1 = 145
    ValueOfHuffmanCode 2 = 114
    ValueOfHuffmanCode 3 = 51
    ValueOfHuffmanCode 4 = 36
    ValueOfHuffmanCode 5 = 182
    nHuffmanCodesOfLength 9 = 4
    ValueOfHuffmanCode 0 = 129
    ValueOfHuffmanCode 1 = 146
    ValueOfHuffmanCode 2 = 162
    ValueOfHuffmanCode 3 = 210
    nHuffmanCodesOfLength 10 = 9
    ValueOfHuffmanCode 0 = 177
    ValueOfHuffmanCode 1 = 130
    ValueOfHuffmanCode 2 = 178
    ValueOfHuffmanCode 3 = 83
    ValueOfHuffmanCode 4 = 99
    ValueOfHuffmanCode 5 = 164
    ValueOfHuffmanCode 6 = 22
    ValueOfHuffmanCode 7 = 54
    ValueOfHuffmanCode 8 = 23
    nHuffmanCodesOfLength 11 = 9
    ValueOfHuffmanCode 0 = 194
    ValueOfHuffmanCode 1 = 163
    ValueOfHuffmanCode 2 = 52
    ValueOfHuffmanCode 3 = 132
    ValueOfHuffmanCode 4 = 101
    ValueOfHuffmanCode 5 = 213
    ValueOfHuffmanCode 6 = 102
    ValueOfHuffmanCode 7 = 118
    ValueOfHuffmanCode 8 = 150
    nHuffmanCodesOfLength 12 = 9
    ValueOfHuffmanCode 0 = 226
    ValueOfHuffmanCode 1 = 67
    ValueOfHuffmanCode 2 = 147
    ValueOfHuffmanCode 3 = 179
    ValueOfHuffmanCode 4 = 116
    ValueOfHuffmanCode 5 = 37
    ValueOfHuffmanCode 6 = 69
    ValueOfHuffmanCode 7 = 38
    ValueOfHuffmanCode 8 = 166
    nHuffmanCodesOfLength 13 = 1
    ValueOfHuffmanCode 0 = 39
    nHuffmanCodesOfLength 14 = 0
    nHuffmanCodesOfLength 15 = 0
    TableClass = 1
    HuffmanTableIdentifier = 1
    nHuffmanCodesOfLength 0 = 0
    nHuffmanCodesOfLength 1 = 2
    ValueOfHuffmanCode 0 = 0
    ValueOfHuffmanCode 1 = 1
    nHuffmanCodesOfLength 2 = 1
    ValueOfHuffmanCode 0 = 2
    nHuffmanCodesOfLength 3 = 2
    ValueOfHuffmanCode 0 = 17
    ValueOfHuffmanCode 1 = 3
    nHuffmanCodesOfLength 4 = 2
    ValueOfHuffmanCode 0 = 33
    ValueOfHuffmanCode 1 = 4
    nHuffmanCodesOfLength 5 = 5
    ValueOfHuffmanCode 0 = 240
    ValueOfHuffmanCode 1 = 49
    ValueOfHuffmanCode 2 = 18
    ValueOfHuffmanCode 3 = 19
    ValueOfHuffmanCode 4 = 5
    nHuffmanCodesOfLength 6 = 7
    ValueOfHuffmanCode 0 = 65
    ValueOfHuffmanCode 1 = 81
    ValueOfHuffmanCode 2 = 209
    ValueOfHuffmanCode 3 = 34
    ValueOfHuffmanCode 4 = 50
    ValueOfHuffmanCode 5 = 21
    ValueOfHuffmanCode 6 = 6
    nHuffmanCodesOfLength 7 = 8
    ValueOfHuffmanCode 0 = 113
    ValueOfHuffmanCode 1 = 129
    ValueOfHuffmanCode 2 = 145
    ValueOfHuffmanCode 3 = 161
    ValueOfHuffmanCode 4 = 177
    ValueOfHuffmanCode 5 = 193
    ValueOfHuffmanCode 6 = 241
    ValueOfHuffmanCode 7 = 20
    nHuffmanCodesOfLength 8 = 7
    ValueOfHuffmanCode 0 = 97
    ValueOfHuffmanCode 1 = 66
    ValueOfHuffmanCode 2 = 82
    ValueOfHuffmanCode 3 = 130
    ValueOfHuffmanCode 4 = 194
    ValueOfHuffmanCode 5 = 51
    ValueOfHuffmanCode 6 = 7
    nHuffmanCodesOfLength 9 = 5
    ValueOfHuffmanCode 0 = 114
    ValueOfHuffmanCode 1 = 162
    ValueOfHuffmanCode 2 = 210
    ValueOfHuffmanCode 3 = 35
    ValueOfHuffmanCode 4 = 36
    nHuffmanCodesOfLength 10 = 8
    ValueOfHuffmanCode 0 = 225
    ValueOfHuffmanCode 1 = 98
    ValueOfHuffmanCode 2 = 146
    ValueOfHuffmanCode 3 = 242
    ValueOfHuffmanCode 4 = 99
    ValueOfHuffmanCode 5 = 115
    ValueOfHuffmanCode 6 = 52
    ValueOfHuffmanCode 7 = 22
    nHuffmanCodesOfLength 11 = 3
    ValueOfHuffmanCode 0 = 178
    ValueOfHuffmanCode 1 = 226
    ValueOfHuffmanCode 2 = 211
    nHuffmanCodesOfLength 12 = 1
    ValueOfHuffmanCode 0 = 227
    nHuffmanCodesOfLength 13 = 0
    nHuffmanCodesOfLength 14 = 0
    nHuffmanCodesOfLength 15 = 0
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	toaster-ijg.jpg 
Views:	250 
Size:	16.2 KB 
ID:	2087   Click image for larger version. 

Name:	toaster-photoshopcs5.jpg 
Views:	243 
Size:	16.2 KB 
ID:	2088  

  12. #12
    Member
    Join Date
    Apr 2012
    Location
    Stuttgart
    Posts
    430
    Thanks
    1
    Thanked 93 Times in 54 Posts
    Quote Originally Posted by caveman View Post
    Just a stupid question, does your lib write all the DHT (Huffman tables) and DQT (Quantization tables) in a single marker (Adobe Photoshop does it this way and it saves a few bytes) or does it only enclose one table per marker (IJG libjpg does it this way)? Sample files attached.
    They go all into one marker, i.e. one marker for all tables, one marker for all quantization matrices. For optimal compression, you should of course use the optimized Huffman option.

  13. #13
    Member caveman's Avatar
    Join Date
    Jul 2009
    Location
    Strasbourg, France
    Posts
    190
    Thanks
    8
    Thanked 62 Times in 33 Posts
    Quote Originally Posted by thorfdbg View Post
    They go all into one marker, i.e. one marker for all tables, one marker for all quantization matrices. For optimal compression, you should of course use the optimized Huffman option.
    Interesting!
    And do you know if trying different permutations of the Huffman values of the same size (they are usually sorted in growing order, I don't think this is mandatory) could hinder the occurrence of FF and subsequent 00 bytes in the compressed data? (this is done to avoid false marker detection)

  14. #14
    Member
    Join Date
    Apr 2009
    Location
    here
    Posts
    200
    Thanks
    161
    Thanked 108 Times in 64 Posts
    windows compile:

    http://rghost.net/41704129

  15. #15
    Member
    Join Date
    Apr 2012
    Location
    Stuttgart
    Posts
    430
    Thanks
    1
    Thanked 93 Times in 54 Posts
    Quote Originally Posted by caveman View Post
    Interesting! And do you know if trying different permutations of the Huffman values of the same size (they are usually sorted in growing order, I don't think this is mandatory) could hinder the occurrence of FF and subsequent 00 bytes in the compressed data? (this is done to avoid false marker detection)
    Actually, I haven't thought about this yet. False markers and bitstuffing may appear as soon as you have a code with more than eight consecutive ones, which - due to the way how the huffman tables are built - appear automatically as soon as you have codes longer than eight bit. One could probably avoid such codes by assigning the corresponding bit sequences to codewords that are very rare or do not appear at all, but that also enlarges the code of the other sequences. Actually, this sounds like a nice optimization problem I haven't thought trough yet. However, please note that not all bits are huffman coded, and ff-bytes can also appear due to the raw bits inserted into the bitstream, thus, this won't avoid byte-stuffing in all cases.

  16. #16
    Member
    Join Date
    Apr 2012
    Location
    Stuttgart
    Posts
    430
    Thanks
    1
    Thanked 93 Times in 54 Posts
    Quote Originally Posted by load View Post
    Thanks!

  17. #17
    Member
    Join Date
    Apr 2012
    Location
    Stuttgart
    Posts
    430
    Thanks
    1
    Thanked 93 Times in 54 Posts
    Just uploaded a new release to git:
    https://github.com/thorfdbg/libjpeg
    This improves the performance of the AC coding modes by using separate contexts for chroma/luma. It also fixes the workaround for the JPEG LS part-2 color transformation which did the overflow check incorrectly.
    The repository also includes a revised version of the ISO working draft. Note, however, that this is an early WD, the software does not conform to this WD, nor has the WD been decided upon.
    The JPEG online test at
    http://jpegonline.rus.uni-stuttgart.de/index.py
    has been improved to also measure mean multiscale SSIM as one of the metrics. Have patience, though, measuring SSIM takes more time than PSNR or peak error.

  18. #18

  19. #19
    Member Alexander Rhatushnyak's Avatar
    Join Date
    Oct 2007
    Location
    Canada
    Posts
    231
    Thanks
    36
    Thanked 78 Times in 42 Posts
    Tested compression quality and bit-exactness on LPCB files.

    Code:
    2,010,200,240 bytes   jpeg -c -p         %1.ppm %1.qvj   http://imagecompression.info/gralic/list-qvj.txt
    1,440,226,595 bytes   jpeg -c -p -a      %1.ppm %1.qva   http://imagecompression.info/gralic/list-qva.txt
    1,290,129,169 bytes   jpeg -c -ls 0      %1.ppm %1.qv0   http://imagecompression.info/gralic/list-qv0.txt
    1,293,859,667 bytes   jpeg -c -ls 1      %1.ppm %1.qv1   http://imagecompression.info/gralic/list-qv1.txt
    1,299,798,722 bytes   jpeg -c -ls 2      %1.ppm %1.qv2   http://imagecompression.info/gralic/list-qv2.txt
    1,210,043,117 bytes   jpeg -c -ls 0 -cls %1.ppm %1.qc0   http://imagecompression.info/gralic/list-qc0.txt
    1,217,125,694 bytes   jpeg -c -ls 1 -cls %1.ppm %1.qc1   http://imagecompression.info/gralic/list-qc1.txt
    1,222,696,140 bytes   jpeg -c -ls 2 -cls %1.ppm %1.qc2   http://imagecompression.info/gralic/list-qc2.txt
    In the latter three cases the decompressed images aren't bit-exact in many cases, e.g. image from http://www.imagecompression.info/gralic/PIA13862ppm.rar :

    000335a5: 00 FF
    // skipped 2239 lines, all ending with 00 FF
    0018f677: 00 FF

    Are there other sets of options for lossless compression?

    This newsgroup is dedicated to image compression:
    http://linkedin.com/groups/Image-Compression-3363256

  20. #20
    Member
    Join Date
    Apr 2012
    Location
    Stuttgart
    Posts
    430
    Thanks
    1
    Thanked 93 Times in 54 Posts
    Quote Originally Posted by Alexander Rhatushnyak View Post
    Tested compression quality and bit-exactness on LPCB files. In the latter three cases the decompressed images aren't bit-exact in many cases, e.g. image from http://www.imagecompression.info/gralic/PIA13862ppm.rar : 000335a5: 00 FF // skipped 2239 lines, all ending with 00 FF 0018f677: 00 FF Are there other sets of options for lossless compression?
    Fixed. I misplaced a > where I should have a >=, only in the blue component. All other components are fine.
    Other options for lossless: "-r -h" uses residual coding, use "-q" to select a "base quality" which will be visible to legacy decoders. Useful values for -q depend on the image. For complex images, q can be as low as 60, for simple images, go higher. I would pick 90 as base layer quality. This does not require "-c", also works with YCbCr. This is a new mode in the JPEG extensions.
    You can also use hierarchical coding for lossless, but only in RGB space, and only with this coder (simply because the DCT isn't standardized in the old standard). For this, you can use the original DCT image as base layer, or a downscaled version of the DCT image as base image. Use -q to select a base quality, -y 0 for a same-resolution base layer, and -y 1 for a downscaled image as base layer. Thus:
    -h -y 0 -c -q 90
    or
    -h -y 1 -c -q 90
    Again, only for this code, not lossless in general. "-r" for residual coding works best.

  21. #21
    Member
    Join Date
    Apr 2012
    Location
    Stuttgart
    Posts
    430
    Thanks
    1
    Thanked 93 Times in 54 Posts
    Quote Originally Posted by Alexander Rhatushnyak View Post
    Tested compression quality and bit-exactness on LPCB files. In the latter three cases the decompressed images aren't bit-exact in many cases, e.g. image from http://www.imagecompression.info/gralic/PIA13862ppm.rar : 000335a5: 00 FF // skipped 2239 lines, all ending with 00 FF 0018f677: 00 FF Are there other sets of options for lossless compression?
    Fixed. I misplaced a > where I should have a >=, only in the blue component. All other components are fine.
    Other options for lossless: "-r -h" uses residual coding, use "-q" to select a "base quality" which will be visible to legacy decoders. Useful values for -q depend on the image. For complex images, q can be as low as 60, for simple images, go higher. I would pick 90 as base layer quality. This does not require "-c", also works with YCbCr. This is a new mode in the JPEG extensions.
    You can also use hierarchical coding for lossless, but only in RGB space, and only with this coder (simply because the DCT isn't standardized in the old standard). For this, you can use the original DCT image as base layer, or a downscaled version of the DCT image as base image. Use -q to select a base quality, -y 0 for a same-resolution base layer, and -y 1 for a downscaled image as base layer. Thus:
    -h -y 0 -c -q 90
    or
    -h -y 1 -c -q 90
    Again, only for this code, not lossless in general. "-r" for residual coding works best. Again, this *is* backwards compatible, though legacy decoders will not be able to decode lossless, but only to the base-layer quality you specified. "-y" "should" work with legacy decoders theoretically, except that there is no legacy decoder I'm aware that implemented hierarchical.

  22. #22
    Member Alexander Rhatushnyak's Avatar
    Join Date
    Oct 2007
    Location
    Canada
    Posts
    231
    Thanks
    36
    Thanked 78 Times in 42 Posts
    Added to LPCB.
    Quote Originally Posted by load View Post
    Win32 compile:
    How was it built, and is the Linux compile ~8 times slower than CharLS too?

    This newsgroup is dedicated to image compression:
    http://linkedin.com/groups/Image-Compression-3363256

  23. #23
    Member Alexander Rhatushnyak's Avatar
    Join Date
    Oct 2007
    Location
    Canada
    Posts
    231
    Thanks
    36
    Thanked 78 Times in 42 Posts
    Okay, will try these five next time:
    jpeg -c -ls 0 -cls
    jpeg -c -ls 1 -cls
    jpeg -c -ls 2 -cls
    jpeg -h -y 0 -c -q 90
    jpeg -h -y 1 -c -q 90

    -r isn't needed for the latter two variants, is it?

    This time only the following five were added to LPCB:
    jpeg -c -p
    jpeg -c -p -a
    jpeg -c -ls 0
    jpeg -c -ls 1
    jpeg -c -ls 2

    This newsgroup is dedicated to image compression:
    http://linkedin.com/groups/Image-Compression-3363256

  24. #24
    Member
    Join Date
    Apr 2012
    Location
    Stuttgart
    Posts
    430
    Thanks
    1
    Thanked 93 Times in 54 Posts
    Quote Originally Posted by Alexander Rhatushnyak View Post
    Okay, will try these five next time: jpeg -c -ls 0 -cls jpeg -c -ls 1 -cls jpeg -c -ls 2 -cls jpeg -h -y 0 -c -q 90 jpeg -h -y 1 -c -q 90 -r isn't needed for the latter two variants, is it?
    For none of them. Residual coding is based on the "still lossy" modes - not required for modes that are already lossless. Thus, simply "jpeg -h -r -q XX" or "jpeg -a -r -q XX", where the latter (ac-coding of legacy and residual data) will likely not end up in the specs.

  25. #25
    Member
    Join Date
    Apr 2012
    Location
    Stuttgart
    Posts
    430
    Thanks
    1
    Thanked 93 Times in 54 Posts
    New version on git. Really only a single misplaced > instead of >=. Bummer!

  26. #26
    Member
    Join Date
    Apr 2012
    Location
    Stuttgart
    Posts
    430
    Thanks
    1
    Thanked 93 Times in 54 Posts
    Quote Originally Posted by Alexander Rhatushnyak View Post
    Added to LPCB. How was it built, and is the Linux compile ~8 times slower than CharLS too?
    Wouldn't surprise me. For example, the Golomb decoder in the current version is "clear", but "outright stupid", and it would be obvious how to improve its performance: It currently reads one bit at a time, counting bits. It's really not necessary to do it this way. However, speed wasn't quite the issue here. Speed might come later.

  27. #27
    Member
    Join Date
    Apr 2009
    Location
    here
    Posts
    200
    Thanks
    161
    Thanked 108 Times in 64 Posts

  28. #28
    Member
    Join Date
    Apr 2012
    Location
    Stuttgart
    Posts
    430
    Thanks
    1
    Thanked 93 Times in 54 Posts
    Just released a tiny fix - apparently, the code picked up the wrong Huffman AC coding table in the progressive mode due to a stupid typo that was completely unnoticed. Fixed that.

  29. #29
    Member
    Join Date
    Apr 2009
    Location
    here
    Posts
    200
    Thanks
    161
    Thanked 108 Times in 64 Posts
    ok, here's the win32 compile. maybe it works.

    http://www.mirrorcreator.com/files/P...jpeg.rar_links

Similar Threads

  1. Quo Vadis JPEG - Another update (2)
    By thorfdbg in forum Data Compression
    Replies: 17
    Last Post: 28th September 2012, 21:10
  2. Quo Vadis JPEG - Another update
    By thorfdbg in forum Data Compression
    Replies: 7
    Last Post: 11th September 2012, 20:09
  3. Quo Vadis JPEG - an update
    By thorfdbg in forum Data Compression
    Replies: 8
    Last Post: 31st July 2012, 17:35
  4. Quo Vadis JPEG - New Movements in Still Image Compression
    By thorfdbg in forum Data Compression
    Replies: 37
    Last Post: 14th June 2012, 20:47
  5. zpaq 1.02 update
    By Matt Mahoney in forum Data Compression
    Replies: 11
    Last Post: 10th July 2009, 00:55

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •