JPEGs optimization

Hi all.

I have found these two papers:
[list]
[]Joint Optimization of Run-Length Coding, Huffman Coding, and Quantization Table With Complete Baseline JPEG Decoder Compatibility | IEEE Journals & Magazine | IEEE Xplore
[
]Joint Optimization of Run-Length Coding, Huffman Coding and Quantization Table with Complete Baseline JPEG Compatibility | IEEE Conference Publication | IEEE Xplore
[/list]
which present a compression method fully compatible with existing JPEG decoders, that achieve a 30% reduction on file size without noticeable degradation and the method is quick (0.2 seconds for typical images).

It is also claimed that this algorithm achives the maximum compression that JPEG encoders fully compatible with existing JPEG decoders can achieve.

As photoshop does not have it, do you know if there is a software that implements this algorithm?

You couldn’t have asked at a better time - this just rolled across TechCrunch: New Startup JPEGmini Reduces Photos’ Size, Not Their Quality – TechCrunch

I’m not sure that jpegmini is using those kind of “free” (i.e. encoding) optimizations (joint optimization of quantization table + run-length encoding + Huffman table). Last time I tried it, it seemed to me that it was optimizing the image, but not the encoding (+30% size reduction). I’ll give it another try and see.

Yeah, I haven’t seen any of the advanced optimizations applied to libjpeg (which is what just about everything except for Photoshop is based on). jpegtran can do some basic huffman table optimizations to squeeze a little space from images but not on the order of what the researchers were seeing.

I believe that jpegmini is not using any optimization of quantization tables + run-length encoding + Huffman encoding, because by using optimized quantization tables alone, I get images without visible artifacts and significantly smaller. At best, I believe that jpegmini is only using some smoothing.

To reach this conclusion, I used the following quantization table optimizer: http://pages.cs.wisc.edu/~ratnakar/rdopt.tar.gz

Which is described on this paper: Extending RD-OPT with Global Thresholding for JPEG Optimization

Curiously, the authors of the two papers I mentioned before have published another one where they get a 40-45% file size reduction by using optimization of quantization tables + run-length encoding + arithmetic encoding:

[url=http://ieeexplore.ieee.org/ielx5/5075165/5090078/05090215.pdf?arnumber=5090215]Joint optimization of run-length coding, context-based arithmetic coding and quantization step sizes | IEEE Conference Publication | IEEE Xplore

Given that the respective patents are 8 and 6 years old:

[list]
[][url]http://www.google.com/patents?id=rKrRAAAAEBAJ&printsec=abstract&zoom=4#v=onepage&q&f=false[/url]
[
][url]http://www.google.com/patents?id=7JKaAAAAEBAJ&printsec=abstract&zoom=4#v=onepage&q&f=false[/url]
[/list]

I start to wonder if the aforementioned research, funded by public money, will ever be translated on a JPEG compressor for sale on the market using those algorithms.

I believe that jpegmini is not using any optimization of quantization tables + run-length encoding + Huffman encoding, because by using optimized quantization tables alone I get images without visible artifacts and significantly smaller. At best, I believe that jpegmini is only using some smoothing.

To reach this conclusion, I used the following quantization table optimizer: http://pages.cs.wisc.edu/~ratnakar/rdopt.tar.gz

Which is described on this paper: Extending RD-OPT with Global Thresholding for JPEG Optimization

Curiously, the authors of the two papers I mentioned before have published another one where they get a 40-45% file size reduction by using optimization of quantization tables + run-length encoding + arithmetic encoding:

[url=http://ieeexplore.ieee.org/ielx5/5075165/5090078/05090215.pdf?arnumber=5090215]Joint optimization of run-length coding, context-based arithmetic coding and quantization step sizes | IEEE Conference Publication | IEEE Xplore

Given that the respective patents are 8 and 6 years old:

[list]
[][url]http://www.google.com/patents?id=rKrRAAAAEBAJ&printsec=abstract&zoom=4#v=onepage&q&f=false[/url]
[
][url]http://www.google.com/patents?id=7JKaAAAAEBAJ&printsec=abstract&zoom=4#v=onepage&q&f=false[/url]
[/list]

I start to wonder if the aforementioned research, funded by public money, will ever be translated on a JPEG compressor for sale on the market using those algorithms.

I believe that jpegmini is not using any optimization of quantization tables + run-length encoding + Huffman encoding, because by using optimized quantization tables alone I get images without visible artifacts and significantly smaller. At best, I believe that jpegmini is only using some smoothing.

To reach this conclusion, I used the following quantization table optimizer: http://pages.cs.wisc.edu/~ratnakar/rdopt.tar.gz

Which is described on this paper: Extending RD-OPT with Global Thresholding for JPEG Optimization

Curiously, the authors of the two papers I mentioned before have published another one where they get a 40-45% file size reduction by using optimization of quantization tables + run-length encoding + arithmetic encoding:

[url=http://ieeexplore.ieee.org/ielx5/5075165/5090078/05090215.pdf?arnumber=5090215]Joint optimization of run-length coding, context-based arithmetic coding and quantization step sizes | IEEE Conference Publication | IEEE Xplore

Given that the respective patents are 8 and 6 years old:

[list]
[][url]http://www.google.com/patents?id=rKrRAAAAEBAJ&printsec=abstract&zoom=4#v=onepage&q&f=false[/url]
[
][url]http://www.google.com/patents?id=7JKaAAAAEBAJ&printsec=abstract&zoom=4#v=onepage&q&f=false[/url]
[/list]

I start to wonder if the aforementioned research, funded by public money, will ever be translated on a JPEG compressor for sale on the market using those algorithms.

I believe that jpegmini is not using any optimization of quantization tables + run-length encoding + Huffman encoding, because by using optimized quantization tables alone I get images without visible artifacts and significantly smaller. At best, I believe that jpegmini is only using some smoothing.

To reach this conclusion, I used the following quantization table optimizer: http://pages.cs.wisc.edu/~ratnakar/rdopt.tar.gz

Which is described on this paper: Extending RD-OPT with Global Thresholding for JPEG Optimization

Curiously, the authors of the two papers I mentioned before have published another one where they get a 40-45% file size reduction by using optimization of quantization tables + run-length encoding + arithmetic encoding:

[url=http://ieeexplore.ieee.org/ielx5/5075165/5090078/05090215.pdf?arnumber=5090215]Joint optimization of run-length coding, context-based arithmetic coding and quantization step sizes | IEEE Conference Publication | IEEE Xplore

Given that the respective patents are 8 and 6 years old:

[list]
[][url]http://www.google.com/patents?id=rKrRAAAAEBAJ&printsec=abstract&zoom=4#v=onepage&q&f=false[/url]
[
][url]http://www.google.com/patents?id=7JKaAAAAEBAJ&printsec=abstract&zoom=4#v=onepage&q&f=false[/url]
[/list]

I start to wonder if the aforementioned research, funded by public money, will ever be translated on a JPEG compressor for sale on the market using those algorithms.

Of the open source tools, I am still a big fan of jpegoptim and jpegtran.

Aaron

Sorry if i am hijacking this thread, but it’s a very closely related question. But what is the best image resizing and compressing library that has been exposed via php? Currently we’re using imagick, any better ones out there?

p.s. Yes i know you can make calls to any on system executable via php, but i have that ability disabled for security sake, so it has to be exposed through php.

super

We all know too well that JPEG images are the main cause of page bloatedness due to the increase use of images and due to the inefficient JPEG compression algorithm.

On the other hand, PackJPG is a lossless compression software, is now open source under the terms of LGPL v3 and typically
it reduces the file size of a JPEG file by 23%…24% [cf. packJPG Usage Information].

Now follow some questions:

[list]
[]Do someone know if there are already browsers implementing this type of compression for JPEG images?
[
]Are there ways of avoiding the IE Varies problem?
[*]Is there a HTTP 2.0 way of allowing cacheable content negotiation and the inclusion of this [PackJPG] lossless compression algorithm?
[/list]
IMO, JPEGs are the elephant in the room and paradoxically remain unoptimized while time is being wasted with higher order optimizations yielding lower returns.

Lena… I am not sure I agree. Jpeg is actually the most optimized lossy compression format I think. The only way to reduce it further is through binary compression. That may shrink the file size, but that will increase the CPU utilization to uncompress it, which will only serve to increase the browser rendering time. Palleted formats like .gif and 8 bit png and others can be more efficient, but only because they are encoding the image data to a limited number of colors in the pallet, and let’s not forget about vector based formats, but as you are probably aware, both of these do not serve well for images like photographs and graphics containing gradations. I would say, it’s more of the choose the best tool for the job than anything else.

The problem with PackJPG (as best as I can tell) is that the output is a compressed file that is no longer a JPEG and needs to be decompressed with PackJPG. At that point you’re either trying to get a new image format adopted or a new content-encoding (akin to gzip for text).

At that point you’re better off using WebP which already has decent browser support and shows better improvement than loslessly compressing JPEGs.

@ jarrod1937

I prize your sense o humor, calling JPEG “the most optimized lossy compression format”, as it can be further compressed losslessly another 23…24%.

@ pmeenan

I was thinking along the line of a new content-encoding. That’s why I also mentioned the IE Varies problem.

Yea, you have a good point concerning decent browser support of WebP.

Concerning WebP, is there a way of getting content negotiation [JPEG <-> WebP] and cacheability at the same time without resorting to the auto-defeating Vary: User Agent header? Alternatively, is there a way of doing it with HTTP 2.0?

The current effort for negotiation is around vary: accept. Opera has always announced webp support in their accept headers and Chrome added it recently so all/both of the browsers that support WebP announce it.

CDN/proxy support for vary: accept is pretty weak right now though it is improving (Akamai is one of the first that I know that is supporting it).

HTTP 2.0 doesn’t really add any mechanisms that help other than being encrypted by default so you can feel safe that you’re bypassing any intermediaries.

Indeed, which is why I said the most optimized lossy format, any format can losslessly be compressed more, but it’s a matter of CPU utilization required to achieve that both on the compression end, and the decompression end. CPU utilization during browser rendering is not something to ignore, and lossless compression techniques that can be tacked on only add to this, not to mention the diminishing returns it takes to compress the file that much. It takes considerably more processing power to compress an image more and more as the low hanging fruit of the binary level compression gets taken. Technically you can tell your browser to do gzip compression right now for images, but it will actually slow down your overall page display time (from my testing) for the little bit of download size it saves. I’m know there are better lossless compression techniques that can be used other than those used by gzip, but I still doubt the tradeoff is worth it, and those would require a new format adoption as mentioned, not an easy task :wink:

Why not use the HTTP response header Link? The URL is the URL of a directory containing the images and the rel attribute [with a chosen and agreed name, for instance jpg->webp] is used as a hint for browsers to ask “image.webp” instead of “image.jpg” for the images located in a sub directory of the aforementioned URL.

It seems to be past proof [no side effects with current browsers and I would guess that proxies already support this response header (i.e. Link)] and future proof [it’s up to the browsers to opt in].

Not sure I understand where you’d pass the header or how you’d use it. Using it on the base page ties all sorts of conventions in with it’s use that are unlikely to work for a lot of sites and applying it to individual resources implies that the image was already requested so it would cause a double fetch.

We considered a srcset style solution that required markup to define alternate image formats but adoption would take forever and it’s not all that clean.

By delivering different file formats for the same URI based on advertised capabilities, it opens up support for automatic image transcoding services and delivery without having to change the pages themselves at all.

New compression algorithms will not help unless the algorithms are implemented in all Browsers. If a new algorithm was added to the major Browsers today it will be many years before that algorithm would be practical to use because older Browsers would not be able to render the image.

Yahoo Smush.it is hard to beat: http://www.smushit.com/ysmush.it/