How to get your Heroku app to optimize your images for you automagically using Paperclip, image_optim and special buildpacks

Site speed can be said to be the number one issue facing web developers today.

Whether it’s this KISS Metrics block post, another KISS Metrics block post, study after study show that delivering your content fast, fast, fast is make-or-break factor in today’s web economony. That’s why it’s so important that your images are optimized for the web.

Photoshop and other tools export notoriously large files — well over 1 MB. This is unacceptable in today’s world, where 33% of mobile users in the US are on 3G connections.

If you’re on Rails using Paperlcip, I’ve got a great solution to explore for you today: Image-Optim. You can automagically compress all your images, inside the Rails pipeline and also the ones you upload with Paperclip. On Heroku, you’ll need to use two special buildpacks to make this work. As well, because Heroku uses an ephemeral file system, Paperclip needs to be configured to use an AWS bucket as its storage.

Before continuing, figure out which Heroku stack you are on. For most Rails apps, that will either be the older Cedar 14 (released in 2014) or the newer Heroku 16 (released in 2016) stack. In all subsequent examples “your-app-name” means the actual name of your Heroku app.

To find out, use heroku stack -a your-app-name

If you’re on Heroku 16, for example, you’ll see this, with “heroku-16” appearing with an asterisks and in green:

=== your-app-name Available Stacks
* heroku-16
 cedar-14
 container

There are slightly different buildpacks for the different stacks so be sure to confirm which one your app is on before continuing.

Cedar 14

First, refer to my blog post from last year, about how to add the ImageMagick buildpack to your Cedar-14 herokubuikd.

The instructions above will direct you to do add this buildpack first:

Then add another buildpack to your Heroku environment

(you’ll note here you are using the index flag to put this buildpack into position 2 because you already should have the Imaegmagick buildpack at position 1)

You should now have 3 buildpacks, which can be check with heroku buildpacks like so:

Heroku 16 (Updated 4/22/2018)

For Heroku 16 the current working solution is slightly different.

Please note here you are counter-intuitively pushing both buildpacks into “position #1” of your buildpacks. Since they are ordered (and the order matters), Heroku will bump the first one you push into position #2 when you push the second one. Needless to say, run them in exactly this order.

and

You should see a message like this:

If you mess it up, use heroku buildpacks:remove -i X (where X is the position # you want to remove to remove the errant ones. The heroku/ruby is there by default.)

Gemfile setup
Then add to your Gemfile these 4 gems, (for the sake of this post I will assume you already have gem ‘paperclip’ in your Gemfile).

To get this working on Heroku, you’ll actually need to work through a few more steps: database setup, AWS. For the lazy, check out the example which you can find at the end of the blog post.

Here’s my has_attached_file. In this example, I’m creating only two styles: a thumbnail, and an optimized version.

Notice that I’ve turned off lossless compression, in other words, allow_lossy: true

With this safeguard on (allow_lossy: false, which is default), I’m usually able to only get an image down to about 75% of its original size.

A large 909KB file was only reduced down to 730 KB; whereas Optimizilla was able to get it down to a whopping 189 KB.

With the safety guard switched off allow_lossy: true, I get much better results but much worse quality.

1st Example
Here, I define a thumb and a optimized.

2nd Example
Here, I define a thumb and a large.

Remember, when configured together the whole thing looks like this, see the “Per style setting” on this paperlclip-optimizer doc:

(this is an example that mimics the paperclip-optimizer docs)

The Magic Sauce

The docs say you should have allow_lossy set to its default, which is is false. Using this setting this way means your images come out with no quality loss. In my tests, I’ve found that this setting should be turned on, overriding the default.

I recommend paying attention to two important settings
jpegoptim max_quality – 0 through 4, with 4 being best quality
jpegrecompress quality – 0 through 100%, with 100% being best quality

In my tests, I’ve found that the following are acceptable for production websites with high-quality images.

Option A
jpegoptim max_quality quality: 4; jpegrecompress quality: 80
this yields 20-40% compress images of the uncompressed JPGS

Option B
jpegoptim max_quality quality: 3; jpegrecompress quality: 60
this yields 10-20% compress images of the uncompressed JPGS

As far as I can tell, jpegoptim max_quality setting appears to have very little effect on the file size, where as the jpegrecompress quality setting has the most dramatic effect, especially on larger files. The values for jpegrecompress quality are 0-4, with 0 being the least quality (most savings) and 4 being the best quality. With a settle of 4, you can’t perceive any quality loss, but you don’t get the benefit of extremely optimized files. I recommend a setting of 3, which is barely noticeable in terms of quality loss but a significant boost in file size.

Test App

I threw together a test demo here. You can read the source of this demo app on Github.

Please note this Heroku (production) app is configured with a few extra goodies:

AWS setup for a basic Amazon S3 bucket
Postgres setup for Heroku

The jpegoptim max_quality and the jpegrecompress max_quality

You’ll notice my example app here creates 5 different versions, using the same jpegoptim setting (jpegoptim: { allow_lossy: true, strip: :all, max_quality: 75 }, but 5 different quality settings on the jpegrecompress setting (be sure to note the jpegrecompress takes a quality parameter of 0-4; the jpegoptim setting takes a max_quality setting of 0-100)

In my example app I’ve split the settings for jpegrecompress and jpegoptim into a global setting and a per-style setting. Its setup differs from the examples above.

In my sample app, I’ve set the jpegoptim max_quality setting to 75 and created five different jpegrecompress settings: 0, 1, 2, 3, and 4, named:

optimized_compress_0
optimized_compress_1
optimized_compress_2
optimized_compress_3
optimized_compress_4

(you’ll see these in the has_attached_file in app/models/asset.rb)

So go ahead, upload a color-rich un-optimized image. In my experiments, I found that quality settings 4, 3, 2, and 1 yield approximately the same file size, with only a small dip in file size when you went down to 0.

However, the noticeable loss in quality begins to happen even at quality setting 3, so it seems to me why not use quality setting 4. You will be baking in an automatic guard against very large un-optimized images coming into your app. You’ll need to play around with these two settings.

Important Addendum (2017-03-09)

I am adding an important addendum to this post. After switching around my buildpacks on Heroku, I ran into a strange Sprockets error:

The only way I found to fix this was to purge my assets in slug compilation. This will mean your 1st push after purging will take an extra long time to slug compile.

If you run into that error, do this before you push to your environment:

Also see this Stack overflow post. I corresponded with the maintainer of Sprockets regarding this issue, and he suggested later versions of Sprockets may have addressed this issue (we are on Rails 4.1 with Sprockets 2.12.4).

Update 08/12/2019

From the PaperclipOptimizer installation: