Antipattern: Fire & Forget

When dealing with an external service, like an API call, you can: 1) Check the response, catching for any and all errors, and gracefully handle any failures or responses from the external API, 2) Check the response for a simple success or failure and don’t gracefully handle anything else, or 3) Don’t check the response whatever; either assume that the request always succeeds or simply don’t care if it fails.

The third strategy is what we like to call “fire and forget.” While it might a valid approach in very rare circumstances, it’s a terrible idea.

Fire-and-forget is an antipattern. It is something done quickly by developers for expedience, and it’s easy to see why.

Your app should have excellent introspection, reporting, and alerting. That means that when you have an API call, you tie that API call to a service, or even maybe a database record. Creating database records for this kind of stuff is non-standard Rails, but it’s something I’ve seen successfully employed many times.

Introspection

Any time that your app can look at itself, or introspect, it does so to tell you something about itself. For example: how backed-up a queue is, or how many requests to an external service have been sent in the last hour, etc— this is an example of introspection. Sometimes you might use an external tool for monitoring you app (that isn’t really “introspection” but it is in the same set of tools because it gives you insight into your app’s process.)

Reporting

This refers to any time an app writes a report to create a log of something that will be looked at later. Often by a developer, the concept of “report” can be extended to business people, too, of course. Business people will want generated business reports from a system. Anything that the system records and then communicates (in any way) to the developer, business owner, or users is a mechanism of reporting.

Alerting

Alerting is what you do when you have a high-traffic production app that can’t go down. In these cases, you use a service like PagerDuty or Onpage to build an alert system that will proactively set off alarm bells telling you there are problems.

An Example

I will present a pattern here that works for uploading CSV files, processing them, and then calling an external service with the results. To do this, I’ll use ActiveJob (introduced in Rails 4.2), a simple job wrapper to create a base-level job. A job is anything that runs in the background of your app.

In our example, the user’s request will create & enqueue the job. The application will run the job on a separate instance without blocking the user’s request from responding.

In this pattern, the results of the job get saved onto the table where the CSV file is attached. However, I don’t store the actual results themselves.

We’re going to upload a CSV that contains four columns: email, first, last, and zipcode. We’ll do something with this data (what we do is not implemented in this example but instead just left as a placeholder).

First let’s start with a FileUpload object

Create it with a migration:

class CreateFileUploads < ActiveRecord::Migration[6.0]
  def change
    create_table :file_uploads do |t|
      t.string :results
      t.datetime :errored_at
      t.datetime :successful_at
    end
  end
end

Then make the model object in models/file_uplaod.rb

class FileUpload < ApplicationRecord
  has_one_attached :file
end

Configure Active Storage according to the Rails guide here.

Testing the File Upload

This CSV File should be specified in 4 columns, with the data in this order: email, first, last, and zipcode. Do not include a header row or a label row.

The Ruby job will then read this CSV and turn it into a hash where the keys are :email, :first, :last, and :zipcode. It will then pass this on to some other process. Note that in this example, I have specifically not saved the actual data collect back to the FileUpload object.

The File Processor Example

class ProcessCsvJob < ApplicationJob
  queue_as :default
  def perform(*args)
    file_upload_id = args[0][:file_upload_id]
    file_upload = FileUpload.find(file_upload_id)

    temp_file_name = 5.times.collect{letters[rand(letters.length)] }.join

    fw_handle = File.open(@temp_file, "w:UTF-8") {|f|
      f.write(file_upload.file.download.force_encoding('UTF-8'))
    }
    temp_file = "/tmp/#{temp_file_name}.#{ file_upload.file.filename.extension}"
    csv = CSV.new(File.open(temp_file, "r:UTF-8"))

    # it is assumed the file upload has columns in this order
    index = [:email, :first, :last, :zipcode]

    begin
      data = []
      first_line = @csv.first

      csv.each_with_index do |row,i|
        if i==0 && file_upload.first_row_is_labels
          next
        end
        this_line = {}
        [0,1,2,3].each do |i|
          unless row[i].nil?
            # transmute the position-based hash to a key-based array
            this_line[index[i].to_sym] = row[i]
          end
        end
        data << this_line
      end
      results << "successful import"

      # do something with data here

    rescue   StandardError => e
      results = "ERROR: #{e.message} #{e.backtrace if Rails.env.development?}"
      file_upload.update({results: results,
                           errored_at:  Time.current})
      raise(e)
    end

    file_upload.update({results: results,
                        errored_at:  nil,
                        successful_at: Time.current})
    File.delete(temp_file) if temp_file
  end
end

Deconstruct

Let’s deconstruct what we are doing

 file_upload_id = args[0][:file_upload_id]   
 file_upload = FileUpload.find(file_upload_id)

Because args is passed as a splat, we pull the :file_upload_id out of its first parameter.

Next, we’ll create a random file name and save the file to the /tmp folder. We do this to force ActiveStorage to download the file from the AWS bucket and save it locally for fastest processing.

    temp_file_name = 5.times.collect{letters[rand(letters.length)] }.join

    fw_handle = File.open(@temp_file, "w:UTF-8") {|f|
      f.write(file_upload.file.download.force_encoding('UTF-8'))
    }
    temp_file = "/tmp/#{temp_file_name}.#{ file_upload.file.filename.extension}"

Don’t be confused— the real file lives however or where-ever you set up ActiveStorage. It is read with file_upload.file.download.force_encoding('UTF-8'). We’ll cleanup the temp file at the end of our method.

Now we’re ready to read from our CSV

csv = CSV.new(File.open(temp_file, "r:UTF-8"))

Next, I’m going to make an assumption that the order of the columns is email, first, last, zip. That’s defined by my index variable, which is assigned to an ordered array containing these keys

index = [:email, :first, :last, :zipcode]

Then we read the lines of the CSV, pulling out the values by index and building a new hash:

      csv.each_with_index do |row,i|
        if i==0 && file_upload.first_row_is_labels
          next
        end
        this_line = {}
        [0,1,2,3].each do |i|
          unless row[i].nil?
            # transmute the position-based hash to a key-based array
            this_line[index[i].to_sym] = row[i]
          end
        end
        data << this_line
      end
      results << "successful import"

Notice that at the end, I’ve simply stuck “successful import” into the results list. Another option here would be to include the number of accepted and rejected lines that were processed (for example, for duplicates).

Then, I’ve left it unimplemented what to do with the data in this form. See the line # do something with data here

The Rescue & Reraise

Finally— the real meat & potatoes of rescue & re-raise:

rescue   StandardError => e
      results = "ERROR: #{e.message} #{e.backtrace if Rails.env.development?}"
      file_upload.update({results: results,
                           errored_at:  Time.current})
      raise(e)
    end

    file_upload.update({results: results,
                        errored_at:  nil,
                        successful_at: Time.current})

What this code does is fairly straightforward— it uses the CSV parser to turn the data that is read from the CSV into a new hash.

That, in and of itself, is not as interesting as the error reporting pattern we see here. First, notice a few things:

• I have both an errored_at and successful_at timestamp on our FileUpload table. So the fileupload object represents both the uploaded file and the concept of uploading a file and processing the job and what its result might be. Conceptually, if you accept that, this all probably makes sense. (Object purists might want to separate the CSV object from the job and results — which is OK— but I would argue premature and unnecessary. In other words, it isn’t wrong to want to do that — it’s just overkill).
• If there’s a hiccup, we rescue and raise. This is important because rescue and reraise is a significant, albeit imperfect, strategy. There are a number of reasons why too many rescues throughout your app are an anti-pattern themselves, but as a beginner to Rails, you need to learn it both ways (using raise and not using raise). In short, when you can use ensure (beyond the scope of this article), do so because raise causes other problems. Sometimes you won’t be able to use ensure, and then the rescue & reraises strategy make sense.
• When something bad happens, I rescue from the code the exception and save the results back to the database for the developer to see.
• In development-only, I also stuffed in the backtrace for the developer. (You can put it in production too, but typically non-developers do not want to see stack traces so it depends on how much this feedback loop goes to the developer or to the end-user.) This is a good example of how introspection can be even more sophisticated for development-only environments.
• Finally, the code saves a message helping the user back to the FileUpload table. I usually implement these with a simple front-end poller to the back end that polls every 4 seconds for either errored_at or successful_at to be true.

This is just one example of a good introspection & error reporting. I hope this brief taste of erroring and reporting has whet your appetite for error capturing, log analysis, and having good insight into what is going on. Having insight into what is going on is essential in quickly diagnosing any issues or debugging problems once they happen.